Governments at every level and around the world are embracing the opportunity to share and open up their data to broader use by individuals, businesses and community organizations. Recent technological advances in connectivity, cloud storage and computing, analytic techniques such as machine learning, and the Internet of Things, are allowing a dramatic expansion in the types of data gathered and especially in the ways to disseminate and use this data.
On Nov. 19, the Microsoft Innovation & Policy Center in Washington, D.C. hosted a discussion on effective and responsible use of open government data. (You can watch the full discussion below.)
The event began with Rep. Derek Kilmer, from the state of Washington, speaking about the potential for federal agencies to use their data to be more efficient and better fulfill their missions in areas such as economic development and veterans’ services. He is working on legislation that would codify the president’s 2013 executive order making data “open by default,” and would support key steps, such as data inventories, by federal agencies.
The event then shifted to a panel discussion of the open government data landscape, with expert perspectives from:
- Jeff Chen, the inaugural chief data scientist at the Department of Commerce.
- Emily Shaw, U.S. Civic Technologies Researcher at MySociety.
- Ryan Calo of the University of Washington School of Law.
- Joel Gurin, the president of the Center for Open Data Enterprise.
During the discussion, the panelists touched on a number of key areas:
Evolving open data landscape: Open government data can increase the quality and understanding of people’s interaction with their governments, but there are other potential implications to its broad availability as well. For example, while focus groups interviewed in Seattle were hopeful that the information could be used for government accountability and economic impact, they also raised concerns about it being used for marketing or impacting the safety of government workers. Moreover, as sharing government data becomes more common, the goals expand – from providing more transparent information about government activities – to using the data to realize economic value in areas like precision agriculture and medicine, housing and siting energy generation.
Responsible use and trust: Governments hold a great deal of information that touches on individuals’ personal lives, health and finances. Shaw referenced how good governance models and robust processes and structures can help governments assess when and how to release individual-level “microdata.” Calo specifically described how there is an opportunity for governments to develop and implement standard clauses governing vendors’ responsibilities for privacy and security practices, as described in a recent study of the City of Seattle’s open data policies and procedures.
Another element is developing methods to weigh the balance between public and private goods in determining how granular personal data might be released and analyzed in support of larger goals, e.g. oversight of discriminatory effects of housing or lending programs, or development of precision medical interventions. This balance will vary by scenario; a Pew Research Center survey found that 62 percent of those surveyed are OK with government sharing information about criminal records of individual citizens online, 60 percent accept government sharing data about the performance of individual teachers at schools online, while only 22 percent are comfortable with government sharing information about mortgages of individual homeowners online.
In situations where analysis of sensitive granular data is important, the options are not just “release” or “do not release;” groups are experimenting with a variety of options on the spectrum between these two extremes. Detailed data can be shared with limited parties in support of specific goals, e.g. for research, or data can be released to small groups to test the data. It’s not just the sensitivity of the data being released that is important. Understanding how the data was gathered and analyzed, potential data biases in the field and figuring out how to conduct audits on the impact the data has on decision-making are also critical concerns. In the end, data gatherers, providers and users all must be cognizant of and sensitive to the ethics and practices that support responsible data use.
Usability and prioritization: More and more attention is being paid to ways to improve the impact of open government data. The recently released Open Data Charter emphasizes the characteristics that increase the value and usability of open data, including timeliness, accessibility, comparability and interoperability.
Chen emphasized that diverse communities (from scientists to small businesses) can benefit from Department of Commerce data, and it is important to empower users of differing levels of sophistication. For instance, users benefit not just from getting satellite images but also from access to the tools to extract information from the image about, say, ground cover characteristics, or from explanations of how to connect different data sets (e.g. demographic information with police activity reports) to put data in context. The new Commerce Data Service will develop products and services, such as tutorials for using data in different scenarios, to help government agencies better deliver information to their customers.
Looking ahead, there are both technology and policy steps needed to allow agencies, businesses, civic organizations and individuals to learn from and build on open government data. The administration has announced plans to develop national guidelines, in consultation with the broader stakeholder community, on key issues for federal open data, in areas such as data quality, privacy and partnerships.
Creating and supporting the infrastructure, information\, and services needed for a data release to be impactful is not trivial. As Gurin noted, not all data sets are equivalent in impact; he estimated that one-fifth of federal data sets may account for four-fifths of the value. Engaged feedback from users is critical to defining priority or common use opportunities, and partnerships to allow agencies to experiment with new approaches and facilitate new ways of people interacting with their data. Microsoft is pleased to be participating in such efforts with the U.S. Department of Agriculture around food resilience and with the National Oceanic and Atmospheric Administration around climate and ocean information.