The prominent role that commercial software plays in government agencies at local, state, and federal levels makes them a powerful force in opening data (or, far more often, not opening data). This is a fact rarely acknowledged in the open government data efforts. We should capitalize on the latent power of vendors, regarding their software as a crucial vector for the expansion of open data.
At some point in discussions about how government should go about opening more data, the word “just” makes its first appearance. As in “they should just publish SQL dumps to the web and let the private sector take it from there” or “they should just export the data as CSV files, since that would be better than nothing.” Generally, this is true—in the abstract, the simple, suggested steps would be easy wins for the sharing of data. But “just” glosses over a real obstacle—legal and practical impossibility. Such suggestions are akin to suggesting that homeowners just cut off the lock on the PG&E power meter on the side of their house and wire in a Zigbee transmitter to monitor their home energy usage, or open up their Comcast-owned cable box to enable a debug mode to monitor signal degradation over time.
There are a handful of reasons why “just” tasks are often much more difficult than it seems like they should be, but the most important one is this: many agencies rely on commercial software that they have neither the tools (the source code) nor the authority (licensing) to change. There are lots of vendors, large and small, who license records management software—e.g., SAS, Competitive Edge, RBDMS, and CDP, to select a few results from Google. The agencies who use that software are rarely in a position to dictate new features, so “just exporting data as CSV” isn’t actually a thing.
We can pass open data laws, enact open data policies, and appoint Chief Innovation Officers all day long, but if the software relied on by government cannot produce open data, then that’s all for nothing.
Imagine if the major records management vendors included open data functionality. Imagine that it’s enabled by default for all types of data without security risks or personally identifying information, creating bulk downloads and an API for each dataset. The number of municipal, state, and federal datasets that would become available would be larger than those resulting from any other initiative.
There is no possible number of Open Data Institutes, no number of Code for America cities, no expansion of the Sunlight Foundation that could have an impact anywhere close to the impact that would result from a handful of commercial vendors adding a File → Export as JSON menu option.
How do we get commercial software vendors to add functionality that faciliates the provision of open data? I don’t have the faintest idea. Let’s get to work on figuring that out.