A debate in the technology world that's been simmering for years, about whether mapping vendor Esri will allow public geographic information systems (GIS) to access government customers' data, finally has an answer: The mapping software giant will take an unprecedented step, enabling thousands of government customers around the U.S. to make their data on the ArcGIS platform open to the public with a click of a mouse.
"Everyone starting to deploy ArcGIS can now deploy an open data site," Andrew Turner, chief technology officer of Esri's Research and Development Center in D.C., said in an interview. "We're in a unique position here. Users can just turn it on the day it becomes public."
Government agencies can use the new feature to turn geospatial information systems data in Esri's format into migratable, discoverable, and accessible open formats, including CSVs, KML and GeoJSON. Esri will demonstrate the ArcGIS feature in ArcGIS at the Federal Users Conference in Washington, D.C. According to Turner, the new feature will go live in March 2014.
If a majority of federal, state and local agencies choose to activate the feature, the net effect will be to make more public geospatial data available to developers and the public. The data will be free and in the public domain.
This release is "very relevant" to OpenStreetMap (OSM), the online Wikipedia of maps, according to Turner.
"We are allowing users to choose open data license," he said. "That could make a lot of new data available. For instance, you could search for all the public domain data in NYC. That could be a great wealth of data to populate OSM."
According to Turner, this feature has been in development for over six months. The first version will launch with the Environmental Protection Agency (EPA) this week, at an open data site in Chesapeake, Virginia. The site will aggregate data across different entities, reflecting how the EPA gathers data to make regulatory decisions.
"They don't make most of their own data," Turner said. "Instead, they work with orgs to gather data. Now, they can spin an open data site up, customize it, choose a URL, and include data that they want to, including from Maryland and other other organizations."
The kernel of the idea, however, goes back much further, to Esri's mid-2012 acquisition of GeoIQ, the geolocation startup that Turner co-founded with Sean Gorman; it can even be traced back further to the period shortly after the 2010 earthquake in Haiti, when Esri founder Jack Dangermond got interested in catching up to the new open data landscape. That long-term strategy is now coming to fruition with a broad push to shift the public perception of Esri in the open government community through action.
"ESRI has gotten a bad moniker," Turner said. "Esri stands for Environmental Systems Research Institute. It's not about trying to lock people in—it's about enabling people to do the work they need. This is not open data for open data's sake. It's the EPA realizing the benefit of organizations outside of the agency to do interesting things, from citizens to nonprofits to media, to developers. All the data are consistent, all exportable in the same way. it's very easy to find and use the data.
A Brief Timeline
In 2011, Esri showed how social data could be integrated into its maps, enabling emergency services to respond more quickly and effectively to floods in Australia and beyond.
“Maps can be a very valuable part of transparency in government,” Dangermond wrote at the end of 2011. “Maps give people a greater understanding of the world around them. They can help tell stories and, many times, be more valuable than the data itself. They provide a context for taxpayers to better understand how spending or decisions are being made in a circumstance of where they work and live. Maps help us describe conditions and situations, and help tell stories, often related to one’s own understanding of content.”
In 2012, Turner described an aspirational vision for how GeoIQ's open source technology would be integrated into Esri at the DC development center. Critics dispute the company's commitment to open government data.
In 2014, Esri enables thousands of government entities to instantly deploy an open data site in its cloud, without additional cost.
"For enterprise accounts, this is just a free option that they can turn on," Turner said. "D.C. government already has an ELA [enterprise license agreement] with us hosting for them. They can customize and spin up multiple sites: DDOT can have its own, the Metropolitan Police Department can have its own data site, or for specific focus areas."
That technical arrangement is the key to this rapid deployment: While government agencies can and do host open data themselves using traditional servers and open source technology, Esri is making a play to make it extremely easy for them to publish data sets using a GIS sytem they're already paying to maintain.
"They can do it in Esri's cloud, hosted on premise, in a relational database" Turner said. "It's Postgres, SQL Server, Oracle and ArcGIS server on top of it. Geoservices is a specification that's been made open; it's essentially REST and JSON. We've been building other open source tools for it. You can pull data into Leaflet or convert to GeoJSON."
He also noted Esri will be promoting hackathons to work with the newly-released datasets, including a workshop on the tools, mapping APIs, and how to convert data.
On that count, Esri and Microsoft are sponsoring the upcoming "Code Across America" campaign on International Open Data Day this February, hosted in collaboration with the Sunlight Foundation, the Open Knowledge Foundation and Code for America.
The deployment of an integrated open data publishing tool is a significant development for public GIS data as infrastructure in the U.S. Not surprisingly, Turner was clearly more than a little excited by the feature.
"It's why I came to Esri," he said. "As a startup, you're cobbling together hang glider from spare parts as you're running around the ground to try to get wind. With a big company, you're about to jump off a cliff with a hang glider: when you're starting to deploy, you're about to have thousands or millions of eyes on the product. Now, everyone starting to deploy ArcGIS can now deploy an open data site."
Image courtesy of Data.gov