“To improve is to change, so to be perfect is to have changed often”—Sir Winston Churchill
The open data movement has been alive at the Department of Commerce for a very long time. The predecessors to the National Weather Service have been providing open weather reports since the founding of our nation and regular weather reports have been around since the early 1900s.
The Census Bureau, one of our nation’s leading statistical agencies, has also been in the open data business for decades. Census has been evolving its efforts to share its data in the best ways possible, including by embracing new technologies.
However, facilitating the use of Commerce data by as many people as possible is a journey. To bring data-driven insights to new audiences, we issued a challenge to the private sector to help build new tools that make our data more actionable. In response, we’re announcing another free, open and public tool built by Socrata that makes a valuable data set from the U.S. Census Bureau more accessible—Small Area Health Insurance Estimates (SAHIE). The SAHIE program annually produces timely estimates for all counties and states, with detailed demographic and income data as well. SAHIE rely heavily on American Community Survey estimates and related administrative records data.
Before the Commerce department worked to deliver on our open data agenda, one way you might have accessed data from the Census was through downloading a csv file, like the one below.
However, csv files can be hard to manage if you’re not an excel guru, and they’re often huge in size. With the advent of open APIs, developers can instead construct their own URLs to query a specific part of a dataset and only see what they want. For example, if you wanted to see the percentage of people insured and the percentage of people not insured in all counties in Delaware in 2014, you could call Census’ API and get back something that looks like this:
The issue is that unless you speak code, you’re still far from deriving insights or making decisions with the data.
However, our agencies haven’t stopped there. The Census website has offered a data visualization tool since 2008 that displays maps of insurance coverage across states, as well as tables that let you filter by demographic characteristics.
But, as any good policymaker knows, explaining what numbers mean in context is a challenge. As Charles Dickins once said, “There are dark shadows on the earth, but its lights are stronger in the contrast.” Socrata’s Open Data Network (ODN) offers the powerful component of comparison, offering side-by-side comparisons of data so users better understand statistics in context and determine how we should define success.
With a few clicks, Socrata’s search engine lets you query, analyze and compare nearly 10 million statistical records over eight years to see health insurance coverage and trends for every state and county in the country. You can compare and contrast the data by age, sex, income and (for states only) by race and Hispanic origin; all accessible through their open platform, that is completely free and open to the public.
U.S. Secretary of Commerce Penny Pritzker assembled the Commerce Data Advisory Council (CDAC) to provide expert guidance and insight to the Department of Commerce’s data leadership, covering a wide range of pressing issues in data and technology. CDAC member and CEO Kevin Merritt, Socrata, demoed the tool at our recent CDAC meeting held in Commerce’s library and walked us through the startling differences in health insurance coverage across various states and counties in the U.S. For example, with a few clicks, one can see that the percentage of uninsured people aged 18–64 in Massachusetts was 4.5 percent in 2014, while in Georgia, it was nearly 5 times that at 22.2 percent. Drilling down to counties within the state of Illinois, you can see that the uninsured rate in Cook County, the home of the Chicago Cubs, was 17.7 percent, while only an hour’s drive away in Kendall County, it was only 8.7 percent.
When Colin Parris, vice president at GE Software Research, saw the tool at the CDAC meeting, he shared that as far as data-driven decision making is concerned, “Beautiful tools solve half the problem.” And he’s right. Understanding these differences has important and impactful implications from both a policy and private sector perspective, and all of that information is now at the fingertips of decision makers.
Beyond bringing data power to help the uninsured, we hope that the combined power of the Open Data Network and U.S. open data will not only illustrate what’s possible, but also inspire others to take Commerce’s public challenge to help with democratizing public data and build the “beautiful tools” that will help Americans put government data to work. –We know this is just the start of how tech firms can fuel public good by innovatively using public data to solve problems.
“Setting an example is not the main means of influencing others,” Albert Einstein said, “it is the only means.” We look forward to seeing even more examples.
Thanks for reading.
– Justin and Kevin