Civil Beat Puts Public Data Behind the Paywall
The headline proclaims, "Civil Beat Shares Hawaii State Employee Salaries" — but there's a catch: you have to pay to get full access to the information.
To be sure, there's a reasonable and deep debate to be had over whether the names and salaries of government employees should be public in the first place. I was most intrigued, however, by how Civil Beat was looking to capitalize on the story.
They deserve a lot of credit for navigating the twisty halls of government to get at the information, but I'm annoyed at how hard they're making it to benefit from their efforts.
Journalism is Hard Work
Civil Beat, the news site backed by renown Internet entrepreneur Pierre Omidyar, triumphantly announced their data acquisition this way:
"The biggest cost in state government is people. They're what makes it tick. But how much we're paying them isn't so well known — even though the information is a matter of public record and should be available to any citizen who asks. At Civil Beat, we're committed to truth and transparency."
As a proud journalism grad who survived the public records class taught by the great Prof. Beverly Keever at UH, reading words like "public record" and "transparency" make my heart sing.
I also know how much work it takes to get information that we the people (not the press) should rightfully have access to. My college memories are peppered with musty basement archives, cranky clerks and half-broken photocopiers.
And for hopefully good reason, the state Department of Human Resources Development took a few months to provide the information to Civil Beat following its initial request in early May. A lot of the time was probably spent consulting lawyers and redacting entries, as the list of salaries doesn't include several major branches of government, including the Department of Education and the University of Hawaii, nor any position protected by labor contracts (which are many).
Ultimately, though, the information was finally freed from the belly of our government and delivered to Civil Beat, apparently as a 300-plus page PDF sorted by department. The online news site was able to start a series of articles based on the data, including "Hawaii's Lowest Paid State Workers" and "Hawaii's Highest Paid State Employees." Of course, you need to be a paying member to read the articles. But they also put the raw, original data online for people to review.
The Free Tease
If you wanted to see the PDF that the state compiled, Civil Beat posted it online via SlideShare. But while you could page through the document, Civil Beat explicitly set the document viewer to prevent people from downloading it.
Yesterday afternoon, the SlideShare document disappeared completely, but it was quickly replaced with a new version. You still couldn't download it. The only difference was that Civil Beat moved its "Honolulu Civil Beat" watermark from the bottom of each page to the dead center of each page.
I asked Civil Beat about the change on Twitter, and they replied: "Moved watermark. Okay to promo CB after all time/effort it took to get the data, no?"
I certainly can't begrudge them some "promo" for their work. But I also realized that the new watermark also stops people from trying to extract the data by printing out screenshots and running the pages through OCR.
Not that anyone would think to do that, of course.
Since a PDF is hardly an efficient way to work with tables of names and numbers, Civil Beat also put together a searchable database on their site that you can use to look up information by department, first name, last name, title, and salary range. Again, though, you have to be a paying member to get the full benefit of this undoubtedly solid coding. Visitors can only play with a "sample" database of only 50 out of the over 14,000 records provided by the state.
"Get a taste for free, pay for the full meal." Certainly not uncommon in business, and definitely not unusual on the web. So why does Civil Beat's approach bother me?
The Big Picture
In short, I don't like that Civil Beat acquired the data under the auspices of "public data," but is then turning around and selling access back to that same public.
Yes, I know that many businesses depend on this very model (including my employer). Like Civil Beat, it's usually not so much the data that's being sold, but rather its presentation and interpretation. And the response to criticism is obvious: you can always get the information yourself.
What makes Civil Beat different, in my view, is that it's a journalistic enterprise. And not just any journalistic enterprise. They explicitly distinguish themselves from conventional media by focusing on civic affairs, aiming to create an informed public via an online "civic square." As they said in announcing the salary information, it's public information acquired in the name of transparency. Crippling access to the information seems to go against this vision.
Can't a Startup Make a Buck?
Of course, Civil Beat is also a business, and needs to make money. But I think they should focus on doing so in the way they've been doing it from the beginning: selling their original, independent, investigative reporting.
Perhaps I'm deluding myself, but I think most news agencies would make any public data they receive freely available. Why? Because their value to the reader and consumer is by providing context and commentary. It's great that you can get the Environmental Impact Statement for a new Mauna Kea telescope, or a list of bankruptcies for the last year, but what does it mean?
Indeed, Civil Beat says it takes "a holistic approach to news," one that helps people "learn about and understand, debate and discover the important issues facing Hawaii." If they do a good job with their coverage, people will pay for it.
Getting a list of 14,000-plus state employees and their salaries is only the beginning. It's also information that belongs to everyone. I'd like Civil Beat to release the public data to the public, then kick ass and grow its membership with top-notch journalism.
Do It Yourself
Civil Beat did what any citizen could do: request information from the government. And while it's not always a simple or quick process, I can tell you that it's considerably easier than it was fifteen years ago.
Sometimes, the simplest plan works, so I called the Department of Human Resources Development and asked, "Could I get the same report you prepared for Civil Beat?"
"If you file the same request they did," they replied.
Okay, not quite as simple, but they helpfully directed me to the Office of Information Practices, where I downloaded the form, "Request to Access a Government Record" [PDF]. It bears noting that you don't have to explain who you are or why you want the information, just provide a way for them to contact you and deliver the information. I described the dataset Civil Beat received as best I could.
Yes, there are fees for processing record requests. The fees are explained as $2.50 for 15 minutes to search for a record, and $5.00 for 15 minutes to review and segregate a record. If Civil Beat had to pay $2.50 per record, they would've spent $35,000 for 14,000 records. Fortunately, you can request a waiver of fees in the public interest. The form explains how, and the criteria for a waiver:
The requested record pertains to the operations or activities of an agency;
The record is not readily available in the public domain; and
The requester has the primary intention and the actual ability to widely disseminate information from the government record to the public at large.
Based on the above, I felt I qualified for a waiver, so I attached a request letter explaining as much. I noted that they had already compiled this information, so that it would hopefully not take much time or effort to fulfill my request. I also couldn't help but note that unlike Civil Beat, I wholly intend to "widely disseminate information from the government record to the public at large," rather than charging for it.
I should hear back from them in 10-20 days. And if I get the information, I'll make it freely available.
Meanwhile, we can argue about whether names and salaries of state workers should be public information in the first place.
Hat tip to @kerryvm, who asked, "Are Capitalism and Transparency mutually exclusive?"