Data Request Denied, Now Plan B
I had a nice conversation this afternoon with Marie Laderta, director of the state Department of Human Resources Development, in response to my request to secure the same data her office prepared for Civil Beat (and which Civil Beat hid behind its paywall). She was professional, articulate, and demonstrated that she understood the background and intent of my request, and even the basis of my objections to Civil Beat's actions.
But ultimately, she said she was denying my request for the information.
She said a number of interesting things, and I disagree with a good many of them. Unfortunately, I'm much more comfortable in the realm of geekery than I am in the realm of public policy. While I'm not giving up on my request, I'm happy to report that the data in question has been freed, anyway.
What's All This, Then?
If you're just tuning in, I recently cheered the work Civil Beat had done to get a list of almost all state employees, their job titles, and their salaries (or salary ranges). At the same time, however, I jeered the counter-intuitive work Civil Beat did to limit access to it: watermarking and adding DRM to the document, and stashing the extracted data behind its paywall. This is a "commitment to transparency"?
When Civil Beat made it clear, in part through chipper responses via Twitter, that they weren't going to release an unencumbered copy of the digital file they got from the state -- a document that, I believe, is itself public information -- I decided to request the data myself.
I filed a "Request to Access a Government Record" request, along with a waiver of fees in the public interest, with the department. (You can find the forms and some guidance at the state Office of Information Practices website.) I crafted the request to mirror the data requested by Civil Beat.
But I also acknowledged that my request could require time and effort to fulfill. So I said that I would be satisified with a copy of the output of a materially similar request recently fulfilled by their office. In other words, I wanted the same data the OHRD gave to Civil Beat, so I'll settle for the same document they compiled for the news site.
I noted, parenthetically, that I also wouldn't charge anyone to access it.
Then, I waited. The department, like all agencies, had ten days to respond. Sure enough, at exactly 5 p.m. yesterday, Laderta called me. Unfortunately, I was on the air at Hawaii Public Radio, so she could only leave a voicemail. I called back this morning, leaving a message for her, and we finally connected late this afternoon.
Not In So Many Words
We spoke for about 10 minutes. She provided thoughtful explanations, and answered several questions, including questions I asked more than once to make sure I understood the substance of her response to my information request. Insofar as Civil Beat was concerned, she said:
The Office of Human Resources Development did not retain a copy of the file provided to Civil Beat. This is common practice. I asked if she could check the "Sent" folder of her email. She said that if they had it, they would have provided it.
The Civil Beat request took several weeks to fulfill, and involved discussions to refine its scope to exclude information that OHRD could not provide. She noted that this process consumed significant resources, therefore costing taxpayer money. She said she knows they could have assessed a fee, but chose not to do so.
She told me that I could go to the Civil Beat website to "get the information for free." I explained that the display of the PDF document was restricted such that the public could only "look at it," and not download it, and that you need to pay to manipulate the data in any useful way. She suggested that the Civil Beat setup should still be sufficient, but she also conceded that it was not what she expected.
Since she could not produce the document provided to Civil Beat, and since I did not feel I had sufficient access to that information anyway, I said that the crux of my request remained: to have the information compiled and provided to me directly. Laderta's response?
My request for the information is being denied because of the onerous "compilation" component. Basically, the amount of work it would take to put together all the data was excessive. Though she did not cite chapter and verse, she said that OHRD had been advised that "compilation" was one of the grounds on which requests could be denied. She again noted the time and effort consumed by the Civil Beat request, and mentioned constraints including RIFs, furloughs, manpower and budget.
Most interestingly, she also said that the Civil Beat request should not have been fulfilled in the first place, and that if Civil Beat made the same request today, it would be denied on the same basis used to deny my request.
I told her that I found many of the points she raised to be troubling, but thanked her for her time. She was similarly gracious as we ended our call. I did tell her that she'd probably hear from me again.
Next Steps and Sidesteps
At the very least, I'm going to ask Laderta for a written response to my request, hopefully addressing the above points. From there, I'll likely ask the Office of Information Practices to determine whether OHRD can indeed say that public information is only public if it's not too much work to make it public.
But I know both OHRD and OIP are understaffed, overworked, and have better things to do that deal with a whiny blogger. Doug White's tale of an information request that dragged out over a year is not encouraging.
Fortunately, as I had hoped when I first raised the issue of putting public data behind a paywall, there was a technical solution to the ideological problem.
For the past few days, I've been able to collaborate with a fellow open data advocate and data liberation ninja. The product of his considerable talents? A public, freely downloadable, usable spreadsheet of the data Civil Beat tried to lock down.
Instead of posting the PDF document provided by the state (or the watermarked, ostensibly copyrighted PDF that Civil Beat created), we've acquired what mattered most: the information contained within. Information provided by the state, and information that belongs to all of us.
Well... most of the information. Here it is:
That's a Google Docs spreadsheet. You can browse or search it online, or download the data to your computer and play with it. Get it as text, as a comma-separated-values (CSV) file, as a Microsoft Excel file... you can even get it as a largely useless PDF, albeit without any pesky watermarks.
But you'll probably notice that there are no names included. This is because the aforementioned data liberation ninja is also a privacy advocate. As I've engaged in debates all week over whether the names are essential data or merely needless provocation, I think it's a fair compromise. I'm hoping we might be able to provide the entire dataset in the near future.
In the mean time, as a bonus, here's an additional batch of information you might find useful:
As you may recall, the original dataset from the state did not include information from the University of Hawaii. Civil Beat separately requested salary information from UH, and got it, posting more stories (and doing the same thing with the data).
Fortunately, UH is required by law to present several annual reports to the Legislature, and this includes an "Annual Report on Salaries" (HRS 304A-1004).
It's posted online as a PDF, but PDFs are hardly user friendly, and UH inexplicably locks its reports down with DRM. So again, the Google Docs spreadsheet linked above is wide open so that journalists, bloggers, or curious citizens can do whatever they want with the data.
The Legislative salary report also does not include names, while Civil Beat's report does. Civil Beat's information is also more current. But this is a good start.
I still have salary information requests pending with the Legislature (state House and Senate) and the University of Hawaii, and again, whatever I obtain, I will make freely available. After today's rejection from the OHRD, however, I'm doubtful that I'll get much.
In the mean time, Civil Beat continues to acquire massive volumes of public data -- efforts that have apparently prompted at least one state office to turn down subsequent requests -- but continues to push people into paying for access to that data. I'm losing faith that the news site is the principled, idealistic journalistic enterprise it claimed to be.
Though I guess the paywall should've been my first clue.