Monthly Archives: November 2013

Rep. Sensenbrenner Opens Drafting of NSA Reform Legislation on the Internet, Powered by Madison [RELEASE]

$0-Cost Online Platform Strengthens Americans’ Ability to Be Heard by Government & Help Congress Write Better Bills

WASHINGTON, DC (November 26, 2013) – The non-partisan, non-profit OpenGov Foundation and US Congressman Jim Sensenbrenner (R-WI) today announced that bipartisan National Security Agency (NSA) reform legislation is now open to all Americans on the Internet for public comment, questions and crowdsourcing, powered by OpenGov’s Madison lawmaking software.  Madison is a free, online tool that helps people unlock the often confusing and opaque work done in Congress by helping to transform today’s closed, inefficient and frequently paper-bound bill drafting into an open, productive and digital one.  Just as Sensenbrenner is partnering on the USA Freedom Act with Senator Patrick Leahy (D-VT) to make the NSA more transparent, Madison makes Congress’ lawmaking process itself more transparent, connecting policy experts, stakeholders and everyday citizens directly to their elected officials…all without having to fly to Washington, D.C. or hire an expensive lobbyist.

“The USA Freedom Act is about accountability and transparency. In that spirit, I’m using Madison to gather as much input as possible from across the political spectrum, listening to the concerns of citizens in Wisconsin’s fifth district and across the country,” said Rep. Sensenbrenner.  “I look forward to an open and constructive debate, while coming together to strengthen oversight of the NSA, rein in abuse and protect every American’s Constitutional right to privacy.”

“Democracy is all about participation, and everyday Americans have a right to be included – and listened to – by their government,” said OpenGov Foundation Executive Director Seamus Kraft.  “Madison makes that possible, removing barriers that simply should not exist between citizens trying to be heard in Washington, and elected officials trying to discover and implement the best ideas to get our country back on track.  We applaud Rep. Sensenbrenner’s commitment to making Congress more accessible, inclusive and user-friendly.”

<<Click here for the USA Freedom Act in Madison>>

Background on the USA Freedom Act

  • According to USA Today, “The Freedom Act would end the NSA’s bulk collection of Americans’ communications records by amending Section 215 of the Patriot Act, making numerous amendments to the Foreign Intelligence Surveillance Act and creating a privacy advocate to the Foreign Intelligence Surveillance Court, who could argue civil liberties concerns and appeal court decisions.”

  • According to The Guardian, the bill has more than 100 bipartisan cosponsors and has earned support from groups as ideologically diverse as the National Rifle Association (NRA) and the American Civil Liberties Union (ACLU).

  • Click here to visit Rep. Sensenbrenner’s dedicated USA Freedom Act website.

Background on Madison

Contact Information 

The “Virtuous Pipeline” Producing Chicago’s Internet-Ready Laws

The headline says it all: “City of Chicago and Public-Spirited Hackers Unveil the Chicago City Code.” Team OpenGov recently travelled to the Windy City to deliver this MVG – “minimum viable gift” – of the laws back to its true owners, the residents of Chicago and the public servants working on their behalf.

It is safe to say that was well received. But in that crowded room, crackling with energy, something far greater than a website walkthrough took place. All of the people and organizations required to birth were there. As Carl Malamud put it, the entire end-to-end “virtuous pipeline” of modern municipal code was present and accounted for – hugging, high-fiving and hungry for more access to public data and more opportunities to open government. And everyone was fired up to accomplish these goals together – in productive partnerships that benefit both Chicago’s public servants and her residents. You can’t dream up a better environment in which truly modern, accessible and open government can succeed.

Watch and listen below to all the people in the “virtuous pipeline” that led to  The city law starts in Chicago City Clerk Susana Mendoza’s office, then flows out to the City Council and American Legal Publishing, then flows to Carl Malamud putting it online in bulk PDF form, then on to the OpenGov Foundation for transforming those PDFs into XML and republishing them online in Waldo Jaquith’s State Decoded format.  It’s a strong – if early – success for open government, open data, and public-private partnerships formed to deliver both.

We at OpenGov are proud to be a part of this virtuous pipeline, and look forward to “going with the flow” as grows to better serve the needs of Windy City residents seeking access to their own laws on the Internet.

Susana Mendoza, Chicago City Clerk

“Not only is the City Clerk’s Office responsible for updating and maintaining that complex, evolving [Municipal Code], but we also play a critical role in facilitating the legislative process and warehousing the outcomes from all of the hearings and hearings that City Council proceedings have every month.  That is by no means a small feat.

“My office wants to open up this trove of municipal data, and turn it into a useful tool for every Chicagoan, be it a community activist, a scholar, or even your neighbor down the block.  It shouldn’t be difficult to access information.  I want our Municipal Code to be as open, transparent and accessible as possible.”

“We can’t do it all ourselves.  We really do need to have an alliance and partners…to make government truly accessible, and useful, to its citizens.”

“When it is easier to decipher the DaVinci Code than it is to find an ordinance that you Alderman introduced, that’s a problem.  It shouldn’t be that way.  In a way, government has actually wanted it to be difficult for the public to have access to the information that affects their everyday life…You shouldn’t have to have a computer science degree to access that information.  Run with it.  Hack at it.  You’re not going to get a no from me!”

Julia Ellis, Policy Director for the Chicago City Clerk

“Chicago has made a policy decision, a sweeping policy decision, to say that we are going to pay for the work product in order to get to this very powerful, very valuable tool…and we’re going to give that tool away.  And American Legal has a similar philosophy.  They say, ‘We are experts in creating this incredibly valuable, legally relevant, tool, and what you do with it is none of our business.  We are happy to help you make this available in any way you want to, to whomever you want to.’”

Carl Malamud, Public.Resource.Org

“What Clerk Mendoza has described is a process of codification that goes on in every city, county, and state in the United States. Codification, and the periodic updates to the codes, is part of a pipeline. The end result of that pipeline has usually been a big thick document you could buy for a few hundred dollars—your municipal code.

“More recently, the codification companies have started selling CD-ROMs and most of them now have a web site where citizens can view their codes. Unfortunately, that pipeline has in the past stopped with these web sites, most of which are frankly pretty bad. Most of them are a frames-based interface. Cross links and navigation and search are sorely lacking. There are no permanent URLs. There is no bulk access facility.

“These code web sites are not valid or accessible, they have not been touched, as surely they should be, by the better angle brackets of our Internet.”

“This law is your law, not some petty profit opportunity. We can send a message to the code people that Chicago cares about code, that this country cares about code, that when it comes to the rules of our society, open source is the only way to ensure the rule of law.

“That is the only way to have equal protection under the law. That is the only way to have due process under the law. That is the only way to ensure access to justice, the right to free speech, and an informed citizenry.”

Waldo Jaquith, The State Decoded

“The State Decoded is software that you feed a bulk copy of a code into – as XML or JSON or you can write a scraper for whatever you need – and in just about 10, 20, 30 minutes it uses that to create a complete, finished website that is beautiful.  A responsive website, an API for those laws, bulk downloads, an individual page is established for each law where you can post comments, an internal tagging system to improve search.  It’s all backed by a Lucene and Solr search system to support natural language processing for anyone who wants to do real geeky, statistical analysis work .  It’s all free and open source software.”

“Laws are interesting to make available to people, but I think they’re more exciting as a platform to do really interesting, innovative things with.”

Seamus Kraft, Executive Director of the OpenGov Foundation

“We’re regifting you the law for the Internet Age.  Bookmark, because this is your open law website that builds on and extends the work of these fine public servants here tonight, and the thousands and thousands of city bureaucrats, lawyers, people who came before and helped produce and maintain the law before the Internet.  Well, the Internet happened and that’s why we’re here today.”

“We took the PDFs [of the law] and transformed those.  Now, PDF’s suck.  We all know that, as civic developers and hackers.  It’s the bane of our existence, but for a very long time, it was the best way to get paper-based laws onto the Internet.”

Opening the massive binder of the City Code, “Don’t you see structure here?  All we did was translate that into computer code.  It’s the same thing.  The source code of any community is the law itself.  All we did was update it to how people communicate today, which is digitally and on the Internet.”

“The whole point of all of this is people’s daily lives.  There are folks out there who need legal assistance, who can’t afford it.  Our first users [for sites like] are legal aid folks who work with poor people.  They spend all of their day messing with PDFs, messing with big binders of the law.  Now, that’s where it matters.  Everything we’ve talked about today has been pretty academic.  The Clerk’s Office, they’re doing alright.  We, we’re doing alright.  The people on the frontlines aren’t.  That’s why we’re doing this.”

Tackling Public Information Act Reform for Today’s Maryland

From the First Amendment to the Freedom of Information Act, the right to access and share public information is central to American society, and how our governments are supposed to operate.  The people of Maryland have secured this right in the Public Information Act (PIA) [Annotated Code of Maryland, §§10-611 through 10-630].  If you want to access state government information, you file a Public Information Act request.  It is often the only way Marylanders can get answers out of government.  So it is crucial that the PIA – and the processes and procedures it mandates – works as efficiently and effectively as possible.  Just as important is making it easier for PIA-producing public servants to sort through avalanches of raw government data, some of which contain sensitive information (like Social Security numbers) or pertain to pending litigation and need to be redacted or removed before public disclosure.  Compliance is no cakewalk: mistakes here can cost government workers their jobs, or worse.

Clearly, upgrading the PIA to meet the needs and the technology of today’s Maryland is no easy undertaking.  That’s why on November 6, Maryland State Senator Bill Ferguson and Delegate Kumar Barve convened the General Assembly’s Joint Committee on Transparency and Open Government in Annapolis to publicly examine the PIA, discuss current compliance costs and procedures, and identify potential ways to make the PIA work better for both citizens seeking state information and public servants tasked with providing it.

If you missed the meeting, OpenGov has you covered.  Below are the meeting documents and videos, hearing highlights and initial OpenGov thoughts on current PIA challenges and possible solutions.  We’d love to hear your take and include it, too.  Click here to drop us a line.

11/6/13 Joint Committee on Transparency & Open Government Meeting Materials

Fees Associated with Maryland Public Information Act Requests

When a Marylander requests public information under the PIA, they are often hit with review or production fees.  Adam Snyder, Chief Counsel in the Office of the Attorney General, testified that some state agencies charge citizens on a per-hour basis for reviewing their PIA request, ranging from $25 per hour at the Department of Homeland Security to $250 per hour at the Department of Motor Vehicles.  And when it is time to actually produce the information to citizens, state agencies tack on per-page copying fees that range from $0.035 per page at the Veterans Affairs agency to $0.51 per page at the Department of Public Works, Snyder added.  He estimated that the average copying fee is $0.25, and noted that fees can be waived if the agency determines that the request is in the “broad public interest.”  He closed with a reasonable defense of charging citizens for reviewing and complying with PIA requests, noting that the fee structure “encourages requesters to think about the materials they’re interested in.”

Delegate Barve asked what percentage of requests are responded to digitally.  Snyder didn’t have a firm number.

Senator Ferguson asked who determines whether a PIA request is “in the public interest,” thereby waiving associated fees.  Snyder responded that it is up to each agency to determine whether a request is in the public interest and merits a fee waiver.

OpenGov Take

Complying with PIA requests takes staff time and government resources, such as copier paper, ink and energy.  That being said, we think it is worth taking a hard look at three things while PIA reform is under consideration.

1. Harmonizing the fee structure across state government, so that at the very least, citizens aren’t charged what appears to be wildly different amounts for the same public information at different agencies.  This isn’t just a Maryland-specific problem.  The National Conference of State Legislatures just released a report looking at national fees and policies that reveals wild discrepancies from state to state.

2. Going digital by default could not only save acres of trees from untimely death, but it could save significant staff time spent pouring over documents, reviewing requests and copying pages for production to the public.

3. Empowering a separate, neutral arbiter to determine whether a PIA request is in the public interest and, if so, whether fees can be waived.  An unbiased third party would probably be a better judge of what is in the public interest and what is not than an involved state agency.

A Look at Current Agency PIA Policies and Procedures: Maryland Departments of the Environment and Transportation

Testimony from representatives of the Maryland Departments of Transportation and the Environment followed.  Delegate Barve asked, “How many [Environment Department] documents are digitized?”  Agency witness Heather Barthel estimated about 25%, but indicated they are working to increase that number.

Barthel testified that in 2012-2013, her agency received about 3,000 PIA requests.  12 included requests for fee waivers.  The Department of the Environment denied all 12 fee waiver requests, she said.  As far as responding to these requests digitally, Barthel highlighted a surprising obstacle: agency email accounts are limited to sending and receiving a mere 8 MB of information per message.  That means an agency staffer can only email a handful of scanned pages, preventing digital production in the majority of cases.

Senator Ferguson asked if either agency keeps a database of every PIA request made and responded to, allowing staff to easily find and reproduce identical information requests (instead of reinventing the wheel each time).  The answer was no.

When PIA requests are denied, citizens can turn to an appeals process.  Yet, at least at the agencies represented, it doesn’t appear that appeals process is widely used or successful.  Cheryl Brown of the Transportation Department testified that, “…In a little over 7 years [we] have never had an appeal.”  Why?  “I think that’s a sign of us doing a good job because they know we’ve made the proper decisions.

OpenGov Take

1. Increasing the amount of documents that are kept in digital form would go a long way towards a more efficient, effective PIA process for everyone involved.  While retrospectively digitizing documents is no easy task, increasing the percentage of documents agency staff can access quickly and easily online could result in lower costs – in time and resources – spent on PIA compliance.  25% is a good start, but only a quarter of the way there.

2. Creating digital PIA archives at each agency would mean that duplicate requests could be complied with fast – simply go to the database, find the original request, and send a copy of the documents electronically to the new requester.  Why reinvent (and recopy) documents multiple times when the work has already been done once?

3. And while the Transportation Department appeals process may be an outlier, couldn’t the astonishingly infrequent use of the appeals process be due to its complexity and cost for citizens, instead of because the requester truly believes he or she was properly denied?  It would be helpful to next gather data on the PIA appeals process use (or disuse) at each agency.

4. The 8 MB limit on email messages at the Maryland Environment Department is understandable, but completely solvable with a cheap or $0-cost solution – Dropbox is our favorite –  to efficiently share large files via the Internet.  Why not start by lifting the email limit only for certain accounts or setting up an FTP or Dropbox account?  Better yet, the City of Oakland has an awesome, proven system to handle citizen-to-government document requests: RecordTrac Oakland.  The software is free to copy and deploy for Maryland, thanks to a successful partnership between Code for America software developers and city government workers facing similar challenges and constraints.

The Maryland PIA from the Local Perspective

Representatives from Maryland cities and rural counties closed the hearing, sharing their local perspectives on the PIA.  Baltimore City Assistant Solicitor Hilary Ruley testified to the PIA’s out-dated nature regarding modern technology everyone uses, summing up much of the earlier testimony.

The law is written for a copier, not a computer…I understand the expectation that I should be able to look up on the Internet and get something, but to figure out how to do that, say, for a speed camera when your vendor changes is very difficult because I only have as an attorney the legal rights given under the purchasing contract for that software.  So it’s very difficult to perform the search function that the citizen really expects to have done.” [Emphasis added.]

A large portion of Maryland is rural, and Heather Price, Caroline County Attorney, asked that the committee be mindful of the large gap in resources and technical literacy that can exist in the state.

“Keep in mind, from my fellow attorneys in some of the rural jurisdictions…we’re less affluent in some instances and don’t have the type of software and technological availability that some of the other jurisdictions have,” Price testified.

OpenGov Take

Hearing from the front-line government attorneys out in Maryland cities and towns brought home some of the largest issues confronting those of us working to improve both a citizen’s ability to access public information and a public servants ability to deliver it.

1. Procurement rules matter a great deal, often far beyond the lifecycle of a particular purchase or platform.  Are Maryland government technology contracts written with an eye towards PIA requests?  We aren’t sure, but if they are not, this may be a good avenue to achieving long-term, low-cost increases in PIA efficiency and effectiveness.

2. A large “digital divide” can exist between urban and rural, affluent and less affluent, areas of Maryland.  Forcing the entire state to comply with PIA requests digitally without taking into account these underlying factors could actually hobble efforts to reform the PIA and strengthen every citizen’s right to know.

A Coder’s Take on Code for America Summit 2013

A few weeks ago, the OpenGov team attended the 2013 Code For America (CfA) Summit in San Francisco. Over two days, several dozen leaders in civic tech showed off digital tools for building more informed and engaged communities, focusing on the ethos of change.  Check out this recap from OpenGov developer Bill Hunt.  


There were far too many great talks to summarize them all, but CFA has posted the full agenda online as well as videos of all of the talks.  Here are a few of our favorites.

Clay Shirky’s Keynote

  • Clay covered a lot of ground, and his whole speech is worth a listen.
  • One of my favorite takeaways:  “The output of a hackathon isn’t a solution to a problem, it’s a better understanding of the problem.”
  • He also covers that idea that Sir Winston Churchill summarized so well: “Success is the ability to go from one failure to another with no loss of enthusiasm.” 

Tim O’Reilly, Seamus Kraft, & Gavin Newsom – Open Gov Crossing Party Lines

  • Tim O’Reilly led this discussion between OpenGov’s own Seamus Kraft (standing in for Congressman Darrell Issa) and California Lt. Governor Gavin Newsom, on how Open Government transcends party politics, and both sides of the aisle come together on this topic.

There were several talks about the problems with procurement in government, and how it is one of the fastest ways to reduce wasteful, inefficient government spending.

Clay Johnson – Procurement

  • Clay has been featured in a lot of interviews lately talking about procurement and the new website. He points out that the vendors that win bids on RFPs typically are those that have good legal departments, not necessarily the ones that have the best  team for the job.  He showed off a few tools designed to fix that problem:
  •  Screendoor – a system for creating readable, usable RFPs
  •  WriteForHumans – a simple tool to gauge how difficult to read a piece of text is.

Jeff Rubenstein – SmartProcure

  • Jeff showed us SmartProcure, a system designed for localities to find and share vendor information.

There were several talks around using data and technology to improve the effectiveness of services.

Nigel Jacobs, Joel Mahoney & Michael Evans – Boston

  • There were a few talks about the improvements being made in Boston.  This clip features Michael Evanstalking about City Hall To Go – a mobile van that brings City Hall services to underserved areas, allowing residents to pay taxes, get handicap parking spaces, register to vote, and do other things on the spot instead of making a trip all the way down to Boston City Hall.

Cris Cristina & Karen Boyd – RecordTrac

  • Cris and Karen discussed the RecordTrac system, used in Oakland to respond to public records requests.  This project is opensource, so other cities can adopt this and start using it immediately.

Saru Ramanan – Family Assessment Form

  • Historically, there have been few tools to actually tell how effective social services are in helping families.  The Family Assessment Form is a tool to record and investigate help to individual families, and track their progression over time.

The second day featured many talks on user-friendly tools for local improvement.  This also focused on user experience as a form of social justice.

Tamara Manik-Perlman – CityVoice

  • City Voice is a tool built on Twilio‘s API to collect feedback from residents over the phone, using a very easy-to-use system.  This was originally implemented in South Bend to address the housing situation, but could be used in any locality for a variety of topics where gathering feedback outside of a single forum is useful.

Lou Huang and Marcin Wichary – StreetMix

  • Lou and Marcin gave one of the most entertaining talks of the conference on StreetMix, which allows citizens to propose layouts for streets, using simple drag and drop tools to create bike lanes, bus stops, medians, and more.

Dana Chisnell – Redesigning Government

  • Dana and her team were set with the task of designing a ballot for *everyone*, from adults who have never used a computer, to those have low literacy and low education, those with cognitive disabilities, and other often ignored groups.  This is a must-watch for anyone who spends time on accessibility issues, or is interested in Good UX For All.

Cyd Harrell, Ginger Zielinskie, Mark Headd, and Dana Chisnell – Redesigning Government

  • Cyd led a panel discussion talking about a variety of topics around UX and accessibility in technology for cities.  This was my favorite talk of the conference, and one covering topics that are often overlooked.

American Heroes [VIDEO]

All of the work we do opening government means nothing if there are not safe, secure American citizens to benefit.  We have our brave men and women in uniform – many who never made it home – to thank for the ability live and work without fear.  Yesterday, the OpenGov team visited memorial sites in Washington, DC to capture Veteran’s Day 2013 for those who could not travel here themselves.  Please take a moment to watch and listen to the stories of soldiers and sailors who gave their last full measure in defense of our freedom.



WATCH: A Veteran’s Day Tribute: WW II Pilot George HW Bush’s Letter to a MIA Pilot’s Mother 



WATCH: A Veteran’s Day Tribute: Fallen Marine Corps Sgt William Stacey’s Last Letter Home

Manufacturing Patent Troll Reform the Modern Way with Madison

Patent reform has arrived in Congress.  Last Tuesday, Chairman Bob Goodlatte’s (R-VA) convened a hearing on the “Innovation Act,” a bipartisan bill to eliminate alleged abuses of America’s patent system by making it far more difficult for people to “troll” for patents by filing frivolous lawsuits against the original inventor or patent owner.

And on the other side of the Capitol, Senator Patrick Leahy (D-VT) today held a Senate Commerce Committee hearing to gather facts on deceptive patent troll practices.  This is all great news for the countless Americans working hard to become the next Steve Jobs or Thomas Edison, instead of becoming the next victim of litigation that robs them of their innovative new products, business methods and intellectual property.

But something far more radical took place that day.  As the hearing room doors were closing on Capitol Hill, they were opened to everyone on the Internet for a fully-interactive online hearing powered by a game-changing piece of open source software called “Madison.”  While only four witnesses and a few dozen audience-members were physically before the Judiciary Committee, 10 times as many were digitally present thanks to Madison, with the unprecedented ability to watch the hearing anywhere in the world, ask questions, make comments and even suggest changes directly on the Innovation Act itself, with the best suggestions rising to the top.

<< Click Here to Read the Full Post >>

[Developers Blog] Solving a Sticky Solr Upgrade

Read how OpenGov developer Bill Hunt is solving a messy Solr upgrade while increasing the power of the open law search engine: 

I ran into an issue with a Solr configuration that was working for me locally, but not on our CentOS 6.4 server. I’ve documented below all of the issues I encountered along the way, as the upgrade from Solr 4.2 to 4.3+ is a pretty nasty one, due to major changes in the logging system (LOG4J / SLF4J)

Lucene/Solr 4.3+ : The Beginning

The awesome guys at Open Source Connections had created a custom tokenizer for Solr for The State Decoded, which was suddenly throwing this new error:

java.lang.IllegalArgumentException: No enum const class org.apache.lucene.util.Version.LUCENE_43

Which I guessed to mean that they’d specified to use Lucene 4.3 (or in my case, Solr 4.3) in the solr_home/conf/solrconfig.xml file


Since CentOS 6.4 only comes with Solr 4.2 out of the box, I tried changing that to LUCENE_42 but received even more intimidating errors:

org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Plugin init failure for [schema.xml] fieldType "descendent_facet_hierarchical": Plugin init failure for [schema.xml] analyzer/tokenizer: Error instantiating class: 'com.o19s.RegexPathHierarchyTokenizerFactory' 

The full stack dump is here

We guessed that the package needed to be rebuilt from source…

<<