Full disclosure, I'm in a federal department that has been pushing for more open source for a while.
This is a great move by the White House. While there are a lot of groups that are trying to push for more openness and release of software, it can often be challenging. A lot of federal groups have been taught over the years to be very risk averse, and open software is viewed by them to be risk. Probably one of the most common concerns is, "What happens if someone takes and misuses our software?" In a highly risk-averse federal environment, these can be challenging arguments to fight against.
If you like and support this kind of thing, one big thing you can do is to contribute and supply feedback. We frequently have to go to our superiors and justify what we are doing with regards to open source. We say things like, "this repository had X pull requests from non-federal contributors". Or, "We got Y comments and questions from non-federal users of our projects".
It could be as simple as an email saying "Hey thanks, I found this useful", to a full-on pull request fixing an issue or with a new feature request. The more fodder we have to say "open source increases engagement and creates positive feedback" the more you will see this kind of thing happening.
I'm pretty sure that, assuming you're a government employee, any work you create that would normally be covered by copyright as part of your job is public domain. There's no license legally possible.
Yeah, absolutely. We make that argument often. In fact, the software we open-source is technically supposed to be released as public domain (as in, no license applied) and often is if people are doing things right.
But that doesn't mean it has to be hosted and released online. Many agencies work under the "not going to release anything until we get a FOIA request" model. Just because it is public domain doesn't mean it is public.
Want my email address? I'll just send FOIAs all day long through Muckrock if I know what I should be asking for (and pushing the results up to Github and the Internet Archive).
Honestly, I'll take it. More than likely it will be something on a CD or DVD, though. And if I get a PDF of 300kloc, then it's an indicator that I've hit pay dirt, and they don't want me to see something, or are very reluctant to give it up.
All kinds of things. For years as a business owner I used FOIAs to gain some insight into government contracting work. It was expensive but I think worth it. Lately I've been asking for information about military operations and projects that I know have happened or exist, but that no one knows about (this is very, very hard to do, btw, and I don't recommend it since you'll probably just waste your time). When I hear about classified projects -- for example from Snowden's release -- I FOIA for information that might be tangential but unclassified. That's how you find bombshells, btw. Most of the time these days, I get the "cannot confirm or deny" replies, or "we could not find material responsive to your request". I have some very cool FOIA responses from from the NSA from the six months before Snowden dropped his bomb. "We can't confirm or deny ... National security" and the like. I'm not sure how to say this, but due to my network, I had some insight into what was happening. Don't get the wrong idea about me, but anyway, those replies (one of them dated on "the apocalypse" of 21 December 2012) are fairly frame-able. As I said above, FOIA'ing as a commercial requester was expensive. As a private person asking for noncommercial reasons, you can waive a lot of the fees.
Overall I think the FOIA is a great utility. Most big name federal agencies actually have FOIA liaisons that will try to help if they can. Some, of course, are just as douche-y as you would imagine (e.g. NSA, NRO, CIA). Though I did have a few good interactions with CIA FOIAs, mostly bad ones though. The FOIA actually says in the law that the reviewing official should look at a FOIA as "what can we release?" and not as "what can we hide?", and you need to tell people this more often than you would like. I actually used a FOIA to get my DNA information from the military's secret-but-not-secret DNA database. It's a database of every service member's DNA, and ostensibly is for identifying remains. That was actually very difficult, because nobody had ever FOIA'ed the Army's "funerary affairs" office for DNA information, and so they were under the impression they could just ignore my requests. That taught me a lot and was actually fun in a weird way. You just have to persist, and escalate if you don't agree with the response or don't get any. FOIAs are serious business, and most people in the US government are people just like you and me, and want to help you find what you're looking for. Though, some people in government -- like any other community -- are bullies on a power trip. Wow, sorry for the wall of text!
Well (as another Federal coder), one problem is the unfunded mandate issue. If this comes with resources to maintain public repos and updates, sure. Otherwise, if someone emails me I share the latest snapshot but only as my time and duties permit.
I realize that setting up an automatic integration between your existing source control system and github might be nontrivial (eg if you are using an unpopular source control system for which no easy conversion to git) so this won't work for everyone without funding (and it would be nice if some were made available for that)
But if you already use git (or something like SVN or hg which can easily interoperate with git) or don't currently use any SCM (!) then just publishing to github is a nobrainer.
I generally agree. I've seen a few challenges to this perspective. 1) Some agencies do not think using 3rd party services like github is allowed. 2) Different groups make the initial code release pretty time consuming (as, in part, it should be) as, through time, the history may have accumulated secrets/keys/other sensitive material. So very thorough reviews are sometimes required. 3) Some agencies allow the use of github (and the like), but make the process of getting code up there extremely challenging. For example, some groups require all changes to code to go through the same internal review process as traditional communication, like a scientific publication or press release (internal review, multiple levels of management, etc).
Of course, the last one is the worst. A lot of groups are getting their act together when it comes to starting open-source, but it is met with a lot of skepticism.
Apparently things are a bit more complicated than this, since works created by government employees are in the public domain in the US, but foreign copyrights are still held by the government. Unless the software is released with a FOSS license where applicable, I don't think public domain government code is compatible with other FOSS licenses (which are generally worldwide).
Even if it weren't the case that the US government held full copyrights abroad, it's still something of a problem because some jurisdictions do not have a concept of public domain - this is why the concept CC-0 licensing comes into play (it's public domain with a permissive license fallback).
Of course, this isn't the case when you are contracted by the government to develop software for them, which is a huge issue in my eyes. I'm not going to say let's ban shrinkwrapped software in the government (as much as I'd like to see everything the government uses be open source), but I think it's reasonable to require any contracted out to be released under on open license to the public, specifically because it ISN'T just off the shelf software.
>>Of course, this isn't the case when you are contracted by the government to develop software for them...I'm not going to say let's ban shrinkwrapped software in the government
Shrinkwrapped software != bespoke software that is contracted to be developed for the government
IMO any software that is contracted to be developed by the government should be public domain, if the government "owns" the software then in reality the public "owns" it... after all the government is "for the people, by the people" right...
This does preclude the government from licensing the USE of software (like Windows, Office, etc) that is a completely different debate. However if the government is contracting with a company for the company to build custom software the exclusive use of the government they yes that should be Public domain.
As a developer, I have been so impressed with the Obama Administrations efforts to put everything online. There is a .gov for anything now and I've been watching organization after organization digitize our commons and put them online.
This "Federal Source Code" policy is a great extension of the project-open-data initiative released a few years back:
I found the db-to-api project in this repository incredibly useful for quickly and safely exposing data from one of my applications to clients.
The only failing I see to these many many initiatives is that so few people realize these powerful free resources are out there to be taken advantage of. I hope that changes in the future.
I just checked out vets.gov to have a snoop around, and it's mostly quite impressive. But I found an SSL configuration error on this page (https://www.ebenefits.va.gov/ebenefits/vonapp) that will probably prevent vets from being able to use the site. Anyone know where to submit a ticket about that? I got yet another SSL error from the contact form on that page.
For me, the data part is at least as important as the code. I work for UC, and we are working on modernizing the interchange of data between Cal. Public Health and the National Institute of Health for vital records (birth, death, health issues around such).
There are a number of open data reference lists (e.g. - health care facilities) that we are going to be able to snapshot from APIs, rather than maintaining them separately.
The cool part of this is it seems like there's a bunch of API's on here. Perhaps that means I can submit pull-requests to get features that perhaps previously that government would not have had resources to develop.
this is cool, I want to use it... I have an eCommerce website where I sell food. It would be cool if I could be proactive in pulling items from my product catalog. The issue is there is no UPC in the API, so there's no easy way to correlate my products to recalled products. A cursory look at the source code shows me the source:
I think this is great. Maybe the code won't be reused, but it adds another dimension to governmental transparency, which is always a good thing. Furthermore, any code produced by the government is effectively being produced for the American people. We should have access to the code to use as we see fit.
I wonder how this will affect bids for government software projects? Will companies be upset that they have to open-source their software? Regardless of whether an individual agency will use it, I can see the initiative saving time and money, since programmers will know they can just find what they need in a repository. If there is one thing you can count on, it's programmers and government employees being lazy.
I'm thinking this will probably not extend to the majority of code produced. Intuitively, I'm guessing the military produces the most code of any branch of the government. And I'm not sure i believe that we should all have access to that code to use as we see fit.
Putting the code out there for other agencies to use is great, but getting them to actually use it will be another battle. Reuse is tantamount to taking away someone's budget, and therefore, status. The occurrence of two different groups implementing very similar things entirely separately happens more often than you'd think. And I'm not convinced they want to talk or work together, because bureaucrat/military officer X doesn't want to lose budget/people.
Of possible interest, I used the GitHub API a few months back to see what was actually happening with the code government agencies were committing to GitHub (reuse, contribution patterns, licensing, etc). I ended up writing up some of my observations in a post [1] and 18F blog [2].
Even if every bureaucrat and official says, "my group is a snowflake and needs a bespoke system", opportunities will arise to change a project from completely out of scope to "possible if we leverage this open system". That's the hook that consistently pushed open source into business IT.
I've been thinking this for a long time, and I'm pleasantly surprised that the US government now says this publicly. I hope this point of view becomes more prevalent in the near future.
As a person outside the government, I'm concerned about some potential problems with this initiative:
For one, Pull Requests: If a government agency gets a PR on some code, I'm concerned there may be pressure not to accept it: auditing requirements that are so high that nobody wants to review PRs (not that audits are bad!), or policies that otherwise don't encourage PRs, meaning that improvements don't get back to the government.
Secondly, us winding up with a repeat of some of the problems that other previously proprietary projects (namely, OpenSolaris) encountered: The code that was opensourced being dependant on code that, for whatever reason, couldn't be opensourced, hampering forks, and further development outside of the organization that developed the software in the first place.
Even if issues like this, or issues that I haven't even thought of, occur, this is a huge step forward.
I was recently discussing open source policy with a friend in the Australian Government whose agency is working on a whole-of-government standard that encourages open source.
She told me that many departments who don't currently have much experience in open source express the concern that if they open source their code, they've got to deal with all the issues that come with maintainership of an open source project.
Her opinion, though, was that the first and foremost reason for encouraging agencies to release their code is transparency and accountability - it's taxpayer money, after all - and that even chucking up a plain old tarball is a great start. As for running an open source community - well, cross that bridge when you get to it. It's entirely possible that no-one actually cares about contributing to your codebase. It's entirely possible that you're not going to have to deal with contributions larger than typo fixes. If you do end up getting larger contributions, write your policy and your process when you need it.
If you're in the parts of government that are encouraging a move towards openness, tell other agencies that it's fine if they don't have a fully developed open source strategy. Get their code up on GitHub somewhere first, prove that the sky won't fall in, and just get them comfortable with the concept of open source. Once we've achieved the first objective of getting source code out in the open at all, then we can worry about the rest.
> The People's * is a prefix used by totalitarian governments.
Indeed. And so are words like "homeland".
The last ~2 decades have seen a widespread change in ideas about the relationship between the U.S. federal govt. and the people. As a result, practices that were once restricted to highly repressive states are now merely a bit controversial in the U.S. And then we also have these ways of naming things that you & I noted.
So I wonder: could it be that thinking more like a totalitarian somehow naturally leads one to talk more like a totalitarian?
I keep watching older TV shows that aired when I was in the military and they keep using the term "War on Terror", which frankly, is something I'd never directly heard in the military (only from politicians) and it's surprising to me that anyone would say such a thing without reactively slapping themselves for gleefully eating up such confusion-inducing propaganda.
Another one to watch for: "bad guys". It's such a common phrase in America I think people are numb to how thought-stopping it is. TV shows, cops, defense contractors, everyone's eager to nasty things to "bad guys" (but it's okay, see, because they're bad).
As helpful as this may be, the real transformative code is often proprietary. As others have mentioned here, not everybody will use every reusable component, whether it be because of ignorance, larger system incompatibility, or simply turf protection.
We should look into solutions for intellectual property that are based on an information economy instead of an industrial economy.
Personally, I think it would make sense (perhaps more for pharmaceuticals than software) to significantly shorten the time a patent is valid and/or strip the protection of monopolistic production rights, and instead allow the free market to sell the product at the lowest cost it can be made at, as long as there is a royalty fee. How the fee is determined, I'm not sure yet.
Still, it's clear that our IP system is creaky, overcomplicated, and is tilted too far in the direction of big business, lawyers, and patent trolls, instead of the actual inventors and consumers.
Not sure why this comment was downvoted. I feel similarly about IP reform, and would love to see more discussion around how we can reward innovators appropriately for inventing things without artificially limiting the production of those inventions (via patent trolling, royalty fees, or any other mechanism).
If the issue is that inventors want to get compensated for their inventions (to a degree commensurate with their value), but we don't want to limit what the market can produce, then maybe we could imagine an IP-buying group that paid a fair price to inventors and then made the IP available for general use among the financial backers of said group. One could imagine this as a governmental function (distributing purchased IP among taxpayers) or a corporation (s/taxpayers/shareholders), and it appears to me[0] either one could fit within the framework of existing IP law.
[0] I am not a lawyer and have no idea if this is true.
Avoiding duplication of source code across Federal agencies is nice, but it would be better to eliminate duplicate agencies and functions within the bureaucracy.
I'm very familiar with the Department of Energy process, and there are a few considerations for every lab in the DOE.
TL;DR: The DOE encourages open source software, but it isn't default and there's some (low) barriers.
In general, though, what you can do with your (non-export-controlled) code consists of (in order of increasing difficulty):
1. Nothing. Keep your code private. If you'd like to stop maintaining code but want to make sure it sticks around, the DOE has a software library, the ESTSC, in Oak Ridge (a division of OSTI). It may also be the case that the entity running the lab wants to claim ownership.
2. Open Source. Due diligence is needed to ensure funding agencies and MOUs are respected. Copyright is typically assigned to the contractor running the lab in question (i.e. for Berkeley lab, -> Copyright goes to UC, SLAC -> Stanford, etc...). Major international collaborations can be a bit tricky because foreign countries have their own rules. I think more work needs to be emphasized on this front going forwards.
The DOE also wants to track the popularity of Open Source software, namely downloads. GitHub has met their requirements for reporting.
The DOE discourages use of the GPL and similar licenses. The reason, as I understand it, is due to the fact that the Government (i.e. Defense) must be able to use and modify software (and give to contractors, etc...) without falling under any additional burden. I believe the BSD license is preferred most widely across the labs.
In some cases, people at labs do release software under GPL. If they didn't get special permission, they are likely violating their lab's contract with the DOE.
3. Commercialize. This is really hard. You have to first perform market research, establish the market, spin off, deal with SBIRs, etc... This is a high barrier.
I've been personally working on streamlining the process for (2) with legal for my lab, so that anybody can open source their software very easily, hopefully by just filling out a web page. I'm hoping the recent white house directives help eliminate some of the bureaucracy involved in the process. I've also been trying to reduce fragmentation across the lab. The lab has never offered an official SCM platform, and grad students/postdocs are notoriously bad at keeping important source code in their personal GitHub and then leaving after some time.
It should be noted that almost all national lab facilities are effectively ran under contract, so nearly all national lab employees are not actually federal employees. So we do have a slightly different set of rules.
Finally, there is already a decent presence on github and bitbucket of labs, in case you are interested:
It should be noted this is an extremely, extremely small slice of the software that drives experiments, projects, and research in the lab. Many times software belongs to the project/research group, so there's likely a project github organization where the code naturally resides. This is sort of a consequence of labs becoming more and more multi-disciplined, i.e. the science missions of labs like SLAC and Fermilab are no longer aligned primarily around their accelerators.
OSTI is supposed to maintain an index of that software if it's reasonably important, but it's not really enforced.
PS: If someone from USDS/data.gov/18f can and would like to help out with this in any way, I'd be happy to collaborate!
The DC municipal government actually has a Github organization (https://github.com/dcgov) and has been attempting to pilot open source development (http://open.dc.gov/open-source-guidelines/) throughout the city. I helped write that document, which was inspired by/borrowed from 18F policy. Take up has so far been limited, though some agencies have been using Github Pages.
Also, if you're interested in issues like that, you'd probably really enjoy joining the http://codefordc.org/ community! I've only attended one Meetup, but it was the most ridiculously welcoming and well-run tech meetup I've ever attended!
And how would that help? How do you know that the code they published is the same as the code installed on those machines? You need voting machines that leave a paper trail (which the voter can verify while voting), that way the machines can provide quick results while the final results can be counted by hand.
We had them, for decades. They involved the use of punched cards and a butterfly ballot. The voter could physically inspect the ballot to determine which holes were punched. Sadly, the "hanging chad" of Bush v. Gore (2000) pushed many states into using electronic machines, with predictable results.
This is a great move by the White House. While there are a lot of groups that are trying to push for more openness and release of software, it can often be challenging. A lot of federal groups have been taught over the years to be very risk averse, and open software is viewed by them to be risk. Probably one of the most common concerns is, "What happens if someone takes and misuses our software?" In a highly risk-averse federal environment, these can be challenging arguments to fight against.
If you like and support this kind of thing, one big thing you can do is to contribute and supply feedback. We frequently have to go to our superiors and justify what we are doing with regards to open source. We say things like, "this repository had X pull requests from non-federal contributors". Or, "We got Y comments and questions from non-federal users of our projects".
It could be as simple as an email saying "Hey thanks, I found this useful", to a full-on pull request fixing an issue or with a new feature request. The more fodder we have to say "open source increases engagement and creates positive feedback" the more you will see this kind of thing happening.