October 24, 2008 by jimstogdill
Open Source in Defense: Consuming it is Nice, but Building it is Better
I’ll be participating in a panel discussion next week at the 4th Annual DoD Open Technology Conference in DC. The Panel is about open source software in defense and will be moderated by John Scott, an author of Sue Payton’s Open Technology Development Roadmap (pdf). Dan Risacher, who recently discussed the DoD CIO’s upcoming policy memo with GCN, will also be on the panel. We’ll be talking about what makes open source valuable to the department – consuming it, contributing to it, and even building it outright. We’ll also be talking about the policy, legal, and accidental process roadblocks that make it more difficult today than it should be.
Yesterday, while I was doing some preparation, I ran across this sources sought on Fed Biz Ops. It got me thinking about the down to Earth practical stuff that is necessary to make a difference in encouraging open source in defense. I am going to come back to the details of this sources sought in a moment.
The DoD has broken the seal when it comes to consuming open source, at least in packaged form. I’m not certain where I got this factoid, but I think the US Army is now Redhat’s single biggest customer. But like I’ve said before, consuming open source is no big deal and really isn’t occasion for a big celebration. Where the DoD stands to gain much more value is in producing open source software.
Every industry has at least some domain-specific software needs. The stuff that makes up their industry-specific “stack” and that isn’t readily provided in the cross-industry products from the major software vendors. For example, the financial services industry depends on things like high speed messaging buses and high availability transaction monitors. Web firms use things like Perlbal, Hadoop, and of course Apache that help them build a massively horizontally scaled web presence. Telecom has specifications like H.323 and now SLEE and SIP and products built on them.
In the old days, if the industry was big enough, domain-specific software vendors would spring up to provide them with the infrastructure that they needed (e.g. Tibco’s Rendezvous). If they were REALLY big, a large software vendor might even offer a domain-specific product, or at least a version of their product.
These days though there is an alternative, Open Source Quasi- Joint Ventures. Well, nobody really calls them that, but that’s how I think of them. They are like accidental joint ventures that do resource sharing the way a traditional joint venture would, but they rely on open source licensing to make the risk of participation low. Plus, they avoid most of the legal and ownership wrangling that happens in a real joint venture.
A great example of this approach is the Advanced Message Queuing Protocol (AMQP) (and associated AMQ implementations such as OpenAMQ). It was initiated by JPMorgan but has grown to include many large banks as participants. The banks don’t give up any competitive advantage by participating because messaging is about passing information to trading partners, but they save money by more efficiently providing for their own infrastructure.
Things like OpenID and Hadoop also fit into this mold. Companies like Yahoo and SixApart are taking active roles in funding and guiding the development of their industry-specific technology. Again, it’s far enough down in their stack that they aren’t giving up a competitive position; but they are saving money by sharing their development resources.
I don’t mean to say that these aren’t normal open source projects in every way. I’m simply making a distinction about how and why they are funded in terms of the specific needs of the industry that is funding them. By joining together to build components for an industry-specific stack and then intentionally commoditizing it within that industry, these projects seem to be filling in where JV’s or domain-specific software companies might have focused before. This open source approach is better than a traditional JV though because new participants can join up at any time and they avoid many of the up front issues of starting a JV.
Back to the DoD. Defense has saved a great deal over the last decade recognizing that it can leverage COTS hardware and software. However, it still has many unique needs for its information technology stack – the DoD operates in a different operational environment and has many specialized requirements. So, while the DoD today is beginning to consume “package” commodity open source projects such as Linux, there is still a great opportunity to steal a page from JPMorgan’s playbook and build defense-specific infrastructure as open source. The DoD builds defense-specific stack components all the time, but they rarely do it in a way that makes it easy for other programs to adopt them (or even know about them). An open source approach would better spread the funding and would also ensure that once the money is spent the pieces could be widely used and adopted.
This brings me back to the WebTAS sources sought.
WebTAS started life years ago to simplify the process of conducting data analysis that spanned database instances and DBMS’s. Over time it evolved to be something like generalized middleware and application framework for data analytics applications (and has been used in even more general applications since then). The sources sought I linked to describes basically a “business as usual” approach to continuing the program as it plans to support continued R+D of the core framework as well as for at least some of the analytics work that will be done with it (which is why top secret clearances are required).
It’s probably worth asking the question whether there is even still a need for a government funded program to build a database connectivity and analytics suite (especially for a program that is expected to cost as much as $300M – that’s almost 25% of the $1.4B value of the Linux Operating System!). A lot of time has passed since the program was started and there are many more commercial and open source technologies available in that space today than there were when WebTAS started. However, for the purposes of this discussion I’m going to assume that WebTAS is continuing to provide unique capabilities to meet DoD-specific requirements. However, with that assumption in place, I’m going to argue that WebTAS should be developed in the fashion of an Open Source Quasi-JV.
Because WebTAS was developed under government contract, it can theoretically be used in any government contract. The government could furnish it to a contractor as Government Furnished Equipment (GFE) for any program. However, in practice this rarely happens. In defense IT, infrastructure software tends to be used only on contracts delivered by the contractor that built the infrastructure. As an example of this tendency, all three of the projects mentioned by the sources sought, SWIC, PANACIA, and MAAP are built primarily by the same contractor that currently delivers WebTAS.
If the government is interested in getting more value out of their investments in infrastructure like WebTAS (and would like to quench the proprietary lock in business model that they are stuck with today) they need to take concrete steps. As the sources sought indicates, the contract is expected to have a five year term. Once that contract is issued, if an open source approach isn’t built in, there won’t be another opportunity to change the approach for five years.
So, it’s great that Ms. Payton’s office wrote the OTD Roadmap and that the DoD CIO is about to issue a clarifying open source policy. However, if I were Ms. Payton, I would take another step and have my staff directly engaging with program managers of programs like WebTAS to ensure they let contracts that would directly support OTD in general and open source in particular.
For example, why not define in the contract’s CDRL’s (basically the stuff that is delivered) that the vendor must establish and govern an open community and the associated code repository? Why not include award fee metrics that incent open community – stuff like the number of other government programs or contractors that are participating in the community and using the software for their projects in order to incent open community development and marketing? While using IDIQ style contracts, why not make the awards to a range of potential contributors so that the contract is positioned to support an eco-system of contributors and ensure that each of them understands the Intellectual Property approach that will be used? Why not split the contract to develop the infrastructure from the use of it for analytics work so that a wider group of contractors (that don’t have to take on the cost of TS clearances) can participate?
The goal should be to establish rules and incentives in the contract that encourage the development of widely available open software with an effective community.
Savvy contractors will realize on their own that taking the initiative to open source infrastructure like WebTAS themselves is good for business. Assuming the code is valuable, aggressively commoditizing it will contribute to wider adoption and more opportunity – after all, these are all services businesses. However, it’s early and we haven’t reached that tipping point yet. So, if the government wants to achieve real strides with open initiatives they need to do more than provide OTD policy guidance. They need to aggressively work programs like WebTAS to establish contractual terms and incentives that will push their contractors past the tipping point before another round of long term contracts freezes progress for half a decade.