Web vs. SOA – room for both in DoD

I’ve been running across a number of articles lately (e.g. here and here) that attempt to reconcile the concepts of SOA and Web2.0; or at least define them adequately to describe how they fit. They obviously share a number of concepts and a lot of terminology, but also differ in some fundamental ways.

It is probably an over simplification to describe SOA as an IT-department driven mechanism for planful standards-based line-of-business application integration and Web2.0 as more of an emergent phenomona, but it is a starting point.

In the commercial world that demarcation between “web” and “enterprise” is much more clearly drawn, and there is more shared language and understanding around what they mean. If it is integrating my line of business applications, connecting my trading partners, and it uses “middleware” to implement “SOA” it is “enterprise.” If it is consumer oriented, customer facing, or based on REST/LAMP it is “web.”

The DoD unfortunately lumps everything together under the SOA and NetCentricity aegis. I think this creates lots of confusion since most people still equate SOA with web services. For example, when we talk about discovery are we talking about web-like real time content discovery, or are we talking about SOA-like design-time service discovery (for example)?

The interesting thing is that the DoD is so big, more of a super enterprise really, that there is room inside of it for both concepts. Inside an enclave, the typical enterprise concerns related to application integration and “trading partner integration” with other enclaves are key. This is IT-driven SOA stuff.

But there is also lots of room in the seams for web and web2.0 constructs. Search, mashups, widgets (www.programmableweb.mil anyone?), rapid composability, dynamic provisioning and the like.

Consider something like NECC to illustrate the point. NECC at the back end is really a collection of capabilities brought about by integrating existing systems (either as SOA integration of existing runtimes, or by actually creating a new runtime from existing pieces – the strategy right now isn’t clear). It needs traditional IT-managed integration to bring together situational awareness feeds, targeting databases, geo-spatial and imagry data, and etc. There will also need to be “traditional” thin and thick client access to all this stuff.

However, the real interesting part starts when NECC exposes a set of REST-based API’s of the type that Yahoo or Amazon would build if they were creating NECC and maybe also implements a widget library. Imagine users being able to add these widgets, subscribe to a variety of RSS feeds from NECC and other systems, use things like Yahoo Pipes to combine them, and etc. All without having to go through another C+A process because no new code has been written.

• • •

Conference Blogging – DoD

Today I’ve been following John Scott’s stream of posts from the Open Source Business Conference in San Francisco and it reminded me of a topic I meant to write about a few weeks ago after I attended the DISA conference.

There were two primary topics that came up over and over again at the conference: open source and cultural change within the agency. This post is a reflection on culture.

The last conference I attended on the west coast was the O’Reilly Web 2.0 Summit at the same venue where the OSBC is going on now. At that conference, every session had at least the first three rows full of real time bloggers. But that wasn’t the only form of wired audience participation. Countless iChat participants discussed the session topics in real time, pictures appeared on flickr almost immediatly, videos of sessions ended up on youtube, and a variety of wiki’s, chat boards, and etc. sprung up for real or near real time discussions.

In contrast, at the DISA conference I rarely saw a laptop anywhere at the venue, and amazingly, no wi-fi connectivity was available anywhere in the facility. My only internet connectivity outside of my hotel room came (slowly) through my Cingular edge cell phone and bluetooth. Can you imagine AT&T sponsoring a conference and offering no wireless in the building?

I’m not surprised really that few people in the industry blog. There is the natural orientation toward secrecy that comes from the nature of the work; but there are more practical reasons. No government employee can blog without running amok of the PAO and, at this particular conference, every single attendee was required to sign an NDA to obtain admission. Unfortunately I wasn’t offered a copy to keep but I’m pretty confident this post doesn’t violate it.

DISA, if you want to spur cultural change start seeking more transparent discussion of the major issues within the organization, and even better, start an exchange program. Send 10% of your employees on targeted one year sabbaticals to work on the west coast. Put them in startups, Amazon, Google, Microsoft, whereever. Just send them out there. DC isn’t the place for it.

• • •

Perspectives on Free and Open Source Software

I’ve been wading through a collection of papers on F/LOSS from MIT press and am finding it to be a great resource. I’m reading it in book form (available from MIT press) but it is also available online here as a pdf.

What I particularly like is the academic balance of the papers. Standing in front of the Open Source Steamroller by Robert L. Glass (page 81) and Assessing Open Source Software Development by Rusovan, Lawford, and Parnas (page 107) offer a sampling of cautionary contrarian views that are valuable to keep in mind.

In the first paper, Rober Glass takes advantage of his 45+ years in the industry to question some of the underlying assumptions of the movement by looking at similar things in his past experience. In the second, the authors dig into the emergent design of the Linux ARP module and ask whether the result is maintainable.

• • •

Cyberwar – a first taste

Estonia is claiming to be under cyber attack from Russia because of a dispute involving a Soviet era war memorial.

In a previous post I suggested that foreign states may be involved in the establishment of and/or compromising of bot nets to use for these kinds of attacks. Interestingly, the attacks on Estonia seem to have started in Russia but are now coming from bot net armies around the world.

• • •

End of LSI’s?

Looks like Congress is looking to legislate the fox out of the hen house by bringing acquisition skills back in house and eliminating the LSI concept.

There is no direct impact on current LSI programs in the bill but the general mood on LSI’s continues to sour.

• • •

Open Source and Ethics?

Richard Stallman, to contrast the term “free software” with “open source”, notes in a recent letter to the MIT Technology Review that:

“Open Source Software refers to a completely different idea, which does not aim for freedom or social solidarity but merely wants to make software more powerful and reliable. Those practical goals are useful, but we must not subordinate ethical issues to engineering.”

I’ve been following a number of threads on similar topics over at the libertarian Technology Liberation Front and I’m struggling with this idea that free software (or other constraining factors on our use of computers) is the encapsulation of an important ethical issue.

As a natural born gear head who was always frustrated when Detroit’s engineers dictated how my car would function and what I could do to change it, I’m sympathetic with the fundamental desire “to own what I own” at the base of the argument. However, I just can’t help but see the kinds of freedoms at stake here as more of a market issue than an ethical “freedom” issue. How can the freedom to do something that has only existed for half a century deserve the same kind of moral / ethical foundation as the freedoms enshrined in the bill of rights for example?

For fifty or more years the Bell system owned the telephones in our houses and controlled or owned every piece of equipment attached to their network. Irritating? Yes. Bad for innovation? Yes. Bad for competition and consumer choice? Yes. Ethically or morally wrong? I just can’t make that leap; maybe it is a matter of degree and it falls below the moral threshold on my continuum of ethical concerns. In fact, you may even argue that without that model in that timeframe the infrastructure build-out that brought so much public good would have never happened in the first place.

With that as my mindset I simply can’t get myself worked up into a lather over the idea that I don’t have complete control over the code running on a machine that I own (I’m drawing the analogy here to not being able to control the phone company gear in my home). Do I desire that kind of control? Yes. Do I think that open and/or free software has many advantages over proprietary? Yes. However, I just don’t see this as an ethical / moral issue. I might change my mind if I keep thinking more about it; but I’m not there right now.

Back to that continuum idea, in some ways I guess I think that in a world full of Darfur’s, Iraq’s, Iran’s, and etc.; peak oil concerns; global climate change and associated micro climate changes; 50 years of dumped plastic beginning to enter our food chain; and etc. the “right” to change a line of code in my personal computer seems like a bit of a shoulder shrug. This isn’t moral relativism (before you yell at me); I just don’t see it as qualifying as a moral issue at all.

• • •

The Follower’s Curse

At the DISA conference last week I spent some time with the people at the Global Command and Control System (GCCS) booth. I wanted to get a better understanding of how the GCCS family of systems gets used in practice. I also wanted to get their perspective on Net-Enabled Command Capability (NECC).

I walked away with a strong sense of Deja Vu.

NECC’s predecessor, Joint Command and Control (JC2) was conceived to provide a more Joint friendly replacement to the GCCS family of systems. The goal was composability of capability for the Joint Commander in order to replace the all or nothing service-specific choices inherent in the current family. Additionally, it was intended to migrate to a modern post- Common Operating Environment architecture that would support more rapid deployment of new capabilities. The change to the NECC nomenclature signals additional emphasis on web-based technologies and recent descriptions of the program focus very heavily on the ability to rapidly develop, certify, and deploy new functionality to the warfighter.

A few years ago the Air Force started a program called Web Enabled Execution Management Capability (WEEMC) (pdf link) based on modern web-enabled architecture to replace the Automated Deep Operations Coordination System (ADOCS) which was based on difficult-to-maintain C++ code in a client server architecture. Unfortunately, the replacement program was hobbled from the beginning by the marching orders to “treat ADOCS like a requirements document; replace the functionality exactly but do it with more maintainable and flexible modern technology.”

The problem was that the warfighter loved the capabilities that they had fought hard battles to obtain in ADOCS and WEEMC’s development approach conspired to put it in a permanent “me to” follower position; and worse, a follower that was likely to be buggier and less capable at any given point in time (especially since the lead system ADOCS continued to find funding to evolve).

NECC will have to think hard about architecture and implementation to make sure that the constraints of a web UX aren’t too constraining in shipboard and similar environments. But more importantly, the “adoption plan” for NECC should ensure that it focuses first on capabilities that aren’t offered within the huge built-out GCCS infrastructure around the world so that NECC doesn’t find itself competing for adoption with a system that everyone is familiar with, does it better (initally) and has been battle proven.

• • •

DISA Conference Notes

I must say that the annual DISA conference this week in Nashville exceeded my expectations. It was a great opportunity to meet people, network, and get a sense for the agency’s pains and initiatives. A good portion of the DISA leadership was making itself available on the floor and in the halls for conversation and discussion, and the relatively compact venue in Nashville worked to enhance the serendipitous pinball collision effect.

The keynote speeches effectlively laid out the challenges faced by the agency – the need for cultural change, the need to unbind themselves from what they think the rules say (instead of what they really say), get “the middle” to evolve through incentives and mindset, and other related themes. The “intransigence of the middle” point was on display throughout the event; hallway conversations after each keynote would be full of overheard snippets like “I’d love to do that but the security/testing/whatever people would never allow it” or “I’d love to do that but the policy/rule/process won’t permit it.” One has to be careful with these kinds of pronouncements though; often once something is named as a problem, the problem itself is somehow reinforced as the now called out “problems” dig in their heels further.

If I had one beef with the keynote speeches it is that I think they tend to be too sugarcoated to create the kind of urgency that is really needed for cultural change. The speeches were all on the “everything is great and moving forward according to plan” variety. The focus is on speed, sharing, and getting out of the box but the culture as it stands is simply too unforgiving of failure and squelches risk taking as a result. Instead of a keynote of “here’s the strategy and here are the successes since last year” I would love to have heard a bit more unvarnished “here’s the strategy, here are the successes, and here is where we are still too slow and too rigid…” The right balance between unvarnished self-assessment and forgiveness for failure. In fairness, maybe public self-flaggelation at an annual conference isn’t the right mechanism.

Maybe what is needed is a DISA version of Bill Gate’s Internet Tidal Wave memo (note to Ray Ozzie: great speech until you got to the open source question / response. Might be time for a Ray Ozzie Open Source Tidal Wave memo). If DISA were a company the urgency necessary to painfully but quickly drive cultural change would come from waning revenues, failing product lines, and loss of confidence on the Street. That mechanism doesn’t work here. So maybe Lt. Gen. Croom can write a memo to the agency, wait, scratch that…. Maybe Lt. Gen. Croom can do a video for the agency in which he describes the impact of the agency’s failure to get important programs like NECC and NCES (for example) done more quickly, in a more agile way, and more cost effectively. How those failures to be fast, if not corrected, will erode the agency’s long term viability and more importantly, are impacting the warfighter’s ability to be effective. Then interview warfighters up and down the chain of command and have them weigh in on what it means to them.

What I did find to be particularly refreshing was the apparent openness among the senior leadership of the organization. Lt. Gen. Croom has obviously built the kind of intellectually open environment among his staff where they feel like they can speak their minds (as evidenced by his CTO advocating loudly for open source during the Q+A after the keynote – calling out his boss directly, if only jokingly). I’m sure this openness will pay tremendous dividends during the ongoing change management process as the tone set at the top will filter through the entire organization’s culture. (By the way, probably 5 of 7 questions after the keynotes were about open source. It was unexpected how frequently it came up).

It was surprising however, in the context of that sense of openness, to have to sign an NDA when I picked up my badge. Who signs an NDA to attend a conference? I recall that it said something about protecting proprietary IP and “sensitive information” but I’m not exactly sure what I promised to do since I wasn’t offered a copy to keep after I signed it. Proprietary is easy, it is usually marked, but I’m less certain how to evalute what would be considered sensitive. Hopefully this isn’t.

You know, it’s kind of strange. The leadership at DISA has sent the message over and over again that they want an organization that will emulate the creativity, innovation and operational excellence of west coast firms like Amazon, Google, and Yahoo. But if you attend a west coast conference like O’Reilly’s Web 2.0 Summit there will be at least 100 people in the front rows real time blogging the event. People with laptops throughout the event will be conversing in chat rooms as the event unfolds to discuss the major themes etc., as the presentations are being given. Many more people will blog more thoughtful posts after the fact to continue the discussions started during the event. And finally, people will participate in sharing information, ideas and contact info before, during, and after the event via a wiki set up for the event. An altogether invigorating and refreshing set of interactions. The DISA conference in contrast asks you to sign an NDA and there is no wireless available anywhere in the building; but it wouldn’t matter because there isn’t a laptop in site. The cultural divide is palpable but hopefully not unbridgable.

• • •

DISA Announces Surprise Acquisition at Annual Conference

In a bold and surprising move, the Defense Information Systems Agency (DISA) used the occasion of its annual customer conference to announce the acquisition of Google’s Commercial Services (GCS) division. In a single stunning stroke, the agency added significant credibility to its claim to be the preeminent provider of remotely hosted services to the defense establishment.

In a joint statement DISA and Google called the transaction strategic and “in the nation’s and both parties mutual best interest.” DISA’s Lt. Gen. Croom was quoted during DISA’s accompanying press conference as saying “Our acquisition of Google Commercial Services represents a turning point in the agency’s transition to an agile, fast moving, and tactically relevant service provider. Commercial IT has been following in the consumer’s footsteps, and we are now better positioned to leverage the same trend.”

DISA appears to have seen in Google’s technology an opportunity to accelerate delivery of its Net Enabled Command and Control system (NECC). Under the terms of the deal DISA will obtain perpetual rights to the entire Google technology stack as of the date of the transaction for the delivery of search, mapping, mail, and personal productivity applications. DISA is further licensed to leverage the underlying technologies including the Google file system and its design for white box computing to support the delivery of NECC from DISA’s own data centers. DISA plans to continue to market the current GCS product suite to commercial businesses in order to maintain cost discipline, keep data center utilization high, and ensure continued technology advances are well funded and available to its defense customer base.

The deal is reported to include the entire staff of GCS though the final details of their conversion to Government Services ranks have not been released. DISA is purported to have been strongly motivated by the staffing aspects of the deal and believes that the external infusion of skills and culture will pay dividends throughout DISA’s business units, though initially the acquired unit will maintain a degree of autonomy as the DISA GCS division. In addition to maintaining and improving the purchased line of products, DISA is expected to deploy some of the new staff on NECC development.

Industry watchers were caught flat footed by Google’s participation in the transaction. No additional explanation was forthcoming from Google, but industry insiders speculate that the deal may have been initiated as one of the terms of a deal with the Department of Justice to deflect wholesale copyright violation troubles within its popular YouTube division.

Ok, obviously this is a farce. However, DISA, like many large organizations, is having its own struggles with cultural transformation, technology delivery, and cost control in an era of technological discontinuity within the DoD. It’s just kind of fun to think about how they might approach the problem if they were not constrained by the reality of being a government agency. Of course, the Google part is just silly, everyone knows that wholesale copyright violation is a civil matter and outside of DoJ’s jurisdiction. 😉

• • •