Thursday, May 26, 2005

Software Darwinism: Why Open Source is the New "Standard"

Traditionally, a "standard" is established when multiple vendors sell essentially the same technology and customers demand that this commoditized technology be accessed via a standard API. Vendors can then compete on implementation. J2EE is a prime example of the success of this approach.

Open source turns all of this on its head. If there are multiple implementations doing roughly the same thing behind the same API, why not simply collaborate on a SINGLE implementation? One or two best-of-breed implementations inevitably are settled on and everyone can move on and work on technology that has not been commoditized. We have seen this happen with operating systems (Linux) and web servers (Apache), and it will soon happen with databases and application servers.

Everyone wins with the open source approach: customers get better technology at lower cost, vendors do not waste brainpower replicating and maintaining commoditized technology, and developers can step through the source when there are problems.

Every time a model evolves there are stragglers, hybrids, and early adopters. An example of the hybrid model is Java. While Java itself is not released under an open source license, there has been a ton of open source innovation in Java, ranging from Struts to Ant to JUnit to Hibernate. Such hybrid models do cause strange occurrences, such as the JCP continually recreating existing and popular open source projects. A cleaner model, which IBM and many folks in the Java community are advocating, would be to open source Java and integrate leading projects such as Hibernate rather than recreating them.

In the early adopter LAMP stack, where everything is pure play open source, there are still a lot of balls in the air with competing projects, such as the zillion web server API's in Python that are finally coming under a single umbrella. Open source is the accelerated Darwinism of software components, where the strong survive and the weak die in the purgatory of one committer with zero download counts.

One thing is certain, open source is a meritocracy where the best-of-breed technology wins. As open source matures, it is now moving beyond replicating existing commoditized software to providing advanced technology beyond the apparent capabilities of the large software vendors. As we have learned time and time again (ie, AOL and CompuServe vs. the Internet), emergent behavior compounded with a network effect is the path to success.


Julian said...

Hi, I listened to your interview that was published on IT Conversations. That was really good. I like that you're embracing Python, which IMHO, is the right way to move.
I tried out your ActiveGrid tutorial too.

Anyway, quick question. You mentioned that "web server API's in Python that are finally coming under a single umbrella". What is that umbrella?

Peter Yared said...

Hi Jullian,

Thanks for your comments. Plesae check out the following:


Mason Jones said...

You're quite right about the JCP and its ongoing confusion. It's clear to everyone, for example, that EJB 3.0 is simply trying to adopt ideas from Hibernate, Spring, and others and coopt them as a new "standard." The mystery is why this so-called standard would be an improvement over simply using tried-and-true open source solutions like Hibernate and Spring.

Open source, while it's certainly *more* of a meritocracy, does still occasionally slip up, so it remains incumbent on us all to not fall for the "latest and greatest" each time, as well. It can be fun to pry open the source and see what's up, but it's not always so enjoyable when you're in the last push on a big project!

Good luck with ActiveGrid, it's very interesting -- nice to see something not married to specific languages, though I'm sure it's a hard sell at times.

Aldo.CastaƱeda said...

I'm researching "open standards" in identity management and your post addresses some of the thoughts I've had lately (prompted in part by your comments in the IT Conversations interview).

I think this concept of "open source" being the new "standard" is appealing but it raises at least two issues:

1. What if an "implementation" reads on a patent? The fact that the GASLAMP is fired by Open Source doesn't necessarily mean a particular implementation doesn't infringe someone's IPR - right?

Arguably the standards setting process provides value in that it least mitigates this risk.

2. What about "embrace and extend" concerns?

I suppose it could be argued that as the net's architecture shifts to web services it's all "embrace and extend" the question is who attracts more "action" around their particular set of features? Perhaps that's your principle point?

I'd like to discuss this area a bit in my research (see therefore your thoughts would be very much appreciated.


Peter Yared said...

Hi Aldo,

When I did the Liberty Alliance we settled on RAND (reasonable and nondiscrimatory) IP terms. But nowadays it looks like whatever you do you step on somebody's patent, standards or not. Rambus is a case in point, where they patented things they submitted to a standards organization and didn't tell anyone!

Embrace and extend is exactly the point. People should fork off, and the best fork will win!