Wednesday, December 14, 2005

LAMP BusinessWeek Article, Slashdotted Again

There was a great article about LAMP and ActiveGrid by Steve Hamm in BusinessWeek today that was Slashdotted and started YAFF (yet another flame war) very reminiscent of when we were Slashdotted last year.

To sum up the article, it lists a few indisputable facts:

  1. Google, Yahoo, and a lot of Web 2.0 companies such as Friendster, Flickr, and Facebook are using open source stacks like LAMP instead of Java.

  2. There are more and more developers and websites using PHP, as evidenced by booksales and Netcraft stats.

  3. Companies like Merrill Lynch write processor intensive code in C/C++ directly on Linux and script it rather than use Java API's, presumably because they want more perfromance and do not care about platform portability out of Linux.

  4. .Net is getting used a lot more in enterprises (something I have seen in quite a few large accounts and which Sun even acknowledges in the article).

  5. "Earlier this year, [IBM] threw its weight behind PHP as a Web programming language."


Well these are all facts, and grouped together like this, I daresay they are a trend. Sun's response?

  1. Java is getting a lot of traction on mobile phones.

  2. Java is still used for "large, complex corporate applications".


Hey, I recommend Java myself if you want to write a cell phone game or a large, complex corporate app. But for a lightweight web app that needs to integrate a bunch of services with a rich user interface? It's LAMP all the way.

Sunday, December 11, 2005

Google - A Network Effect on Ads

There have been rumblings through the valley about how Google is arrogant, going to fail, not partner friendly, etc. At ActiveGrid, for example, we are bummed that our Googlezon demo is exceeding Google's maximum allowable web service calls.

Matt Asay recently wrote a post about how Google's properties beyond search are all relative failures. I frankly don't think Google's search is all that hot either, I use MSN Search when I do complex queries.

What Google has, and the reason it will be around for a long long time, is a network effect on advertising. Just like EBay cornered auctions, PayPal cornered point to point transactions, and Flickr cornered photo sharing, there is a huge community of people and companies who advertise hundreds of thousands if not millions of terms of Google. Users are accustomed to searching for very nichy (I dare say long tail) things on Google and finding relevant ads.

While search has clearly become commoditized and matched by Yahoo! and MSN, the chances of anyone successfully competeting against the advertising juugernaut that Google has created is not likely. It is as challenging as convincing a critical mass of EBay buyers and sellers to move to something new.

Google is definitely helping a lot of folks make money by advertising their products to new customers. I just hope that Google starts to enable software companies to create new services that use their platform without a bunch of hassle - this is one thing that Microsoft actually has always done very well.

Friday, December 09, 2005

Googlezon - SOA in Action

Today's applications require rich user interfaces and tie together multiple systems. It was hard enough to build a simple HTML app to front-end a single database - most of these types of projects are delivered over budget and late, if at all. Clearly, using today's technologies, developers can't deliver these types of applications in a time-efficient and cost-effective way.

As many of you know, we built ActiveGrid from the ground up with AJAX as well as SOA standards like BPEL, XML Schema, WSDL, and XPath. The new 1.2 version of the ActiveGrid Application Builder and LAMP Application Server offer numerous rich interface and SOA features that make it very easy to build rich applications that tie together a slew of services, ranging from databases to RSS feeds to REST services to Web services.

Check out the following demo app, Googlezon (named after the Googlezon Flash movie that made the rounds last year. Googlezon lets you search both Google and Amazon at the same time by calling Google's SOAP service and Amazon's REST service. It uses scrollbars that asynchronously fetch data from Google or Amazon as you scroll, so you can simply scroll back up to see the previous results. And it previews pages in pop-ups when you mouse over the search results. We like to call it a better Google than Google. :)

Imagine integrating your CRM and ERP results into a single page, just like this ActiveGrid demo integrates Amazon and Google.

Update: Apparently we have hit the Google connection limit, so here's a screenshot for those of you who are getting an error message!

Sunday, August 14, 2005

Why Grids Make Sense

The term "grid" means a lot of different things to a lot of different people, so first off let's define "grid" within the context of this post: a cluster of commodity machines in the same geographic location running a single transaction-oriented application.

The traditional way to run transaction-oriented applications is with a three tier architecture, where web servers running on a cluster of 1 or 2 processor commodity machines mitigate connections to application servers running on a small cluster of 4, 8, or 16 processor machines. This is a great architecture and made a lot of sense when software was written to scale vertically and IT organizations wanted to run small clusters of UNIX machines.


Traditional Three Tier Architecture



The web tier serves static content and load balances connections to the application server tier, which serves dynamic content and connects to backends.

As part of the trend towards commodity machines, people are starting to run 1 or 2 processor commodity machines in the application tier as well as the web tier.


Modern Three Tier Architecture



When you step back and look at this evolving architecture, it begs the question:

Why have a cluster of commodity machines mitigate connections to... another cluster of commodity machines?

Consider the following architecture, where the web and application tiers are collapsed:


Grid Architecture



Since static content is now served directly from the Linux kernel, there is no performance impact of serving the static and dynamic content from the same machines. By including a reverse proxy, which most people are running now, a DMZ can be maintained in place of the web server tier's DMZ.

This architecture where we move from a triple decker club sandwich to a pancake architecture where the web and application tiers are merged together is the exact architecture deployed at every large scale web site, ranging from Amazon to E*TRADE to Google to Yahoo.

In order to deploy this kind of grid architecture in an enterprise environment, the nodes need to be able to call each other to share sessions or data. More on this evolved load balancing in a subsequent post.

Friday, August 12, 2005

Here Come the Java Guys

We threw a small party last night to celebrate the ActiveGrid 1.0 release, which was a fun time. We really pushed it to the limit getting this release out, and I was worried that it was going to be a trophy wife release (definition: looks good but does not work), but the 1.0 is actually very stable and applications like our pilot at Pfizer are running well. Kudos to the OpenCrowd guys for getting the Pfizer app done in record time.

The party was one of those unfortunately few and far between times when you get the former CTO/architects of WebLogic, Kiva, and NetDynamics together. This of course led to some good natured ribbing about how scripting languages suck and that I had better see the light about Java. It was pointed out that even the SpikeSource and SourceLabs guys now have Java stacks!

I shared the questions I ask the audience nowadays whenever I give talks, like the last couple of weeks at OSCON and LinuxWorld:

Q: Does anyone remember back in 1996 and 1997 why everyone started running Java on the server? What was the big compelling thing about Java that Sun had up on its home page?

A: WORA (Write Once Run Anywhere)!

Q: What operating system and processor type are the vast majority of new applications deployed on today?

A: Linux/x86!

Q: So do we still need a big abstraction layer above the operating system?

A: Hmmm...

When I was at Sun there was a guy who had this great way to evaluate technology. He said, if it was invented today, would it win? So today's requirements for an application platform are: "Linux/x86 application platform that processes HTML and XML text efficiently and handles unstructured data well." If J2EE came out today would it win this market?

In other news, I was blown away by a demo of Zimbra (formerly Liquid Systems). This is the nicest AJAX application I have seen yet. Yes, yes, the backend is in Java... :)

Tuesday, July 19, 2005

ActiveGrid Announcements: $10M Funding, LAMP Partners

ActiveGrid has made a couple of announcements in the past few weeks. Today, we are announcing that we have closed a $10 million round of funding led by Worldview Technology Partners. Irwin Gross of Worldview has joined ActiveGrid's board. Irwin understood the potential of ActiveGrid from our first meeting. He started Wilson Sonsini's IP practice and has a great understanding of open source - in short, a VC who knows the differences between BSD, ASL, GPL, LGPL, and MPL!

A couple of weeks ago, we announced partnerships with the LAMP ecosystem principals:

L - Novell & Red Hat
A - Covalent
M - MySQL
P - Zend & Python Software Foundation

Our upcoming 1.0 product is certified to run on Novell SuSe & Red Hat Linux and includes technology from Covalent, MySQL, and Zend. We have signed third tier support arrangements with these vendors so that we will be able to resolve any customer issues without passing the buck.

An interesting side note to this was that we had a great round table interview with Martin LaMonica of CNET with the all of the LAMP principals. Mark Brewer, CEO of Covalent, Marten Mickos, CEO of MySQL, Doron Gerstel, CEO of Zend, and Angie Angerson, VP Platforms of Novell participated. It was a lively discussion, and to everyone's recollection, it was the first time there has been an event with all of the LAMP principals.

The $10 million in additional funding and the LAMP ecosystem partnerships are a big boost for us to bring LAMP into the enterprise.

Thursday, June 16, 2005

The End of Software

Our engineering team has grown quite a bit, and I needed to better track our 1.0 release process. It has been a while since I have directly managed an engineering project, but I have done it before with Microsoft Project. So I order it and it arrives a couple of days later.

I then had a very bizarre experience, almost like an archaelogical dig. It was a shrink wrapped box entitled "Microsoft Project 2003". 2003??? This code hadn't been touched in three years! I opened the box (yes, it is still a pain) and took out the CD and installed the software. I haven't installed software like this in years.

I then entered everybody's tasks, task hierarchies, and such. Project was incredibly difficult to use, stuff that should have been obvious was very obtuse. Then I wanted to publish the Gantt chart in HTML for our intranet. Tried for hours. Couldn't do it. Finally I printed out the chart, taped six sheets of paper together, and posted it on the wall in our bullpen. Unbelievable.

We have now found an online project manager that we are going to move to after the 1.0 release. Everyone gets an account and can log in and update their tasks. The Gantt chart is online updated in real time. We did not have to buy any software. We are subscribed to a service that is easy and intuitive to use with a rich, Web 2.0 user interface.

[Update: the online project manager we are using is Basecamp. It rocks.]

I think it is pretty clear that software as we know it is now over. No one uses traditional desktop software anymore. Any desktop software people use is meant to enable publishing and communication, ranging from photo managers to IM. Even ActiveGrid's Application Builder is essentially a publishing tool. You build an application that ties together existing services, and then you deploy it to your ISP (virtually all ISPs run LAMP) with the deployment wizard. There is no need to install or configure any software. It's Do-it-Yourself Sofware-as-a-Service!

Thursday, May 26, 2005

Software Darwinism: Why Open Source is the New "Standard"

Traditionally, a "standard" is established when multiple vendors sell essentially the same technology and customers demand that this commoditized technology be accessed via a standard API. Vendors can then compete on implementation. J2EE is a prime example of the success of this approach.

Open source turns all of this on its head. If there are multiple implementations doing roughly the same thing behind the same API, why not simply collaborate on a SINGLE implementation? One or two best-of-breed implementations inevitably are settled on and everyone can move on and work on technology that has not been commoditized. We have seen this happen with operating systems (Linux) and web servers (Apache), and it will soon happen with databases and application servers.

Everyone wins with the open source approach: customers get better technology at lower cost, vendors do not waste brainpower replicating and maintaining commoditized technology, and developers can step through the source when there are problems.

Every time a model evolves there are stragglers, hybrids, and early adopters. An example of the hybrid model is Java. While Java itself is not released under an open source license, there has been a ton of open source innovation in Java, ranging from Struts to Ant to JUnit to Hibernate. Such hybrid models do cause strange occurrences, such as the JCP continually recreating existing and popular open source projects. A cleaner model, which IBM and many folks in the Java community are advocating, would be to open source Java and integrate leading projects such as Hibernate rather than recreating them.

In the early adopter LAMP stack, where everything is pure play open source, there are still a lot of balls in the air with competing projects, such as the zillion web server API's in Python that are finally coming under a single umbrella. Open source is the accelerated Darwinism of software components, where the strong survive and the weak die in the purgatory of one committer with zero download counts.

One thing is certain, open source is a meritocracy where the best-of-breed technology wins. As open source matures, it is now moving beyond replicating existing commoditized software to providing advanced technology beyond the apparent capabilities of the large software vendors. As we have learned time and time again (ie, AOL and CompuServe vs. the Internet), emergent behavior compounded with a network effect is the path to success.

Monday, May 02, 2005

Web 2.0 - Don't Try this at Work

One of the big drivers of Internet applications in the corporate world was the stark contrast of what people could do at home and at work:

At Home - Buy books, buy movie tickets, look up the weather, etc.

At Work - Call HR to change health plan, call factory floor to find out what happened to customer's order, call invoicing to find out what happened to a bill, etc.

Well clearly something at work was wrong, and soon enough everything was online. :)

Now look at the difference between home and work today:

At Home: Web applications are using JavaScript and DHTML to enhance the user experience and decrease server interactions. Some random guy can combine Google Maps and Craigslist apartment listings in a useful way without talking to either Google or Craigslist (http://www.paulrademacher.com/housing/).

At Work: Everything has a Netscape 3 level of UI interactivity and nothing works with anything else.

This difference in functionality and the increasing expectations of users will lead to significant change in the enterprise. As more and more Web 2.0 applications deploy, pretty soon even the CEO will be saying "some random guy can get Google to work with Craigslist, and we can't get CRM to work with ERP?" Web 2.0 = SOA, and an enterprise's customers, employees and partners are going to expect it to happen ASAP.

Monday, April 25, 2005

Application Servers on Atkins

The trend towards lightweight servers has clearly moved beyond the LAMP open source stack into the enterprise Java space as well. More and more new Java deployments run on clusters of Apache Tomcat or on the web containers of J2EE servers. EJB 3 will introduce EntityBeans with POJOs (plain old Java objects), so the J2EE standard will finally catch up with popular open source technologies like Hibernate (although of course unnecessarily reinventing it instead of just using the popular open source that is already established). I think we can safely say that the trend in both the LAMP and Java worlds favors lightweight servers running on commodity hardware, rather than small clusters of heavyweight servers running on SMP hardware.

Unfortunately I can not attribute these quotes, but an SVP at IBM shared with me that he knows of only five WebSphere customers that are coordinating multiphase commits from WebSphere. A technologist I know at BEA knew of NO customers coordinating multiphase commits from WebLogic. For the bit-twiddlers out there, I mean using JTA/JTS. XA or a message queue both work just as well from lightweight servers.

Clearly the pain point for the majority of middle tier applications is throughput, especially as more and more web services come online. In short, workloads that accept a lot of connections from clients and make a lot of connections to backends, where systems need to deliver high levels of throughput computing around HTML and XML, rather than traditional processor intensive computing.

On the polar opposite of this trend towards lightweight servers are companies like Azul Systems that are staunchly in the heavyweight server camp. Azul has invested a rumored $100 million on a proprietary software/hardware appliance system for J2EE applications that need to do a lot of processing other than throughput. The Azul appliance is used by existing J2EE clusters to offload compute-intensive processing. The Azul appliance does not have any application level I/O capabilities, and uses the host J2EE cluster to connect to databases and backend systems, so even things like writing to a log file needs to go back to the J2EE cluster. In my opinion there aren't too many applications like this beyond heavy duty ERP and supply chain systems, and those are already over-provisioned with big honking Sun and IBM boxes.

But here's the really funny part: the Azul hardware runs on their custom "Vega" cores, 24 of which fit on a chip, which I am guessing are roughly equivalent in processing power to a MIPS or ARM core. We designed similar systems when I was at SunLabs, so I am speaking from experience here. There is a reason we didn't end up building them, the math didn't add up in terms of price/performance compared to the AMD and Intel roadmaps.

Here is a back-of-the-envelope calculation for Azul's price/performance. Azul's 16 chip, 384 core machine with 256GB of RAM costs $800,000, or $2,083 per core with 680MB RAM. Each Azul core looks like it is barely more powerful than a low power RISC chip from a high-end PDA. At the system level, each $2,083 Azul core costs more than a dual processor Xeon system, which can smoke a low power RISC core.

It would be great to do a real price/performance comparison, but this "fundamentally new approach to delivering compute resources" for "unbound compute" unfortunately is the first new chip in my recollection that shipped with no SPECint numbers. Nor are there any SPECjbb or ECperf numbers. J2EE is a very mature platform with well known benchmarks that can be used to show how much better a particular platform is. Clearly there is a reason for the lack of metrics here. :)

The bottom line is that there is no software/hardware stack on the planet more vertically optimized than the Apache/Linux/x86/GigE "commodity stack", and everything from LAMP to Java to animation rendering will run on it. Anything that tries to compete with the commodity stack is going to have a tough, tough time. Anything that builds on top of the commodity stack is riding the wave to the future.

Monday, April 11, 2005

The Bits are Over the Wall!

I am excited to announce that we have shipped the early access release of the ActiveGrid Grid Application Server and ActiveGrid Application Builder, which are shipping under the Apache 2 license. It took almost two years to get here, and we feel that the early access is a solid foundation on which we can evolve ActiveGrid's technology. The early access release is focused on the development and deployment of basic database-oriented web applications. We will be shipping regular releases through our 1.0 release which is targeted for the June/July timeframe. The 1.0 release will include CSS style editors, DHTML and JavaScript support, a full BPEL editor, and full web services integration.

We have made a concerted effort to ensure that (at least on Windows) downloading the ActiveGrid Application Builder and getting started is a simple and painless experience. It should take no more than 15 minutes to install the Application Builder, run the included demos and start building applications that tie into your existing backends.

To download the release, go to:

Download Early Access

To check out screenshots, go to:

Screen Shots

For our DeveloperGrid developer community, go to:

DeveloperGrid

We hope that you enjoy checking out ActiveGrid and consider joining our community as either a contributing user or as a developer. Let us know what you think, the good, the bad, and the ugly!

Sunday, March 06, 2005

WS-Nothing

A couple of weeks ago I participated on an application server panel at Web Services Edge hosted by Anne Thomas Manes. There was quite a bit of discussion about WS-This and WS-That and WS-Foo and WS-Bar. Essentially, this Web Services thing has devolved into a huge unintelligible morass. The panel was covered today in LinuxWorld, which liked an ad-hoc term I used: WS-Nothing.

Vendors Have Been Arguing for So Long That You Have to Send Text Files Around

On the panel, I said that we vendors have lost sight of the fact that we have been bickering so long that the only way our products can talk to each other is to send text files around. We are now in the process of screwing even that up with all of these competing specifications about how to mark up self-describing text files.

It is of no supprise that Web Services have not had a big uptake with customers; they don't even know where to start. The point of web services is interoperability, and if the Web Service produced by a particular application server only works with that application server, you might as well use a binary protocol.

WS-Nothing

If you ignore the vendors and get back to the core theme of a Web Service and what it is meant for, it is actually pretty easy to get started. Developers should use the WS-Nothing methodology: Concentrate on the definition of the data coming and the data going out of the service. Everything else, such as security, routing, transaction context, etc. should be defined by policy and layered in later.

Even if someone does manage to figure out which ones to use and hardcoded them in (say SAML Artifact and WS-Routing), they will probably end up having to change it around for future consituents. Partner A uses SAML Artifact but then Partner B is signed up, which mandates a certificate check, and you end up having to do a big if/then/else at a minimum or probably two hard coded services.

So use WS-Nothing and simply implement what your service needs to do from a data in and out perspective. Layer in the WS-Gunk later as a wrapper. Or even better, at runtime based on policies.

PS: Calling this "aspect oriented" is already making it too complicated. Data in, data out, WS-Gunk added later. No big words. No complexity. Remember, it's just text files! :)

Thursday, February 24, 2005

IBM Endorses PHP

IBM has endorsed the use of PHP for enterprise web development. This is huge for PHP and LAMP - IBM's endorsement of Java and Linux were what made those technologies acceptable to enterprise customers. Here's a link to an article by Martin LaMonica on News.com: Big Blue backs PHP for Web development.

It is always fun to see the industry shift. I started ActiveGrid almost two years ago on the premise that scripting languages would gain acceptance in the enterprise and that there was an opportunity to deliver a next generation application platform for the LAMP stack. I remember that back in the early Java days we used to spend quite a bit of time attempting to convince customers that Java was a viable enterprise language. After IBM, with Ian Brackenbury's leadership, endorsed server side Java, those discussions were over. PHP has now, in marketing speak, "crossed the chasm."

Congrats to the guys at Zend for having the foresight to make PHP 5 use a lot of the same syntax as Java (try/catch/finally, public/protected/private). PHP 5 will be a very easy transition for people who know Java. I remember when I first coded in Java in 1995 I was productive in minutes because I already knew C++ syntax and Smalltalk dynamic OO principles. If you know Java and BASIC you will be all set to go with PHP 5.

PS: A joke we had last summer with the IBM emerging technology folks is that they don't like LAMP, rather PLAD (PHP, Linux, Apache, and DB2).