The Apache HTTP Server Project (http//www.apache.org/) has become a poster-child for the Open Source Software (OSS) movement by creating and maintaining a robust, secure, efficient, extensible, and Open Source Web server. The project is managed by the Apache Group, a geographically distributed set of volunteers who use the Internet to communicate, develop, and distribute the server. In addition, hundreds of users have contributed ideas, code, and documentation to the project. According to the Netcraft survey of March 1999 (http//www.netcraft.com/survey/), the Apache server and its derivatives are used by more than 58 percent of publicly accessible Internet Web sites, more than double that of its nearest competitor.

Last year, in a surprise move that rocked the entire software industry, IBM announced that it was discontinuing its efforts to create a proprietary Web server product and would instead base its new offering on Apache. This was a substantial endorsement of both the OSS development and business models.

Where did Apache come from? How has a diverse group of programmers spread out all over the world created a program that captured the majority market share in this important software category? Why would a major corporation like IBM discard its own proprietary software in favor of an Open Source system like Apache?

Perhaps the best way to answer these questions is to look at the Apache Project through the eyes of three key contributors to the Apache cause; Rob McCool, Roy Fielding and Brian Behlendorf.

In The Beginning...

- Rob McCool

Before there was an Apache Project, there was the Web server I built and tested at the National Center for Supercomputing Applications (NCSA) at the University of Illinois Urbana Champaign campus. At that time, the most widely deployed Web server was produced by the CERN group in Switzerland, which is also where the World Wide Web had been invented. Marc Andreesen was working on the Mosaic Web browser (the precursor to the Netscape browser) at the NCSA and found the existing CERN server to be large and cumbersome. I took a look at the source code and agreed. My work as a systems administrator at the NCSA had prepared me for the task of writing a server, so when Marc asked me to write a new one, I agreed to.

The CERN server was based upon a protocol called "hypertext transport protocol" (HTTP). I downloaded the specifications for HTTP and started working on the NCSA HTTPD. The D in HTTPD stands for "daemon". Under UNIX, a daemon is a server program -- a program that runs continuously and waits for other programs to connect to it. The protocol at that time was around version 0.9 or 1.0, and was very simple. I found that it wasn't the protocol itself that was interesting, but rather all the things that could be built around it. For example, one day Marc was tossing around some ideas and he started talking about having a URL (a "Uniform Resource Locator", which is just a standardized method for addressing items on the Web) that could run a program. At first I didn't understand what he was talking about, but when it hit me, it was a huge idea! It was the idea that eventually led to clickable image-maps and made HTML fill-out-forms possible.

After that discussion I began working on the "htbin" interface, which later evolved into the "Common Gateway Interface", or CGI. These interfaces specified a standard method for a Web server and an external program to pass information between each other. The "external program" can perform any sort of arbitrary processing on the data passed to it and then return the processed data to the Web server for transmission back to a Web browser. Thus the server became almost infinitely extensible and flexible in its ability to process data.

Nowadays, when the Apache guys want to modify the server, they discuss their ideas on the Apache mailing lists, but at that time, changes to the servers or the protocols were discussed on the "www-talk" list. That is where htbin and CGI were formally worked out.

Although it was a lot of work, and could be stressful, the NCSA server was a very enjoyable project. I often looked back on it fondly as I worked at Netscape. Especially on the www-talk list, I felt a sense of community. There was also a sense of that in the newsgroups, and in the e-mail I would receive from the people using the software. I got a lot of ideas and feedback from them. One of the best ideas I received was from Charles Henrich. He had developed a mechanism for embedding program output in HTML files. The server would scan a page as it was sent out, and replace certain tokens with the contents of a file or with the output of a program. He called the mechanism "server side includes", although I later renamed it "server parsed HTML". Unfortunately I renamed it after it had been released for some time, and both names stuck.

At first I was somewhat leery about the idea of a server parsing HTML files, but this mechanism has now become one of the key components of Web servers. It is widely used for advertising and a variety of other applications. It was the connection with the users of NCSA HTTPD through the www-talk mailing list and the newsgroups that made these innovations possible. Creating the server in that environment was a very enjoyable experience.