News from today’s Hadoop Summit ’09 got me thinking about the importance of open community again.  According to Joey Echeverria’s tweet from Hadoop Summit ’09, Yahoo! representatives feel:

“Yahoo!: Easier to take an open source project and add steam to it rather than write something from scratch. #hadoopsummit09”

What a difference a year and a half makes.  Back in December 2007, there were roughly 10 Yahoo! employees working on Hadoop, and only five or six outside contributors.  Hadoop found said at the time:

“It’s dominated by Yahoo, it would be great for the project to have a more balanced team.”

Today, Hadoop core has 185 contributors, only 30% of whom are Yahoo! employees.

Also, Cloudera, a commercial company aiming to bring Hadoop to the enterprise, has just contributed a new database tool for Hadoop.  The tool, SQOOP, enables users to directly import large database tables into Hadoop.  According to Cloudera founder:

“SQOOP is a tool that enterprise customers were demanding,” Bisciglia said. “Enterprises have lots of data in existing databases, and if you can’t give them a way to interact with that data, Hadoop isn’t as useful as it could be.”

Much like, Apache HTTPD, and Eclipse before it, a meritocratic, open community is unlocking opportunities for the ecosystem, which in turn is helping Hadoop evolve a lot faster than within any one vendor’s corporate walls..

Today, it appears that most of the contributing vendors are collaborators, with little, if any, head to head competition.  That will surely change over time.  But that’s a good thing. More vendors, more developers, more ideas, more innovation.

One can’t help but wonder what Google thinks of Hadoop’s progress.