Home Yahoo Is Discontinuing Its Hadoop Distribution to Focus on Apache Hadoop

Yahoo Is Discontinuing Its Hadoop Distribution to Focus on Apache Hadoop

Yesterday Yahoo announced that it will discontinue The Yahoo Distribution of Hadoop and refocus its development efforts on the main Apache Hadoop distribution. In a blog post, Yahoo VP of Software Engineering Eric Baldeschwieler writes that Yahoo will work more closely with Apache in the future. This leaves only two major distributions of Hadoop: Apache Hadoop and Cloudera’s enterprise-focused Hadoop.

“Unfortunately, Apache is no longer the obvious place to go for Hadoop releases,” writes Baldeschwieler. “The Yahoo! team wants to return to a world where anyone can download and directly use releases of Hadoop from Apache.”

This begins a long process of contributing Yahoo’s Hadoop code back into the main project. Baldeschwieler writes that Yahoo is in the process of contributing much of its work to the branch hadoop/common/branches/branch-0.20-security. Yahoo has proposed this be released as Apache Hadoop 20.100.

Yahoo has created another branch called Hadoop-future, a place for its proposed new features, including:

  • HDFS-1052 – Federation, the ability to support much more storage per Hadoop cluster.
  • HADOOP-6728 – A the new metrics framework
  • MAPREDUCE-1220 – Optimizations for small jobs

Cloudera has remained an active contributor to Apache Hadoop. We haven’t heard back from Cloudera yet, but we don’t expect Yahoo’s announcement to change anything there.

Also of interest is Baldeschwieler’s history of Yahoo’s involvement in Hadoop.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.