Ubuntu Server + Hadoop and Bigdata

Registered by James Page

Apache Hadoop has gained widespread adoption; the various flavours of Hadoop appear to be consolidating and Cloudera have transferred a number of their Hadoop related projects to Apache including Bigtop (the Cloudera packaging for Redhat, Debian and SuSE).

Packaging Hadoop for Ubuntu would help support developing a set of rock solid Juju charms for Hadoop by providing a well integrated version of the packaging for Ubuntu.

Collaboration with Apache Bigtop would also potentially help support packaging the wider family of Hadoop related projects.

Blueprint information

Antonio Rosales
Ubuntu Server
Needs approval
James Page
Series goal:
Accepted for precise
Milestone target:
milestone icon ubuntu-12.04-beta-1
Started by
Robbie Williamson
Completed by
James Page

Related branches



Status Update:

This spec has been superceeded by https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-hdp-hadoop; packaging hadoop from source will not be targetted for this release - a broader range of hadoop packages will be made avaliable in PPA/Partner.

Summary of objectives for Precise:

1) Ubuntu will target packaging Apache Hadoop
- Help drive support for running under OpenJDK
- Packaging will retain flavour of most popular upstream packaging
- Focus will be on most recent stable ( validate with upstream release schedule
- Thrift support will not be included
- LZO and snappy compression options will be investigated
- universe target this release.

2) Juju Charms will by default align to distro packaging for precise

Full sessions notes from UDS-P: http://pad.ubuntu.com/uds-p-servercloud-p-hadoop

Work items precise-alpha-1:
[mark-mims] Hadoop community input (what about no thrift? etc): DONE
[mark-mims] Attend HadoopWorld: DONE
[james-page] Check on release schedule for Apache Hadoop between now and Feature Freeze: DONE
[james-page] Investigate upstream co-operation from Hortonworks/Cloudera to ensure ongoing collaboration going forward: DONE

Work items precise-beta-1:
[negronjl] adjust hadoop charms to have a configurable backend hadoop, get one into the charm repository: DONE
[james-page] Package KFS for Ubuntu: POSTPONED
[james-page] Package Apache ftp-server for Ubuntu: POSTPONED
[james-page] Package Hadoop for Ubuntu: POSTPONED

Work Items:
[james-page] Active backport packaging post 12.04 release: POSTPONED
[james-page] Feed back all work to Debian: POSTPONED


Work Items

Dependency tree

* Blueprints in grey have been implemented.