Web Scale packaging enhancements

Registered by Clint Byrum on 2010-10-15

Target new versions of existing stabilized web-scale storage, acceleration, and supporting technologies for Natty, and identify good candidates for backport to lucid and/or "server conduit" blessed PPA's that users can use to leverage these advanced Natty features in the LTS Lucid environment.

Blueprint information

Status:
Complete
Approver:
Robbie Williamson
Priority:
High
Drafter:
Clint Byrum
Direction:
Approved
Assignee:
Clint Byrum
Definition:
Approved
Series goal:
Accepted for natty
Implementation:
Implemented
Milestone target:
milestone icon ubuntu-11.04-beta-1
Started by
Clint Byrum on 2010-11-29
Completed by
Clint Byrum on 2011-04-08

Related branches

Sprints

Whiteboard

Status: Cassandra 0.7.0 is available for Lucid/Maverick/Natty in stable cassandra-ubuntu PPA.

Work Items for natty-alpha-1:
[clint-fewbar] Cassandra: prepare packages for 0.7 release -- 0.7.0~rc1 done, still using embedded jars: DONE
[clint-fewbar] Cassandra: rework 0.7 package to use already-packaged system java libs: DONE
[clint-fewbar] Cassandra: re-open dialog with upstream about proper packaging: DONE

Work Items for natty-alpha-2:
[clint-fewbar] Cassandra: backport natty work to Lucid in PPA: DONE
[clint-fewbar] Cassandra: run test suite during 0.7 build (intermittently fails on 0.7.0 rc1, fixed in rc2 due next week): DONE
[clint-fewbar] Cassandra: identify and package dependencies that overlap with other projects: DONE
[clint-fewbar] ZeroMQ: package -- packaged already in universe since Lucid: DONE
[clint-fewbar] gearman: Add php5-gearman PECL module to Ubuntu: POSTPONED
[clint-fewbar] gearman: Finish gearman-interface packaging for ruby and lua (pushed to branch, awaiting squeeze release to upload to debian): DONE
[clint-fewbar] MySQL: package HandlerSocket plugin (lp:~clint-fewbar/+junk/handlersocket): DONE
[clint-fewbar] assist Membase getting Membase packaged -- initial help via email, no response for 2 weeks: DONE
[clint-fewbar] assist Percona with packaging and uploading XtraDB: POSTPONED
[clint-fewbar] assist Percona with packaging and uploading XtraBackup (Feedback sent via email): DONE

Work Items for natty-alpha-3:
[clint-fewbar] Polish handlersocket plugin packaging and upload to universe: DONE
[clint-fewbar] MySQL: investigate and resolve conflicts between mariadb and mysql's libmysqlclient: POSTPONED
[clint-fewbar] mod_pagespeed: package for debian/ubuntu: POSTPONED

- Queueing:
  - rabbitmq
  - zeromq
  - gearman
  - Starling

- Storage:
  - cassandra
  - riak
  - hdfs
  - MongoDB
  - redis
  - membase
  - BigCouch (scaleout on top of couchdb)
  - MogileFS
  - Drizzle
  - mariadb
  - - For MariaDB and MySQL itself: HandlerSocket (plugin). See http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html
  - percona/xtradb

- Filesystems:
  - CEPH (already in universe, make sure to keep up w/ upstream)
  - Gluster (already in universe, look into packaging glusterweb)

- Acceleration:
  - varnish (already awesome packaging -- but keep up the sync/merge)
  - Others?

- For all the above, assess status for language bindings:
  - Essential: C, python, perl
  - Important: ruby, php, java

- Keep Ubuntu the best distro for use with cloud by packaging AWS command line software for:
  - Cloud Watch [1]
  - Auto Scaling [2]
  - Elastic Load Balancing [3]
  - Identity and Access Management [4]
  - Relational Database Service [5]

- Get Amazon's Net::Amazon::SimpleDB Perl software [6] into CPAN and package for Ubuntu as, say, libnet-amazon-simpledb-perl. Note: This requires renaming the Perl package or booting the existing Net::Amazon::SimpleDB on CPAN (which does not work and has been abandoned). EricHammond has discussed this with JosBoumans.

[1] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=2534&categoryID=88
[2] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=2535&categoryID=88
[3] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=2536&categoryID=88
[4] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=4143&categoryID=322
[5] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=2928&categoryID=294
[6] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1136&categoryID=189

 * memcached and cassandra packages (or updates) ([[http://groups.google.com/group/ec2ubuntu/msg/eb4d5ac6d40eced8|1]])
 * libmysqlclient - MariaDB and Mysql living in harmony.

(?)

Work Items