Store bundles in a compressed form
Result bundles are currently just dumped on disk, but JSON is highly redundant and even gzip with default settings achieves a compression factor >90%. If we store them compressed, having them around on developer machines becomes less of a burden and if we don't have to compress as part of backing up, backups will be faster and more intelligent strategies than "tar up the whole media" directory become possible.
Blueprint information
- Status:
- Started
- Approver:
- Paul Larson
- Priority:
- Medium
- Drafter:
- None
- Direction:
- Approved
- Assignee:
- None
- Definition:
- Approved
- Series goal:
- Accepted for trunk
- Implementation:
-
Blocked
- Milestone target:
- None
- Started by
- Paul Larson
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
[2012-12-20, mwhudson] after casually asserting that django had a built-in compressing file storage class a while ago, I found out that I was wrong. So we'll have to write one -- it shouldn't be too hard though.
[2012-02-14, mwhudson] setting status to blocked until celery is deployed.
Work Items
Work items:
write a storage class that gzip's the content: DONE
add compressed filefield to model: DONE
change code to look for compressed bundle first: DONE
write celery job to compress a bundle: TODO
run job to compress existing bundles: TODO
change bundle-receiving code to kick off compressing job: TODO
Dependency tree
* Blueprints in grey have been implemented.
