Script Bundle Format Specification

Registered by Nathan Osman on 2011-08-15

JS Studio uses a proprietary format for storing 'script bundles' which are essentially a single binary file with the zlib-compressed contents of files to be extracted when the bundle is installed.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
None
Direction:
Needs approval
Assignee:
Nathan Osman
Definition:
Drafting
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

A few key properties of script bundles:
- they are a single binary file with no external dependencies
- files are compressed / decompressed using zlib
- file metadata is stored in a header that precedes the files' compressed contents
- all bundles begin with a version number indicating what version of the format spec they adhere to

The current format spec for script bundles can be found in the LP branch at 'bundle/format_spec'. The current format is very rigid and lacks flexibility in a number of areas. The only file metadata currently stored is the filename of the file - which is relative to the root install directory for the bundle.

Considerations:
- We probably want to move away from the term 'script bundles' and switch to 'addon bundles' to indicate that the bundles can be used for more than simply storing scripts but also for storing any arbitrary file including images, icons, etc. Addon bundles should contain some indication of what type of content they provide. Current ideas for this include: scripts, icon sets, themes, and project templates (which are basically just JSON files). A real consideration when implementing this is the fact that the code is heavily centered around 'script' instead of 'addon' so a lot of renaming would be in order.
- For future scripts, there may arise a need to store file permission settings (such as whether the file is executable). Although it is difficult to come up with a rationale for why a script would need this level of control, it would be wise to err on the side of caution and provide the capability.
- If scripts become 'addons' then there will be a need for the script engine (er, that is the addon engine) to keep track of what files the bundle installs so that they can be removed when the addon is uninstalled. Before this was not necessary since each script stored its contents exclusively in a separate folder. However, it is reasonable to expect that some scripts would install both an icon set AND a theme, etc. Therefore the addon engine needs to keep track of what files belong to what script. Considerations for implementing this include a mini-database that the engine maintains of all files installed by bundles.
- The current format wastes a LOT of space by forcing filenames to be exactly 256 bytes long. Anything longer than this is unsupported (not a good thing) and anything shorter wastes space. Therefore it would be a good idea to store strings by writing an integer with their length followed by the string itself. Obviously we would never store filenames beyond 65k bytes, so an unsigned short would be sufficient for storing length.

There is currently a PHP script that can convert a ZIP file to a bundle in the bundle folder of the branch. It requires some specific command line args - details can be found by running the app without any arguments.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.