OpenQuake Engine Server

Registered by Lars Butler

A lightweight web application with a REST API for driving and interacting with oq-engine calculations. Includes job queuing and cluster resource monitoring.

Whiteboard

General:
- Upload files/run job
  - job.ini
  - XML input models
- List jobs
  - past
  - currently running
- List job results
- Stop job
- Get cluster info
  - Number of jobs running, task count, worker info, cluster load, etc.

Hazard/Risk:
- Run calc
- List calcs
- List outputs
- Export outputs
- Delete jobs/outputs

UI:
- REST API
- Non-graphical
  - GUI could be an add-on or separate app
- Mimic the CLI interface. --risk-calculations translate to /risk-calculations/ (or similar). Then, we could make the CLI be a client of the api and allow remote oq-engine calls by CLI (e.g. ./bin/openquake --host my-host --run-hazard myjob.ini )

API:
- "List X" endpoints return a list of items, each includes metadata, (ID, etc.) and the URL of the resource
  - Clients don't have to guess/know where to find the resources, given an ID
- JSON is preferred format for reponses

Dependencies:
- Django
- RDBMS (postgres, etc.)
- oq-engine

Authentication:
- Only authenticated users can send API requests
- Use Django user auth

Quotas:
- The oq-engine-server is not responsible for maintaining quotas, but simply providing a REST interface to run and interact with the oq-engine.
  - The client (such as the oq-platform) will be responsible for calculating quotas, permissions, etc.

----------
Authentication
----------

The Django views which service these API endpoints (see below) will require an authenticated user to execute them (particularly for POST/DELETE operations). Per IRC discussion, oq-engine-server will maintain it's own authentication (using the built-in Django auth/account system).

----------
REST API sketch
----------
https://docs.google.com/a/openquake.org/spreadsheet/ccc?key=0AgmeiGIi49FLdGU2Nnp0ejJZcUxUM0hmWjhiS1pHWWc#gid=0

----------
oq-engine API additions
----------

Since the oq-engine-server will need to interact with the oq-engine database to query/modify calculations and results, it would be good to add an API layer to the oq-engine so this part can be mocked for easy testing (and to reduce or eliminate the dependency on the database for test fixtures). This could be accomplished by adding the module `openquake.engine.api`. The `--list-hazard-calculations`, etc. options in bin/openquake can be modified to instead use this API, for consistency. This API would basically just be a simple set of functions which provide a layer between the client and the database, instead of calling `models.HazardCalculation.objects.filter(...)`, etc. directly. Such a change would make the API endpoint views very easy to test in a lightweight manner.

----------
oq-engine-server / oq-engine architecture diagram
----------
https://docs.google.com/a/openquake.org/drawings/d/1YqVW7nRlzqZ_6NZgPiySfJJS8ecvt-ZKZoO7RdeNAEs/edit

----------
OQ Platform / OQ Engine Server interaction
---------
https://docs.google.com/a/openquake.org/drawings/d/1oCXtOrrkk4YDi-vF2xLOjh6s7ZZKtoiKAM9-nX4QTB0/edit

----------
Related blueprints
----------
https://blueprints.launchpad.net/oq-platform/+spec/icebox

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.