Angular Needs to Support Theme Template Overrides

Registered by Diana Whitten on 2016-05-18

Problem:

Currently, Angular does not support Theme housed template overrides. This is a regressions from the current functionality available in Horizon through the Django templates.

Gotchas:

* Angular's client-side templates are loaded into a template cache. When themes are changed, this cache must be cleared.
* The solution must work with 'collectstatic' turned on and off
* We can't just hit a 404 and fallback to the default theme path, this is very costly

Proposed Solutions:

=-=-=-=-=-= Use Django to return the correct static asset =-=-=-=-=-=

Explanation:
  Exactly like Dynamic Themes is working currently for 'themable assets', we let Django figure out which asset to return based on the request in the cookie.

Pros:
  * Very little logic needs to change in the Angular
  * Same logic is already written in the Django side

Cons:
  * Concerns about speed if Django has to serve up static assets now, Apache can't just do it

=-=-=-=-=-= Django to return list of templates (via json) per theme with base page =-=-=-=-=-=

Explanation:
  Django has access to the directory structure, so its easy to have it return a list of templates per theme. Then Angular can source that list when loading templates.

Pros:
  * The static directory will not grow in size

Cons:
  * Slightly more logic needed in the Angular side than currently exists

=-=-=-=-=-= Collect ALL the static and set the Angular Basepath =-=-=-=-=-=

Explanation:
  If we collect all the static into every single theme, then we just need to update the basepath set by Angular.

Pros:
  * Very little logic needs to change in the Angular

Cons:
  * The static directly will grow in size linearly with the number of themes. This can get HUGE.
  * We will have to write our own collectstatic function in Django, which means we have to fork that functionality.
     It will be a pain to maintain in the future.

Blueprint information

Status:
Complete
Approver:
Rob Cresswell
Priority:
High
Drafter:
Diana Whitten
Direction:
Approved
Assignee:
None
Definition:
Approved
Series goal:
Accepted for 10.0.0-newton
Implementation:
Implemented
Milestone target:
milestone icon newton-3
Started by
Rob Cresswell on 2016-07-21
Completed by
Rob Cresswell on 2016-07-21

Related branches

Sprints

Whiteboard

[robcresswell 2016-05-23]

I have some ideas on this. We should be able to use Angulars templateCache service, and populate it with any templates in the selected theme dir on app initialisation. Once the templates are cached, when angular looks for them it should immediately hit the cached template. Because there is a full reload on theme swap anyway, this should work with theme swapping too (it may make the reload slightly longer, but not a significant issue).

[tyr & hurgleburgler 2016-05-27]

Collecting some ideas for initial implementation. Success looks like:

- Ideally Newton-2 (Jul 11-15) to give plugins a chance to respond to required changes
- for any angular template *file* that exists a deployer can override that template using an override that is part of theme
- solution doesn't duplicate static files that aren't overridden (which causes bloat in static directory)
- use of overrides doesn't degrade user's experience (speed, extra user actions needed to see overrides, etc) (aka using a theme with overrides isn't any slower at run time than just using the default theme)
- must also work for OFFLINE_COMPRESSION and if COMPRESS=False
- Note: automated test of solution or lint enforcement can come as a follow-on
- Note: is it acceptable to increase static file collection / compression times if necessary
- Note: current theme and available theme are persistent across sessions for a user using the same browser (browser cookies)
- Note: in-line or generated templates are not considered overridable...only .html files can be customized

Open Questions:
- what about image overrides?
  - for example, one could override a template to include a different logo...but how does that new logo get collected into the static file directory?
  - currently Django only allows override of very limited images (logo.png & logo_splash.png) so support for image overrides in Angular side can be lower priority
- does changing a theme need to be any faster than visiting Horizon for the first time? (e.g. if a template cache was cleared on theme change, it would be *nice* to not need to re-fetch templates that have not been overridden).
- if a "pathService" is used to generate templateUrls...how could this work when paths are needed in config functions (like when establishing routes to templates). One possibility is to create the core path lookup logic in a horizon global function on window (similar to STATIC_URL). This logic can then be used by legacy javascript code that are part of Django templates, used in config functions of Angular modules, or wrapped with a simple service that also uses the function as its implementation to make the behavior injectable into other Angular code.

Simplest thing that could possibly work?
- override http service to detect a *.html load, and if so, lookup that string in a file generated during compression that lists any overridden .html. Return that .html file instead.

Possible Alternatives:
- pre-seed the template cache with overridden templates
- creates theme directories in /static and clone all files into each theme directory
- use Django to serve static files, and serve the overridden file
- create a 'lookup file' similar to translations, that is served to the application. Angular refers to this file before loading a static file (perhaps from templateCache, perhaps from some new service, from http, etc)
- do this entirely client side through some kind of template file registration that client side uses before fetching a template
- maybe something useful in $stateProvider.templateProvider? http://angular-ui.github.io/ui-router/site/#/api/ui.router.state.$stateProvider

Proposed Solution Summary:
1) Use the existing Django static file collection and place HTML overrides into a "themes/my_theme/static/templates/" directory.
2) Write Python code executed from settings.py, very similar to, or in the same path as the current Javascript static file collection. This outputs data into the HORIZON_CONFIG object.
3) This new data is then used by a new template tag (e.g. % theme_overrides %) that is placed in _scripts.html. This new template tag generates JSON data that allows a simple lookup of a full HTML file path (e.g. '/static/framework/widgets/toast/toast.html') and convert it to the appropriate override file path (if overridden) (e.g. '/static/themes/my_theme/static/templates/framework/widgets/toast/toast.html').
4) Create a javascript function in _scripts.html available on window that can do the above file lookup. This can be used in Django provided javascript as well as Angular. This function also prepends the necessary WEB_ROOT / STATIC_URL values.
5) Wrap the file lookup javascript in an Angular injectable for convenience.
6) All locations that currently use templateUrl will use this new service instead. This has the added benefit of preventing accidental problems by code that forgets to use WEB_ROOT / STATIC_URL
7) Consider lint rule for enforcement

Example of Use:
To override the "toast" template in "my_theme"
- Create:
/horizon/openstack_dashboard/themes/my_theme/static/templates/framework/widgets/toast/toast.html
- This will be collected today (no changes needed) by Django and placed at:
/horizon/static/themes/my_theme/static/templates/framework/widgets/toast/toast.html

- Somehow(tm) when the toast directive loads its template using:
templateUrl: path + 'toast/toast.html',

- that path of '/static/framework/widgets/' is converted to '/static/themes/my_theme/static/templates/framework/widgets'

Example of a possible config object that can be passed to the Angular side:
config_object = {'material': ['/static/framework/widgets/toast/toast.html']}
Note: It is probably good to use the FULL web root path to the html.

What if 'basePath' was simply a function or a 'path' service?

[tyr 2016-05-31]

Solving template overrides appears directly related to pre-populating the Angular template cache to improve load perfomance (e.g. 1 http call to get all templates vs 1 http call per template). I'm going to first look into a method to generate the needed <script type="text/ng-template"> tags, probably as part of _scripts.html, then address how to use template overrides in those tags.

Gerrit topic: https://review.openstack.org/#q,topic:bp/angular-template-overrides,n,z

Addressed by: https://review.openstack.org/323534
    WIP Pre-populate the Angular template cache

Gerrit topic: https://review.openstack.org/#q,topic:schema-form,n,z

(?)

Work Items

Work items:
Structure of JSON file: INPROGRESS
[hurgleburgler] Python code to generate JSON file: INPROGRESS
Javascript file lookup function: INPROGRESS
[tyr] Angular service to get file path: INPROGRESS

This blueprint contains Public information 
Everyone can see this information.