Rewrite YamlParser to improve error messages and JJB yaml extensibility

Registered by Wayne Warren

The goal of this blueprint is to articulate the pressing need for a rewrite of YamlParser in jenkins-job-builder.

I have attempted to improve defaults handling without much success due to the strange and difficult-to-grok, organically-grown data structure traversal within exandYaml. With a better-conceived yaml parsing class such improvements will become easier to make.

Error messages (for example when a JJB variable is missing) are almost impossible to understand because of all the data munging going on at different points within YamlParser. With a yaml parsing class that attachs information such as the filename where each piece of data is obtained we can at least point users to the relevant files containing the tidbit that is failing to be expanded or interpolated.

In addition to that, it is obvious that functionality has creeped into YamlParser that does not really fit the role of "yaml parsing". I was very surprised, for example, to find out that XML generation itself occurs within YamlParser... Also see the somewhat grotesque inclusion of a ModuleRegistry object in YamlParser's attributes.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
Wayne Warren
Direction:
Needs approval
Assignee:
Wayne Warren
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

(?)

Work Items

Work items:
Remove XML generation from YamlParser https://review.openstack.org/148087: INPROGRESS
Remove ModuleRegistry object from YamlParser https://review.openstack.org/148087: INPROGRESS
Create parser stub object to pass to gen_xml https://review.openstack.org/148087: INPROGRESS
Clarify YamlParser API using underscores to indicate private functions https://review.openstack.org/148089: INPROGRESS
Move YamlParser into separate python module https://review.openstack.org/148088: INPROGRESS
Move Builder.load_files function into YamlParser: TODO

This blueprint contains Public information 
Everyone can see this information.