[EDP][Spark] Configure cluster for external hdfs

Registered by Trevor McKay

The Oozie EDP engine will update /etc/hosts files as necessary when a job execution references an external hdfs. The Spark EDP engine should include this functionality as well.

(Originally this feature was only associated with URLs referenced via data_sources, which is why it was omitted in the case of Spark since Spark does not reference data_sources. However, now that data_source reference substitution is supported in job_configs, Spark should implement this as well.

Also see https://bugs.launchpad.net/sahara/+bug/1425731. Spark should handle manually entered URLs in job_configs as well.)

Blueprint information

Status:
Not started
Approver:
Sergey Lukjanov
Priority:
Undefined
Drafter:
Trevor McKay
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/edp-spark-external-hdfs,n,z

Addressed by: https://review.openstack.org/161371
    [EDP][Spark] Configure cluster for external hdfs

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.