Solum input validation

Registered by Devdatta Kulkarni on 2014-03-06

This blueprint deals with validating input data sent to Solum.

We want:
- generic checks for each resource's input data
  e.g.: attribute name should not be empty

- specific checks for each resource's input data
  e.g.: In POST to /assembly, the code artifact value should adhere to following rules:
  - ^http://github.com/[a-z/-]+$

Done:
1) Generic check for 'name' attribute
https://review.openstack.org/#/c/77251/ (MERGED) (this uses wsme to add generic checks).

To Do:
1) Add generic checks for other attributes

2) Add a "framework" to add specific checks for different resource data:
    - suggested approach 1: Use WSME's @validate decorator
    - suggested approach 2: Use a library such as Voluptuous (bug for this is available):
      https://bugs.launchpad.net/solum/+bug/1286400

Other related information:
https://groups.google.com/forum/#!searchin/python-wsme/validation%7Csort:relevance%7Cspell:true/python-wsme/XR06D76CXRA/FM-TwvJCKzsJ

See Task: https://bugs.launchpad.net/solum/+bug/1286400

Blueprint information

Status:
Complete
Approver:
Adrian Otto
Priority:
Medium
Drafter:
Devdatta Kulkarni
Direction:
Approved
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Implemented
Milestone target:
None
Started by
Adrian Otto on 2014-03-11
Completed by
Adrian Otto on 2015-06-11

Related branches

Sprints

Whiteboard

paulmo: I see no reason for WSME to be involved in any data validation.

(asalkeld) because that is why we use it (serialization, validation and working with
                   objects/classes not blobs of json). Our user input is via rest so it makes sense for
                  the rest framework to do the user validation. There is a large overlap between voluptuous
                  and wsme validation:
                                       type checking | required | range | custom
                  wsme: <yes> <yes> <no> <yes>
                  voluptuous <yes> <yes> <yes> <yes>

                 I think the heat project could benefit from using it, but for us I think it doesn't seem to make
                 sense (The Heat project does not have WSME and deals a lot with json templates).
                 We would have to convert the typed objects we get from wsme and convert back to
                 json and re-validate - that just seems weird to me.

                 +1 on more validation and either using the validate() method on the WSME objects or
                 just raising exceptions within the controller. (I have no problem with the below example,
                  use WSME where it makes sense and custom code as needed).

                  -1 on voluptuous

paulmo: Thanks for the explanation Angus, I'll look into WSME more.

We may want to use this regex for git URLS as a sample: ^https{0,1}://github.com/[a-z/_-]+$
Here is some example code to consider:

import re

from oslo.config import cfg

CONF = cfg.CONF

VALIDATION_OPTS = [
    cfg.IntOpt('git_url_max_len',
               default=255,
               help='The longest git url acceptable'),
    cfg.StrOpt('git_url_regex',
               default='^https{0,1}://github.com/[a-z/_-]+$',
               help='The git url validation regex')
]
opt_group = cfg.OptGroup(name='validation',
                         title='User input validation settings')
CONF.register_group(opt_group)
CONF.register_opts(VALIDATION_OPTS, opt_group)

def valid_git_url(url):
    if len(url) > CONF.validation.git_url_max_len:
        return False
    if re.match(CONF.validation.git_url_regex, url) is not None:
        return True

(I need to add exceptions and such but that is one possibility to just Oslo config the url validation regex.)

Pull request: https://review.openstack.org/#/c/79042/

Gerrit topic: https://review.openstack.org/#q,topic:bp/solum-input-validation,n,z

paulmo: Devdatta and I believe that once this code is used to verify the git url that we've met the M1 requirements. This topic certainly needs to be expanded for all input though and I will continue working this.

Addressed by: https://review.openstack.org/79042
    Git url validation

Addressed by: https://review.openstack.org/81644
    Git pull url validation

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.