std.http action should be able to work with binary data

Currently, we can't use "std.http" action to download binary data. We can work only with JSON-compatible strings, meaning that once we downloaded content (accessible via "content" field of the response) it must be a JSON-compatible string to save it to the workflow context. If it's a binary data it won't work.

As an approach, we can see the HTTP response content type and if it's binary then we can encode the content into base64 (or similar) so that we can save a result in the workflow context. However, Mistral somehow then need know that it's a base64 encoded string so that it could be used for anything useful. "response enconding" header could also be used to force the needed enconding.

IRC discussion:
...
btw, I have something to discuss
rakhmerov
https://blueprints.launchpad.net/mistral/+spec/mistral-http-action-binary-data
rakhmerov
just created
rakhmerov
my colleagues would like to have this ability in some form
d0ugal
Interesting
rakhmerov
but I have doubts about how to implement it properly
rakhmerov
yeah
rakhmerov
working with a file system doesn't look right to me
rakhmerov
so base64 encoding maybe
d0ugal
yeah, I would prefer base64
d0ugal
but I guess binary data can sometimes be large? that might be bad for the database?
rakhmerov
d0ugal: of course..
rakhmerov
yes
d0ugal
Using Mistral with another service in the case might be best? Something like Swift could work
rakhmerov
may be setting some limit would be reasonable
rakhmerov
or something like this, yes..
rakhmerov
something that's specifically designed for that
rakhmerov
for storing large data
d0ugal
Is this something you hope to do for rocky?
rakhmerov
no
rakhmerov
I don't think it's feasible
rakhmerov
given all other things that i'm hoping to achieve
rakhmerov
next cycle
d0ugal
:)
d0ugal
but that is good, then we have time to think about it more
rakhmerov
yeah
rakhmerov
using an external storage seems the best solution for me BUT
rakhmerov
we need to have that external storage :)
rakhmerov
lots of people (including us) don't care about OpenStack that much
d0ugal
Indeed
rakhmerov
:)
d0ugal
I guess it could be pluggable
rakhmerov
yes
rakhmerov
through some interface
d0ugal
Yeah, and I guess the interface could be very simple
rakhmerov
otherwise, our DB would be abused
d0ugal
(so easy to implement on other backends)
rakhmerov
yep
d0ugal
The Mistral DB could even be a backend, but with with warnings and limitations
rakhmerov
so it could look like just an additional option in std.http to store content into that storage
15:10 rakhmerov
and then if it's needed we could have a YAQL/JINJA function to extract it
rakhmerov
yeah, agree with your latest thought
rakhmerov
yes
d0ugal
Sounds good
rakhmerov
architecturally it would be more proper solution
rakhmerov
I think I'll copy our conversation to the BP description
rakhmerov
do you mind?
d0ugal
rakhmerov: sure, go for it
...