Fetch and fetch all have different default behavior for the many_handler

Registered by vdloo

I noticed that the default many handler for fetch returns the first value, while the default many handler for fetch_all returns the entire set if the len > 0. I think it would be nice to have both many handlers behave like the fetch_all version because the difference between fetch and fetch_all is that fetch all returns all results from the store and fetch only returns result. It should say nothing about how much entries to return for each result.

https://github.com/openstack/taskflow/blob/master/taskflow/storage.py#L692
https://github.com/openstack/taskflow/blob/master/taskflow/storage.py#L811

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
vdloo
Direction:
Needs approval
Assignee:
vdloo
Definition:
Discussion
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

JH: So just a thought, I know folks are using the fetch version and expecting to only get one result (because mainly only one result really matters to them), I think your suggestion is to change fetch to work like fetch_all? Perhaps we just need to make the docstrings on fetch() and fetch_all() better to make this more understandable? Thoughts?

One example is: https://github.com/openstack/cinder/blob/master/cinder/volume/manager.py#L470 (and I'm not sure how many others are depending on that doing the above to so it is somewhat hard to change that...).

vdloo:
My suggestion was to make the default behavior of the many_handler not differ between fetch and fetch_all, because I understood 'fetch' to mean 'get a specific item from the store' and 'fetch_all' to mean 'get all items from the store'. The difference between fetch and fetch_all says nothing about how much entries to get for each item in the store, it only says which item(s) to get from the store. The amount of entries to return for that item is something that the many handler should decide, and I don't think the default behavior of that handler should differ between fetch and fetch all (because those functions denote whether to get one item or all items and not how much entries for each item).

I get that for fetch_all it would make sense to return everything from the store (so also all entries for each item), that's why I suggested to change the many_handler for fetch to behave like the fetch_all one (and return a list if there is more than one entry for that item in the store). I also understand that this is difficult to change because as you say many people are probably already depending on this behavior. Perhaps it is an idea to do it the other way around and make the fetch_all many handler behave like the fetch many handler and only return 1 entry for each result by default?

class ReturnOne(task.Task):
    def execute(self):
        return 1

f = linear_flow.Flow('returnone-flow')
f.add(
    ReturnOne('t2', provides='a')
)
e = engines.load(f)
e.run()

print(e.storage.fetch('a')) # fetch is 1
print(e.storage.fetch_all()['a']) # fetch_all a is 1

store = {'a': 1}
f = linear_flow.Flow('inc-flow')
f.add(
    ReturnOne('t1', provides='a')
)
e = engines.load(f, store=store)
e.run()

print(e.storage.fetch('a')) # fetch a is 1
print(e.storage.fetch_all()['a']) # fetch_all a is [1, 1], maybe have this return 1 because fetch_all means 'get all items from the store' and not 'get all entries for all items from the store' because that is behavior you can specify by overwriting the many_handler

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.