Extend pagination into the identity backends wherever possible to improve scaling and performance

Registered by Henry Nash

In v3, our identity APIs support pagination. However, such pagination support is only skin deep - in that the controller re-fetches the entire list upon each "get next page" request from the client. We should therefore extend the pagination into the backends (e.g. SQLAlchemy supports such a concept) to improve the scaling and performance of keystone.

A good pagination system will still require good filtering. The addition of this is covered by a separate blueprint (https://blueprints.launchpad.net/keystone/+spec/filtering-backend-support)

Related wishlist bug: https://bugs.launchpad.net/keystone/+bug/1009023

Blueprint information

Status:
Complete
Approver:
None
Priority:
Low
Drafter:
Henry Nash
Direction:
Approved
Assignee:
Henry Nash
Definition:
Obsolete
Series goal:
None
Implementation:
Not started
Milestone target:
None
Completed by
Lance Bragstad

Related branches

Sprints

Whiteboard

This was one of the items discussed during the Havana keystone scaling and performance summit session: https://etherpad.openstack.org/havana-keystone-performance

Not all backends (e.g. LDAP) can be expected to respond to pagination hints, so the work should still be duplicated at the controller layer.

Gerrit topic: https://review.openstack.org/#q,topic:bp/pagination-backend-support,n,z

Addressed by: https://review.openstack.org/43581
    Implement pagination support in driver backends

"Let's Kill Pagination!" http://summit.openstack.org/cfp/details/6

procedural change of milestone target, feel free to target to mitaka-2 or mitaka-3, or update this to reflect a more current update on this feature

(lbragstad) 19-02-15: I'm marking this as obsolete based on the historical context captured in the whiteboard here and in the bug report [0] (which doesn't currently fit in the project plans). If we choose to pursue this we will reopen the existing bug report since it already captures the historical context of the problem.

[0] https://bugs.launchpad.net/keystone/+bug/1009023

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.