Send as much DB traffic as possible from periodic tasks to DB slaves

Registered by Michael H Wilson on 2013-07-29

Periodic tasks are some of the most consistent load that any deployment will experience. As such, and because of the nature of most periodic tasks, their reads are prime targets for DB slaves.

Blueprint information

Status:
Complete
Approver:
Russell Bryant
Priority:
Medium
Drafter:
Michael H Wilson
Direction:
Approved
Assignee:
Michael H Wilson
Definition:
Approved
Series goal:
Accepted for icehouse
Implementation:
Implemented
Milestone target:
milestone icon 2014.1
Started by
Michael H Wilson on 2013-07-31
Completed by
John Garbutt on 2014-03-05

Related branches

Sprints

Whiteboard

I would have given a higher priority than Low to this normally, but it's just that it's a new blueprint toward the end of the cycle. I don't want to block if this doesn't make it. If this spills into Icehouse, I'll up the priority. --russellb

Gerrit topic: https://review.openstack.org/#q,topic:bp/periodic-tasks-to-db-slave,n,z

Addressed by: https://review.openstack.org/38872
    Allow _sync_power_states periodic task to hit slave DB

I think it's fine if the above patch lands in Havana. Since this blueprint covers much more than this first periodic task, I updated it for Icehouse, since that's when the vast majority of the changes will end up. --russellb

Addressed by: https://review.openstack.org/57358
    Allow all instance polling periodic tasks to hit db slave

Addressed by: https://review.openstack.org/57360
    Periodic task _poll_unconfirmed_resize can now use slave db

Addressed by: https://review.openstack.org/57372
    WIP Periodic task _heal_instance_info_cache can now use slave db

Addressed by: https://review.openstack.org/60992
    Allow _check_instance_build_time to hit db slave

Addressed by: https://review.openstack.org/61009
    Fix bug #1259383

Addressed by: https://review.openstack.org/61011
    Allow run_image_cache_manager_pass to hit db slave

I'm going to wait on Nikola Dipanov's object implementation of BDMS before I slaveify _cleanup_running_deleted_instances. The non-object way is gross, so I'd rather wait
for it to be pretty and workable. --geekinutah

Addressed by: https://review.openstack.org/72797
    Task cleanup_running_deleted_instances can now use slave

We're passed the deadline for blueprint code to be proposed for Icehouse, but I think this one should get an exception since most of it has been completed. Let's leave this targeted to Icehouse for now. --russellb

I removed update_available_resource task because there is more work to be done in the Service object that will likely take a while. I'll revisit this in Juno. As it stands this bp should just need code review and fixups. --geekinutah

Gerrit topic: https://review.openstack.org/#q,topic:bp/periodic-tasks-to-db-slave-cleanup_running_deleted_instances,n,z

patch moved to juno:
===========================
https://review.openstack.org/#/c/72797/
(Note: stuck behind BDM changes)

Apologies, this missed the deadline for Feature Freeze. Marking this one as Implemented, so please open a new blueprint for the remaining patches. Please rebase patches as soon as Juno opens, and we will try to get this in during that period. --johnthetubaguy (5th March 2014)

This was subsequently merged as BDM got FFE. -geekinutah

(?)

Work Items

Work items:
slaveify _check_instance_build_time: DONE
slaveify _heal_instance_info_cache: DONE
slaveify _poll_rebooting_instances: DONE
slaveify _poll_rescued_instances: DONE
slaveify _poll_unconfirmed_resizes: DONE
slaveify _poll_shelved_instances: DONE
slaveify _poll_bandwidth_usage: DONE
slaveify _sync_power_states: DONE
slaveify _reclaim_queued_deletes: DONE
slaveify _cleanup_running_deleted_instances: DONE
slaveify _run_image_cache_manager_pass: DONE

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.