Add a 'nova-manage db purge_task_log --before' command

Registered by melanie witt on 2019-10-16

As discussed on the ML thread last month:

http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009199.html

We have a 'task_log' database table which is appended to whenever the server usage audit log API [1] is enabled: "This API is dependent on the instance_usage_audit configuration option being set on all compute hosts where usage auditing is required."

From the thread, we know that other systems are using the API in the wild (example: cASO https://github.com/IFCA/caso).

It's also enabled by OpenStack Telemetry [2], which is how we have customers running into issues where the 'task_log' database table is growing without bound.

We have no database API methods for ever deleting 'task_log' records, so it is not a defect in cleanup methods as there simply aren't any.

In this blueprint, we propose adding a 'nova-manage db purge_task_log --before <date>' command to permanently delete 'task_log' records that are from before the specified <date>, else purge them all if --before is not specified. This will provide a way for operators to clean up the 'task_log' table and prevent it from growing indefinitely.

[1] https://docs.openstack.org/api-ref/compute/?expanded=#server-usage-audit-log-os-instance-usage-audit-log
[2] https://docs.openstack.org/ceilometer/train/install/install-compute-rdo.html#configure-compute-to-use-telemetry

Blueprint information

Status:
Complete
Approver:
None
Priority:
Undefined
Drafter:
melanie witt
Direction:
Needs approval
Assignee:
melanie witt
Definition:
Obsolete
Series goal:
None
Implementation:
Unknown
Milestone target:
None
Completed by
Eric Fried on 2019-10-24

Related branches

Sprints

Whiteboard

I think the thing to consider with this is, do we consider this a bug such that task log entries build up over time and we have no supported interface to purge them which makes operators have to clean up the database table(s) directly? If so, then we don't really need a blueprint for this and can just fix it as a bug, which might also mean it could be backported, though I'd really consider it low priority to backport since I'm not aware of anyone using the task log stuff too much (it's legacy audit stuff tied to ceilometer and disabled by default). -- mriedem 20191017

[mriedem 20191017] We briefly discussed this in the nova meeting today:

http://eavesdrop.openstack.org/meetings/nova/2019/nova.2019-10-17-14.00.log.html#l-306

and a bit after in the nova channel. It sounds like we're leaning toward just calling this a wishlist bug and then sort out backport possibilities later.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.