Add local time zone support for colleting and reporting

Registered by Xiangjun Li

Currently, cloudkitty processes datetime-related data based on UTC time zone like other openstack components. But as a rating system this manner should cause problems in some special scenarios.

For example, collector will process the metering data of ceilometer that was generated in current month (of course, the time stamp of meter data stored in ceilometer database is in UTC format) if this is the first time running cloudkitty-processor. Suppose current UTC date time is 2015-11-15 15:00:00, the time period of the meter data retrieved from ceilometer should be from 2015-11-01 00:00:00 to 2015-12-01 00:00:00 according to cloudkitty processing logic. This is not a problem if the server running cloudkitty is placed in UTC time zone, but if the server is placed in other time zone, e.g. CST (UTC+8), the boundary of November in UTC format should be from 2015-10-31 16:00:00 to 2015-11-30 16:00:00, that is, in this scenario, collector should collect ceilometer meter data between 2015-10-31 16:00:00 and 2015-11-30 16:00:00 instead of 2015-11-01 00:00:00 to 2015-12-01 00:00:00. Similarly, API layer of cloudkitty will face the same issues (especially in report API) since the time frame is also stored as UTC format in cloudkitty database. So, the situation that cloudkitty deployed in different time zones must be take into consideration when the data time boundary is being calculated (in both cloudkitty-api and cloudkitty-processor).

The main work of this specification is: refactor time calculations module (utils.py) so that necessary transform logic will be added when calculating data time boundary or doing addition and subtraction for date time. Of course, a set of wrapper functions concerning local time zone support will be provided in utils.py

For now, just following places will be affected if the local time zone support is applied:
Cloudkitty-processor: retrieve meter data from ceilometer for a specific time periods
Cloudkitty-api: get_total, get_tenants
But in the feature, it is recommended that if a new feature that include date time boundary calculation (it would be a very common scenario in a rating/billing system, e.g. calculate total cost of a tenant for day/month/year) will be added, you can call corresponding wrapper functions in utils.py module to do necessary preprocessing first.

Blueprint information

Status:
Started
Approver:
Stéphane Albert
Priority:
Low
Drafter:
Xiangjun Li
Direction:
Approved
Assignee:
Xiangjun Li
Definition:
Approved
Series goal:
Accepted for 0.6
Implementation:
Needs Code Review
Milestone target:
None
Started by
Xiangjun Li

Related branches

Sprints

Whiteboard

Time calculation in CloudKitty are made with UTC timezone. Local time is always converted to UTC and requests are made with this UTC time. So there should be no shifting in dates, if it's the case then it's a bug.
About API request, it's the client that is responsible of date normalization. Client should convert local time to UTC time.
Every OpenStack component is working on UTC time and requests should be made so that requests are in UTC zone.

Yes, there is no problem that data is stored as UTC time in storage and end user/client should normalize the requests with UTC time first before requests send to CK-API.
But some internal processes about data time boundary calculation within CK-API and CK-processor maybe generate unexpected result. For example:
Duration (UTC time format) for 2015.11 in UTC time zone: 2015-11-01 00:00:00 to 2015-12-01 00:00:00
Duration (UTC time format) for 2015.11 in CST time zone: 2015-10-31 16:00:00 to 2015-11-30 16:00:00
As an end user’s point view, if the user in CST time zone and current month is 2015.11, he expects CloudKitty to rate the ceilometer metering data which was generated at the beginning of 2015.11(should be UTC:2015-10-31 16:00:00) when CloudKitty is the first running. But If I understood correctly, the beginning that CloudKitty calculated is always UTC:2015-11-01 00:00:00, either CloudKitty was deployed in UTC time zone or others time zone.
Also get_total API need to do a data time boundary calculation for current month if the data time boundary isn’t specified in requests. So I think some proper transformations (i.e. rightly figure out the duration of day/month/year for different time zones) is needed when the data time boundary is being calculated within CK-API and CK-processor.
Shall we consider implement some billing-related feature in the future like providing query of total cost for a day/month/year (or many extension scenarios)? For the sake of performance, pre-calculation for total cost of past day/month/year rather than do runtime calculation after request is arrived is needed, in these scenarios, data time boundary calculation are very common too.
                                                                                                                              --lixiangjun (12.04.2015)

Ok, now I see what you mean. Sorry I forgot to subscribe to the BP so my answer was late.
              - Stephane (12.08.15)

Gerrit topic: https://review.openstack.org/#q,topic:bp/local-time-zone-support,n,z

Addressed by: https://review.openstack.org/282661
    Enhance utils.py to support local time zone

Addressed by: https://review.openstack.org/282672
    Refactor boundary calculation logic

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.