Compare DB time with local to ensure that there is no mistiming
All tables in nova DB have 3 columns (deleted_at, updated_at, created_at) that have type datetime.
There are a lot of places where they are used. For example when we are checking is service available we are comparing local time with value in updated_at column.
So if our nodes become mistiming we will get a lot of different problems (for example unavailable servieses). And for administrators there could be not so clear why services are unavailable or something else is not working..
So to reduce problems with time we should implement 2 next things:
1. When nova starts, during init, compare local time to db time, and raise exception if the difference is too large
2. In periodic task, compare local time to db time, and if difference is to large do one of the next thing (it should be specified in conf):
a) nothing - do nothing
b) log - log warning
c) sync - sync time using ntpdate
d) die - kill nova process
Blueprint information
- Status:
- Not started
- Approver:
- Russell Bryant
- Priority:
- Undefined
- Drafter:
- Boris Pavlovic
- Direction:
- Needs approval
- Assignee:
- Boris Pavlovic
- Definition:
- Drafting
- Series goal:
- None
- Implementation:
-
Not started
- Milestone target:
- None
- Started by
- Completed by
Related branches
Sprints
Whiteboard
Boris, to tackle these, I'd like to help.
How about we do the following:
1. Identify all the 'problem' places
2. Split out fixing some 'problem' places (and document the fix)
3. Profit
Moving to Icehouse since this is still marked as not started. --russellb
Hi Boris,
I am writing because I would start working on this blueprint that you have assigned “Compare DB time with local to ensure that there is no mistiming”
Are you going to start working on this soon? Any problem if I assign it to me?
Thanks
--sjcazzol
Boris, could you please assign to me this blueprint?
Boris, to tackle these, I'd like to help.
How about we do the following:
1. Identify all the 'problem' places
2. Split out fixing some 'problem' places (and document the fix)
3. Profit
Moving to Icehouse since this is still marked as not started. --russellb
Hi Boris,
I am writing because I would start working on this blueprint that you have assigned “Compare DB time with local to ensure that there is no mistiming”
Are you going to start working on this soon? Any problem if I assign it to me?
Thanks
--sjcazzol
Boris, could you please assign to me this blueprint?
Marking this blueprint as definition: Drafting. If you are still working on this, please re-submit via nova-specs. If not, please mark as obsolete, and add a quick comment to describe why. --johnthetubaguy (20th April 2014)Boris, to tackle these, I'd like to help.
How about we do the following:
1. Identify all the 'problem' places
2. Split out fixing some 'problem' places (and document the fix)
3. Profit
Moving to Icehouse since this is still marked as not started. --russellb
Hi Boris,
I am writing because I would start working on this blueprint that you have assigned “Compare DB time with local to ensure that there is no mistiming”
Are you going to start working on this soon? Any problem if I assign it to me?
Thanks
--sjcazzol
Boris, could you please assign to me this blueprint?
Boris, to tackle these, I'd like to help.
How about we do the following:
1. Identify all the 'problem' places
2. Split out fixing some 'problem' places (and document the fix)
3. Profit
Moving to Icehouse since this is still marked as not started. --russellb
Hi Boris,
I am writing because I would start working on this blueprint that you have assigned “Compare DB time with local to ensure that there is no mistiming”
Are you going to start working on this soon? Any problem if I assign it to me?
Thanks
--sjcazzol
Boris, could you please assign to me this blueprint?
Marking this blueprint as definition: Drafting. If you are still working on this, please re-submit via nova-specs. If not, please mark as obsolete, and add a quick comment to describe why. --johnthetubaguy (20th April 2014)
Marking this blueprint as definition: Drafting. If you are still working on this, please re-submit via nova-specs. If not, please mark as obsolete, and add a quick comment to describe why. --johnthetubaguy (20th April 2014)
