(Alerts) Surveillance and monitoring

Registered by Saptarshi on 2010-10-18

DHIS needs to support requirements for data surveillance and monitoring. When certain types of data is entered into the system, e.g. data for notifiable diseases, the system should provide mechanisms for alerting a target group of users.

More specifically, the system should support automatically dispatched alerts based on criteria/checks for the incoming data. These criteria/checks are defined as mathematical formulas including data element values much in the same way as the existing validation rules. These formulas can act as thresholds, when the threshold is exceeded the system should respond with an alert.

It should be possible to check these thresholds automatically at regular intervals through scheduled jobs. These jobs should be configurable in terms of which checks to include and at what intervals to run the job. The default interval should be nightly.

The mechanism for sending alerts should be integrated with the existing messaging system. This system is able to dispatch messages to the internal message inbox, to user's email address and to user's mobile phones over SMS.

It should be possible to set a level of importance for the checks to indicate the seriousness of a violation. There should be three levels: High, medium, low.

Alerts of violations should be dispatched through the messaging system along the lines of the organisational hierarchy. For instance, if a violation occurs in a district, the users associated with the district, the parent province and national organisation units should be notified.

The checks should be run for a configurable number of days back in time. The default should be 7 days. It is in the nature of notifiable diseases that if action is not taken during a relatively short time, it will be too late and notifications become irrelevant.

---
Implementation notes:

Alerts are based on validation rules, with which they have a lot in common. Like a validation rule, an alert "defines a relationship between a number of data elements. The expression has a left side and a right side and an operator which defines whether the former must be less than, equal to or greater than the latter." As with a validation rule, these left and right sides can consist of a single data element, or a numeric value (for example 0, 1, 2, etc.) Or these sides may contain an expression made up of data elements and/or constants using the operators +, -, *, /, and using () for grouping.

For example an alert may be triggered if a data element value is greater than a fixed number (like 0, 1, 2, ...). In general, an alert is triggered when one expression of data (and/or numeric) values meets the comparison with another expression of data (and/or numeric) values. Also like a validation rule, an alert specifies the data entry period type: Daily, Weekly, Monthly, etc. The data entry period determines which data elements are available to form the left and right side of the expression.

Alerts also have the following properties that go beyond validation rules:

a. Importance: High, Medium or Low. The importance level will be present in any messages triggered by this alert.

b. Organisation unit level: The organisation unit level to which any data elements in the expression are aggregated. For example, if the organization unit level represents an individual facility, the alert is computed by taking the data for that facility, or summing all data elements within that facility if there are lower-level organisations. If the organisation unit level represents a district, the alert is computed by summing all the data elements within each district. Etc.

c. Period extent: The number of time periods over which any data elements in the expression are aggregated. For example, a period extent of 1 with a weekly data element of "number of new malaria cases" means the weekly number of new malaria cases most recently reported. Whereas a period extent of 2 means that number of malaria cases from the most recent two weeks will be added together before used in the expression. The period extent applies to data elements in both the left and right sides of the expression. Data elements with a "sum" aggregation operator are summed throughout the time periods, while those with an "average" aggregation operator are averaged.

d. Preceding periods: This value affects how data elements are interpreted in the RIGHT SIDE of the equation. If the value is not specified, the most recent period data is used for each data element (on both sides of the equation.) If the value is 1, the next-to-last data is used for each data element appearing on the right side. For example this lets you compare the number of new cases of malaria in the week just reported with the number of new cases in the previous week. If the value is greater than 1, it specifies the number of previous periods that are averaged together on the right side to get the value. For example, this lets you compare the number of new cases of malaria in the week just reported with the weekly average over the previous 5 weeks.

Note that preceding periods and period extent can work together. For example, let's say that week 9 of the year has just been reported. A period extent of 2 with no preceding periods means to sum data elements over weeks 8-9. If preceding periods is 2, the sum will be over weeks 7-8 and averaged with the sum over weeks 6-7. (This example assumes a Sequential preceding period type, as described next.)

e. Preceding period type: This affects how the preceding period is computed. The possible values are Sequential or Annual. If Sequential, the preceding periods are the ones immediately preceding the latest data. For example the Sequential preceding periods for 2013 week 9 are 2013 week 8, 2013 week 7, etc. If Annual, the preceding periods will be the same period in preceding years. For example the Annual preceding periods for 2013 week 9 are 2012 week 9, 2011 week 9, etc.

f. High outliers: This affects the way preceding periods are averaged. It gives the number of the previous highest values that will be discarded before computing the average for preceding periods. For example, if the preceding periods count is 5 and high outliers count is 2, this takes values from five preceding periods, but throws out the highest two values before averaging them. This can be useful, for example, if there were some outbreaks in previous periods that you wish to ignore when considering the baseline data to compare the latest period against.

g. Low outliers: This is similar to high outliers. It gives the number of previous LOWEST values that will be discarded before computing the average for preceding periods. Note that the high outliers count plus the low outliers count must be less than the preceding periods count.

h. How often the alert expression is tested. The choices are daily, weekly or monthly. [Question: do we need longer test intervals, or are these sufficient because alerts are supposed to be timely?] Note that this testing interval may be greater or less than the data entry period. For example, the data could be entered weekly but the test run daily. In this case, a scan is made every day for new data that may be entered for the week. If data is entered that triggers an alert, the alert is generated that day without waiting for the end of the week.

i. If the test is weekly or monthly, the day on which the test occurs. If the test is weekly, the choices are Sunday, Monday, ... Saturday. If the test is monthly, this is the day of month: 1 means the test is run on the first day of the month, 2 on the second day, etc. If the number is in the range 29-31 and that month happens to have fewer days, the test is run on the last day of the month.

[Questions about alert test scheduling:]

   1) Are properties (h) and (i) important for a first release of alerts -- or can the tests just always run daily?

   2) Is there a need to specify that alerts will be tested for only a subset of the organisation tree -- or can the alerts always be run for the whole organisation tree.

   3) Should scheduling parameters (if any) be configured on a separate screen, or can they be on the same screen with the rest of the alert definition?

Delivering alerts:

As said above, alert delivery will "be integrated with the existing messaging system. This system is able to dispatch messages to the internal message inbox, to user's email address and to user's mobile phones over SMS." Alerts will "be dispatched through the messaging system along the lines of the organisational hierarchy. For instance, if a violation occurs in a district, the users associated with the district, the parent province and national organisation units should be notified."

[Question: are there cases in which it is not appropriate (or necessary) to send alerts to all users associated with an organisation unit? If so, we can define an authority such as ALERT DELIVERY to be used only in roles where alerts should be delivered.]

To control access to defining alerts, three new authorities will be added: ALERT ADD, ALERT DELETE and ALERT UPDATE.

Blueprint information

Status:
Complete
Approver:
Lars Helge Øverland
Priority:
Undefined
Drafter:
None
Direction:
Approved
Assignee:
Jim Grace
Definition:
Approved
Series goal:
Accepted for trunk
Implementation:
Implemented
Milestone target:
milestone icon 2.13
Started by
Lars Helge Øverland on 2013-08-26
Completed by
Lars Helge Øverland on 2013-10-14

Sprints

Whiteboard

We also need to consider sending out alters for reported individual cases.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.