New feature: service double-check and differential update
Occasionally, the rescuetime-
1. Alert the user that data hasn't been accepted.
2. Provide the user with a mechanism to upload the rejected data at a later time.
With the current issue (see bug #522046) the service acknowledges "0" records to be imported. That points to a bug early in their infrastructure and is easily detected. However the bug might as well be deeper in their infrastructure where the services acknowledges our submissions, but they just don't end up in their database.
Rescuetime are providing a data API, where we can check which data actually made it into their database. This allows it to check end-to-end whether records the client submitted actually make it where they are needed (into their database). As in the current implementation (version 99) we keep all the data records, we know what we have sent to them and thus can look up these records as a low-priority job in the background and check whether we can find them in the database. If that is the case we can either finally delete them (I think that'd be the wisest action) or move them into yet another state-directory called "verified" or something similar.
The software infrastructure necessary for the above end-to-end test then can easily tell us, which of these files have to be re-submitted again. This resubmission could either be done on a manual basis, where the user is displayed a selectable list failed records or on an automatical basis. In the latter case, however, we need a way to not do it forever, so we need to put an upper limit on the number of resubmissions. Also an intelligent trigger of resubmission would be nice. For example in a situation like the current one, where no records make it into the database there isn't really any point in attempting a resubmission. But once we detect some records are actually making it again into the database we should start the resubmission process.
