Implement sensible versioning of congregation & group data

Databases uniquely identify records by assigning a unique ID (normally an autoincremented integer) in an 'id' column in every table. This works fine when you have one database. But, we're dealing with federated data--data in many databases, each of which has its own set of IDs, and normally the RCL database will have no access to non-RCL databases' ID columns. The best way to deal with this situation at present is to use UUIDs - universally unique identifiers, which are basically long randomly generated integers that are thereby highly probable to be unique. But non-RCL databases (feed sources) normally won't provide us UUIDs.

When we update one feed's cache with new data from the feed, in the case where the feed does provide its own set of IDs (though not UUIDs) for each congregation/group (e.g., as a URL parameter like ?id=123), we can record that ID not as the RCL ID, but as the feed-specific ID for that congregation/group, and thereby avoid creating duplicates in that feed's data. This will not prevent duplicates in other cases, though. It also might fail if a feed source unexpectedly rearranges its IDs.

So, in addition to what is described in other blueprints already, we need to decide how to match

a) a congregation/group coming from a feed that doesn't already associate the congregation/group with a RCL UUID with

b) the right RCL congregation/group UUID.

Perhaps do it via similarity heuristics (like difflib http://docs.python.org/library/difflib.html, Levenshtein http://code.google.com/p/pylevenshtein/, or Damerau-Levenshtein http://stackoverflow.com/questions/682367/good-python-modules-for-fuzzy-string-comparison), but this is bound to create duplicate data unless we automate the matching heuristic to progressively decrease its sensitivity until a match is made (but then, we'll still need to determine if a NEW congregation was added in the source feed--how do we do that? Some sources won't flag congregations as new. We might be able to approximate a flag by counting the number of congregations.) So do we need a way to clean up duplicates manually?

We may also need to prioritize different fields - a congregation's name is more likely to remain the same than its street address. Likewise, we can consider prioritizing the beginning & end of some fields differently. E.g., changes after the first two words in the street address are less likely to be significant than changes to the first two words. Changes after the first word of the congregation name are less likely to be significant than changes to the first word.