Remove psycopg1 dependency in favor of psycopg2

Registered by Morten Brekkevold

psycopg1 has been outdated for several years already, this dependency should be removed from NAV. NAV has already gained a dependency on psycopg2 due to the introduction of Django, so legacy code should be refactored to become psycopg2-compatible.

Blueprint information

Status:
Complete
Approver:
Morten Brekkevold
Priority:
Medium
Drafter:
Morten Brekkevold
Direction:
Approved
Assignee:
Morten Brekkevold
Definition:
Approved
Series goal:
Accepted for 3.6
Implementation:
Implemented
Milestone target:
milestone icon 3.6.0b1
Started by
Morten Brekkevold
Completed by
Morten Brekkevold

Related branches

Sprints

Whiteboard

I am working on patches to do this switchover. psycopg2 is more Python DB-API compliant than psycopg1. In practice, this means that connection objects do not have an autocommit method, and cursor objects do not have the dictfetch*-methods.

Psycopg2 has several extension possibilities, and one of them is support for custom connection and cursor objects. The API provides a DictCursor, whose return values for the fetch* methods are custom row objects that can be treated both as lists and dictionaries. I'm using DictCursors to transition code that uses the deprecated dictfetch* methods.

The transition has turned out to be a lot more complex than first thought. Several subsystems expect cursor result rows to be updateable dictionaries, which DictCursor result rows are not. Lots of code expect dates to be mx.DateTime objects, but they will in all likelihood be datetime.datetime objects. Some code expects psycopg2 to support the full range of python format specifiers when doing parameterized SQL, but psycopg2 only supports %s.

There is also the problem with mixing unicode and str objects in Cheetah templates. As NAV is integrating Django, strings returned from psycopg2 will be unicode objects.

A couple of serious bugs in psycopg2 could be showstoppers. These will kill some of NAV's daemons, specifically snmptrapd. psycopg 2.0.5 (available in Debian Etch) has one bug, psycopg 2.0.7 (available in Debian Lenny) has another. Both are related to connections that have been dropped. Seems we need 2.0.8 to make sure NAV won't crash.

Feature branch here: http://metanav.uninett.no/hg/features/deprecate-psycopg1/

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.