Ubuntu Apps Directory

Merge lp:~michael.nelson/ubuntu-webcatalog/load-logging-config-in-wsgi into lp:ubuntu-webcatalog

  1. load-logging-config-in-wsgi
  2. Merge into trunk
Proposed by Michael Nelson
Status: Merged
Approved by: Łukasz Czyżykowski
Approved revision: 189
Merged at revision: 188
Proposed branch: lp:~michael.nelson/ubuntu-webcatalog/load-logging-config-in-wsgi
Merge into: lp:ubuntu-webcatalog
Diff against target: 198 lines (+66/-52)
5 files modified
django_project/config/main.cfg (+1/-50)
django_project/wsgi.py (+2/-2)
src/webcatalog/management/commands/import_sca_apps.py (+23/-0)
src/webcatalog/schema.py (+4/-0)
src/webcatalog/wsgi.py (+36/-0)
To merge this branch: bzr merge lp:~michael.nelson/ubuntu-webcatalog/load-logging-config-in-wsgi
Reviewer Review Type Date Requested Status
Łukasz Czyżykowski (community) Approve
Review via email: mp+183586@code.launchpad.net

Commit message

Separate logging configs for webapp and import command.

Description of the change

Related to RT 61147 - providing a log file for the import_sca_apps job that can be monitored.

After switching to a dictConfig and adding a separate logger for the import command, the deploy to staging errored because every cron/setup/whatever script which would use ./manage.py would automatically try to touch the webapp.log file (as Django loads the logging config directly after settings are loaded), and doesn't have perms on the filesystem.

This branch separates the logging configs and only initialises them when they are used (ie. in this case when the webapp is started).

As the "dictConfig" defined in a configglue ini file was getting hard to follow, rather than adding a separate dict config there, I've moved them into a more readable (ie. normal) dict where they're used, and just used settings for the relevant pieces we'd want to change (format, file location).

Finally, and strangely, I had to specify a django logger (with no handlers) just to ensure that the msgs were propagated to the root logger.

To post a comment you must log in.
Revision history for this message
Łukasz Czyżykowski (lukasz-czyzykowski) wrote :

I like the idea of pulling only the relevant config bits into configglue file.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'django_project/config/main.cfg'
2--- django_project/config/main.cfg 2013-08-29 10:38:27 +0000
3+++ django_project/config/main.cfg 2013-09-03 08:55:21 +0000
4@@ -6,7 +6,6 @@
5 django.contrib.auth.backends.ModelBackend
6
7 databases = django_databases
8-logging = django_logging
9
10 debug = true
11 media_root = django_project/media_root_dev/
12@@ -73,55 +72,6 @@
13 engine = django.db.backends.sqlite3
14 name = webcatalog.db
15
16-[django_logging]
17-version = 1
18-disable_existing_loggers = True
19-handlers = django_logging_handlers
20-loggers = django_logging_loggers
21-root = django_logging_root
22-formatters = django_logging_formatters
23-
24-[django_logging_root]
25-level = WARNING
26-
27-[django_logging_loggers]
28-django = django_logger
29-webcatalog.management.commands.import_sca_apps = import_sca_logger
30-
31-[django_logger]
32-handlers = webapp
33-level = INFO
34-
35-[django_logging_handlers]
36-webapp = webapp_logging_handler
37-importer = import_sca_apps_handler
38-
39-[webapp_logging_handler]
40-class = logging.handlers.WatchedFileHandler
41-level = INFO
42-formatter = simple
43-filename = ./tmp/webapp.log
44-
45-[import_sca_apps_handler]
46-class = logging.StreamHandler
47-level = INFO
48-formatter = simple_no_name
49-
50-[import_sca_logger]
51-handlers = importer
52-propagate = false
53-level = INFO
54-
55-[django_logging_formatters]
56-simple = django_logging_simple_format
57-simple_no_name = django_logging_simple_no_name_format
58-
59-[django_logging_simple_format]
60-format = %%(asctime)s - %%(name)s - %%(levelname)s - %%(message)s
61-
62-[django_logging_simple_no_name_format]
63-format = %%(asctime)s - %%(levelname)s - %%(message)s
64-
65 [openid]
66 openid_sso_server_url = https://login.staging.ubuntu.com
67 openid_create_users = true
68@@ -142,6 +92,7 @@
69 default_distro = natty
70 convoy_root = ./src/webcatalog/static/
71 ubuntu_series_for_versions = series_for_versions
72+webapp_log_path = ./tmp/webapp.log
73
74 [series_for_versions]
75 10.04 = lucid
76
77=== modified file 'django_project/wsgi.py'
78--- django_project/wsgi.py 2013-04-23 08:05:18 +0000
79+++ django_project/wsgi.py 2013-09-03 08:55:21 +0000
80@@ -10,6 +10,6 @@
81 Longer-term, we should bring our own wsgi app from configs into
82 the dev environment.
83 """
84-from django.core.wsgi import get_wsgi_application
85+from webcatalog.wsgi import make_app
86
87-application = get_wsgi_application()
88+application = make_app()
89
90=== modified file 'src/webcatalog/management/commands/import_sca_apps.py'
91--- src/webcatalog/management/commands/import_sca_apps.py 2013-08-29 10:38:27 +0000
92+++ src/webcatalog/management/commands/import_sca_apps.py 2013-09-03 08:55:21 +0000
93@@ -43,6 +43,29 @@
94 __all__ = []
95
96
97+logging_dict = {
98+ 'version': 1,
99+ 'disable_existing_loggers': True,
100+ 'handlers': {
101+ 'importer_handler': {
102+ 'class': 'logging.StreamHandler',
103+ 'level': 'INFO',
104+ 'formatter': 'simple',
105+ },
106+ },
107+ 'root': {
108+ 'handlers': ['importer_handler'],
109+ 'level': 'INFO',
110+ },
111+ 'formatters': {
112+ 'simple': {
113+ 'format': settings.LOG_FORMAT,
114+ }
115+ }
116+}
117+
118+
119+logging.config.dictConfig(logging_dict)
120 logger = logging.getLogger(__name__)
121
122
123
124=== modified file 'src/webcatalog/schema.py'
125--- src/webcatalog/schema.py 2013-08-27 14:53:21 +0000
126+++ src/webcatalog/schema.py 2013-09-03 08:55:21 +0000
127@@ -75,6 +75,10 @@
128 'software-center.ubuntu.com',
129 'myapps.developer.ubuntu.com',
130 ])
131+ webapp_log_path = schema.StringOption(
132+ default='./tmp/webapp.log')
133+ log_format = schema.StringOption(
134+ default='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
135
136 class google(schema.Section):
137 google_analytics_id = schema.StringOption()
138
139=== modified file 'src/webcatalog/wsgi.py'
140--- src/webcatalog/wsgi.py 2013-08-27 14:53:21 +0000
141+++ src/webcatalog/wsgi.py 2013-09-03 08:55:21 +0000
142@@ -17,6 +17,7 @@
143
144 """wsgi stack for the Apps Directory."""
145
146+import logging.config
147 import oops_dictconfig
148 import oops_wsgi
149 import oops_wsgi.django
150@@ -27,6 +28,37 @@
151 from django.core.handlers import wsgi
152
153
154+logging_dict = {
155+ 'version': 1,
156+ 'disable_existing_loggers': True,
157+ 'handlers': {
158+ 'webapp': {
159+ 'class': 'logging.handlers.WatchedFileHandler',
160+ 'level': 'INFO',
161+ 'formatter': 'simple',
162+ 'filename': settings.WEBAPP_LOG_PATH,
163+ },
164+ },
165+ 'root': {
166+ 'handlers': ['webapp'],
167+ 'level': 'INFO',
168+ },
169+ 'loggers': {
170+ # Ensure django log messages are propagated to our
171+ # above root logger.
172+ 'django': {
173+ 'handlers': [],
174+ 'propagate': True,
175+ }
176+ },
177+ 'formatters': {
178+ 'simple': {
179+ 'format': settings.LOG_FORMAT,
180+ }
181+ }
182+}
183+
184+
185 class EagerOOPSWSGIHandler(wsgi.WSGIHandler):
186 """Pre-generates an oops id on exception."""
187 def handle_uncaught_exception(self, request, resolver, exc_info):
188@@ -62,6 +94,10 @@
189 if settings.SHOULD_SERVE_HTTPS:
190 os.environ['HTTPS'] = 'on'
191
192+ # We configure logging here rather than the default settings
193+ # because not every process using ./manage.py has file perms
194+ # for the webapp log.
195+ logging.config.dictConfig(logging_dict)
196 non_oops_app = EagerOOPSWSGIHandler()
197
198 config = oops_dictconfig.config_from_dict(settings.OOPSES)

Subscribers

People subscribed via source and target branches