Ubuntu Apps Directory

Merge lp:~michael.nelson/ubuntu-webcatalog/load-logging-config-in-wsgi into lp:ubuntu-webcatalog

  1. load-logging-config-in-wsgi
  2. Merge into trunk
Proposed by Michael Nelson
Status: Merged
Approved by: Łukasz Czyżykowski
Approved revision: 189
Merged at revision: 188
Proposed branch: lp:~michael.nelson/ubuntu-webcatalog/load-logging-config-in-wsgi
Merge into: lp:ubuntu-webcatalog
Diff against target: 198 lines (+66/-52)
5 files modified
django_project/config/main.cfg (+1/-50)
django_project/wsgi.py (+2/-2)
src/webcatalog/management/commands/import_sca_apps.py (+23/-0)
src/webcatalog/schema.py (+4/-0)
src/webcatalog/wsgi.py (+36/-0)
To merge this branch: bzr merge lp:~michael.nelson/ubuntu-webcatalog/load-logging-config-in-wsgi
Reviewer Review Type Date Requested Status
Łukasz Czyżykowski (community) Approve
Review via email: mp+183586@code.launchpad.net

Commit message

Separate logging configs for webapp and import command.

Description of the change

Related to RT 61147 - providing a log file for the import_sca_apps job that can be monitored.

After switching to a dictConfig and adding a separate logger for the import command, the deploy to staging errored because every cron/setup/whatever script which would use ./manage.py would automatically try to touch the webapp.log file (as Django loads the logging config directly after settings are loaded), and doesn't have perms on the filesystem.

This branch separates the logging configs and only initialises them when they are used (ie. in this case when the webapp is started).

As the "dictConfig" defined in a configglue ini file was getting hard to follow, rather than adding a separate dict config there, I've moved them into a more readable (ie. normal) dict where they're used, and just used settings for the relevant pieces we'd want to change (format, file location).

Finally, and strangely, I had to specify a django logger (with no handlers) just to ensure that the msgs were propagated to the root logger.

To post a comment you must log in.
Revision history for this message
Łukasz Czyżykowski (lukasz-czyzykowski) wrote :

I like the idea of pulling only the relevant config bits into configglue file.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'django_project/config/main.cfg'
--- django_project/config/main.cfg 2013-08-29 10:38:27 +0000
+++ django_project/config/main.cfg 2013-09-03 08:55:21 +0000
@@ -6,7 +6,6 @@
6 django.contrib.auth.backends.ModelBackend6 django.contrib.auth.backends.ModelBackend
77
8databases = django_databases8databases = django_databases
9logging = django_logging
109
11debug = true10debug = true
12media_root = django_project/media_root_dev/11media_root = django_project/media_root_dev/
@@ -73,55 +72,6 @@
73engine = django.db.backends.sqlite372engine = django.db.backends.sqlite3
74name = webcatalog.db73name = webcatalog.db
7574
76[django_logging]
77version = 1
78disable_existing_loggers = True
79handlers = django_logging_handlers
80loggers = django_logging_loggers
81root = django_logging_root
82formatters = django_logging_formatters
83
84[django_logging_root]
85level = WARNING
86
87[django_logging_loggers]
88django = django_logger
89webcatalog.management.commands.import_sca_apps = import_sca_logger
90
91[django_logger]
92handlers = webapp
93level = INFO
94
95[django_logging_handlers]
96webapp = webapp_logging_handler
97importer = import_sca_apps_handler
98
99[webapp_logging_handler]
100class = logging.handlers.WatchedFileHandler
101level = INFO
102formatter = simple
103filename = ./tmp/webapp.log
104
105[import_sca_apps_handler]
106class = logging.StreamHandler
107level = INFO
108formatter = simple_no_name
109
110[import_sca_logger]
111handlers = importer
112propagate = false
113level = INFO
114
115[django_logging_formatters]
116simple = django_logging_simple_format
117simple_no_name = django_logging_simple_no_name_format
118
119[django_logging_simple_format]
120format = %%(asctime)s - %%(name)s - %%(levelname)s - %%(message)s
121
122[django_logging_simple_no_name_format]
123format = %%(asctime)s - %%(levelname)s - %%(message)s
124
125[openid]75[openid]
126openid_sso_server_url = https://login.staging.ubuntu.com76openid_sso_server_url = https://login.staging.ubuntu.com
127openid_create_users = true77openid_create_users = true
@@ -142,6 +92,7 @@
142default_distro = natty92default_distro = natty
143convoy_root = ./src/webcatalog/static/93convoy_root = ./src/webcatalog/static/
144ubuntu_series_for_versions = series_for_versions94ubuntu_series_for_versions = series_for_versions
95webapp_log_path = ./tmp/webapp.log
14596
146[series_for_versions]97[series_for_versions]
14710.04 = lucid9810.04 = lucid
14899
=== modified file 'django_project/wsgi.py'
--- django_project/wsgi.py 2013-04-23 08:05:18 +0000
+++ django_project/wsgi.py 2013-09-03 08:55:21 +0000
@@ -10,6 +10,6 @@
10Longer-term, we should bring our own wsgi app from configs into10Longer-term, we should bring our own wsgi app from configs into
11the dev environment.11the dev environment.
12"""12"""
13from django.core.wsgi import get_wsgi_application13from webcatalog.wsgi import make_app
1414
15application = get_wsgi_application()15application = make_app()
1616
=== modified file 'src/webcatalog/management/commands/import_sca_apps.py'
--- src/webcatalog/management/commands/import_sca_apps.py 2013-08-29 10:38:27 +0000
+++ src/webcatalog/management/commands/import_sca_apps.py 2013-09-03 08:55:21 +0000
@@ -43,6 +43,29 @@
43__all__ = []43__all__ = []
4444
4545
46logging_dict = {
47 'version': 1,
48 'disable_existing_loggers': True,
49 'handlers': {
50 'importer_handler': {
51 'class': 'logging.StreamHandler',
52 'level': 'INFO',
53 'formatter': 'simple',
54 },
55 },
56 'root': {
57 'handlers': ['importer_handler'],
58 'level': 'INFO',
59 },
60 'formatters': {
61 'simple': {
62 'format': settings.LOG_FORMAT,
63 }
64 }
65}
66
67
68logging.config.dictConfig(logging_dict)
46logger = logging.getLogger(__name__)69logger = logging.getLogger(__name__)
4770
4871
4972
=== modified file 'src/webcatalog/schema.py'
--- src/webcatalog/schema.py 2013-08-27 14:53:21 +0000
+++ src/webcatalog/schema.py 2013-09-03 08:55:21 +0000
@@ -75,6 +75,10 @@
75 'software-center.ubuntu.com',75 'software-center.ubuntu.com',
76 'myapps.developer.ubuntu.com',76 'myapps.developer.ubuntu.com',
77 ])77 ])
78 webapp_log_path = schema.StringOption(
79 default='./tmp/webapp.log')
80 log_format = schema.StringOption(
81 default='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
7882
79 class google(schema.Section):83 class google(schema.Section):
80 google_analytics_id = schema.StringOption()84 google_analytics_id = schema.StringOption()
8185
=== modified file 'src/webcatalog/wsgi.py'
--- src/webcatalog/wsgi.py 2013-08-27 14:53:21 +0000
+++ src/webcatalog/wsgi.py 2013-09-03 08:55:21 +0000
@@ -17,6 +17,7 @@
1717
18"""wsgi stack for the Apps Directory."""18"""wsgi stack for the Apps Directory."""
1919
20import logging.config
20import oops_dictconfig21import oops_dictconfig
21import oops_wsgi22import oops_wsgi
22import oops_wsgi.django23import oops_wsgi.django
@@ -27,6 +28,37 @@
27from django.core.handlers import wsgi28from django.core.handlers import wsgi
2829
2930
31logging_dict = {
32 'version': 1,
33 'disable_existing_loggers': True,
34 'handlers': {
35 'webapp': {
36 'class': 'logging.handlers.WatchedFileHandler',
37 'level': 'INFO',
38 'formatter': 'simple',
39 'filename': settings.WEBAPP_LOG_PATH,
40 },
41 },
42 'root': {
43 'handlers': ['webapp'],
44 'level': 'INFO',
45 },
46 'loggers': {
47 # Ensure django log messages are propagated to our
48 # above root logger.
49 'django': {
50 'handlers': [],
51 'propagate': True,
52 }
53 },
54 'formatters': {
55 'simple': {
56 'format': settings.LOG_FORMAT,
57 }
58 }
59}
60
61
30class EagerOOPSWSGIHandler(wsgi.WSGIHandler):62class EagerOOPSWSGIHandler(wsgi.WSGIHandler):
31 """Pre-generates an oops id on exception."""63 """Pre-generates an oops id on exception."""
32 def handle_uncaught_exception(self, request, resolver, exc_info):64 def handle_uncaught_exception(self, request, resolver, exc_info):
@@ -62,6 +94,10 @@
62 if settings.SHOULD_SERVE_HTTPS:94 if settings.SHOULD_SERVE_HTTPS:
63 os.environ['HTTPS'] = 'on'95 os.environ['HTTPS'] = 'on'
6496
97 # We configure logging here rather than the default settings
98 # because not every process using ./manage.py has file perms
99 # for the webapp log.
100 logging.config.dictConfig(logging_dict)
65 non_oops_app = EagerOOPSWSGIHandler()101 non_oops_app = EagerOOPSWSGIHandler()
66102
67 config = oops_dictconfig.config_from_dict(settings.OOPSES)103 config = oops_dictconfig.config_from_dict(settings.OOPSES)

Subscribers

People subscribed via source and target branches