forked from mattmakai/fullstackpython.com
-
Notifications
You must be signed in to change notification settings - Fork 0
/
logging.html
317 lines (316 loc) · 22.7 KB
/
logging.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="author" content="Matt Makai">
<meta name="description" content="Logging saves error, warning and event output to persistent storage for debugging purposes. Learn more about logging on Full Stack Python.">
<link rel="shortcut icon" href="//static.fullstackpython.com/fsp-fav.png">
<title>Logging - Full Stack Python</title>
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
<script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
<![endif]-->
<link href="theme/css/f.min.css" rel="stylesheet">
</head>
<body>
<div style="padding: 0 0 20px 0; margin: 0 0 20px 0; background-color: #22B24C;">
<div class="container">
<p class="banner"><a href="https://www.gumroad.com/l/python-deployments" style="color: #fff">The Full Stack Python Guide to Deployments December book update just released</a>!</p>
</div>
</div> <a href="https://github.com/makaimc/fullstackpython.com"><img style="position: absolute; top: 0; right: 0; border: 0;" src="/theme/img/fork.png" alt="Fork me on GitHub"></a>
<div class="container">
<div class="row">
<div class="col-md-12">
<div class="logo-header-section">
<a href="/" style="text-decoration: none; border: none;"><img src="theme/img/logo.png" height="52" width="52" class="logo-image" style="padding-top: 1px;" alt="Full Stack Python logo"></a>
<span class="logo-title"><a href="/">Full Stack Python</a></span>
</div>
</div>
</div><div class="row">
<div class="col-md-8">
<h1>Logging</h1>
<p>Logging saves output such as errors, warnings and event information to
persistent storage for debugging purposes. </p>
<h2>Why is logging important?</h2>
<p>Runtime exceptions that prevent code from running are important to log to
investigate and fix the source of the problems. Informational and debugging
logging also helps to understand how the application is performing even if
code is working as intended.</p>
<h2>Logging levels</h2>
<p>Logging is often grouped into several categories:</p>
<ol>
<li>Information</li>
<li>Debug</li>
<li>Warning</li>
<li>Error</li>
</ol>
<p>Logging errors that occur while a web framework is running is crucial to
understanding how your application is performing. </p>
<h2>Logging aggregators</h2>
<p>When you are running your application on several servers, it is helpful
to have a monitoring tool called a "logging aggregator". You can configure
your application to forward your system and application logs to one location
that provides tools for viewing, searching, and monitoring logging events
across your cluster. </p>
<p>Another advantage of log aggregation tools is they allow you to set up
custom alerts and alarms so you can get notified when error rates breach a
certain threshold.</p>
<h3>Open source log aggregators</h3>
<ul>
<li>
<p><a href="https://github.com/getsentry/sentry">Sentry</a> started as a Django-only
exception handling service but now has separate logging clients to cover
almost all major languages and frameworks. It still works really well for
Python-powered web applications and is often used in conjunction with other
monitoring tools. <a href="http://raven.readthedocs.org/en/latest/">Raven</a> is open
source Python client for Sentry.</p>
</li>
<li>
<p><a href="http://graylog2.org/">Graylog2</a> provides a central server for log
aggregation as well as a GUI for browsing and searching through log events.
There are libraries for most major languages, including python. Saves data
in Elasticache.</p>
</li>
<li>
<p><a href="http://logstash.net/">Logstash</a> Similar to Graylog2, logstash offers
features to programmatically configure log data workflows.</p>
</li>
<li>
<p><a href="https://github.com/facebook/scribe">Scribe</a> A project written by Facebook
to aggregate logs. It's designed to run on multiple servers and scale with
the rest of your cluster. Uses the Thrift messaging format so it can be
used with any language. </p>
</li>
</ul>
<h3>Hosted logging services</h3>
<ul>
<li>
<p><a href="https://www.loggly.com/">Loggly</a> is a third party cloud based
application that aggregates logs. They have instructions for every major
language, including python. It includes email alerting on custom searches. </p>
</li>
<li>
<p><a href="http://www.splunk.com/">Splunk</a> offers third party cloud and self
hosted solutions for event aggregation. It excels at searching and data
mining any text based data. </p>
</li>
<li>
<p><a href="https://papertrailapp.com/">Papertrail</a> is similar to both
Loggly and Splunk and provides integration with S3 for long term storage.</p>
</li>
<li>
<p><a href="http://raygun.io/">Raygun</a> logs errors and provides immediate notification
when issues arise.</p>
</li>
<li>
<p><a href="https://www.scalyr.com/">Scalyr</a> provides log aggregation, dashboards,
alerts and search in a user interface on top of standard logs.</p>
</li>
<li>
<p>There is a <a href="https://www.getsentry.com/welcome/">hosted version of Sentry</a>
in case you do not have the time to set up the open source project yourself.</p>
</li>
</ul>
<h2>Logging resources</h2>
<ul>
<li>
<p>This
<a href="http://www.blog.pythonlibrary.org/2012/08/02/python-101-an-intro-to-logging/">intro to logging</a>
presents the Python logging module and how to use it.</p>
</li>
<li>
<p><a href="http://www.hybridcluster.com/blog/logging-storytelling/">Logging as Storytelling</a>
is a multi-part series working the analogy that logs should read like
a story so you can better understand what's taking place in your web
application.
<a href="http://www.hybridcluster.com/blog/logging-storytelling-lets-add-action/">Part 2 describes actions</a>
and
<a href="http://www.hybridcluster.com/blog/logging-storytelling-3-types/">part 3 talks about types</a>.</p>
</li>
<li>
<p><a href="https://lukasa.co.uk/2014/05/A_Brief_Digression_About_Logging/">A Brief Digression About Logging</a>
is a short post that gets Python logging up and running quickly.</p>
</li>
<li>
<p><a href="https://hynek.me/articles/taking-some-pain-out-of-python-logging/">Taking the pain out of Python logging</a>
shows a logging set up with uWSGI.</p>
</li>
<li>
<p>Django's 1.3 release brought unified logging into project configurations.
This <a href="http://www.djm.org.uk/how-to-log-file-django-13-and-above/">post shows how to set up logging</a>
in a project's settings.py file. Caktus Group also has a nice tutorial on
<a href="http://www.caktusgroup.com/blog/2013/09/18/central-logging-django-graylog2-and-graypy/">central logging with graypy and Graylog2</a>.</p>
</li>
<li>
<p><a href="http://www.caktusgroup.com/blog/2015/01/27/Django-Logging-Configuration-logging_config-default-settings-logger/">Django Logging Configuration: How the Default Settings Interfere with Yours</a>
explains a problem with the default Django logging configuration and what
to do about in your project.</p>
</li>
<li>
<p><a href="https://www.loggly.com/blog/exceptional-logging-of-exceptions-in-python/">Exceptional Logging of Exceptions in Python</a>
shows how to log errors more accurately to pinpoint the problem instead of
receiving generic exceptions in your logs.</p>
</li>
</ul>
<h2>Logging learning checklist</h2>
<ol>
<li>
<p>Read how to integrate logging into your web application framework. </p>
</li>
<li>
<p>Ensure errors and anomalous results are logged. While these logs can be
stored in <a href="/monitoring.html">monitoring</a> solutions, it's best to have your
own log storage location to debug issues as they arise to complement other
monitoring systems.</p>
</li>
<li>
<p>Integrate logging for system events you may need to use for debugging
purposes later. For example, you may want to know the return values on
functions when they are above a certain threshold. </p>
</li>
</ol>
<h3>What's next after setting up logging for your app?</h3>
<div class="row">
<div class="col-md-4">
<div class="well select-next">
<a href="/monitoring.html" class="btn btn-success btn-full"><svg width="36" height="30" viewBox="0 0 1792 1792" xmlns="http://www.w3.org/2000/svg"><path d="M512 896v512h-256v-512h256zm384-512v1024h-256v-1024h256zm1024 1152v128h-2048v-1536h128v1408h1920zm-640-896v768h-256v-768h256zm384-384v1152h-256v-1152h256z" fill="#fff"/></svg></a>
<p class="under-btn">What tools exist for monitoring a deployed web app?</p> </div>
</div>
<div class="col-md-4">
<div class="well select-next">
<a href="/web-analytics.html" class="btn btn-success btn-full"><svg width="28" height="30" viewBox="0 0 1792 1792" xmlns="http://www.w3.org/2000/svg"><path d="M384 1152q0-53-37.5-90.5t-90.5-37.5-90.5 37.5-37.5 90.5 37.5 90.5 90.5 37.5 90.5-37.5 37.5-90.5zm192-448q0-53-37.5-90.5t-90.5-37.5-90.5 37.5-37.5 90.5 37.5 90.5 90.5 37.5 90.5-37.5 37.5-90.5zm428 481l101-382q6-26-7.5-48.5t-38.5-29.5-48 6.5-30 39.5l-101 382q-60 5-107 43.5t-63 98.5q-20 77 20 146t117 89 146-20 89-117q16-60-6-117t-72-91zm660-33q0-53-37.5-90.5t-90.5-37.5-90.5 37.5-37.5 90.5 37.5 90.5 90.5 37.5 90.5-37.5 37.5-90.5zm-640-640q0-53-37.5-90.5t-90.5-37.5-90.5 37.5-37.5 90.5 37.5 90.5 90.5 37.5 90.5-37.5 37.5-90.5zm448 192q0-53-37.5-90.5t-90.5-37.5-90.5 37.5-37.5 90.5 37.5 90.5 90.5 37.5 90.5-37.5 37.5-90.5zm320 448q0 261-141 483-19 29-54 29h-1402q-35 0-54-29-141-221-141-483 0-182 71-348t191-286 286-191 348-71 348 71 286 191 191 286 71 348z" fill="#fff"/></svg></a>
<p class="under-btn">I want to learn more about app users via web analytics.</p> </div>
</div>
<div class="col-md-4">
<div class="well select-next">
<a href="/web-application-security.html" class="btn btn-success btn-full"><svg width="28" height="30" viewBox="0 0 1792 1792" xmlns="http://www.w3.org/2000/svg"><path d="M640 768h512v-192q0-106-75-181t-181-75-181 75-75 181v192zm832 96v576q0 40-28 68t-68 28h-960q-40 0-68-28t-28-68v-576q0-40 28-68t68-28h32v-192q0-184 132-316t316-132 316 132 132 316v192h32q40 0 68 28t28 68z" fill="#fff"/></svg></a>
<p class="under-btn">What can I do to mitigate security vulnerability in my web app?</p> </div>
</div>
</div><div id="mc_embed_signup">
<form action="//mattmakai.us2.list-manage.com/subscribe/post?u=b7e774f0c4f05dcebbfee183d&id=b22335388d" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="validate" target="_blank" novalidate>
<div id="mc_embed_signup_scroll">
<h4>Sign up here to receive an email with major updates to this site and Python tutorials delivered to your inbox once a month.</h4>
<div class="row">
<div class="col-md-9">
<input type="email" value="" name="EMAIL" class="email form-control" id="mce-EMAIL" placeholder="email address" required>
<div style="position: absolute; left: -5000px;"><input type="text" name="b_b7e774f0c4f05dcebbfee183d_b22335388d" tabindex="-1" value=""></div>
</div>
<div class="col-md-3">
<div class="clear"><input type="submit" value="Subscribe" name="subscribe" id="mc-embedded-subscribe" class="btn btn-success" style="font-family: 'Helvetica Neue';"></div>
</div>
</div>
</div>
</form>
</div>
</div>
<div class="col-md-offset-1 col-md-3" id="sidebar">
<div class="panel panel-success">
<div class="panel-body">
<a href="http://www.deploypython.com/"><img src="//static.fullstackpython.com/fsp-deployment-guide.png" alt="The Full Stack Python Guide to Deployments" width="100%"></a>
<p style="font-size: .8em; margin-top: 10px;">Searching for a complete, step-by-step deployment walkthrough? Learn more about <a href="http://www.deploypython.com/">The Full Stack Python Guide to Deployments book</a>.
</p>
</div>
</div> <div class="panel panel-success">
<div class="panel-heading">
<h3 class="panel-head"><a href="/table-of-contents.html" style="color: #fff;">Table of Contents</a></h3>
</div>
<div class="list-group">
<a href="/introduction.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Introduction</a>
<a href="/learning-programming.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Learning Programming</a>
<a href="/why-use-python.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Why Use Python?</a>
<a href="/python-2-or-3.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Python 2 or 3?</a>
<a href="/enterprise-python.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Enterprise Python</a>
<a href="/best-python-resources.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Best Python Resources</a>
<a href="/best-python-videos.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Best Python Videos</a>
<a href="/development-environments.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Development Environments</a>
<a href="/vim.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Vim</a>
<a href="/emacs.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Emacs</a>
<a href="/python-programming-language.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Python Programming Language</a>
<a href="/generators.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Generators</a>
<a href="/comprehensions.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Comprehensions</a>
<a href="/web-development.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Development</a>
<a href="/web-frameworks.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Frameworks</a>
<a href="/django.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Django</a>
<a href="/flask.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Flask</a>
<a href="/bottle.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Bottle</a>
<a href="/pyramid.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Pyramid</a>
<a href="/morepath.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Morepath</a>
<a href="/other-web-frameworks.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Other Web Frameworks</a>
<a href="/web-design.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Design</a>
<a href="/cascading-style-sheets.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Cascading Style Sheets</a>
<a href="/javascript.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>JavaScript</a>
<a href="/websockets.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>WebSockets</a>
<a href="/template-engines.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Template Engines</a>
<a href="/web-application-security.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Application Security</a>
<a href="/static-site-generator.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Static Site Generators</a>
<a href="/data.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Data</a>
<a href="/databases.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Databases</a>
<a href="/no-sql-datastore.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>NoSQL Data Stores</a>
<a href="/object-relational-mappers-orms.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Object-relational Mappers (ORMs)</a>
<a href="/application-programming-interfaces.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Application Programming Interfaces</a>
<a href="/api-integration.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>API Integration</a>
<a href="/api-creation.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>API Creation</a>
<a href="/deployment.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Deployment</a>
<a href="/servers.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Servers</a>
<a href="/platform-as-a-service.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Platform-as-a-service</a>
<a href="/operating-systems.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Operating Systems</a>
<a href="/web-servers.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Servers</a>
<a href="/wsgi-servers.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>WSGI Servers</a>
<a href="/source-control.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Source Control</a>
<a href="/application-dependencies.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Application Dependencies</a>
<a href="/static-content.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Static Content</a>
<a href="/task-queues.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Task Queues</a>
<a href="/configuration-management.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Configuration Management</a>
<a href="/continuous-integration.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Continuous Integration</a>
<a href="/logging.html" class="list-group-item smaller-item active" style='font-family: "Helvetica Neue",sans-serif;'>Logging</a>
<a href="/monitoring.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Monitoring</a>
<a href="/web-analytics.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Analytics</a>
<a href="/docker.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Docker</a>
<a href="/caching.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Caching</a>
<a href="/microservices.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Microservices</a>
<a href="/testing.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Testing</a>
<a href="/unit-testing.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Unit Testing</a>
<a href="/integration-testing.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Integration Testing</a>
<a href="/code-metrics.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Code Metrics</a>
<a href="/debugging.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Debugging</a>
<a href="/what-full-stack-means.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>What "Full Stack" Means</a>
<a href="/change-log.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Change Log</a>
<a href="/future-directions.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Future Directions</a>
<a href="/about-author.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>About the Author</a>
</div>
</div>
<div class="panel panel-success">
<div class="panel-heading"><h3 class="panel-head">Logging</h3></div>
<div class="panel-body">
Major updates are tweeted via
<a href="https://twitter.com/fullstackpython">@fullstackpython</a>.
<hr/>
Need more detailed tutorials than you see here?
<a href="http://www.deploypython.com/">Learn more about The Full Stack Python Guide to Deployments book.</a>
</div>
</div>
</div></div>
<hr/>
</div>
<div class="container">
<div class="footer pull-right">
<a href="http://www.mattmakai.com/" class="underline">Matt Makai</a>
2015
</div>
</div>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-19910497-7', 'auto');
ga('send', 'pageview');
</script>
<script type='text/javascript'>
var trackOutboundLink = function(url) { ga('send', 'event', 'outbound', 'click', url, {'hitCallback': function () { document.location = url; } }); }
</script>
</body>
</html>