Error Tracker¶

Error Tracker is a python app plugins for Flask and Django, that provides many of the essentials features of system exceptions tracking.

Features¶

Mask all the variables, including dict keys, HTTP request body which contain password and secret in their name.

Recorded exceptions will be visible to the configured path

Send notification on failures

Record exceptions with frame data, including local and global variables

Raise bugs or update ticket in Bug tracking systems.

Provide customization for notification, context building, ticketing systems and more

Exception Listing

Detailed Exception

Quick start¶

Installation¶

To install Error Tracker, open an interactive shell and run:

pip install error-tracker

Error Tracker can be used with

Standalone Python application
Flask Application
Django Application

Using Error Tracker as simple as plugging any other module.

Recording exception/error¶

An error/exception can be recorded using decorator or function call.

To record the error using decorator, decorate a function with track_exception or auto_track_exception
Where as to record error using function call use record_exception function.

All the data will be stored in the configured data store and these data will be available at configure URL path.

Flask App setup¶

An instance of AppErrorTracker needs to be created and have to be configured with the correct data. Monitoring feature can be configured either using object based configuration or app-based configuration, the only important thing here is we should have all the required key configs in the app.config otherwise it will fail.

For object based configuration add settings.py

...
APP_ERROR_SEND_NOTIFICATION = True
APP_ERROR_RECIPIENT_EMAIL = ('example@example.com',)
APP_ERROR_SUBJECT_PREFIX = "Server Error"
APP_ERROR_EMAIL_SENDER = 'user@example.com'

app.py

from flask import Flask
from flask_mail import Mail
import settings
from error_tracker import AppErrorTracker, NotificationMixin
from flask_sqlalchemy import SQLAlchemy
...
app = Flask(__name__)
app.config.from_object(settings)
db = SQLAlchemy(app)
class Notifier(Mail, NotificationMixin):
    def notify(self, request, exception,
               email_subject=None,
               email_body=None,
               from_email=None,
               recipient_list=None):
        message = Message(email_subject, recipient_list, email_body, sender=from_email)
        self.send(message)
mailer = Notifier(app=app)
error_tracker = AppErrorTracker(app=app, db=db, notifier=mailer)

....

....
# Record exception when 404 error code is raised
@app.errorhandler(403)
def error_403(e):
    error_tracker.record_exception()
    # any custom logic

# Record error using decorator
@app.errorhandler(500)
@error_tracker.track_exception
def error_500(e):
    # some custom logic
....

Here, app, db and notifier parameters are optional. Alternatively, you could use the init_app() method.

If you start this application and navigate to http://localhost:5000/dev/error, you should see an empty page.

Django App Setup¶

We need to update settings.py file as

Add app error_tracker.DjangoErrorTracker to installed apps list
Add Middleware error_tracker.django.middleware.ExceptionTrackerMiddleWare for exception tracking [1].
Other configs related to notification
Add URLs to the list of URL patterns

[1]	This should be added at the end so that it can process exception 1st in the middleware call stack.

...
APP_ERROR_RECIPIENT_EMAIL = ('example@example.com',)
APP_ERROR_SUBJECT_PREFIX = "Server Error"
APP_ERROR_EMAIL_SENDER = 'user@example.com'

INSTALLED_APPS = [
    ...
    'error_tracker.DjangoErrorTracker'
]
MIDDLEWARE = [
    ...
    'error_tracker.django.middleware.ExceptionTrackerMiddleWare'
]

We need to add URLs to the urls.py so that we can browse the default pages provided by Error Tracker

from error_tracker.django import urls

urlpatterns = [
    ...
    url("dev/", include(urls)),
]

Using With Python App (NO WEB SERVER)¶

Choose either of the preferred framework, flask or Django and configure the app as per their specifications. For example, if we want to use Flask then do

Flask App
- Create Flask App instance
- Create Error Tracker app instance
- DO NOT call run method of Flask app instance
- To track exception call error_tracker.record_exception method
Django App
- Create Django App with settings and all configuration
- Set environment variable DJANGO_SETTINGS_MODULE
- call django.setup()
- from error_tracker.django.middleware import error_tracker
- To track exception do error_tracker.record_exception(None, exception)

Flask App Usage¶

Lazy initialization¶

Use error_tracker.init_app method to configure

error_tracker = AppErrorTracker()
...
error_tracker.init_app(app=app, db=db, notifier=notifier)

Config details¶

Enable or disable notification sending feature
APP_ERROR_SEND_NOTIFICATION = False
Email recipient list
APP_ERROR_RECIPIENT_EMAIL = None
Email subject prefix to be used by email sender
APP_ERROR_SUBJECT_PREFIX = ""
Mask value with following string
APP_ERROR_MASK_WITH = "**************"
Masking rule
App can mask all the variables whose lower case name contains one of the configured string .. code:
APP_ERROR_MASKED_KEY_HAS = ("password", "secret")
Above configuration will mask the variable names like
password secret PassWord THis_Is_SEcret
Note

Any variable names whose lower case string contains either password or secret
Browse link in your service app
List of exceptions can be seen at /dev/error, but you can have other prefix as well due to some securities or other reasons.
APP_ERROR_URL_PREFIX = "/dev/error"

Email address used to construct Message object

APP_ERROR_EMAIL_SENDER = "prod-issue@example.com"

Manual Exception Tracking¶

Error can be tracked programmatically using AppErrorTracker’s record_exception method.

error_tracker = AppErrorTracker(...)
...
try
    ...
catch Exception as e:
    error_tracker.record_exception()

Decorator based exception recording, record exception as it occurs in a method call.

Note

Exception will be re-raised so it must be caught in the caller or ignored.

error_tracker = AppErrorTracker(...)
@error_tracker.auto_track_exception
def fun():
    pass

Django App Settings¶

Error Tracker fits nicely with Django framework, error tracker can be configured in different ways. Multiple settings are available, these settings can be configured using settings file.

Setting details¶

Home page list size, display 10 exceptions per page

EXCEPTION_APP_DEFAULT_LIST_SIZE = 10

What all sensitive data should be masked

APP_ERROR_MASKED_KEY_HAS =  ("password", "secret")

Note

This means any variables whose name have either password or secret would be masked

Sensitive data masking value
APP_ERROR_MASK_WITH = '*************'

Exception email subject prefix

APP_ERROR_SUBJECT_PREFIX = get('APP_ERROR_SUBJECT_PREFIX', '')

Email sender’s email id

APP_ERROR_EMAIL_SENDER = "server@example.com"

Whom email should be sent in the case of failure

APP_ERROR_RECIPIENT_EMAIL = ('dev-group1@example.com', 'dev@example.com')

By default only 500 errors are tracked but HTTP 404, 401 etc can be tracked as well
TRACK_ALL_EXCEPTIONS = True

Note

Below configurations are required path to some class.

Custom Masking Module

APP_ERROR_MASKING_MODULE = "path to Masking class"

Ticketing/Bugging module
APP_ERROR_TICKETING_MODULE = "path to Ticketing class"
Note

Class must not have any constructor arguments

Notifier module

APP_ERROR_NOTIFICATION_MODULE = "path to Notification class"
Note

Class must not have any constructor arguments

Context Builder module
APP_ERROR_CONTEXT_BUILDER_MODULE = "path to ContextBuilder class"
Note

Class must not have any constructor arguments
Custom Model used for exceptions storage
APP_ERROR_DB_MODEL = "path to Model class"
Note

Class must implements all abstract methods

Manual Exception Tracking¶

Error can be tracked programmatically using ErrorTracker’s object available in middleware module. For tracking exception call error_tracker.record_exception method.

from error_tracker.django.middleware import error_tracker

...
try
    ...
catch Exception as e:
    error_tracker.record_exception(request, e)

Decorator based exception recording, record exception as it occurs in a method call.

Note

Exception will be re-raised so it must be caught in the caller or ignored.

from error_tracker.django.middleware import track_exception

@track_exception
def do_something():
    ...

Notification notify feature¶

Notifications are very useful in the case of failure, in different situations notification can be used to notify users using different channels like Slack, Email etc. Notification feature can be enabled by providing a NotificationMixin object.

from error_tracker import NotificationMixin
 class Notifier(NotificationMixin):
     def notify(self, request, exception,
                email_subject=None,
                email_body=None,
                from_email=None,
                recipient_list=None):
         # add logic here

Flask App Usage¶

error_tracker = AppErrorTracker(app=app, db=db, notifier=Notifier())

Django App Usage¶

settings.py

APP_ERROR_NOTIFICATION_MODULE = "path to Notifier class"

Ticketing¶

Ticketing interface can be used to create tickets in the systems like Jira, Bugzilla etc, ticketing can be enabled using ticketing interface.

Using TicketingMixin class

implement raise_ticket method of TicketingMixin interface

from error_tracker import TicketingMixin
class Ticketing(TicketingMixin):
    def raise_ticket(self, exception, request=None):
        # Put your logic here

Flask App Usage¶

app = Flask(__name__)
db = SQLAlchemy(app)
error_tracker = AppErrorTracker(app=app, db=db, ticketing=Ticketing() )
db.create_all()

Django App Usage¶

settings.py

APP_ERROR_TICKETING_MODULE = "path to Ticketing class"

Masking Rule¶

Masking is essential for any system so that sensitive information can’t be exposed in plain text form. Flask error monitor provides masking feature, that can be disabled or enabled.

Disable masking rule: set APP_ERROR_MASKED_KEY_HAS = ()
To set other mask rule add following lines

#Mask all the variables or dictionary keys which contains from one of the following tuple
APP_ERROR_MASKED_KEY_HAS = ( 'secret', 'card', 'credit', 'pass' )
#Replace value with  `###@@@@@###`
APP_ERROR_MASK_WITH = "###@@@@@###"

Note

Masking is performed for each variable like dict, list, set and all and it’s done based on the variable name
Masking is performed on the dictionary key as well as e.g. ImmutableMultiDict, QueryDict standard dict or any object whose super class is dict.

Custom masking rule using MaskingMixin

Note

implement __call__ method of MaskingMixin

from error_tracker import MaskingMixin
class MyMaskingRule(MaskingMixin):
    def __call__(self, key):
        # Put any logic
        # Do not mask return False, None
        # To mask return True, Value

Flask App Usage¶

error_tracker = AppErrorTracker(app=app, db=db,
                                 masking=MyMaskingRule("#########", ('pass', 'card') ) )

Django App Usage¶

settings.py

APP_ERROR_MASKING_MODULE="path to MyMaskingRule"
APP_ERROR_MASKED_KEY_HAS = ('pass', 'card')
APP_ERROR_MASKED_WITH = "############"

Custom Context Builder¶

Having more and more context about failure always help in debugging, by default this app captures HTTP headers, URL parameters, any post data. More data can be included like data-center name, server details and any other, by default these details are not captured. Nonetheless these details can be captured using ContextBuilderMixin. Error Tracker comes with two type of context builders DefaultFlaskContextBuilder and DefaultDjangoContextBuilder for Flask and Django respectively. We can either reuse the same context builder or customize them as per our need.

Using ContextBuilderMixin

Note

Implement get_context method of ContextBuilderMixin, default context builders capture request body, headers and URL parameters.

from error_tracker import ContextBuilderMixin
class ContextBuilder(ContextBuilderMixin):
     def get_context(self, request, masking=None):
          # return context dictionary

Flask App Usage¶

This custom context builder can be supplied as parameter of AppErrorTracker constructor.

error_tracker = AppErrorTracker(app=app, db=db,
                                context_builder=ContextBuilder())

Django App Usage¶

Add path of the custom builder class to the settings file, this class should not take any arguments for constructions.

settings.py

APP_ERROR_CONTEXT_BUILDER_MODULE = "path to ContextBuilder class"

Using Mongo or other data store¶

Using any data store as easy as implementing all the methods from ModelMixin

from error_tracker import ModelMixin
class CustomModel(ModelMixin):
     objects = {}

     @classmethod
     def delete_entity(cls, rhash):
         ...

     @classmethod
     def create_or_update_entity(cls, rhash, host, path, method, request_data, exception_name, traceback):
         ...

     @classmethod
     def get_exceptions_per_page(cls, page_number=1):
         ...

     @classmethod
     def get_entity(cls, rhash):
         ...

Flask App Usage¶

Create app with the specific model

error_tracker = AppErrorTracker(app=app, model=CustomModel)

Django App Usage¶

Add path to the model in settings file as

APP_ERROR_DB_MODEL = core.CustomModel