Logging contextual data using LoggerAdapter in Python

Skill - Logging contextual data using LoggerAdapter in Python

Table of Contents

Skills Required

• Setup python development environment
• Basic Printing in Python
• Managing Variables in python
• Logging in Python

Please make sure to have all the skills mentioned above to understand and execute the code mentioned below. Go through the above skills if necessary for reference or revision

---

In this post we will learn how to add extra contextual data to logs using LoggerAdapter in python

Use cases

• Adding contextual data in logs may be useful for easy debugging of the logs and also add new attributes for efficient querying and filtering of the logs
• For example if additional attributes like process Id, application name, etc. are added in each log, searching and debugging logs can be more efficient

Add context in a single log using “extra”

import logging
import os

logging.basicConfig(level=logging.INFO,
                    format="%(asctime)s - %(pid)s - %(org_name)s - %(levelname)s - %(message)s")
logger = logging.getLogger("root")

logger.info("Hello World!!!", extra={"org_name": "Acme", "pid": os.getpid()})

• In the above example, the log format is configured to show additional attributes named pid, org_name along with the message
• The additional attributes are supplied at the time of logging as a dictionary using the extra argument

Add context data in all logs using “LoggerAdapter”

• Using the extra input argument for generating each log is susceptible to human errors
• So we can use a LoggerAdapter to create a logger that can add context information to all the logs by default

import logging
from logging import LoggerAdapter, StreamHandler

# create a logger object
logger = logging.getLogger("root")
logger.setLevel(logging.INFO)

# create a log handler, set the log format and add to the logger object
consoleHandler = StreamHandler()
fmt = "%(asctime)s - %(org_name)s - %(levelname)s - %(message)s"
consoleHandler.setFormatter(logging.Formatter(fmt))
logger.addHandler(consoleHandler)

# create a logger adapter with the logger object
loggerAdapter = LoggerAdapter(logger, extra={"org_name": "Acme"})

# generate logs using the logger adapter
loggerAdapter.info("Hello World!!!")

• In the above example, the context is configured once and need not be mentioned explicitly in each log
• This can help to generate logs with context in a clean and less error prone way

Global logger adapter for usage across multiple files

• A practical python application can contain more than one file and logs can be generated in more than one python file

## app_logger.py
import logging
from logging import LoggerAdapter

class AppLogger:
    __instance: LoggerAdapter = None

    @staticmethod
    def getInstance():
        """ Static access method. """
        if AppLogger.__instance == None:
            AppLogger.initLogger()
        return AppLogger.__instance

    @staticmethod
    def initLogger():
        # get a named global logger
        appLogger = logging.getLogger("root")
        appLogger.setLevel(logging.INFO)

        # configure console logging
        streamHandler = logging.StreamHandler()
        fmt = "%(asctime)s - %(org_name)s - %(levelname)s - %(message)s"
        streamHandler.setFormatter(logging.Formatter(fmt))
        appLogger.addHandler(streamHandler)

        # setup the static variable
        AppLogger.__instance = LoggerAdapter(
            appLogger, extra={"org_name": "Acme"})

# index.py
from app_logger import AppLogger

logger = AppLogger.getInstance()
logger.info("Hello World!!!")

• In the above example, a python file named app_logger.py exposes a class named AppLogger that maintains a global logger adapter instance that can be accessed from multiple python files just by calling AppLogger.getInstance()
• This AppLogger class uses static methods and static variables for maintaining a single global logger adapter object

Video

You can see the video for this post here

---

References

• logging cookbook - https://docs.python.org/3/howto/logging-cookbook.html#using-loggeradapters-to-impart-contextual-information
• Official documentation - https://docs.python.org/3/library/logging.html

Table of Contents