Logging Effectively in Python

When working on any coding project observability is so important for faster debugging & less time running around in loop trying to solve problems.

Please note the basis of this code was copied from
https://towardsdatascience.com/the-reusable-python-logging-template-for-all-your-data-science-apps-551697c8540

Here is a one file implementation of a logging module for python projects.

Logging Module

python
# logging.py
import logging
import os

APP_LOGGER_NAME = 'dd'
 
def setup_applevel_logger(logger_name = APP_LOGGER_NAME, file_name=None): 
    logger = logging.getLogger(logger_name).getChild(__name__)
    logger.setLevel(logging.DEBUG)
    # formatter = logging.Formatter(f"%(asctime)s - {__name__} - %(levelname)s - %(message)s")
    formatter = logging.Formatter(f"%(asctime)s - {APP_LOGGER_NAME}:%(filename)-12s: - %(levelname)s - %(message)s")
    sh = logging.StreamHandler(sys.stdout)
    sh.setFormatter(formatter)
    logger.handlers.clear()
    logger.addHandler(sh)

    ## error file handler
    error_fh = logging.FileHandler('logs/error.log')
    error_fh.setFormatter(formatter)
    error_fh.setLevel(logging.ERROR)
    logger.addHandler(error_fh)

    if file_name:
        fh = logging.FileHandler(file_name)
        fh.setFormatter(formatter)
        logger.addHandler(fh)
    return logger

def get_logger(module_name):
    log = setup_applevel_logger(file_name = 'logs/test.log') 
    return logging.getLogger(APP_LOGGER_NAME).getChild(module_name)



def test_logger():
    # log = setup_applevel_logger(file_name = 'logs/test.log')
    log = get_logger(__name__)

    log.debug('Calling module function.')
    multiply(5, 2)
    log.debug('Finished.')

    log.warning('This is a warning')
    log.error('This is an error')
    log.debug('This is an debug')
    log.info('This is an info')
    try: 
        lst = ['dummyvalue']
        second_value = lst[2]
    except Exception as ex:
        log.exception("Exception occured")
        log.critical("Exception occured")

# test_logger()

Simply add the above file and then import where it is needed like

Implementation

log = setup_applevel_logger(file_name='logs/runtime.log')
log.info("Info Message")

Just remember to add the path to the logging file. Make sure the logs folder is already created, then the file will automatically be saved there. Assume that it will be saved from the entry point of the python application.