Intro to Python logging module - An alternative to print() function

 Logging is generally used to debug our Python code. Many of us (like me) have a habit of using the print() function extensively during the development of a program. Nothing bad in that per se, just that when we want to promote this program to the production environment we must necessarily remove the print() statements. 

There are other reasons too why the print statement is not recommended and its better to use the logging module:

• The print statement works only if we have access to the console
• We have no mechanism to write the print statements to a text file for further perusal

Python Logging Levels

There are six log levels in Python; each level is associated with an integer that indicates the log severity:.

Value	Level	Logging Function	Description
0	NOTSET	logging.notset()	By default, a new logger has the NOTSET level (root logger default is WARNING)
10	DEBUG	logging.debug()	Providing information to diagnosing problems.
20	INFO	logging.info()	Tracking the normal operation of a program.
30	WARNING	logging.warning()	Although the code is still working as expected, something unexpected is happening
40	ERROR	logging.error()	The code was unable to run some parts
50	CRITICAL	logging.critical()	The code cannot run.

Except for NOTSET, all the levels are rather straightforward (DEBUG < INFO < WARN ).

If you set your logging level to CRITICAL, only logging messages of the CRITICAL level will be shown. To set your logging level to CRITICAL, you can use

logging.basicConfig()

OR

logging.basicConfig(level=50)

Let us see an example of how logging works

This is our output:

DEBUG:root:This is DEGUG level of logging
INFO:root:Now we are at the INFO level of logging
WARNING:root:and next comes the WARNING level of logging
ERROR:root:now we are at the ERROR level of debugging
CRITICAL:root:OOPS!!! This is the CRITICAL level of logging!!!

Now as an experiment, change the logging level to WARNING and see how the code runs.

This is our output:

WARNING:root:and next comes the WARNING level of logging
ERROR:root:now we are at the ERROR level of debugging
CRITICAL:root:OOPS!!! This is the CRITICAL level of logging!!!

As you have seen, only messages with severity of WARNING or higher are displayed.

Formatting the logging messages

One advantage of using the logging module is the ability to format the message as we deem necessary. I like to format my messages with the Date Time stamp, logging level, name of the program and the actual message. So this is the line that I insert into my code:

logging.basicConfig(format='%(asctime)s - %(levelname)s - %(filename)s - %(message)s',level=logging.ERROR)

And this is the output that I receive:

2022-11-15 21:26:43,909 - ERROR - TestProgram.py - now we are at the ERROR level of debugging
2022-11-15 21:26:43,911 - CRITICAL - TestProgram.py - OOPS!!! This is the CRITICAL level of logging!!!

There are a lot of useful attributes that can be used for formatting . All of them are detailed in the LogRecord attributes section of logging documentation in the Python docs . A snapshot is given below for quick reference:

If you have noticed, filename is one of the attribute names. This is what I use as it helps me to store logs on any program which can be referenced back any time in future. I will dwell more on the filename latter on.

What we have used till now is the root logger. This is the default logger but we need to specify and create our own logger in production. If I call python program A from python program B, the use of root logger in both programs will lead to problems.( The root logger configuration takes place only once and so the log file will be created for only of the two programs.) Hence, as a rule of thumb,  it is recommended to create our own logger for each and every individual program.

Configuring our own logger

This is how I would create my own logger and use in my program. 

# Handling the Logger
logger = logging.getLogger(__name__) # Create the logger
logger.setLevel(logging.INFO) # set logger level
f = logging.Formatter('%(asctime)s - %(levelname)s - %(filename)s - %(message)s') # define the format 
fh = logging.FileHandler('saveToFile.log') # Set file handler
fh.setFormatter(f) # Set the formatting
logger.addHandler(fh) # Add filehandler to logger

# Using the logger created
logging.info('Now we are at the INFO level of logging')