How to configure Python logging for both file and stdout?

General Discussions
,
1

How can I configure Python logging to ensure all log messages are output to both a log file and stdout? Specifically, I want messages logged using logger.warning, logger.error, and logger.critical to go to their respective log file destinations while also being mirrored to stdout, without manually duplicating calls like:

logger.critical("something failed")
print("something failed")

How can this be achieved using the logging module in Python?

2

Hey All!

Alright, let’s start with a basic setup. If you want to add a StreamHandler for logging to stdout, you can easily do that in Python. Here’s how:

import logging
import sys

# Configure root logger
root = logging.getLogger()
root.setLevel(logging.DEBUG)

# Create a StreamHandler for stdout
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)

# Set a formatter for the handler
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)

# Add the handler to the root logger
root.addHandler(handler)

With this setup, you’ve got your logs going to stdout. It’s a simple and effective way to see your logs in real time without complicating things

3

Great start, @shashank_watak! Now, let’s take it a step further. If you need multiple logging outputs (like both to stdout and a file), a good approach is to use a logging configuration dictionary. It keeps your code neat and can scale well as your logging needs grow. Here’s how you can configure that:

import logging
import logging.config
import sys

# Logging configuration dictionary
logging_config = {
    'version': 1,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'stream': sys.stdout,
            'level': 'DEBUG',
            'formatter': 'detailed',
        },
        'file': {
            'class': 'logging.FileHandler',
            'filename': 'app.log',
            'level': 'INFO',
            'formatter': 'detailed',
        },
    },
    'formatters': {
        'detailed': {
            'format': '%(asctime)s - %(name)s - %(levelname)s - %(message)s',
        },
    },
    'root': {
        'level': 'DEBUG',
        'handlers': ['console', 'file'],
    },
}

# Apply the logging configuration
logging.config.dictConfig(logging_config)

# Example usage
logger = logging.getLogger()
logger.debug('This is a debug message.')
logger.info('This is an info message.')

With this, you’ve got a structured way to log messages to both stdout and a file with minimal code overhead. It’s a really powerful setup, especially when your application grows!

4

Nice one, @alveera.khn! And for those of you who want even more control over the logging behavior, consider creating a custom logger function. This gives you flexibility to configure the logging behavior in different parts of your application while keeping it clean. Here’s how you can set it up:

import logging
import sys

def get_custom_logger(name):
    # Create a logger
    logger = logging.getLogger(name)
    logger.setLevel(logging.DEBUG)

    # Create a StreamHandler for stdout
    stream_handler = logging.StreamHandler(sys.stdout)
    stream_handler.setLevel(logging.DEBUG)

    # Create a FileHandler for logging to a file
    file_handler = logging.FileHandler('app.log')
    file_handler.setLevel(logging.INFO)

    # Set formatters
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    stream_handler.setFormatter(formatter)
    file_handler.setFormatter(formatter)

    # Add handlers to the logger
    logger.addHandler(stream_handler)
    logger.addHandler(file_handler)

    return logger

# Example usage
logger = get_custom_logger('MyAppLogger')
logger.info('This message will appear in both stdout and the log file.')

This way, you can reuse this custom logger wherever you need, and it handles both stdout and file logging for you. It’s a great choice when you have different loggers for different modules or classes, ensuring everything is nicely organized.