fedbiomed.common.logger

Module: fedbiomed.common.logger

Global logger for fedbiomed

Written above origin Logger class provided by python.

Following features were added from to the original module:

  • provides a logger instance of FedLogger, which is also a singleton, so it can be used "as is"
  • provides a dedicated file handler
  • provides a JSON/MQTT handler: all messages with priority greater than error are sent to the MQQT handler (this permit to send error messages from a node to a researcher)
  • works on python scripts / ipython / notebook
  • manages a dictionary of handlers. Default keys are 'CONSOLE', 'MQTT', 'FILE', but any key is allowed (only one handler by key)
  • allow changing log level globally, or on a specific handler (using its key)
  • log levels can be provided as string instead of logging.* levels (no need to import logging in caller's code) just as in the initial python logger

A typical usage is:

from fedbiomed.common.logger import logger

logger.info("information message")

All methods of the original python logger are provided. To name a few:

  • logger.debug()
  • logger.info()
  • logger.warning()
  • logger.error()
  • logger.critical()

Contrary to other Fed-BioMed classes, the API of FedLogger is compliant with the coding conventions used for logger (lowerCameCase)

Dependency issue

Please pay attention to not create dependency loop then importing other fedbiomed package

Attributes

DEFAULT_LOG_FILE module-attribute 

DEFAULT_LOG_FILE = 'mylog.log'

DEFAULT_LOG_LEVEL module-attribute 

DEFAULT_LOG_LEVEL = logging.WARNING

DEFAULT_LOG_TOPIC module-attribute 

DEFAULT_LOG_TOPIC = 'general/logger'

logger module-attribute 

logger = FedLogger()

Classes

FedLogger

CLASS 
FedLogger(level=DEFAULT_LOG_LEVEL)

Base class for the logger. it uses python logging module by composition (only log() method is overwritten)

All methods from the logging module can be accessed through the _logger member of the class if necessary (instead of overloading all the methods) (ex: logger._logger.getEffectiveLevel() )

Should not be imported

An initial console logger is installed (so the logger has at minimum one handler)

Parameters:

Name Type Description Default
level str

initial loglevel. This loglevel will be the default for all handlers, if called without the default level

 DEFAULT_LOG_LEVEL
Source code in fedbiomed/common/logger.py 
def __init__(self, level: str = DEFAULT_LOG_LEVEL):
    """Constructor of base class

    An initial console logger is installed (so the logger has at minimum one handler)

    Args:
        level: initial loglevel. This loglevel will be the default for all handlers, if called
            without the default level


    """

    # internal tables
    # transform string to logging.level
    self._nameToLevel = {
        "DEBUG": logging.DEBUG,
        "INFO": logging.INFO,
        "WARNING": logging.WARNING,
        "ERROR": logging.ERROR,
        "CRITICAL": logging.CRITICAL,
    }

    # transform logging.level to string
    self._levelToName = {
        logging.DEBUG: "DEBUG",
        logging.INFO: "INFO",
        logging.WARNING: "WARNING",
        logging.ERROR: "ERROR",
        logging.CRITICAL: "CRITICAL"
    }

    # name this logger
    self._logger = logging.getLogger("fedbiomed")

    # Do not propagate (avoids log duplication when third party libraries uses logging module)
    self._logger.propagate = False

    self._default_level = DEFAULT_LOG_LEVEL  # MANDATORY ! KEEP THIS PLEASE !!!
    self._default_level = self._internalLevelTranslator(level)

    self._logger.setLevel(self._default_level)

    # init the handlers list and add a console handler on startup
    self._handlers = {}
    self.addConsoleHandler()

    pass

Functions

addConsoleHandler(format='%(asctime)s %(name)s %(levelname)s - %(message)s', level=DEFAULT_LOG_LEVEL)

Adds a console handler

Parameters:

Name Type Description Default
format str

the format string of the logger

 '%(asctime)s %(name)s %(levelname)s - %(message)s'
level Any

initial level of the logger for this handler (optional) if not given, the default level is set

 DEFAULT_LOG_LEVEL
Source code in fedbiomed/common/logger.py 
def addConsoleHandler(self,
                      format: str = '%(asctime)s %(name)s %(levelname)s - %(message)s',
                      level: Any = DEFAULT_LOG_LEVEL):

    """Adds a console handler

    Args:
        format: the format string of the logger
        level: initial level of the logger for this handler (optional) if not given, the default level is set
    """

    handler = logging.StreamHandler()

    handler.setLevel(self._internalLevelTranslator(level))

    formatter = logging.Formatter(format)
    handler.setFormatter(formatter)
    self._internalAddHandler("CONSOLE", handler)

    pass

addFileHandler(filename=DEFAULT_LOG_FILE, format='%(asctime)s %(name)s %(levelname)s - %(message)s', level=DEFAULT_LOG_LEVEL)

Adds a file handler

Parameters:

Name Type Description Default
filename str

File to log to

 DEFAULT_LOG_FILE
format str

Log format

 '%(asctime)s %(name)s %(levelname)s - %(message)s'
level any

Initial level of the logger (optionnal)

 DEFAULT_LOG_LEVEL
Source code in fedbiomed/common/logger.py 
def addFileHandler(self,
                   filename: str = DEFAULT_LOG_FILE,
                   format: str = '%(asctime)s %(name)s %(levelname)s - %(message)s',
                   level: any = DEFAULT_LOG_LEVEL):
    """Adds a file handler

    Args:
        filename: File to log to
        format: Log format
        level: Initial level of the logger (optionnal)
    """

    handler = logging.FileHandler(filename=filename, mode='a')
    handler.setLevel(self._internalLevelTranslator(level))

    formatter = logging.Formatter(format)
    handler.setFormatter(formatter)

    self._internalAddHandler("FILE", handler)
    pass

addMqttHandler(mqtt=None, node_id=None, topic=DEFAULT_LOG_TOPIC, level=logging.ERROR)

Adds a mqtt handler, to publish error message on a topic

Parameters:

Name Type Description Default
mqtt Any

already opened MQTT object

 None
node_id str

id of the caller (necessary for msg formatting to the researcher)

 None
topic Any

topic to publish to (non-mandatory)

 DEFAULT_LOG_TOPIC
level Any

level of this handler (non-mandatory) level must be lower than ERROR to ensure that the research get all ERROR/CRITICAL messages

 logging.ERROR
Source code in fedbiomed/common/logger.py 
def addMqttHandler(self,
                   mqtt: Any = None,
                   node_id: str = None,
                   topic: Any = DEFAULT_LOG_TOPIC,
                   level: Any = logging.ERROR
                   ):

    """Adds a mqtt handler, to publish error message on a topic

    Args:
        mqtt: already opened MQTT object
        node_id: id of the caller (necessary for msg formatting to the researcher)
        topic: topic to publish to (non-mandatory)
        level: level of this handler (non-mandatory) level must be lower than ERROR to ensure that the
            research get all ERROR/CRITICAL messages
    """

    handler = _MqttHandler(
        mqtt=mqtt,
        node_id=node_id,
        topic=topic
    )

    # may be not necessary ?
    handler.setLevel(self._internalLevelTranslator(level))
    formatter = _MqttFormatter(node_id)

    handler.setFormatter(formatter)
    self._internalAddHandler("MQTT", handler)

    # as a side effect this will set the minimal level to ERROR
    self.setLevel(level, "MQTT")
    pass

delMqttHandler()
Source code in fedbiomed/common/logger.py 
def delMqttHandler(self):
    self._internalAddHandler("MQTT", None)

log(level, msg)

Overrides the logging.log() method to allow the use of string instead of a logging.* level

Source code in fedbiomed/common/logger.py 
def log(self, level: Any, msg: str):
    """Overrides the logging.log() method to allow the use of string instead of a logging.* level """

    level = logger._internalLevelTranslator(level)
    self._logger.log(
        level,
        msg
    )

setLevel(level, htype=None)

Overrides the setLevel method, to deal with level given as a string and to change le level of one or all known handlers

This also change the default level for all future handlers.

Remark

Level should not be lower than CRITICAL (meaning CRITICAL errors are always displayed)

Example: 

setLevel( logging.DEBUG, 'FILE')

Parameters:

Name Type Description Default
level

level to modify, can be a string or a logging.* level (mandatory)

 required
htype

if provided (non-mandatory), change the level of the given handler. if not provided (or None), change the level of all known handlers

 None
Source code in fedbiomed/common/logger.py 
def setLevel(self, level: Any, htype: Any = None):
    """Overrides the setLevel method, to deal with level given as a string and to change le level of
    one or all known handlers

    This also change the default level for all future handlers.

    !!! info "Remark"

        Level should not be lower than CRITICAL (meaning CRITICAL errors are always displayed)

        Example:
        ```python
        setLevel( logging.DEBUG, 'FILE')
        ```

    Args:
        level : level to modify, can be a string or a logging.* level (mandatory)
        htype : if provided (non-mandatory), change the level of the given handler. if not  provided (or None),
            change the level of all known handlers
    """

    level = self._internalLevelTranslator(level)

    if htype is None:
        # store this level (for future handler adding)
        self._logger.setLevel(level)

        for h in self._handlers:
            self._handlers[h].setLevel(level)
        return

    if htype in self._handlers:
        self._handlers[htype].setLevel(level)
        return

    # htype provided but no handler for this type exists
    self._logger.warning(htype + " handler not initialized yet")