Filter PII

Python Logging is a centralized place to filter out sensitive Personally Identifiable Information.

The flow chart shows how a LogRecord propagates through the logging frameworks' layers. We can leverage the Filter layer (over the formatter) to redact information.

You can use the convenient Loggingredactor library to do so. The redaction acts on

record.msg which is the format str line that's match across regex patterns
record.args are any parameters that's passed to the log which are interpolated against the format string msg. This checks any dictionary for keys that needs to be redacted.

def filter(self, record):
    try:
        record.msg, record.args = (
            self.redact(record.msg), self.redact(deepcopy(record.args)
        )
    except Exception:
        pass
    return True

deepcopy to not mutate the shared (by reference) args object

Always logs content by returning True

Wesley's Page provides some nice example of the popular python-json-logger

Filter PII

Comments

More from this blog

Get harsh feedback

Database Isolation levels

8 Simple reminders

Design a url shortner

No regrets

References

Command Palette

Comments

More from this blog

References