Send logging output above a running bar

redirect_stdout/redirect_stderr catch print() and raw writes, but calls through the stdlib logging module go via a StreamHandler that writes directly to whatever stream it was bound to – redirecting the raw stream alone doesn’t touch it.

"""Route stdlib `logging` output above a progress bar instead of corrupting it.

`streams.wrap_stderr()` redirects raw writes the same way `redirect_stderr=
True` does; `streams.wrap_logging()` additionally retargets every
`StreamHandler` already pointed at stdout/stderr so calls to `logging.info(
...)` -- not just `print()` -- land above the bar cleanly. Construction
order does not matter: a bar defaulting to `sys.stderr` resolves that to
the unwrapped stream either way, which is what stops its own redraws
recursing through the capture. Always unwrap (and, here, remove the
handler) in a `finally`, since both mutate process-global state.
"""

import logging
import sys
import time

import progressbar

logger = logging.getLogger(__name__)

STEPS = 24


def main() -> None:
    handler = logging.StreamHandler()
    logger.setLevel(logging.INFO)
    logger.propagate = False
    logger.addHandler(handler)
    # `fd=sys.stderr` is read at call time, not bound once when this module
    # was first imported -- unlike the parameter's own default. Passing it
    # explicitly is a no-op in a real run (nothing has touched `sys.stderr`
    # yet, so it is the same object either way); it only matters to a
    # caller -- such as a test -- that has already reassigned `sys.stderr`
    # before this line runs.
    bar = progressbar.ProgressBar(max_value=STEPS, fd=sys.stderr)
    try:
        progressbar.streams.wrap_stderr()
        progressbar.streams.wrap_logging()
        try:
            with bar:
                for step in range(STEPS):
                    if step in {8, 16}:
                        logger.info('completed step %d', step)
                    bar.update(step + 1)
                    time.sleep(0.005)
        finally:
            progressbar.streams.unwrap_logging()
            progressbar.streams.unwrap_stderr()
    finally:
        logger.removeHandler(handler)


if __name__ == '__main__':
    main()

streams.wrap_stderr() redirects raw writes, the same way redirect_stderr=True does on a single bar. streams.wrap_logging() additionally retargets StreamHandler instances already attached to a logger: it walks every logger’s handlers once, and any handler currently writing to the real stdout/stderr (or to whatever stdout/stderr already is) gets pointed at the wrapped stream instead, so logging.info(...) lands above the bar the same way print() does. Construction order does not matter here: a bar defaulting to sys.stderr resolves that to the unwrapped stream either way, which is what keeps its own redraws from recursing back through the capture. Unwind both in a finally: unwrap_logging() restores each handler’s original stream, then unwrap_stderr() restores sys.stderr. Both mutate process-global state, so a leaked wrapper affects every bar built afterward in the same process.

Caveats

wrap_logging() only touches handlers it can already see: it matches each handler’s current stream against the process’s real stdout/stderr (captured once, when progressbar first loads) and whatever stdout/stderr are right now. A handler already pointed at some other stream object – because another tool had already substituted sys.stderr before this one ran, or because the handler was built with an explicit file argument – isn’t one wrap_logging() recognizes, and is left untouched, still writing wherever it already was.