Creating multiple log files at log rotation time '00:00'

[NannoSilver]

Hi guys!

This is my log_sys.py code:

from loguru import logger

logger.remove()

formato = '{time:YYYY-MM-DD HH:mm:ss.SSSSS!UTC}   {level:15}  {message}  [{file} : {line}]'

logger.add('logs_loguru/loguru_UTC_time_WARNING_{time:YYYY-MM-DD!UTC}.log', rotation='00:00', format=formato, level='WARNING', catch=True)
logger.add('logs_loguru/loguru_UTC_time_DEBUG_{time:YYYY-MM-DD!UTC}.log', rotation='00:00', format=formato, level='DEBUG', catch=True)

It is a web app and is running in a Cloudlinux server.
At the time the app was started, two log files were created, as expected:
loguru_UTC_time_WARNING_2022-08-23.log
loguru_UTC_time_DEBUG_2022-08-23.log

Problem started after a few hours (when the server clock was at 00:00 at the server timezone?). Instead to rotate the files, creating loguru_UTC_time_WARNING_2022-08-24.log and loguru_UTC_time_DEBUG_2022-08-24.log, it started to create many log files, with different file name and overwriting the content of some of them, as you can see in the following image:

File time shown in the image is my local time rather than server time.

I tried to add enqueue=True as
logger.add('logs_loguru/loguru_UTC_time_DEBUG_{time:YYYY-MM-DD!UTC}.log', rotation='00:00', format=formato, level='DEBUG', catch=True, enqueue=True) but that leads my web app to freezing/crashing, that seems to be a separated issue.

I could not find a way to set the rotation to UTC time. Can that be the issue?

[Delgan]

Hi @sukhoi47. Are you using any framework like Uvicorn to implement your web app?

The issue you're facing often happen because multiple sub-processes are started which results with logger.add() being called multiple times, causing problems when rotation occurs.

[NannoSilver]

Hi @Delgan,

I am not using Uvicorn, gunicorn nor anything similar in my code, however the Cloudlinux with Litespeed server seems to have its own built-in WSGI.

Strangely, the parameter enqueue=True is leading to issues, or there is something else I should include in my code to make it work properly?

[NannoSilver]

I just completed a test with file rotation... for the test I removed the !UTC reference in the file name. The updated code is logger.add('logs_loguru/loguru_UTC_time_WARNING_{time:YYYY-MM-DD}.log', rotation='00:00', format=formato, level='WARNING', catch=True) and seems it rotated properly at 00:00 hours at the server internal clock, with the creation of only two files, as expected:
loguru_UTC_time_DEBUG_2022-08-25.log
loguru_UTC_time_WARNING_2022-08-25.log

While we cannot rule out another additional problem, that seems to be at least part of the issue, that is, date reference in the file name and rotation shall be the same (in my code it was not, one was UTC and the other, no). I searched in the loguru documentation, but could not find any reference on how to set rotation to UTC.
rotation='00:00!UTC' does not works.

Is there a way to set rotation to UTC?

[Delgan]

Hey @sukhoi47, thanks for the investigations and the reported edge case.

I think you can try using a datetime.time() object with tzinfo set as UTC for rotation:

logger.add(..., rotation=datetime.time(0, 0, 0, tzinfo=datetime.timezone.utc))

May bad it's not working, I need to address this bug.

Currently, I think you have to implement a custom rotation function or convert your UTC time to local time.

[NannoSilver]

In recent weeks I am using this code, that is working fine so far.

My server has internal clock set to UTC +02:00. So, by rotating at 02:00 it will be rotating at UTC +00:00 time.

from loguru import logger
from datetime import time, timezone

logger.remove()

formato = '{time:YYYY-MM-DD HH:mm:ss.SSSSS!UTC}   {level:8}  {message}  [{file} : {line}]'

# server internal clock is set to UTC +02:00
logger.add('logs_loguru/loguru_UTC_time_WARNING_{time:YYYY-MM-DD!UTC}.log', rotation='02:00', format=formato, level='WARNING')
logger.add('logs_loguru/loguru_UTC_time_DEBUG_{time:YYYY-MM-DD!UTC}.log', rotation='02:00', format=formato, level='DEBUG')

[Delgan]

Fixed on master branch.

It's now possible to use aware datetime.time object for file rotation:

logger.add("file.{time:YYYY-MM-DD!UTC}.log", rotation=datetime.time(0, 0, 0, tzinfo=datetime.timezone.utc)))

This handler will rotate every day at midnight UTC, and the file will be created according to the UTC day.

Strings passed to the rotation argument (like "00:00" or "daily") are interpreted as naive local time.

[NannoSilver]

Thank you for this great job, Delgan!

Look forward for the updated Loguru library at https://pypi.org/