Support arbitrary timestamp formats

[Hammit]

Another instance of this error

║ <HOSTAME_REDACTED>:22:/var/log/httpd/error.log: unable to detect time format from "[Tue Jun 03 14:42:42.007002 2025]  [pid 2803271:tid 139754874960192] A ║
║ H00094: Command line: '/usr/sbin/httpd -D FOREGROUND'"                                                                                                     ║

Simple auto-detection of time format will probably work for the majority of cases, but as you see here, it won't work all the time.
One suggestion I have is to have a configuration item for specifying date/time format strings for when it doesn't work.

[dimonomid]

Yeah, currently Nerdlog really focuses on parsing syslog, and mostly ignores other kinds of logs.

Parsing other kinds of logs is not too trivial, especially since in some cases the timestamp components are not at the fixed offsets from the beginning of the string (e.g. in nginx logs we first have IP address and only then time), and I don't want to implement it quickly in a way which will have to be later reimplemented in some incompatible way.

Also keep in mind that parsing timestamps on the awk side is the hottest path in the whole thing, so it needs to be fast. Currently, Nerdlog gets various timestamp components by just getting substrings at fixed offsets like substr($0, 9, 2) etc, but if we go to some complex conditions and/or regular expressions, it'll affect the performance directly.

So, I'm aware of this issue and would like to address it, but it requires some thought and I don't want to rush.

Thanks for filing the issue.

[dimonomid]

But as a minimal change to support this particular case (Apache logs), I just pushed this commit to master. Could you check if it works for you?

[Hammit]

Wow. Thanks for the quick reply. Unfortunately I cannot build Go projects as I'm behind a corporate proxy/firewall that blocks Go dependency downloading. Thankyou anyway. I'll wait until this makes its way into the release.

I will note however though that I was intending to use nerdlog for custom webapp logs as well and didn't realize it was only intended for syslog/messages/journalctl.

[dimonomid]

I will note however though that I was intending to use nerdlog for custom webapp logs as well and didn't realize it was only intended for syslog/messages/journalctl.

A couple notes here:

1. I, too, use it for custom webapp, but that webapp runs as a systemd service, therefore its logs are naturally printed to syslog, which then Nerdlog can read. Just as a workaround to keep in mind;
2. Hopefully it's a temporary limitation not to be able to parse arbitrary timestamps. Just requires some time to think and to implement it, as mentioned above.

[dimonomid]

I'll wait until this makes its way into the release.

Forgot to mention before, but fyi this change to add basic support for reading Apache logs is part of 1.10.0 released last week.

[eMPee584]

This tool looks really cool! sadly, it doesn't like something about my journalctl's data stream 🤔

localhost:journalctl: parsing mstats:
parsing time "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" as "Jan _2 15:04":
cannot parse "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" as "Jan"