commas, quotes and CR/LFs in log description field prevents easy parsing

[dalers]

Hi all, this morning's descent into a rabbit hole was log file parsing and interpretation.

It seems log files can't be easily parsed for analysis because of commas, quotes and CR/LF's in description field text.

IIuc,

• a log file record has 4 comma-delimeted fields: "[time],[type],[user],[description]"
• the "description" field is from the 3rd comma delimeter to the end of line and may include both comma and quote characters
• the "description" field text is not quoted

It's hard to follow a particular user through the log entries because:

• if viewed in a text editor, the user name moves left or right in the line depending on how long the "type" field is
• the file can't be readily parsed by LibreOffice Calc (e.g. to use Smart Filtering on the user or type columns) because a) the description field isn't quoted (many logs have commas in the description) and b) logs from SelectProduct.php can also include both CR/LFs AND quote characters (i.e. simply wrapping the description field text in quote characters won't work).

Am I missing an obvious solution? How do you analyze webERP log files?

Also, is the syntax for log records documented somewhere? I searched the manual for "logging" and "log file" but didn't find anything useful.

Cheers,
Dale

Here are the offending logs from SelectProduct.php with CR/LF and quote characters in the description field/column.

2024-11-20 11-11-27,SUCCESS Report,bjenks,New Item <a href="SelectProduct.php?StockID=100-0001">100-0001</a> has been added to the database<br />NB: The item cost and pricing must also be setup<br /><a target="_blank" href="StockCostUpdate.php?StockID=100-0001">Enter Item Cost</a>
							<br /><a target="_blank" href="Prices.php?Item=100-0001">Enter Item Prices</a> 
2024-11-21 03-11-52,SUCCESS Report,mnestor,New Item <a href="SelectProduct.php?StockID=100-0001">100-0001</a> has been added to the database<br />NB: The item cost and pricing must also be setup<br /><a target="_blank" href="StockCostUpdate.php?StockID=100-0001">Enter Item Cost</a>
							<br /><a target="_blank" href="Prices.php?Item=100-0001">Enter Item Prices</a> 
2024-11-21 03-11-42,SUCCESS Report,bjenks,New Item <a href="SelectProduct.php?StockID=100-0001">100-0001</a> has been added to the database<br />NB: The item cost and pricing must also be setup<br /><a target="_blank" href="StockCostUpdate.php?StockID=100-0001">Enter Item Cost</a>
							<br /><a target="_blank" href="Prices.php?Item=100-0001">Enter Item Prices</a> 
2024-11-27 12-11-52,SUCCESS Report,bjenks,New Item <a href="SelectProduct.php?StockID=100-0002">100-0002</a> has been added to the database<br />NB: The item cost and pricing must also be setup<br /><a target="_blank" href="StockCostUpdate.php?StockID=100-0002">Enter Item Cost</a>
							<br /><a target="_blank" href="Prices.php?Item=100-0002">Enter Item Prices</a> 
2024-12-10 11-12-55,SUCCESS Report,bjenks,New Item <a href="SelectProduct.php?StockID=100-0003">100-0003</a> has been added to the database<br />NB: The item cost and pricing must also be setup<br /><a target="_blank" href="StockCostUpdate.php?StockID=100-0003">Enter Item Cost</a>
							<br /><a target="_blank" href="Prices.php?Item=100-0003">Enter Item Prices</a> 

[timschofield]

Hi Dale, I have stripped all html tags from the log file entries.

Thanks
Tim

[dalers]

stripped all html tags from the log file entries.

seems reasonable... I was concerned useful information in the tags would be lost, but at least in this case they don't add value. I need to generate new log entries and review before saying more.

[timschofield]

The log messages are essentially the prnMsg() messages with a few bits added. The HTML tags are just there to add a bit of formatting.