Skip to content

Commit 75f5653

Browse files
aiekickaiekick
committed
[ADD] : add a way to specify the format to convert string <> epoch
1 parent 140bc3c commit 75f5653

10 files changed

Lines changed: 163 additions & 13863 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,11 +103,15 @@ ltg:addSignalEndZone("category", "signal_name", epoch, "label")
103103
ltg:addSignalTag(epoch, r, g, b, a, "name", "tooltip") -- point event, color is linear [0:1]
104104
```
105105

106-
Time conversion (ISO-like "YYYY-MM-DD HH:MM:SS,MS" or "YYYY-MM-DD HH:MM:SS.MS", hour offset as 2nd arg):
106+
Time conversion. Default pattern is `"yyyy-MM-dd HH:mm:ss,SSS"` (Joda-style). Pass a custom pattern as the 3rd argument to parse / format other layouts. Supported tokens: `yyyy`/`yy` (year), `MM` (month), `dd` (day), `HH`/`hh` (hour 24/12), `mm` (minute), `ss` (second), and a trailing `S` run (N digits, fractional seconds — must be at the end of the pattern, preceded by exactly one delimiter character):
107107

108108
```lua
109109
local epoch = ltg:stringToEpoch("2023-01-16 15:24:26,464", 0)
110110
local s = ltg:epochToString(epoch, 0)
111+
112+
-- custom pattern (microseconds with a dot delimiter, or a day-first layout):
113+
local e2 = ltg:stringToEpoch("2023-01-16 15:24:26.123456", 0, "yyyy-MM-dd HH:mm:ss.SSSSSS")
114+
local s2 = ltg:epochToString(e2, 0, "dd/MM/yyyy HH:mm:ss")
111115
```
112116

113117
Regex (`boost::regex` engine, PCRE-like — supports `|`, `{m,n}`, lookahead, etc., much more expressive than Lua patterns):

‎bin/log_parsing.lua‎

Lines changed: 0 additions & 35 deletions
This file was deleted.

‎bin/log_sample.log‎

Lines changed: 0 additions & 13807 deletions
This file was deleted.

‎doc/log_parsing.lua‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,12 @@
88
-- ltg:addSignalValue(signal_category, signal_name, signal_epoch_time, signal_value, description_string_optional) : will add a signal numerical value
99
-- ltg:addSignalStartZone(signal_category, signal_name, signal_epoch_time, signal_string) : will add a signal start zone
1010
-- ltg:addSignalEndZone(signal_category, signal_name, signal_epoch_time, signal_string) : will add a signal end zone
11-
-- get/set epoch time from datetime in format "YYYY-MM-DD HH:MM:SS,MS" or "YYYY-MM-DD HH:MM:SS.MS" with hour offset in second param
11+
-- get/set epoch time from datetime. default Joda pattern "yyyy-MM-dd HH:mm:ss,SSS" (3rd arg = custom pattern, hour offset in 2nd)
12+
-- supported tokens: yyyy yy MM dd HH hh mm ss + trailing SSS... (N digits, fractional second, preceded by one delimiter char)
1213
-- double ltg:stringToEpoch("2023-01-16 15:24:26,464", 0)
14+
-- double ltg:stringToEpoch("16/01/2023 15:24:26.123", 0, "dd/MM/yyyy HH:mm:ss.SSS")
1315
-- string ltg:epochToString(18798798465465.546546, 0)
16+
-- string ltg:epochToString(epoch, 0, "yyyy-MM-dd HH:mm:ss.SSSSSS")
1417
-- regex (boost::regex engine, PCRE-like syntax — supports |, {m,n}, lookahead, etc.):
1518
-- local re = ltg:regex(pattern) -- compile once at script init, reuse on every row
1619
-- re:test(input) -> bool
@@ -28,8 +31,7 @@ local re = ltg:regex([[<profiler section="([^"]*)" epoch_time="([^"]*)" name="([
2831
function parse(buffer)
2932
local section, time, name, value = re:match(buffer)
3033
if section and time and name and value then
31-
local epoch = ltg:stringToEpoch(time, 0);
32-
ltg:addSignalValue(section, name, epoch, tonumber(value))
34+
ltg:addSignalValue(section, name, tonumber(time), tonumber(value))
3335
end
3436
end
3537

‎doc/samples.ltg‎

0 Bytes
Binary file not shown.

‎plugins/LuaScripting/README.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -47,11 +47,13 @@ Each call goes through the host's Messaging singleton and shows up in the Consol
4747

4848
### Date / time conversion
4949

50-
ISO-like strings (`"YYYY-MM-DD HH:MM:SS,MS"` or `"YYYY-MM-DD HH:MM:SS.MS"`), with an hour offset as the second argument:
50+
Default Joda-style pattern `"yyyy-MM-dd HH:mm:ss,SSS"`. An optional 3rd argument overrides the pattern. Supported tokens: `yyyy`/`yy` (year), `MM` (month), `dd` (day), `HH`/`hh` (hour 24/12), `mm` (minute), `ss` (second). A trailing run of `S` marks fractional seconds (N digits, must be at the end of the pattern, preceded by exactly one delimiter character). Hour offset is the 2nd argument:
5151

5252
```lua
53-
local epoch = ltg:stringToEpoch("2023-01-16 15:24:26,464", 0) -- -> double (epoch seconds, ms preserved)
54-
local s = ltg:epochToString(18798798465465.546546, 0) -- -> string
53+
local epoch = ltg:stringToEpoch("2023-01-16 15:24:26,464", 0) -- -> double (epoch seconds, ms preserved)
54+
local s = ltg:epochToString(18798798465465.546546, 0) -- -> string
55+
local e2 = ltg:stringToEpoch("16/01/2023 15:24:26.123456", 0, "dd/MM/yyyy HH:mm:ss.SSSSSS")
56+
local s2 = ltg:epochToString(e2, 0, "yyyy-MM-dd HH:mm:ss") -- no fractional part
5557
```
5658

5759
Both throw a `std::runtime_error` on a bad format. With the auto-bp-on-error toggle on, that throw pauses the worker at the exact line of the call, with the actual boost message visible.

‎plugins/LuaScripting/src/modules/LuaDatasModel.cpp‎

Lines changed: 128 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -51,38 +51,159 @@ double LuaDatasModel::luaModuleGetRowCount() {
5151
return static_cast<double>(m_RowCount);
5252
}
5353

54+
namespace {
55+
56+
// Default Joda-style pattern when the 2-arg form is used — backward-compatible with the
57+
// previously hardcoded "YYYY-MM-DD HH:MM:SS,MS" / ".MS" inputs.
58+
constexpr const char* sDefaultJodaPattern = "yyyy-MM-dd HH:mm:ss,SSS";
59+
60+
// Translate a Joda-style pattern (yyyy / MM / dd / HH / mm / ss + optional trailing 'S' run)
61+
// into a strftime format string for std::get_time / std::put_time, plus the fractional-seconds
62+
// delimiter and digit count. The 'S' run, if present, MUST be at the end of the pattern and
63+
// MUST be preceded by exactly one literal delimiter character.
64+
void translateJodaPattern(const std::string& vPattern, std::string& aoStrftimeFmt, char& aoFracDelim, int32_t& aoFracDigits) {
65+
if (vPattern.empty()) {
66+
throw std::invalid_argument("pattern: empty");
67+
}
68+
// walk back from the end to count the trailing 'S' run (Joda fractional seconds marker)
69+
std::size_t fracStart = vPattern.size();
70+
while (fracStart > 0 && vPattern[fracStart - 1] == 'S') {
71+
--fracStart;
72+
}
73+
std::string dateOnly;
74+
if (fracStart == vPattern.size()) {
75+
dateOnly = vPattern;
76+
aoFracDelim = 0;
77+
aoFracDigits = 0;
78+
} else if (fracStart == 0) {
79+
throw std::invalid_argument("pattern: fractional 'S' must be preceded by a delimiter");
80+
} else {
81+
dateOnly = vPattern.substr(0, fracStart - 1);
82+
aoFracDelim = vPattern[fracStart - 1];
83+
aoFracDigits = static_cast<int32_t>(vPattern.size() - fracStart);
84+
}
85+
aoStrftimeFmt = dateOnly;
86+
// longer tokens first so 'yyyy' is consumed before 'yy' tries to match
87+
// clang-format off
88+
const std::pair<std::string, std::string> tokens[] = {
89+
{"yyyy", "%Y"}, {"yy", "%y"},
90+
{"MM", "%m"},
91+
{"dd", "%d"},
92+
{"HH", "%H"}, {"hh", "%I"},
93+
{"mm", "%M"},
94+
{"ss", "%S"},
95+
};
96+
// clang-format on
97+
for (const auto& tk : tokens) {
98+
std::size_t pos = 0;
99+
while ((pos = aoStrftimeFmt.find(tk.first, pos)) != std::string::npos) {
100+
aoStrftimeFmt.replace(pos, tk.first.size(), tk.second);
101+
pos += tk.second.size();
102+
}
103+
}
104+
}
105+
106+
// Convert a fractional-seconds integer with `vDigits` digits into microseconds (6 digits).
107+
int64_t scaleFractionToMicros(int64_t vValue, int32_t vDigits) {
108+
if (vDigits == 6) {
109+
return vValue;
110+
}
111+
int64_t factor = 1;
112+
if (vDigits < 6) {
113+
for (int32_t i = 0; i < 6 - vDigits; ++i) {
114+
factor *= 10;
115+
}
116+
return vValue * factor;
117+
}
118+
for (int32_t i = 0; i < vDigits - 6; ++i) {
119+
factor *= 10;
120+
}
121+
return vValue / factor;
122+
}
123+
124+
// Convert microseconds (6 digits) into a fractional-seconds integer with `vDigits` digits.
125+
int64_t scaleMicrosToFraction(int64_t vMicros, int32_t vDigits) {
126+
if (vDigits == 6) {
127+
return vMicros;
128+
}
129+
int64_t factor = 1;
130+
if (vDigits < 6) {
131+
for (int32_t i = 0; i < 6 - vDigits; ++i) {
132+
factor *= 10;
133+
}
134+
return vMicros / factor;
135+
}
136+
for (int32_t i = 0; i < vDigits - 6; ++i) {
137+
factor *= 10;
138+
}
139+
return vMicros * factor;
140+
}
141+
142+
} // namespace
143+
54144
double LuaDatasModel::luaModuleStringToEpoch(const std::string& vDateTime, double vHourOffset) {
145+
return luaModuleStringToEpochWithPattern(vDateTime, vHourOffset, sDefaultJodaPattern);
146+
}
147+
148+
double LuaDatasModel::luaModuleStringToEpochWithPattern(const std::string& vDateTime, double vHourOffset, const std::string& vPattern) {
149+
std::string strftimeFmt;
150+
char fracDelim = 0;
151+
int32_t fracDigits = 0;
152+
translateJodaPattern(vPattern, strftimeFmt, fracDelim, fracDigits);
153+
55154
struct tm timeStruct = {};
56-
int microseconds = 0;
57155
std::istringstream dateStream(vDateTime);
58-
char delimiter;
59-
dateStream >> std::get_time(&timeStruct, "%Y-%m-%d %H:%M:%S");
156+
dateStream >> std::get_time(&timeStruct, strftimeFmt.c_str());
60157
if (dateStream.fail()) {
61158
throw std::invalid_argument("Invalid date format");
62159
}
63160
timeStruct.tm_hour += static_cast<int32_t>(vHourOffset);
64161
timeStruct.tm_isdst = 1;
65-
dateStream >> delimiter >> microseconds;
162+
163+
int64_t fracValue = 0;
164+
if (fracDigits > 0) {
165+
char delimiter = 0;
166+
dateStream >> delimiter >> fracValue;
167+
if (dateStream.fail() || delimiter != fracDelim) {
168+
throw std::invalid_argument("Invalid fractional seconds");
169+
}
170+
}
171+
66172
std::time_t epochSeconds = std::mktime(&timeStruct);
67173
if (epochSeconds == -1) {
68174
throw std::runtime_error("Failed to convert to epoch time");
69175
}
70176
epochSeconds -= static_cast<time_t>(std::difftime(std::mktime(std::gmtime(&epochSeconds)), std::mktime(std::localtime(&epochSeconds))));
177+
178+
const int64_t microseconds = (fracDigits > 0) ? scaleFractionToMicros(fracValue, fracDigits) : 0;
71179
return static_cast<double>(epochSeconds) + static_cast<double>(microseconds) / 1000000.0;
72180
}
73181

74182
std::string LuaDatasModel::luaModuleEpochToString(double vEpochTime, double vHourOffset) {
183+
return luaModuleEpochToStringWithPattern(vEpochTime, vHourOffset, sDefaultJodaPattern);
184+
}
185+
186+
std::string LuaDatasModel::luaModuleEpochToStringWithPattern(double vEpochTime, double vHourOffset, const std::string& vPattern) {
187+
std::string strftimeFmt;
188+
char fracDelim = 0;
189+
int32_t fracDigits = 0;
190+
translateJodaPattern(vPattern, strftimeFmt, fracDelim, fracDigits);
191+
75192
std::time_t seconds = static_cast<std::time_t>(vEpochTime);
76-
int microseconds = static_cast<int>((vEpochTime - seconds) * 1000000.0);
193+
const int64_t microseconds = static_cast<int64_t>((vEpochTime - seconds) * 1000000.0);
77194
struct tm* timeStruct = std::gmtime(&seconds);
78195
if (!timeStruct) {
79196
throw std::runtime_error("Failed to convert epoch time to struct tm");
80197
}
81198
timeStruct->tm_hour += static_cast<int32_t>(vHourOffset);
82199
timeStruct->tm_isdst = 1;
200+
83201
std::ostringstream dateStream;
84-
dateStream << std::put_time(timeStruct, "%Y-%m-%d %H:%M:%S");
85-
dateStream << '.' << std::setfill('0') << std::setw(6) << microseconds;
202+
dateStream << std::put_time(timeStruct, strftimeFmt.c_str());
203+
if (fracDigits > 0) {
204+
const int64_t fracValue = scaleMicrosToFraction(microseconds, fracDigits);
205+
dateStream << fracDelim << std::setfill('0') << std::setw(fracDigits) << fracValue;
206+
}
86207
return dateStream.str();
87208
}
88209

‎plugins/LuaScripting/src/modules/LuaDatasModel.h‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,9 @@ class LuaDatasModel {
3535
void luaModuleLogError(const std::string& vKey);
3636
void luaModuleLogDebug(const std::string& vKey);
3737
double luaModuleStringToEpoch(const std::string& vDateTime, double vHourOffset);
38+
double luaModuleStringToEpochWithPattern(const std::string& vDateTime, double vHourOffset, const std::string& vPattern);
3839
std::string luaModuleEpochToString(double vEpochTime, double vHourOffset);
40+
std::string luaModuleEpochToStringWithPattern(double vEpochTime, double vHourOffset, const std::string& vPattern);
3941
void luaModuleAddSignalTag(double vEpoch, double r, double g, double b, double a, const std::string& vName, const std::string& vHelp);
4042
void luaModuleAddSignalValue(const std::string& vCategory, const std::string& vName, double vEpoch, double vValue);
4143
void luaModuleAddSignalValueWithDesc(const std::string& vCategory, const std::string& vName, double vEpoch, double vValue, const std::string&);

‎plugins/LuaScripting/src/modules/Module.cpp‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -213,8 +213,12 @@ bool Module::load(Ltg::IDatasModelWeak vDatasModel) {
213213
);
214214
m_luaPtr->new_usertype<LuaDatasModel>(
215215
"LuaDatasModel", sol::constructors<std::shared_ptr<LuaDatasModel>()>(),
216-
"stringToEpoch", &LuaDatasModel::luaModuleStringToEpoch,
217-
"epochToString", &LuaDatasModel::luaModuleEpochToString,
216+
"stringToEpoch", sol::overload(
217+
&LuaDatasModel::luaModuleStringToEpoch,
218+
&LuaDatasModel::luaModuleStringToEpochWithPattern),
219+
"epochToString", sol::overload(
220+
&LuaDatasModel::luaModuleEpochToString,
221+
&LuaDatasModel::luaModuleEpochToStringWithPattern),
218222
"addSignalTag", &LuaDatasModel::luaModuleAddSignalTag,
219223
"addSignalStatus", &LuaDatasModel::luaModuleAddSignalStatus,
220224
"addSignalValue",sol::overload(
@@ -371,8 +375,12 @@ void Module::m_ensureCompletionState() {
371375
);
372376
m_completionLuaPtr->new_usertype<LuaDatasModel>(
373377
"LuaDatasModel", sol::constructors<std::shared_ptr<LuaDatasModel>()>(),
374-
"stringToEpoch", &LuaDatasModel::luaModuleStringToEpoch,
375-
"epochToString", &LuaDatasModel::luaModuleEpochToString,
378+
"stringToEpoch", sol::overload(
379+
&LuaDatasModel::luaModuleStringToEpoch,
380+
&LuaDatasModel::luaModuleStringToEpochWithPattern),
381+
"epochToString", sol::overload(
382+
&LuaDatasModel::luaModuleEpochToString,
383+
&LuaDatasModel::luaModuleEpochToStringWithPattern),
376384
"addSignalTag", &LuaDatasModel::luaModuleAddSignalTag,
377385
"addSignalStatus", &LuaDatasModel::luaModuleAddSignalStatus,
378386
"addSignalValue", sol::overload(
@@ -628,8 +636,8 @@ const std::vector<SignatureEntry>& s_signatureCatalog() {
628636
// clang-format off
629637
static const std::vector<SignatureEntry> catalog = {
630638
// -------- ltg: usertype methods (cf. new_usertype<LuaDatasModel> in Module::load) --------
631-
{"ltg", "stringToEpoch", {{"dateTime","string"}, {"hourOffset","number"}}},
632-
{"ltg", "epochToString", {{"epochTime","number"}, {"hourOffset","number"}}},
639+
{"ltg", "stringToEpoch", {{"dateTime","string"}, {"hourOffset","number"}, {"pattern","string?"}}},
640+
{"ltg", "epochToString", {{"epochTime","number"}, {"hourOffset","number"}, {"pattern","string?"}}},
633641
{"ltg", "addSignalTag", {{"epoch","number"}, {"r","number"}, {"g","number"}, {"b","number"}, {"a","number"}, {"name","string"}, {"help","string"}}},
634642
{"ltg", "addSignalStatus", {{"category","string"}, {"name","string"}, {"epoch","number"}, {"status","string"}}},
635643
{"ltg", "addSignalValue", {{"category","string"}, {"name","string"}, {"epoch","number"}, {"value","number"}, {"desc","string"}}},

‎src/project/ProjectFile.cpp‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,9 +112,12 @@ R"lua(-- UserDatas ltg (LogToGraph valid only from LogToGraph)
112112
-- ltg:addSignalValue(signal_category, signal_name, signal_epoch_time, signal_value, description_string_optional) : will add a signal numerical value
113113
-- ltg:addSignalStartZone(signal_category, signal_name, signal_epoch_time, signal_string) : will add a signal start zone
114114
-- ltg:addSignalEndZone(signal_category, signal_name, signal_epoch_time, signal_string) : will add a signal end zone
115-
-- get/set epoch time from datetime in format "YYYY-MM-DD HH:MM:SS,MS" or "YYYY-MM-DD HH:MM:SS.MS" with hour offset in second param
115+
-- get/set epoch time from datetime. default Joda pattern "yyyy-MM-dd HH:mm:ss,SSS" (3rd arg = custom pattern, hour offset in 2nd)
116+
-- supported tokens: yyyy yy MM dd HH hh mm ss + trailing SSS... (N digits, fractional second, preceded by one delimiter char)
116117
-- double ltg:stringToEpoch("2023-01-16 15:24:26,464", 0)
118+
-- double ltg:stringToEpoch("16/01/2023 15:24:26.123", 0, "dd/MM/yyyy HH:mm:ss.SSS")
117119
-- string ltg:epochToString(18798798465465.546546, 0)
120+
-- string ltg:epochToString(epoch, 0, "yyyy-MM-dd HH:mm:ss.SSSSSS")
118121
-- regex (boost::regex engine, PCRE-like syntax — supports |, {m,n}, lookahead, etc.):
119122
-- local re = ltg:regex(pattern) -- compile once at script init, reuse on every row
120123
-- re:test(input) -> bool

0 commit comments

Comments
 (0)