historize every node value even if node value didn't change?

[andreasKarle]

So i have a server up and running and want to historize every value that has been set on the nodes.
The historize_node_data_change() method, as the name already says, only historizes new, changed data values.
Is it possible to historize every data value that is set with the set_value() method of the nodes? (even if it equals the actual value)
I use this for data analysis and therefor need the complet timeseries of a node.

[zerox1212]

I don't remember if UA Spec actually calls out if data should be time series or not...but I don't think that is the intention.

Anyways, maybe you could hack the code to historize a "tick" node that changes ever second, then in the call back write all other nodes to history. Perhaps another way is to just have the server write to every node with the same value on a tick. Bother are probably bad ideas though haha.

[AndreasHeine]

@zerox1212 if i remember right its an internal subscription which needs a valuechange to fire a notification in order to get logged into the db!?

[zerox1212]

Yes that is how it works, but there isn't any reason why you can't make a workaround and access the history interface directly. As long as you do not have a large number of nodes I don't see why you couldn't create a bunch of DataValue objects for the nodes and insert them into the DB yourself every X seconds.

[andreasKarle]

Sure this would work but my intention was to just get every value that was set ever, no matter if it changed.
So my usecase is to simulate a machine tool which pushs its values to the server and i want to do some data processing with it.
Therefore the original sample rate might be better than some reconstruction. Especally some sensors/signals might have different sample rates than others.
To historize all node.set_value() values is quite specific and not really efficient for the database but would be quite simple for my application😄

Do you guys think its worth creating a contribution with a feature to historize all set values or should i just solve it with some preprocessing in my client because this is not a common usecase?

[AndreasHeine]

no! only datachanges will be added to the History... if you need a whole timeseries (sampled timeseries) you must fill the gap assuming the value has not changed between two records

but you can also copy the historymanagerclass and change it to a cyclic sampling historizer and contribute it back as a new feature 😉

[AndreasHeine]

https://machinelearningmastery.com/resample-interpolate-time-series-data-python/

maybe the upsample part is somthing for you 🙂

[andreasKarle]

ok so a good solution is to just get the raw history, put it in a pandas series and resample and interpolate it with method='previous' to "hold" the values till a new historized data change occured in the series.

i just thought that this might be a usefull feature to others to, so a server side solution to get all set data points might be quite convenient.
I will have a look at it and maybe make my first contribute at all 😃
You will give it a review any ways rigth? So this might also be quite teachfull for me.

Thanks for the fast response 👍

[zerox1212]

It's always good to experiment and learn new things, but I don't think many people need/want time series history. In fact I do not think many people even use history at all.

[AndreasHeine]

in my eyes the most important usecase for historizing is if you want to avoid dataloss caused by network issues (like wireless) so you can subscribe and if the sub/con breaks for whatever reason you can make a history read from the last known timestamp (sometimes its not possible to get back or transfer the subscription) client crash... if you save the timestamp on filesystem ^^

[andreasKarle]

ok, so a making a feature that isn't used wont help anyone. so i will just go for the upsampling.
Thanks for the help again!