Merging C3D Files

[lukas-lbm]

Hi,

thanks for your fast respsonse to my mail. Here is the full conversation for other people that might have a similar problem:

My original question:

Hi pariterre,

I am trying to use the ezc3d package in Python (version 1.6.0) to merge two c3d files containing marker data and force plate data from the same trial. Unfortunately I am having some troubles/questions and thought maybe you could help me out.

Is there a way to access the raw analog data from a file? Using file['header']['analogs'] returns an array that appears to be downsampled to the point frame_rate.
When I artificially increase the number of samples in the analog data to fit the frame rate of the analog data collection there are no forces in the resulting file? Is this because the data is not real or do I need to define some more parameters even when using the raw analog data?

Please find attached my files and the code I am using.

Thank you!

Best regards

And your answer:

Thanks for reaching out. If possible, I would appreciate if you could copy-paste our conversation to "https://github.com/pyomeca/ezc3d/issues" so everyone else can benefit from what we discuss :)

In the meantime, I am answering here your questions so far:

1. Is there a way to access the raw analog data from a file? Using file['header']['analogs'] returns an array that appears to be downsampled to the point frame_rate.

I assume you meant file['data']['analogs']? If so, that is correct. These are the actual unmodified data. However, the provided sampling ratio between points and analogs must be sound, that is, ANALOG:RATE / POINT:RATE must be an integer. Otherwise, the data won't be stored properly. Apart from that, file['data']['analogs'] is the raw data.

2. When I artificially increase the number of samples in the analog data to fit the frame rate of the analog data collection there are no forces in the resulting file? Is this because the data is not real or do I need to define some more parameters even when using the raw analog data?

The "force" data are not stored in a C3D file, only analogs. Analyses must be applied to get the force data. Depending on if the analog data are of type TYPE-2, TYPE-4, TYPE-7 and TYPE-20 (I may be misremembering the accepted format), analogs must be stored accordingly. Changing the "force" field of the data will not change anything as they are computed values. If you want to merge data, you must append them to the file['data']['analogs'] field (meaning you must provide the corrected formatting which may or may not be possible).

Additional note : "header" field is NEVER read by ezc3d, it is always generated. If you want to modify something, you MUST do it in the PARAMETER field. That is because all information in "header" are redundant with either the parameters or the data themselves. So changing frame_rate in heading will be discarded to match the one in parameter when writing the new C3D file

So specifically for you case, you can drop anything related to header. I also had a quick look at your data and your POINT:RATE is 60.00239944 which is not ideal to create a nice and neat ratio, and you ANALOG:RATE (in the other file) is 1200. So we expect 20X more data in the second file, but we get:
file_forceplate.data.analogs.shape => (1, 16, 1922)
file_marker.data.points.shape => (4, 84, 2104)
which obvious does not make sense (for 2104 frames of points, there should be 42080 frames of analogs). So combining these file don't really make sense anyway...

Hope this helps! Again, please copy-paste this conversation in a Github issue

Have a nice day

Thanks again, this already helps me a lot. But I still have some follow up questions:

Some additional information regarding the number of frames in the files. Yes they dont match because the recordings are not synchronized but I know the offset between the files and the relevant part of the recordings so I can just crop my data accordingly and then combine the relevant parts.

Regarding the first point, yes I meant file['data']['analogs']. I dont quite understand the thing about the frame rate though. If I open FileForcePlate.c3d in Mokka it shows 1922 frames. If I export FileForcePlate.c3d to a csv file using Mokka, the csv file contains 38440 (1922 * 20) analog samples, which makes sense since the analog data was sampled at 1200 Hz, 20 times the marker data at 60 Hz. So I am assuming Mokka only shows data for each point frame and not for each analog frame? Regardless the data is in the file otherwise I couldnt see it in the csv export right?

When I use the data from the csv export and create a new c3d file that contains this data in file['data']['analogs'] and use the marker data from FileMarker.c3d in file['data']['points'] and declare the parameters from FileForcePlate.c3d in file['parameters']['FORCE_PLATFORM'] and file['parameters']['ANALOG'] and the parameters from FileMarker.c3d in file['parameters']['POINT'] I actuall get the wanted result. The new file contains the marker trajectories, the analog data and computed force data. And the analog data is sampled at 1200 Hz (even though I only see that in the csv export, in Mokka directly it again only shows data at 60 Hz).

The problem here is that I have several trials and would need to manually export the c3d files to csv files for each trial. So using a script to access the analog data from FileForcePlate.c3d would of course be nicer, but I am still facing the issue that I only get 1922 frames of analog data and not the full 38440 frames using file_forceplate['data']['analogs']. Am I still missing something obvious here?

c3d and csv files:
MergeC3D.zip

[pariterre]

Thanks for moving the conversation here :)

After you explained your confusion, I realize that ezc3d may just have a bug for your specific case. I've therefore had a look and this is indeed the case.
When no points are found in the file, I was readjusting the number of subframes to 1. Doing so, ezc3d would simply ignore most of the values (actually it would read the first nframes values and ignore the rest).

I will push a fix in few minutes ( #378 ). During the day, ezc3d version 1.6.1 should be made available which will include all the frames that were ignored in your case

[lukas-lbm]

Hi, I was just wondering is the new version already available? The newest version I can install using pip or conda is still 1.6.0.

[pariterre]

Hi @lukas-lbm

The anaconda website is showing 1.6.1 https://anaconda.org/channels/conda-forge/packages/ezc3d/files

I think PIP still has a problem though... so it may be behind

[pariterre]

After fighting a hard fight with Github's CI, I finally managed to push a version to pip :)

[lukas-lbm]

Hi,

so I just tried the new versions and 1.6.1 works :)

Version 1.6.2 also reads all the analog frames, but I cannot save the combined file. I've experimented a bit and it turns out that I cannot even save an unaltered file. So if I just read a file and write the exact same file again, without manipulating any of the contents, I still get same error:

TypeError                                 Traceback (most recent call last)
File ~\Anaconda3\envs\C3DFiles\Lib\site-packages\ezc3d\__init__.py:659, in c3d.write(self, path, first_frame_as_zero, collapse_string_matrices_to_vector, keep_trailing_spaces)
    658 try:
--> 659     new_param.set(ezc3d.VecString(old_param["value"]), dim)
    660 except:

File ~\Anaconda3\envs\C3DFiles\Lib\site-packages\ezc3d\ezc3d.py:2495, in Parameter.set(self, *args)
   2494 def set(self, *args):
-> 2495     return _ezc3d.Parameter_set(self, *args)

TypeError: Wrong number or type of arguments for overloaded function 'Parameter_set'.
  Possible C/C++ prototypes are:
    ezc3d::ParametersNS::GroupNS::Parameter::set(int)
    ezc3d::ParametersNS::GroupNS::Parameter::set(size_t)
    ezc3d::ParametersNS::GroupNS::Parameter::set(float)
    ezc3d::ParametersNS::GroupNS::Parameter::set(double)
    ezc3d::ParametersNS::GroupNS::Parameter::set(std::vector< double,std::allocator< double > > const &,std::vector< size_t,std::allocator< size_t > > const &)
    ezc3d::ParametersNS::GroupNS::Parameter::set(std::vector< double,std::allocator< double > > const &)
    ezc3d::ParametersNS::GroupNS::Parameter::set(std::string const &)


During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[2], line 1
----> 1 file_marker.write('file_marker_out.c3d')

File ~\Anaconda3\envs\C3DFiles\Lib\site-packages\ezc3d\__init__.py:661, in c3d.write(self, path, first_frame_as_zero, collapse_string_matrices_to_vector, keep_trailing_spaces)
    659         new_param.set(ezc3d.VecString(old_param["value"]), dim)
    660     except:
--> 661         raise ValueError(f"Value in parameters {group}:{param} could not be converted to string")
    662 else:
    663     raise NotImplementedError("Parameter type not implemented yet")

ValueError: Value in parameters POINT:UNITS could not be converted to string

I dont know if this has something to do with my files or if it is a general issue, just wanted to let you know.

But version 1.6.1 works for my use case so I will be using that for now. Thanks for your support!

[pariterre]

This is a bug I introduced trying to fix a warning yesterday. I will quickly push a 1.6.3 to revert that!
Thanks for reporting!

[pariterre]

Fixed :)