Add more preprocessing: combine trajectories of multiple camera views

[JuleAdrian]

Since we started to add preprocessing steps in PedPy (projecting invalid trajectories), I would like to propose to allow adding more preprocessing functions.

For example, I propose that an updated version of the existing "combine script", which is combining trajectories from different camera views to a combined trajectory set, is included here.

Two trajectory sets from different camera views:

Interpolation between two trajectories:

In the current version, the main purpose of the script is to find a time shift between the different trajectory sets.
Possible partner trajectories are searched without previously knowing who is who. Only after finding (multiple) possible partner trajectories, reasonability is checked by comparing, e.g., height and marker IDs to decide which potential partner has to be the real partner..

Since our new standard is to timely synchronize cameras by timecode and to detect individual markers in each camera view, the script should be "turned around" to first check for the presence of identical marker IDs and then search for overlapping partner trajectories in space and time.
Furthermore, we properly need to take care of trajectories in which the height-component is not constant, but varies over time.

[chraibi]

I think something important to have, but I have no experience with this problem. In Simulations, trajectories don't have this problem.
However, can you tell more about the constraints of the scripts, meaning: what are the things the scripts expect the trajectories, setup of the cameras, etc fulfil.

[JuleAdrian]

It is a quite complex script that enables to create one combined trajectory dataset based on trajectory datasets of several camera views, even if people walk back and forth between cameras. For every trajectory it is checked, if a potential partner trajectory is present in the other camera views that has an overlap in space and time. Sometimes, more than one potential partner trajectory is found, then some reasonability checks are done to determine which trajectory is the real partner.
While doing so, the script also checks for a possible time shift between the datasets and corrects the datasets according to an optimum time shift that is automatically determined.
When partner trajectories are found, the scripts spatially interpolates between the separate trajectory in order to create a smooth combined trajectory.
The script can handle present markerIDs and person heights (z-component). If MarkerIDs and heights are only present in one of the camera views, the script assigns markerIDs and heights to the other datasets according to the found partnerships.
The main combine script is executed several times, to allow the combination of multiple camera views (more than two) and to handle the combination of trajectories of persons who walk back and forth between cameras.

[JuleAdrian]

Most important pre-requisite is that we have trajectory datasets that overlap in space and time.
And in the current version we assume that the spatial camera calibration is as exact as possible. Therefore, we only check for possible time-shifts between camera data, but not for spatial shifts.

[JuleAdrian]

The main question to answer is, do we want to open up PedPy to include more of these pre-processing procedures or not?
Since we aim to include smoothing of trajectories etc. besides the already closed PR about trajectory correction, my answer would be "yes, let's include more pre-processing".
(Initially the purpose of PedPy was to focus on analysis only, I think.)