Skip to content

Commit 6061e5a

Browse files
authored
Update doc quickstart (#226)
* Clarify certain points of the quick start, fix typos, add schema
1 parent c06f7ec commit 6061e5a

5 files changed

Lines changed: 29 additions & 16 deletions

File tree

doc/QuickStart.md

Lines changed: 29 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ A Linux machine (CC7 or Ubuntu) or a Mac. See the O2 instructions below for the
3131
3. Prepare the QualityControl development package
3232
* `aliBuild init QualityControl@master --defaults o2`
3333

34-
4. Install GLFW to have GUIs in the DPL (optional, DPL GUIs do not work in containers).
34+
4. Install GLFW to have GUIs in the DPL (optional, DPL GUIs do not work in containers nor over SSH).
3535
* On CC7 : `sudo yum install glfw-devel --enablerepo=epel`
3636
* On Mac : `brew install glfw`
3737

@@ -52,49 +52,62 @@ To make sure that your system is correctly setup, we are going to run a basic QC
5252

5353
### Basic workflow
5454

55-
We will run a basic workflow with attached QC described in the following schema.
55+
We are going to run a basic workflow whose various processes are shown in the following schema.
5656

57-
![alt text](images/basic-schema.png)
57+
![basic-schema](images/basic-schema.png)
5858

59-
The _Producer_ is a random data generator. In a more realistic setup it would be a processing device or the _Readout_. The _Data Sampling_ is the system in charge of dispatching data samples from the main data flow to the _QC tasks_. It can be configured to dispatch different proportion or different types of data. The _Checker_ is in charge of evaluating the _MonitorObjects_ produced by the _QC tasks_, for example by checking that the mean is above a certain limit. It can also modify the aspect of the histogram, e.g. by changing the background color or adding a PaveText. Finally the _Checker_ is also in charge of storing the resulting _MonitorObject_ into the repository where it will be accessible by the web GUI. It also pushes it to a _Printer_ for the sake of this tutorial.
59+
The _Producer_ is a random data generator. In a more realistic setup it would be a processing device or the _Readout_. The _Data Sampling_ is the system in charge of dispatching data samples from the main data flow to the _QC tasks_. It can be configured to dispatch different proportion or different types of data. The __tasks__ are in charge of analyzing the data and preparing QC objects, often histograms, that are then pushed forward every cycle. A cycle is 10 second in this example. In production it is closer to 1 minute. The _Checker_ is in charge of evaluating the _MonitorObjects_ produced by the _QC tasks_. It runs _Checks_ defined by the users, for example checking that the mean is above a certain limit. It can also modify the aspect of the histogram, e.g. by changing the background color or adding a PaveText. Finally the _Checker_ is also in charge of storing the resulting _MonitorObject_ into the repository where it will be accessible by the web GUI. It also pushes it to a _Printer_ for the sake of this tutorial.
6060

6161
To run it simply do:
6262

6363
o2-qc-run-basic
6464

6565
Thanks to the Data Processing Layer (DPL, more details later) it is a single process that steers all the _devices_, i.e. processes making up the workflow. A window should appear that shows a graphical representation of the workflow. The output of any of the processes is available by double clicking a box. If a box is red it means that the process has stopped, probably abnormally.
6666

67-
![alt text](images/basic-dpl-gui.png)
67+
![basic-dpl-gui](images/basic-dpl-gui.png)
6868

69-
The presented example consists of one DPL workflow which has both the main processing and QC infrastructure declared inside. In the real case, we would usually prefer to attach the QC without modifying the original topology. It can be done by merging two (or more) workflows, as below:
69+
The example above consists of one DPL workflow which has both the main processing and the QC infrastructure declared inside. In the real case, we would usually prefer to attach the QC without modifying the original topology. It can be done by merging two (or more) workflows, as shown below:
7070

7171
o2-qc-run-producer | o2-qc --config json://${QUALITYCONTROL_ROOT}/etc/basic.json
7272

73-
This command uses two executables. The first one contains the _Producer, which represents a main data flow. The second executable generates the QC infrastructure based on given configuration file. These two workflows are joined together using the pipe | character. This example illustrates how to add QC to any DPL workflow by using `o2-qc` and passing it a configuration file.
73+
![basic-schema-2-exe](images/basic-schema-2-exe.png)
74+
75+
This command uses two executables. The first one contains only the _Producer (see Figure above), which represents the data flow to which we want to apply the QC. The second executable generates the QC infrastructure based on the given configuration file (more details in a few sections). These two workflows are joined together using the pipe `|` character. This example illustrates how to add QC to any DPL workflow by using `o2-qc-run-qc` and passing it a configuration file.
7476

7577
__Repository and GUI__
7678

7779
The data is stored in the [ccdb-test](ccdb-test.cern.ch:8080/browse) at CERN. If everything works fine you should see the objects being published in the QC web GUI (QCG) at this address : [https://qcg-test.cern.ch/?page=objectTree](https://qcg-test.cern.ch/?page=objectTree). The link brings you to the hierarchy of objects (see screenshot below). Open "QcTask" (the task you are running) and click on "example" which is the name of your histogram. The plot should be displayed on the right. If you wait a bit and hit "REFRESH NOW" in the far left menu you should see it changing from time to time (see second screenshot below).
78-
Please note that anyone running o2-qc-run-basic publishes the same object and you might see the one published by someone else.
80+
Please note that anyone running o2-qc-run-basic publishes the object under the same name and you might see the one published by someone else.
7981

8082
![alt text](images/basic-qcg1.png)
8183
![alt text](images/basic-qcg2.png)
8284

83-
TODO add a link to the user documentation of the QCG
85+
TODO add a link to the user documentation of the QCG when it is written.
8486

8587
__Configuration file__
8688

87-
The devices are configured in the config file named `basic.json`. It is installed in `$QUALITYCONTROL_ROOT/etc`. Each time you rebuild the code, `$QUALITYCONTROL_ROOT/etc/basic.json` is overwritten by the file in the source directory (`~/alice/QualityControl/Framework/basic.json`).
89+
In the example above, the devices are configured in the config file named `basic.json`. It is installed in `$QUALITYCONTROL_ROOT/etc`. Each time you rebuild the code, `$QUALITYCONTROL_ROOT/etc/basic.json` is overwritten by the file in the source directory (`~/alice/QualityControl/Framework/basic.json`).
90+
91+
The configuration for the QC is made of many parameters described in an [advanced section of the documentation](https://github.com/AliceO2Group/QualityControl/blob/master/doc/Advanced.md#configuration-files-details). For now we can just see below the definition of a task. `moduleName` and `className` specify respectively the library and the class to load and instantiate to do the actual job of the task.
92+
```json
93+
(...)
94+
"tasks": {
95+
"QcTask": {
96+
"active": "true",
97+
"className": "o2::quality_control_modules::skeleton::SkeletonTask",
98+
"moduleName": "QcSkeleton",
99+
"cycleDurationSeconds": "10",
100+
(...)
101+
```
102+
Try and change the name of the task by replace `QcTask` by a name of your choice. Relaunch the workflows. You should now see the object published under a different directory in the QCG.
88103

89104
### Readout chain
90105

91-
In this second example, we are going to use the Readout as data source.
106+
In this second example, we are going to use the Readout as our data source.
92107

93108
![alt text](images/readout-schema.png)
94109

95-
This workflow is a bit different from the basic one. The _Readout_ is not a device and thus we have to have a _proxy_ to get data from it. This is the extra box going to the _Dispatcher_, which then injects data to the task. This is handled in the _Readout_ if you enable the corresponding configuration flag.
96-
97-
TODO make the qc task use the daq code
110+
This workflow is a bit different from the basic one. The _Readout_ is not a DPL, nor a FairMQ, device and thus we have to have a _proxy_ to get data from it. This is the extra box going to the _Data Sampling_, which then injects data to the task. This is handled in the _Readout_ as long as you enable the corresponding configuration flag.
98111

99112
To do so, open the readout config file located at `$READOUT_ROOT/etc/readout.cfg` and make sure that the following properties are correct :
100113

@@ -116,7 +129,7 @@ Start Readout :
116129
readout.exe file://$READOUT_ROOT/etc/readout.cfg
117130
```
118131

119-
Start the proxy, DS (DataSampling) and QC workflows :
132+
Start the proxy, DataSampling and QC workflows :
120133
```
121134
o2-qc-run-readout | o2-qc --config json://${QUALITYCONTROL_ROOT}/etc/readout.json
122135
```
@@ -138,7 +151,7 @@ equipment-player-* fillPage int 1 If 1, content of data file is copied multi
138151

139152
#### Readout data format as received by the Task
140153

141-
The header is a O2 header populated with data from the header built by the Readout.
154+
The header is an O2 header populated with data from the header built by the Readout.
142155
The payload received is a 2MB (configurable) data page made of CRU pages (8kB).
143156

144157
__Configuration file__

doc/images/basic-schema-2-exe.png

34.2 KB
Loading

doc/images/basic-schema.graffle

2.77 KB
Binary file not shown.

doc/images/basic-schema.png

5.19 KB
Loading

doc/images/readout-schema.png

7.73 KB
Loading

0 commit comments

Comments
 (0)