From 345feefe7e911a40e4150bd60572e9b05b5cb9a2 Mon Sep 17 00:00:00 2001 From: Steffengreiner Date: Thu, 26 Feb 2026 16:35:49 +0100 Subject: [PATCH 1/5] docs: apply AI-generated documentation overhaul from proposals Replace all 22 documentation pages and mkdocs.yml with the reviewed proposal versions generated by the multi-agent debate system. Changes include plain-language rewrites across every section, consistent step-by-step formatting, What's next navigation footers, and a restructured nav in mkdocs.yml (workflow order, User account section, Ontology search flattened, Developers & Stewards renamed to Reference). --- docs/batch/sample-batch.md | 127 +++----- docs/developers/api.md | 20 +- docs/experiment/confounding-variables.md | 49 ++- docs/experiment/experiment_creation.md | 111 +++---- docs/experiment/experiment_introduction.md | 37 ++- docs/get_started/process_overview.md | 86 ++++- docs/index.md | 194 +++++------ docs/measurement/measurement_edit.md | 157 +++------ docs/measurement/measurement_introduction.md | 30 +- docs/measurement/measurement_registration.md | 144 ++++----- docs/metadata/concepts.md | 255 +++++++-------- .../ontology_search_introduction.md | 61 ++-- docs/project/project_access.md | 85 ++--- docs/project/project_edit.md | 102 +++--- docs/project/project_introduction.md | 53 +-- docs/project/project_registration.md | 97 +++--- docs/rawdata/raw_data_download.md | 178 ++++------- .../rawdata/raw_data_request_server_access.md | 95 +++--- docs/rawdata/raw_data_upload.md | 301 ++++++------------ docs/user/password_reset.md | 59 ++-- docs/user/user_edit.md | 64 ++-- docs/user/user_registration.md | 104 +++--- mkdocs.yml | 43 ++- 23 files changed, 1056 insertions(+), 1396 deletions(-) diff --git a/docs/batch/sample-batch.md b/docs/batch/sample-batch.md index 3827982..cd431f2 100644 --- a/docs/batch/sample-batch.md +++ b/docs/batch/sample-batch.md @@ -1,111 +1,80 @@ -# Sample Batch -[//]: # (What is a sample batch?) -## Definition +# Sample batches -A sample batch forms a logical container of samples that are going to be shipped together to the -measurement facility with the intention of being processed under the same conditions. +## What is a sample batch? -[//]: # (What is the purpose of grouping samples into batches?) -## Intention +A sample batch is a group of samples shipped together to a measurement facility and processed under the same conditions. Tracking batches is how you identify and correct for **batch effects** — technical variation introduced when samples are processed at different times or under different conditions. -Grouping and processing samples as distinct sample batches is key in properly tracking and avoiding batch effects. -Click [here](https://pmc.ncbi.nlm.nih.gov/articles/PMC3880143/) for a quick introduction into batch effect and their impact on research data. +Undocumented batches are one of the most common causes of irreproducible results. [Learn more about batch effects](https://pmc.ncbi.nlm.nih.gov/articles/PMC3880143/). -[//]: # (How do I add samples to my experiment?) ## Creating and registering sample batches -We call the process of adding samples to an experiment "sample registration". -When you want to link samples to an experiment you need to register metadata for those samples. +1. In the experiment summary, click **Register sample batch**. -To **start** with the sample registration process, click on the button `Register sample batch` to open the sample registration dialog. +2. **Download the template** — an Excel spreadsheet with one row per sample. -Please go ahead and **download the metadata template file** from the dialog. -In this file you can fill out information for the samples you want to register. + !!! note "Mandatory fields" + Columns marked with an asterisk (`*`) are mandatory. -!!! note - **Mandatory** information for the sample registration is marked by an asterix `*` after the column name. +3. Fill in the template. For each sample, record its species, specimen, analyte, experimental group, and any other required details. -**Once you filled out** all the information, you can go back to the dialog. In case you closed the dialog, simply re-open it. -
-Please go ahead and **upload your filled metadata file** in the dialog. -Once the Data Manager validated the information in your file, -go ahead and **choose a name for your batch**. -
-Once you named your batch and uploaded the file with the necessary information, go ahead and **click the -`Register`** button. The Data Manager will now go through the process of creating samples within your -experiment. + !!! tip "Sample IDs are assigned automatically" + Don't fill in the Sample ID column. The system generates a unique, permanent ID for each sample (e.g. `Q2ABCD001AA`) when you register the batch. -!!! info "Email Notification" - Upon successful batch registration, - all [project collaborators](../project/project_access.md#add-collaborator) will automatically receive - an email with a link to the created batch. +4. Upload the completed file in the registration dialog. The system validates it and highlights any errors. -After the Data Manager is done registering your samples, you can **close** the dialog **by clicking the -`Finish`** button. -Now samples annotated with the provided metadata are registered to your experiment. You can see the -newly created batch next to the other batches in the samples view. +5. Enter a **batch name** (e.g. "Pilot cohort — January 2026"). -!!! info "SampleId" - Upon successful batch registration, each sample will be associated with a unique SampleId - distinguishing it from other samples within the system - an email with a link to the created batch. +6. Click **Register**. The system processes your samples in the background. -[//]: # (How do I edit existing samples in my experiment?) -## Editing sample batches +7. Click **Finish** when done. + +!!! info "Email notification" + All [project collaborators](../project/project_access.md#add-collaborator) receive an email with a link to the new batch. -You might need to edit sample metadata after registering the sample batch to the experiment. -Editing sample information is restricted to editing the sample metadata. Adding or removing samples from a batch is not possible. +??? info "Sample ID format" + Sample IDs follow the pattern `Q2XXXXNNNCC`: the six-character project code, a three-digit running number, and a two-character random suffix (e.g. `Q2ABCD001AA`). They are permanent and cannot be changed after registration. + +## Editing sample batches -To **start** with the sample edit process, click on the edit button next to the batch you want to edit. This will open the batch editing dialog. +You can update sample metadata after registration. You cannot add or remove individual samples from a batch. -!!! info "Project role" - Should you not see the action column, - please make sure that you have been granted the "write" or "admin" role to it by the project owner/admin. +1. Click the **edit** button next to the batch. -Please go ahead and **download the metadata template file** from the dialog. -In this file you can fill out information for the samples you want to register. + !!! info "Required role" + You need **write** or **admin** role. -!!! note - **Mandatory** information for the sample editing is marked by an asterix `*` after the column name. +2. **Download the current metadata** — the template is pre-filled with existing values. -Note that the information in greyed out columns such as e.g. `Sample Id` is immutable and changes made within will not be registered during the sample editing process. +3. Make your changes. Grey columns (e.g. `Sample Id`) are locked — changes there are ignored. -!!! note - Information shown in greyed out columns are immutable and cannot be changed. + !!! note "Mandatory fields" + Columns marked with `*` remain mandatory. -**Once you filled out** all the information, you can go back to the dialog. In case you closed the dialog, simply re-open it. -
-Please go ahead and **upload your filled metadata file** in the dialog. -Once the Data Manager validated the information in your file, -go ahead and **choose a name for your batch** if you want to change it. -
-Once you named your batch and uploaded the file with the necessary information, go ahead and **click the -`Edit batch`** button. The Data Manager will now go through the process of updating the sample information within your -experiment. +4. Upload the edited file, optionally rename the batch, and click **Edit batch**. -After the Data Manager is done registering your samples, you can **close** the dialog **by clicking the -`Finish`** button. +5. Click **Finish**. -[//]: # (How do I delete existing samples in my experiment?) -## Delete a sample batch +## Deleting a sample batch -!!! warning "Batch Deletion" - Keep in mind, that deleting a batch will also delete all sample metadata of the samples within the batch +!!! danger "This is irreversible" + Deleting a batch permanently removes all sample metadata for every sample in that batch. -To **start** with the sample batch deletion process, click on the **`delete`** button next to the batch in question within the action column. +!!! warning "Measurements must be removed first" + A batch can only be deleted if none of its samples are referenced by a measurement. [Delete the measurements](../measurement/measurement_edit.md#delete-measurements) first. -!!! info "Project role" - Should you not see the action column, - please make sure that you have been granted the "write" or "admin" role to it by the project owner/admin. +1. Click the **delete** button next to the batch. -This will open the batch deletion dialog requiring **confirmation** of the batch deletion process by clicking the **`Confirm`** button. + !!! info "Required role" + You need **write** or **admin** role. -!!! warning "Attached Measurements" - Keep in mind, that batches can only be deleted if none of the samples within the batch have been used in a measurement.) - Otherwise, you need to delete the measurements in question before the batch can be deleted. +2. Click **Confirm** in the dialog. -[//]: # (How do I download sample metadata) ## Download sample metadata -To **download** the sample metadata for all registered samples click on the **`Download sample metadata`** button. -This will export all registered metadata as an `.xlsx` file to your local download directory. +Click **Download sample metadata** to export all registered sample metadata as an `.xlsx` file. This is useful for copying sample IDs into [measurement registration spreadsheets](../measurement/measurement_registration.md). + +--- + +## What's next + +➡ [Register measurements](../measurement/measurement_registration.md) · [Back to experiment](../experiment/experiment_introduction.md) diff --git a/docs/developers/api.md b/docs/developers/api.md index 5f6ca08..04d1a46 100644 --- a/docs/developers/api.md +++ b/docs/developers/api.md @@ -1,14 +1,20 @@ -# Data Manager API +# API reference -Access to resources in Data Manager via a RESTful API. +Access Data Manager resources programmatically via a RESTful API. -!!! info "API endpoints" - Currently, only one endpoint exists, which allows for a download of raw data. It is planned to extend the API in - the future to interact with various resources in Data Manager. +!!! info "Available endpoints" + The API currently provides a single endpoint for raw data download. Additional endpoints are planned. +## Documentation -## Swagger API docs +Interactive API documentation is available via Swagger UI: -Detailed API documentation is available via Swagger UI and [hosted on the web service](https://download.qbic.uni-tuebingen.de/swagger-ui/index.html). +**[Open Swagger UI](https://download.qbic.uni-tuebingen.de/swagger-ui/index.html)** +Authentication uses the same [personal access token](../rawdata/raw_data_download.md#personal-access-token) as the command-line download. +--- + +## What's next + +➡ [Download raw data](../rawdata/raw_data_download.md) · [Metadata glossary](../metadata/concepts.md) diff --git a/docs/experiment/confounding-variables.md b/docs/experiment/confounding-variables.md index 3d3ab40..30b79a7 100644 --- a/docs/experiment/confounding-variables.md +++ b/docs/experiment/confounding-variables.md @@ -1,41 +1,32 @@ -# Confounding Variables +# Confounding variables -!!!info - _"Confounding, sometimes referred to as confounding bias, is mostly described as a ‘mixing’ or - ‘blurring’ of effects. It occurs when an investigator tries to determine the effect of an - exposure on the occurrence of a disease (or other outcome), but then actually measures the effect - of another factor, a confounding variable."_ - - Jager, K. J., Zoccali, C., MacLeod, A., & Dekker, F. W. (2008). Confounding: What it is and how to deal with it. Kidney International, 73(3), 256-260. [https://doi.org/10.1038/sj.ki.5002650](https://doi.org/10.1038/sj.ki.5002650) +A confounding variable is something you didn't control in your experiment but that might still affect your results — the time of day samples were collected, which lab member processed them, or whether a reagent lot changed mid-experiment. Recording these factors lets you account for them during analysis. -### Define confounding variables in your experiment +!!! info "Why this matters" + *"Confounding occurs when an investigator tries to determine the effect of an exposure on the occurrence of a disease (or other outcome), but then actually measures the effect of another factor."* + — Jager et al. (2008), *Kidney International*, 73(3), 256–260. [DOI](https://doi.org/10.1038/sj.ki.5002650) -In order to work with confounding variables, first you must define them. -Confounding variables are experiment-specific. Currently, it is not possible to reuse the same -variable in other experiments metadata in Data Manager. +Confounding variables are defined per experiment and cannot currently be shared across experiments. -Confounding variables are defined in your experiment, so navigate to your experiment first. -There you can find the controls to add new confounding variables. +## Define a confounding variable -### Assign levels of a confounding variable to your sample +Navigate to your experiment and locate the confounding variables section. Add a variable by providing its name (e.g. `Collection time`, `Operator`, `Passage number`). -To get the full potential and account for effects of confounding variables in your experiment, you -can now annotate the samples with your observation of the variables' manifestation. -In contrast to experimental variables, not all samples need to have a level in one confounding variable. +## Assign values to samples -After [defining the confounding variables in your experiment](#define-confounding-variables-in-your-experiment), -the confounding variables are added in the sample registration and sample editing process. -In the sample registration and sample editing processes, you have the possibility to annotate -samples with levels for the confounding variables. +After defining variables, annotate individual samples during [batch registration](../batch/sample-batch.md#creating-and-registering-sample-batches) or [batch editing](../batch/sample-batch.md#editing-sample-batches). Not every sample needs a value — fill in only what you observed. -### Rename a confounding variable +## Rename a variable -To remove a confounding variable, navigate to your experiment, locate the confounding variables editing and rename your confounding variable. -After renaming a variable, the levels are preserved and only the name of the confounding variable are changed. +Open the confounding variables editing controls, enter the new name, and save. Existing sample values are preserved. -### Delete a confounding variable +## Delete a variable -To remove a confounding variable navigate to your experiment, locate the confounding variables editing and remove the variables you want to delete. +!!! danger "This is irreversible" + Deleting a confounding variable permanently removes **all values assigned to that variable across every sample in the experiment**. Export your [sample metadata](../batch/sample-batch.md#download-sample-metadata) first if you need a record. -!!!warning - Removing a confounding variable will delete all annotated levels of that variable on all samples in your experiment. +--- + +## What's next + +➡ [Register sample batches](../batch/sample-batch.md) · [Back to experiment overview](experiment_introduction.md) diff --git a/docs/experiment/experiment_creation.md b/docs/experiment/experiment_creation.md index de1aada..2d48fe6 100644 --- a/docs/experiment/experiment_creation.md +++ b/docs/experiment/experiment_creation.md @@ -1,83 +1,74 @@ -# Experiment Creation +# Create an experiment -Start by [navigating](../project/project_introduction.md#project-navigation) -to the project summary view of your project of interest. -To create an experiment in the selected project, click the add button within the experiment component on the top right. -![project_summary](../project/images/project_summary.png){.screenshot} +[Navigate](../project/project_introduction.md#find-and-open-a-project) to the project summary. Click the **add** button in the experiment section (top right). -!!! info "Project role" - Should you not see the add button, - ensure that you have been granted the "write" or "admin" project role by the project owner/admin! +![Project summary](../project/images/project_summary.png){.screenshot} -This will open the create experiment dialog, in which you can specify the name of the experiment and the included species, specimen and analyte -information of the involved organisms. -To provide these details you can select one or more entries via the provided search fields. -Start by providing an easily identifiable name and at least 2 letters of your species, specimen or analyte in their respective inputs -fields and possible selection options will appear. -![experimental_information](images/create_experiment_search.png){.screenshot} +!!! info "Required role" + You need **write** or **admin** role to create experiments. -!!! info "Ontology id" - Behind each selectable option within the species, specimen and analyte input fields - , the unique ontology identifier from one of several ontologies is stored. +In the dialog, provide: -Optionally you can also select one of the provided icons most fitting for your selected species and specimen. +- **Experiment name** — unique within the project, easy to identify +- **Species** — the organism(s) your samples come from (type at least 2 characters to search) +- **Specimen** — the biological material type (e.g. blood, liver tissue) +- **Analyte** — the molecular class being measured (e.g. protein, mRNA) -Once all the required information has been provided you can create your experiment via the "Add" -button below. -![experimental_information](images/create_experiment.png){.screenshot} +You can select multiple entries per field. Optionally choose an icon for the species and specimen. +![Search active](images/create_experiment_search.png){.screenshot} -## Experimental Variable Creation +Click **Add**. -Experimental variables can be considered as the building blocks for the condition with which the samples in your experiments were treated. -One or more of these variables can be added in the experiment summary screen by selecting the experimental variables tab and pressing the "add variables" button within the disclaimer. -![experiment_summary](images/experimental_summary_no_variables.png){.screenshot} +![Completed](images/create_experiment.png){.screenshot} -!!! info "Project role" - Should you not see the add button, - ensure that you have been granted the "write" or "admin" project role by the project owner/admin! +## Define experimental variables -Once clicked, the add variables dialog will appear, in which one or more variables can be defined and linked to your experiment. -![add_experimental_variables](images/add_experimental_variables.png){.screenshot} +Experimental variables are the conditions you're testing — drug concentration, temperature, time point, and so on. They're the building blocks of your experimental groups. -Defined variables can also be deleted easily via a click on the cross icon next to the variable -![add_experimental_variables_deleted](images/add_experimental_variables_deleted.png){.screenshot} +In the experiment summary, select the **Experimental Variables** tab and click **Add variables**. -Once all variables have been defined, they can be linked to the experiment by clicking the "Add" button within the dialog. -![experiment_summary](images/add_experimental_variables_summary.png){.screenshot} +![No variables](images/experimental_summary_no_variables.png){.screenshot} -Finally, already defined experimental variables can be edited by clicking on the edit button within the Experimental Variables tab, -which will open the edit experimental variable dialog in which the necessary changes can be made and saved via a click on the save button. -![edit_experimental_variables](images/edit_experimental_variables.png){.screenshot} +!!! info "Required role" + You need **write** or **admin** role. -!!! info "Editing Variables" - Keep in mind, that variables can only be edited/deleted if no experimental groups are defined within the experiment. - Otherwise, you need to delete the experimental groups before the variables can be edited. +For each variable, provide: -## Experimental Group Creation +- **Name** — e.g. `Temperature` +- **Levels** — the values it can take: e.g. `0`, `10`, `100` +- **Unit** *(optional)* — e.g. `°C` -Experimental groups contain the biological replicates which experienced the same condition defined by its experimental variables within your experiment. -You can define your samples into distinct groups via a selection of the experimental groups tab within the experiment summary screen. -In this tab press the "Add groups" button within the disclaimer. -![experiment_summary](images/experimental_summary_no_groups.png){.screenshot} +![Add variables](images/add_experimental_variables.png){.screenshot} -!!! info "Project role" - Should you not see the add button, - ensure that you have been granted the "write" or "admin" project role by the project owner/admin! +Click **Add** to save. To edit later, click **edit** in the tab and then **Save**. -Once clicked, the add experimental group dialog will appear, in which one or more groups can be defined via a selection of the previously defined experimental variable and linked to your experiment. -![add_experimental_variable](images/add_experimental_groups.png){.screenshot} +![Variables saved](images/add_experimental_variables_summary.png){.screenshot} -Defined groups can also be deleted easily via a click on the cross icon next to the group -![add_experimental_variables_deleted](images/add_experimental_groups_deleted.png){.screenshot} +!!! warning "Variables are locked once groups exist" + Delete all experimental groups before editing variables. -Once all variables have been defined, they can be linked to the experiment by clicking the "Add" button within the dialog. -![experiment_summary](images/add_experimental_groups_summary.png){.screenshot} +## Define experimental groups -Finally, already defined experimental groups can be edited by clicking on the edit button within the Experimental Groups tab, -which will open the edit experimental groups dialog in which the necessary changes can be made and saved via a click on the save button. -![edit_experimental_groups](images/edit_experimental_groups.png){.screenshot} +Groups represent unique combinations of variable levels — the distinct conditions your samples experienced. Each group holds a set of biological replicates. -!!! info "Editing Groups" - Keep in mind, that groups can only be edited/deleted if no samples have been registered to the experiment. - Otherwise, you need to remove the registered samples from the experiment before the groups can be edited. +Select the **Experimental Groups** tab and click **Add groups**. + +![No groups](images/experimental_summary_no_groups.png){.screenshot} + +For each group, select the variable levels that define its condition and enter the number of biological replicates. + +![Add groups](images/add_experimental_groups.png){.screenshot} + +Click **Add** to save. + +![Groups saved](images/add_experimental_groups_summary.png){.screenshot} + +!!! warning "Groups are locked once samples are registered" + Remove all registered samples before editing groups. + +--- + +## What's next + +➡ [Register sample batches](../batch/sample-batch.md) · [Define confounding variables](confounding-variables.md) diff --git a/docs/experiment/experiment_introduction.md b/docs/experiment/experiment_introduction.md index 28cc450..c65bdd2 100644 --- a/docs/experiment/experiment_introduction.md +++ b/docs/experiment/experiment_introduction.md @@ -1,23 +1,28 @@ -# Experiment Introduction +# Experiments -After a successful login you will be redirected to your personal landing page. -From there, [navigate](../project/project_introduction.md#project-navigation) into your project of interest to see the project summary view from which you can -[navigate](#experiment-navigation) into the experiment summary view. -Within this view you are able to [create](experiment_creation.md) new experiments. +An experiment captures the biological design of a study within a project — the species, specimens, analytes, variables, and groups that structure your sample and measurement metadata. -## Experiment Navigation +Each project can contain multiple experiments (e.g. a pilot run and a full study). -From the project summary you can find your experiments via the experiment list on the top right. -Click on the name of an experiment to load the respective experiment. -![project_summary](../project/images/project_summary.png){.screenshot} +## Find and open an experiment -Alternatively you can also select your experiment of interest via clicking on its name in the -application drawer to the left. All of the project's experiments are also available when you are already in one of the experiments. -![project_summary_drawer](../project/images/project_summary_drawer.png){.screenshot} +From the project summary, find your experiments in the list on the top right. Click an experiment name to open it. + +![Project summary](../project/images/project_summary.png){.screenshot} + +Or open the application drawer and select an experiment by name. + +![Drawer](../project/images/project_summary_drawer.png){.screenshot} !!! info "Application drawer" - The application drawer can be used to navigate between projects and experiments once a project has been selected. - It can be opened and closed via it's button on the top left. + The drawer lists all experiments in the current project. Use it to switch between experiments without going back to the project summary. + +This opens the **experiment summary view**. + +![Experiment summary](images/experimental_summary_no_variables.png){.screenshot} + +--- + +## What's next -This will take you to the experiment summary view -![experiment_summary](images/experimental_summary_no_variables.png){.screenshot} +➡ [Create a new experiment](experiment_creation.md) · [Define confounding variables](confounding-variables.md) · [Register sample batches](../batch/sample-batch.md) · [View measurements](../measurement/measurement_introduction.md) diff --git a/docs/get_started/process_overview.md b/docs/get_started/process_overview.md index ef57029..37d512e 100644 --- a/docs/get_started/process_overview.md +++ b/docs/get_started/process_overview.md @@ -1,10 +1,76 @@ -# Process Overview - -1. Register an [account](../user/user_registration.md) -2. Register your first [project](../project/project_introduction.md) -3. Specify the [experimental design](../experiment/experiment_introduction.md) -4. Register [Sample Batches](../batch/sample-batch.md) -5. Specify the performed [measurements](../measurement/measurement_introduction.md) -6. Upload your [raw data](../rawdata/raw_data_request_server_access.md). -7. Fine tune the accessibility of your project to [collaborators](../project/project_access.md)! -8. Share your measurement data [easily](../rawdata/raw_data_download.md) +# Process overview + +Here's the complete Data Manager workflow — from creating your account to sharing data with collaborators. + +```mermaid +graph LR + A(1. Create account) --> B(2. Register project) + B --> C(3. Design experiment) + C --> D(4. Register samples) + D --> E(5. Register measurements) + E --> F(6. Upload raw data) + F --> G(7. Manage access) + G --> H(8. Share data) +``` + +## Step by step + +### 1. Create your account + +Register a free Data Manager account using your ORCID (recommended) or email address. ORCID is a free, permanent researcher identifier — linking it means one fewer password to remember. + +➡ [Register your account](../user/user_registration.md) + +### 2. Register a project + +A project is the top-level container for your research — it holds experiments, samples, measurements, and data files in one place. You'll provide a title, objective, contacts, and funding details. + +➡ [Register a project](../project/project_registration.md) + +### 3. Design your experiment + +Define the biological design of your study: which organisms, tissues, and analytes you're working with, and what experimental conditions you're testing. + +➡ [Create an experiment](../experiment/experiment_creation.md) + +!!! tip "Track confounding variables" + Factors you didn't control — like sample collection time or operator — can be recorded as [confounding variables](../experiment/confounding-variables.md) so you can account for them during analysis. + +### 4. Register your samples + +Samples are registered in groups called **batches** — the set of samples shipped together to a measurement facility. Download the Excel template, fill in your sample metadata, and upload it. + +➡ [Register a sample batch](../batch/sample-batch.md) + +### 5. Register your measurements + +After your samples have been measured, record what was done: which instrument, which protocol, and other domain-specific details. Each measurement gets a unique identifier that links back to its sample. + +➡ [Register measurements](../measurement/measurement_registration.md) + +### 6. Upload your raw data + +Transfer instrument output files to the QBiC upload server via SFTP. Each file gets linked to its measurement record. + +!!! info "External collaborators" + Uploading from outside the University of Tübingen network? [Request server access](../rawdata/raw_data_request_server_access.md) first. + +➡ [Upload raw data](../rawdata/raw_data_upload.md) + +### 7. Manage project access + +Invite collaborators and assign roles — viewer, writer, or admin — to control who can see and edit your data. + +➡ [Manage access](../project/project_access.md) + +### 8. Share your data + +Generate download URLs so collaborators or reviewers can retrieve your raw data files directly. + +➡ [Download data](../rawdata/raw_data_download.md) + +--- + +## Need help? + +Contact the QBiC support team at [support@qbic.zendesk.com](mailto:support@qbic.zendesk.com). diff --git a/docs/index.md b/docs/index.md index 9b84e65..cbbfd90 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,137 +1,107 @@ # Life Science Data Management -Behind every great research project should be great research data management! -Start your voyage towards a __FAIR__ and __Open Data__ future and include -the [Data Manager](https://rdm.qbic.uni-tuebingen.de/login) platform early on in your research! +Good science starts with good data management. The [QBiC Data Manager](https://rdm.qbic.uni-tuebingen.de/login) helps you keep your research data organised, findable, and ready to share — from your first experiment through to publication. -## What's new? +Not sure where to begin? Head to the **[Get started guide](get_started/process_overview.md)**. -### Data Submission from outside the University Tübingen network +--- -
October 10th, 2025
+## What's new -- External Data Submission: Check out how - to [request access ](rawdata/raw_data_request_server_access.md) - and [submit your data](rawdata/raw_data_upload.md) from outside the University Tübingen network -- Orcid Linkage documentation: Learn how to [link your orcid account](user/user_edit.md#link-orcid) - within the data manager +### Data submission from outside the University of Tübingen network + +
October 10th, 2025
+ +- **External data submission:** Collaborators outside the university network can now upload measurement data. See how to [request server access](rawdata/raw_data_request_server_access.md) and [upload your data](rawdata/raw_data_upload.md). +- **ORCID linking:** Connect your ORCID account to an existing Data Manager profile at any time from your [user profile](user/user_edit.md#link-orcid). + +--- ## Update history -### Orcid Linkage and Template Registration Overhaul +### ORCID linking and simpler template registration -
September 18th, 2025
+
September 18th, 2025
-📌 Checkout the -complete [release notes](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.11.0) on -GitHub. +📌 Full [release notes (v1.11.0)](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.11.0) on GitHub. -💡 Highlights: +Highlights: -- Orcid Linkage: If you didn't add your orcid account during account registration you can - catch up now by linking your orcid account in your user profiles -- Simplify sample registration: Downloading the batch registration template can now be done within - the sample registration dialog - directly. -- Simplify measurement registration: Specify the domain within the measurement registration dialog - directly download the domain - specific template -- Optional measurement name: The measurement registration sheets for proteomics and genomics now - feature a column "measurement - name" to optionally store the internally assigned lab identifier -- Raw data filtering: The registered datasets can now be filtered by their properties via a - dedicated search field -- Download Urls for filtered raw data: Triggering the download of dataset URLS provides only the - URLs of the filtered dataset +- **ORCID linking:** Link your ORCID from your user profile — no need to re-register. +- **Simpler sample registration:** Download the batch template directly from the registration dialog. +- **Simpler measurement registration:** Pick your domain (proteomics or genomics) and download the template in one step. +- **Optional measurement name:** Both registration sheets now include a `Measurement Name` column for your internal lab identifier. +- **Raw data filtering:** Filter registered datasets by properties using the search field. +- **Filtered download URLs:** URL export now includes only the datasets matching your current filter. -### Metadata Examples and Descriptions +### Metadata glossary -
September 18th, 2025
+
September 18th, 2025
-- Glossary: Introduction of a glossary explaining the terms with example values in the research data - management documentation. Check it out [here](metadata/concepts.md) +- A new [Metadata glossary](metadata/concepts.md) explains every field with plain-language descriptions and example values. -### Enable Asynchronous Sample and Experiment Update +### Faster sample and experiment updates -
June 10th, 2025
+
June 10th, 2025
-- Enable the user to trigger sample creation and update asynchronously -- Enable the user to trigger experiment update processes asynchronously. -- Keep him informed of the progress via dedicated toast notifications +- Sample and experiment updates now run in the background. A notification confirms when the process is complete. -### Streamline service API within the Data Manager backend to enable CRUD processes asynchronously +### Background project creation -
March 4th, 2025
+
March 4th, 2025
-- Enable the user to trigger project creation asynchronously -- Keep him informed of the progress via dedicated toast notifications -- Update the Data Manager to Java 21 +- Project creation now runs in the background with progress notifications. ### Confounding variable support -
February 12th, 2025
- -- Experimental design: The management of confounding variables on experiment and sample level is now - possible -- Ontology: Ontology terms are now queried from TIB's new [APIv3](https://terminology.tib.eu/ts/api) - by default, which uses the improved OLS4 backend -- Measurements: Technical replicate information is now shown for measurements -- Measurements: Sample pool names are now displayed in the measurement overview -- Metadata: Improved Excel file export that also displays the ontology terms CURIE - -Check out the -full [release notes](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.8.0) on GitHub. - -### New project summary layout - -
November 14th, 2024
- -- A completely new design of the project summary, that targets many improvements for accessing and - updating high level project information -- Enhances the spreadsheet templates with examples using what Microsoft - calls [Input Messages](https://support.microsoft.com/en-us/office/more-on-data-validation-f38dee73-9900-4ca6-9301-8a5f6e1f0c4c) - and links to further information resources -- Spreadsheet templates are no static documents anymore, but generated dynamically - -### Excel spreadsheets now supported for sample batch registration - -
October 23rd, 2024
- -- Sample Batches can now be registered and updated - directly [with XLSX spreadsheets](batch/sample-batch.md). -- RO-Crate Export: The project summary information can now - be [downloaded](project/project_edit.md#download-project-metadata) as an - RO-Crate to your local filesystem within the project summary. - For more information on RO-Crates visit [here](https://www.researchobject.org/ro-crate/). -- Some smaller [bug fixes](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.5.0). - -### Excel spreadsheets now supported for measurements - -
September 4th, 2024
- -- Measurements can now be registered and updated - directly [with XLSX spreadsheets](measurement/measurement_introduction.md). TSV is still - supported. -- Sample metadata: the term `Organism ID` has been renamed to `Biological Replicate` to match its - purpose of use. -- The Data Manager is now connected to the [TIB terminology service](https://terminology.tib.eu). - The [queried ontologies](ontology_search/ontology_search_introduction.md) are restricted to life - science specific ones. You miss one? Please let us - know and submit - a [feature request](https://github.com/qbicsoftware/data-manager-app/issues/new/choose). Currently - included ontologies are: - - Bio-assay Ontology (__BAO__) - - Brenda Tissue Ontology (__BTO__) - - Chemical Entities of Biological Interest (__CHEBI__) - Bioinformatics operations, data types, formats, identifiers and topics (__EDAM__) - - Experimental Factor Ontology (__EFO__) - - Environmental Factor Ontology (__ENVO__) - - Gene Ontology (__GO__) - - Molecular Interaction (__MI__) - - PSI Mass Spectrometry Ontology (__MS__) - - National Cancer Institute Thesaurus (__NCIT__) - - Plant Ontology (__PO__) -- For - species, [terms can be selected](ontology_search/ontology_search_introduction.md) - from [NCBI's tree of life](https://doi.org/10.1371/journal.pgen.1005912). -- Some smaller [bug fixes](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.4.0). +
February 12th, 2025
+ +- **Confounding variables:** Track factors that might influence your results at the experiment and sample level. +- **Updated ontology search:** Terms now come from the improved [TIB Terminology Service](https://terminology.tib.eu/ts/api). +- **Measurement details:** Technical replicate information and sample pool names are now visible in the measurement overview. +- **Better metadata export:** Excel exports now include the ontology identifier (CURIE) alongside each human-readable label. + +Full [release notes (v1.8.0)](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.8.0). + +### Redesigned project summary + +
November 14th, 2024
+ +- New project summary layout with easier access to project information. +- Spreadsheet templates now include helpful examples and validation tooltips. +- Templates are generated dynamically — always up to date. + +### Excel support for sample batch registration + +
October 23rd, 2024
+ +- Register and update sample batches with [Excel spreadsheets](batch/sample-batch.md). +- **RO-Crate export:** Download your project metadata as a [structured, machine-readable package](project/project_edit.md#download-project-metadata). Learn more about [RO-Crate](https://www.researchobject.org/ro-crate/). +- [Bug fixes](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.5.0). + +### Excel support for measurements + +
September 4th, 2024
+ +- Register and update measurements with [Excel spreadsheets](measurement/measurement_introduction.md). TSV still supported. +- The sample field `Organism ID` has been renamed to `Biological Replicate` to better reflect its purpose. +- Connected to the [TIB Terminology Service](https://terminology.tib.eu) for standardised scientific vocabulary. Currently supported ontologies: + + | Abbreviation | Full name | + |---|---| + | **BAO** | Bio-assay Ontology | + | **BTO** | Brenda Tissue Ontology | + | **CHEBI** | Chemical Entities of Biological Interest | + | **EDAM** | Bioinformatics operations, data types, formats, identifiers and topics | + | **EFO** | Experimental Factor Ontology | + | **ENVO** | Environmental Ontology | + | **GO** | Gene Ontology | + | **MI** | Molecular Interaction | + | **MS** | PSI Mass Spectrometry Ontology | + | **NCIT** | National Cancer Institute Thesaurus | + | **PO** | Plant Ontology | + + Missing an ontology? [Submit a feature request](https://github.com/qbicsoftware/data-manager-app/issues/new/choose). + +- Species terms come from [NCBI's tree of life](https://doi.org/10.1371/journal.pgen.1005912). +- [Bug fixes](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.4.0). diff --git a/docs/measurement/measurement_edit.md b/docs/measurement/measurement_edit.md index e00a920..00312ba 100644 --- a/docs/measurement/measurement_edit.md +++ b/docs/measurement/measurement_edit.md @@ -1,150 +1,67 @@ -# Measurement Edit +# Edit measurements -!!! tip "Excel file support" - The Data Manager now supports measurement registration and updates with Excel files (*.xlsx). +!!! tip "Excel is supported" + Upload edited metadata as an Excel file (`.xlsx`). Metadata must be on the first sheet. - __Only requirement__: metadata must be in the first sheet of your workbook. +[Navigate](measurement_introduction.md#navigate-to-measurements) to the measurement summary. +!!! info "Required role" + You need **write** or **admin** role to edit or delete measurements. -To edit measurement metadata, start by [navigating](measurement_introduction.md#measurement-navigation) into the measurement summary view. -![measurement_summary](images/measurement_summary_with_measurements.png){.screenshot} +## Edit measurement metadata -Once within the measurement summary view, measurements can be edited via the following steps: +1. Select the domain tab (Proteomics or Genomics). -1. Select the tab containing the [registered metadata](measurement_registration.md#measurement-registration) of the domain of interest. -2. [Download](#download-metadata) the measurement metadata -3. [Prepare](#prepare-metadata) the edits of the downloaded domain specific metadata sheet -4. [Upload](#upload-metadata) the edited metadata sheet via the edit measurement dialog +2. Click **Download Metadata** to get the current metadata as `.xlsx`. -Additionally, within the measurement summary view you're also able to [delete](#delete-metadata) measurement metadata. +=== "Proteomics" -!!! info "Project role" - Should you not see the edit, deletion and download buttons, - please make sure that you have been granted the "write" or "admin" role to the project by its owner or an admin! + ![Download](images/measurement_edit_proteomics_download_metadata.png){.screenshot} -## Download Metadata +=== "Genomics" -### Proteomics + ![Download](images/measurement_edit_ngs_download_metadata.png){.screenshot} -Start by selecting the **Proteomics** Tab within the measurement metadata tab sheet. -![measurement_edit_proteomics_download_metadata.png](images/measurement_edit_proteomics_download_metadata.png){.screenshot} +3. Open the file and make your changes. Grey columns (e.g. `Measurement Id`) are locked — changes there are ignored. -Press the "Download Metadata" button, which will download the metadata for all your proteomic measurements in a _xlsx_ file. -![measurement_edit_proteomics_downloaded_metadata.png](images/measurement_edit_proteomics_downloaded_metadata.png){.screenshot} + !!! warning "Locked fields" + To correct a locked field, [delete the measurement](#delete-measurements) and [re-register it](measurement_registration.md). -### Genomics +4. Save the file. You can upload the `.xlsx` directly or export as tab-separated UTF-16BE text. -Start by selecting the **Genomics** Tab within the measurement metadata tab sheet. -![measurement_edit_genomics_download_metadata.png](images/measurement_edit_ngs_download_metadata.png){.screenshot} +5. Click **Edit** in the measurement summary to open the upload dialog. -Press the "Download Metadata" button, which will download the metadata for all your genomic measurements in a _xlsx_ file. -![measurement_edit_genomics_downloaded_metadata.png](images/measurement_edit_ngs_downloaded_metadata.png){.screenshot} + ![Upload](images/measurement_edit_upload_metadata.png){.screenshot} -## Prepare Metadata +6. Upload your file. The dialog validates it and highlights errors. -### Proteomics + !!! warning "File requirements" + `.xlsx`, `.txt`, or `.tsv`. Maximum 16 MB. -Start by opening the downloaded proteomics measurement metadata _xlsx_ file containing the domain specific metadata sheet. -![measurement_edit_proteomics_downloaded_metadata.png](images/measurement_edit_proteomics_downloaded_metadata.png){.screenshot} +7. Click **Save**. -The _xlsx_ lists the registered measurements according to their measurement Ids with the provided property values during the registration. +## Delete measurements -!!! warning "Fixed Properties" - Specific measurement properties such as _Measurement Id_ **cannot** be changed after a measurement has been registered. - Cells of these properties are marked as grey, meaning changes to these cells will be ignored. - Errors made within these cells require a [deletion](#delete-metadata) and [reregistration](measurement_registration.md#measurement-registration) of the measurement metadata in question. +1. In the measurement summary, select the measurements to delete by clicking their checkboxes. Use the header checkbox to select all in a domain tab. -Within this sheet make the necessary property changes for the measurements. -![measurement_edit_proteomics_measurement_edited.png](images/measurement_edit_proteomics_measurement_edited.png){.screenshot} + ![Individual selection](images/measurement_deletion_individual_selection.png){.screenshot} -Finally, use your workbook directly (xlsx file) or export the sheet into a tab seperated UTF-16BE Unicode Text (*.txt) text file. -![edit_measurement_proteomics_measurement_export.png](images/measurement_edit_proteomics_measurement_export.png){.screenshot} + ![All selected](images/measurement_deletion_all_selection.png){.screenshot} -**Notes:** -The "Instrument" column expects an ontology [CURIE](https://link.springer.com/article/10.1007/s12599-022-00744-0) of the instrument. -You can use our [ontology search](../ontology_search/ontology_search_introduction.md#ontology-search) to find the CURIE + !!! note "Domain-specific selection" + Selections reset when you switch domain tabs. Delete measurements for each domain separately. -The "Organisation Id" column expects the full [RoR Id](https://ror.org/about/) URL of the organisation. -Use the [ROR registry search](https://ror.org/search) to determine the URL of the organisation RoR Id. +2. Click **Delete**. A confirmation dialog shows the count and domain. -Finally, [upload](#upload-metadata) the exported text file into the Data Manager application. + ![Confirm](images/measurement_deletion_confirm_dialog.png){.screenshot} -### Genomics +3. Click **Confirm**. -Start by opening the downloaded proteomics measurement metadata _xlsx_ file containing the domain specific metadata sheet. -![measurement_edit_proteomics_downloaded_metadata.png](images/measurement_edit_proteomics_downloaded_metadata.png){.screenshot} +!!! warning "Raw data must be removed first" + Measurements with attached raw data cannot be deleted. Delete the raw data first. -The _xlsx_ file lists the registered measurements according to their measurement Ids with the provided property values during the registration. +--- -!!! warning "Fixed Properties" - Specific measurement properties such as _Measurement Id_ **cannot** be changed after a measurement has been registered. - Cells of these properties are marked as grey, meaning changes to these cells will be ignored. - Errors made within these cells require a [deletion](#delete-metadata) and [reregistration](measurement_registration.md#measurement-registration) of the measurement metadata in question. +## What's next -Within this sheet make the necessary property changes for the measurements. -![measurement_edit_ngs_measurement_edited.png](images/measurement_edit_ngs_measurement_edited.png){.screenshot} - -Finally, use your workbook directly (xlsx file) or export the sheet into a tab seperated UTF-16BE Unicode Text (*.txt) text file. -![edit_measurement_ngs_measurement_export.png](images/measurement_edit_ngs_measurement_export.png){.screenshot} - -**Notes:** -The "Instrument" column expects an ontology [CURIE](https://link.springer.com/article/10.1007/s12599-022-00744-0) of the instrument. -You can use our [ontology search](../ontology_search/ontology_search_introduction.md#ontology-search) to find the CURIE - -The "Organisation Id" column expects the full [RoR Id](https://ror.org/about/) URL of the organisation. -Use the [ROR registry search](https://ror.org/search) to determine the URL of the organisation RoR Id. - -Finally, [upload](#upload-metadata) the exported text file into the Data Manager application. - -## Upload Metadata - -Once the measurement metadata has been [prepared](#prepare-metadata) according to the domain specifications, -the exported _txt_ file can be uploaded into the Data Manager application. - -To start the update process press the "Edit" button within the measurement summary view. -![measurement_summary](images/measurement_summary_with_measurements.png){.screenshot} - -This will open the measurement edit dialog with which the edited metadata can be uploaded. -![edit_measurement_upload_metadata.png](images/measurement_edit_upload_metadata.png){.screenshot} -Within the dialog you are able to upload your measurement files either via clicking the upload files button and selecting the files of interest in your file system -or by drag and dropping the files into the dashed box saying "drop your files here". -![measurement_edit_upload_metadata_filled.png](images/measurement_edit_upload_metadata_filled.png){.screenshot} - -Should you have uploaded one or more files by accident, you can easily delete them via a press of the cross icon next to their respective file names - -!!! warning "File constraints" - Please adhere to the file format and maximum file size outlined in the dialog. - Currently, we support the _txt_, _tsv_ or _xlsx_ file formats with a maximum file size of 16Mb - -The edit dialog will validate the provided metadata information and show invalid properties below the file name. - -Once all metadata properties are valid, upload the measurement metadata files to the experiment via pressing the "Save" button on the bottom right of the dialog. -The changed measurement metadata will be shown in the grid within their domain specific tab in the measurement summary view. - -# Delete Metadata - -To delete measurement metadata, start by [navigating](measurement_introduction.md#measurement-navigation) into the measurement summary view. -![measurement_summary](images/measurement_summary_with_measurements.png){.screenshot} - -Once within the measurement summary view, start the measurement metadata deletion process by selecting the measurements within the domain of interest. -![measurement_deletion_individual_selection.png](images/measurement_deletion_individual_selection.png){.screenshot} - -Alternatively, can also select all measurements of a domain via a press of the checkbox within the column header. -![measurement_deletion_all_selection.png](images/measurement_deletion_all_selection.png){.screenshot} - -You are also able to use the Search field above to filter measurements before selection. Keep in mind that changing the filter will reset the selection of measurements. - -!!! Note "Domain Specific Selection" - The selected measurements are reset if a different domain tab is clicked. - Therefore, delete the measurements for each domain individually. - -Afterwards, press the "Delete" button to start the measurement deletion. A dialog will open asking you to confirm the deletion and informing you about the number and domain of the measurements that will be deleted. -![measurement_deletion_confirm_dialog.png](images/measurement_deletion_confirm_dialog.png) - -Press the "Confirm" button to trigger the measurement deletion. -If all measurements for a specific domain have been deleted, the domain specific tab will disappear as well until new measurements are registered. -![measurement_deletion_all_selection_deleted.png](images/measurement_deletion_all_selection_deleted.png) - -!!! warning "Attached Raw Data" - Keep in mind, that measurements can only be deleted if they have no raw data attached to them. - Otherwise, you need to delete the raw data before the measurement can be deleted. +➡ [Register new measurements](measurement_registration.md) · [Upload raw data](../rawdata/raw_data_upload.md) diff --git a/docs/measurement/measurement_introduction.md b/docs/measurement/measurement_introduction.md index 6209221..5a43412 100644 --- a/docs/measurement/measurement_introduction.md +++ b/docs/measurement/measurement_introduction.md @@ -1,12 +1,24 @@ -# Measurement Introduction +# Measurements -Start by [navigating](../project/project_introduction.md#project-navigation) to the project summary view of your project of interest. -[Navigate](../experiment/experiment_introduction.md#experiment-navigation) into the experiment of interest and within the experiment [navigate](#measurement-navigation) into the measurement summary view. -Within this view you are able to [register](measurement_registration.md) or [edit](measurement_edit.md) new measurements. +A measurement records how a sample was analysed: which instrument, which protocol, and where the work was done. Registering measurements creates the link between your biological samples and the raw data files produced by the instrument. -## Measurement Navigation +The Data Manager supports two measurement domains: -From the experiment summary you can navigate into the measurement summary view. -![experiment_summary](../experiment/images/experimental_summary.png){.screenshot} -To do so, click on the "View Measurements" tab within the experiment navigation bar on the top. -![measurement_summary](images/measurement_summary_no_measurements.png){.screenshot} +- **Proteomics** — mass spectrometry, measurement IDs prefixed with `MS` +- **Genomics** — next-generation sequencing, measurement IDs prefixed with `NGS` + +## Navigate to measurements + +[Navigate](../project/project_introduction.md#find-and-open-a-project) to your project, then [open the experiment](../experiment/experiment_introduction.md#find-and-open-an-experiment). Click the **View Measurements** tab. + +![Experiment summary](../experiment/images/experimental_summary.png){.screenshot} + +This opens the measurement summary view with separate tabs for Proteomics and Genomics. + +![Measurement summary](images/measurement_summary_no_measurements.png){.screenshot} + +--- + +## What's next + +➡ [Register measurements](measurement_registration.md) · [Edit measurements](measurement_edit.md) diff --git a/docs/measurement/measurement_registration.md b/docs/measurement/measurement_registration.md index e3e2558..5b1d70c 100644 --- a/docs/measurement/measurement_registration.md +++ b/docs/measurement/measurement_registration.md @@ -1,128 +1,96 @@ -# Measurement Registration +# Register measurements -!!! tip "Excel file support" - The Data Manager now supports measurement registration and updates with Excel files (*.xlsx). - - __Only requirement__: metadata must be in the first sheet of your workbook. +!!! tip "Excel is supported" + Upload measurement metadata as an Excel file (`.xlsx`). Metadata must be on the first sheet of the workbook. -To register measurements, start by [navigating](measurement_introduction.md#measurement-navigation) into the measurement summary view. -![measurement_summary](images/measurement_summary_no_measurements.png){.screenshot} -Once within the measurement summary view, measurements can be registered via the following steps: +[Navigate](measurement_introduction.md#navigate-to-measurements) to the measurement summary. -1. [Download](#download-template) the domain specific metadata template spreadsheet -2. [Prepare](#prepare-metadata) the downloaded metadata sheet with the domain specific mandatory information -3. [Upload](#measurement-upload) the filled in measurement metadata sheet +Register measurements in three steps: -!!! info "Project role" - Should you not see the registration and download buttons, - please make sure that you have been granted the "write" or "admin" role to the project by the owner/admin! +1. [Download the template](#download-template) +2. [Fill in the metadata](#prepare-metadata) +3. [Upload the completed file](#upload) -## Download Template +!!! info "Required role" + You need **write** or **admin** role to see the registration buttons. -### Proteomics +## Download template -Within this view download the proteomic specific template's _.xlsx_ file via the template component on the top right by clicking -on the download icon (down arrow). -![register_measurements_proteomics_download_template.png](images/measurement_registration_proteomics_download_template.png){.screenshot} +=== "Proteomics" -### Genomics + Click the download icon (↓) in the template section (top right). -Within this view download the genomic specific template's xlsx file via the template component on the top right by clicking -on the download icon (down arrow). + ![Proteomics template](images/measurement_registration_proteomics_download_template.png){.screenshot} -Once downloaded open the template file in Microsoft Excel, which contains the two sheets "Property Information" and "Metadata". -![register_measurements_ngs_measurement_template.png](images/measurement_registration_ngs_measurement_template.png){.screenshot} +=== "Genomics" -## Prepare Metadata + Click the download icon (↓) in the template section (top right). -### Proteomics + ![Genomics template](images/measurement_registration_ngs_measurement_template.png){.screenshot} -Start by opening the downloaded proteomics template file in Microsoft Excel. It contains the two sheets "Property Information" and "Metadata". -![register_measurement_proteomics_measurement_template.png](images/measurement_registration_proteomics_measurement_template.png){.screenshot} +## Prepare metadata -The "Property Information" Sheet provides detailed information about which properties are required and what values are permitted for each property within the sheet. -![register_measurement_proteomics_measurement_template_property_sheet.png](images/measurement_registration_proteomics_measurement_template_property_sheet.png){.screenshot} +The template has two sheets: **Property Information** (reference) and **Metadata** (fill this in). Mandatory columns are marked with `*`. -Use these Guidelines to fill in the "Metadata" sheet with the mandatory necessary information for the performed measurement. -![register_measurement_proteomics_measurement_filled.png](images/measurement_registration_proteomics_measurement_filled.png){.screenshot} +=== "Proteomics" -!!! tip "Sample Id" - You can copy the Sample IDs from the [downloadable](../batch/sample-batch.md#download-sample-metadata) batch metadata sheet + Key fields: -**Notes:** -Mandatory metadata properties are marked with an asterisk next to the column header + - **Sample ID** — copy from your [batch metadata download](../batch/sample-batch.md#download-sample-metadata) + - **Instrument** — an ontology code (CURIE) for your mass spectrometer, e.g. `BAO:0002733`. Use the [ontology search](../ontology_search/ontology_search_introduction.md) to find the right code. + - **Organisation Id** — the full [ROR](https://ror.org/) URL of your institution, e.g. `https://ror.org/03a1kwz48`. Search at [ror.org](https://ror.org/search). + - **Digestion enzyme**, **Digestion method**, **LC column** — required proteomics-specific fields. -The "Instrument" column expects an ontology [CURIE](https://link.springer.com/article/10.1007/s12599-022-00744-0) of the instrument. -You can use our [ontology search](../ontology_search/ontology_search_introduction.md#ontology-search) to find the CURIE + ![Filled template](images/measurement_registration_proteomics_measurement_filled.png){.screenshot} -The "Organisation Id" column expects the full [RoR Id](https://ror.org/about/) URL of the organisation. -Use the [ROR registry search](https://ror.org/search) to determine the URL of the organisation RoR Id. +=== "Genomics" -Once the mandatory measurement metadata has been provided, export "Metadata" sheet into a tab seperated UTF-16BE Unicode Text (*.txt) text file. This encoding ensures that special symbols like 'μ' (think volumes in sample preparation) are correctly transferred and can be displayed with your sample information. -![register_measurement_proteomics_measurement_export.png](images/measurement_registration_proteomics_measurement_export.png){.screenshot} + Key fields: -Finally, [upload](#measurement-upload) the exported text file into the Data Manager application. + - **Sample ID** — copy from your [batch metadata download](../batch/sample-batch.md#download-sample-metadata) + - **Instrument** — an ontology code (CURIE) for your sequencer, e.g. `OBI:0002750`. Use the [ontology search](../ontology_search/ontology_search_introduction.md) to find the right code. + - **Organisation Id** — the full [ROR](https://ror.org/) URL of your institution. + - **Read type** — `paired-end` or `single-end`. + - **Index I7 / I5** — required for pooled (multiplexed) measurements. These are the DNA index sequences used to identify which sample is which when multiple libraries are sequenced together. -### Genomics + ![Filled template](images/measurement_registration_ngs_measurement_filled.png){.screenshot} -Start by opening the downloaded genomic template file in Microsoft Excel. It contains the two sheets "Property Information" and "Metadata". -![register_measurements_ngs_measurement_template.png](images/measurement_registration_ngs_measurement_template.png){.screenshot} +??? info "What is a CURIE?" + A CURIE (Compact URI) is a short, standardised code that identifies a specific term in a scientific database — for example, `OBI:0002750` for an Illumina sequencer. The format follows the [W3C CURIE Syntax](https://www.w3.org/TR/curie/): `PREFIX:REFERENCE`. You don't need to memorise these. Search by instrument name using the [ontology search](../ontology_search/ontology_search_introduction.md) and copy the code. -The "Property Information" Sheet provides detailed information about which properties are required and what values are permitted for each property within the sheet. -![register_measurements_ngs_measurement_template_property_sheet.png](images/measurement_registration_ngs_measurement_template_property_sheet.png){.screenshot} +!!! tip "XLSX vs TSV" + You can upload `.xlsx` directly (recommended) or export the Metadata sheet as a tab-separated `.txt` file in **UTF-16BE Unicode Text** encoding. UTF-16BE preserves special characters like `μ`. A different encoding will corrupt them. -Use these Guidelines to fill in the "Metadata" sheet with the mandatory necessary information for the performed measurement. -![register_measurement_proteomics_measurement_filled.png](images/measurement_registration_proteomics_measurement_filled.png){.screenshot} +## Upload -!!! tip "Sample Id" - You can copy the Sample IDs from the [downloadable](../batch/sample-batch.md#download-sample-metadata) batch metadata sheet +1. Click **Register Measurements**. -**Notes:** -Mandatory metadata properties are marked with an asterisk next to the column header + ![Register button](images/measurement_summary_no_measurements.png){.screenshot} -The "Instrument" column expects an ontology [CURIE](https://link.springer.com/article/10.1007/s12599-022-00744-0) of the instrument. -You can use our [ontology search](../ontology_search/ontology_search_introduction.md#ontology-search) to find the CURIE +2. Drag and drop your file or click **Upload files**. -The "Organisation Id" column expects the full [RoR Id](https://ror.org/about/) URL of the organisation. -Use the [ROR registry search](https://ror.org/search) to determine the URL of the organisation RoR Id. + ![Upload dialog](images/measurement_registration_upload_template_filled.png){.screenshot} -Once the mandatory measurement metadata has been provided, export "Metadata" sheet into a tab seperated UTF-16BE Unicode Text (*.txt) text file. This encoding ensures that special symbols like 'μ' (think volumes in sample preparation) are correctly transferred and can be displayed with your sample information. -![register_measurement_proteomics_measurement_export.png](images/measurement_registration_proteomics_measurement_export.png){.screenshot} + !!! warning "File requirements" + `.xlsx`, `.txt`, or `.tsv`. Maximum 16 MB. -Finally, [upload](#measurement-upload) the exported text file into the Data Manager application. +3. The dialog validates your file and flags any errors. -## Measurement Upload +4. Click **Register**. -Once the measurement metadata has been [prepared](#prepare-metadata) according to the domain specifications, -the exported _txt_ file can be uploaded into the Data Manager application. -To start the measurement registration process press the "Register Measurements" button within the measurement summary view. -![measurement_summary](images/measurement_summary_no_measurements.png){.screenshot} +??? info "Measurement ID format" + Each measurement gets a unique ID that encodes its provenance: `[DOMAIN][SAMPLE_ID]-[TIMESTAMP]`. For example, `MSQ2ABCD001AA-118569093700875` (proteomics) or `NGSQ2ABCD001AA-118569093700875` (genomics). The embedded sample ID means you can always trace a measurement back to its source sample. -This will open the measurement registration dialog with which the metadata can be registered. -![register_measurements_upload_template.png](images/measurement_registration_upload_template.png){.screenshot} +Registered measurements appear in the domain-specific tab. -Within the dialog you are able to upload your measurement files either via clicking the upload files button and selecting the files of interest in your file system -or by drag and dropping the files into the dashed box saying "drop your files here". -![register_measurements_proteomics_upload_template_filled.png](images/measurement_registration_upload_template_filled.png){.screenshot} +![Measurements registered](images/measurement_summary_with_measurements.png){.screenshot} -Should you have uploaded one or more files in error, you can easily delete them via a press of the cross icon next to their respective file names +## Download measurement metadata -!!! warning "File constraints" - Please adhere to the file format and maximum file size outlined in the dialog. - Currently, we support the _txt_ or _tsv_ file formats with a maximum file size of 16Mb +Click **Download Metadata** to export the metadata for all measurements in the currently selected tab as an `.xlsx` file. -The registration dialog will validate the provided information and show invalid properties below the file name. +--- -Finally, upload the measurement metadata files to the experiment via pressing the "Register" button on the bottom right of the dialog. +## What's next -!!! note "Measurement Id" - During measurement registration each measurement will be assigned a unique measurement ID, - distinguishing it from other measurement within the system. - -Your uploaded measurement metadata will be shown in the grid within their domain specific tab in the measurement summary view. -![measurement_summary](images/measurement_summary_with_measurements.png) - -## Measurement Metadata Download - -You can download the measurement specific metadata via the "Download Metadata" button, which will download the metadata of all measurements of the **currently selected tab** in an _xlsx_ file. -![batch_registration_downloaded_metadata.png](images/measurement_registration_downloaded_metadata.png) +➡ [Upload raw data](../rawdata/raw_data_upload.md) · [Edit measurements](measurement_edit.md) diff --git a/docs/metadata/concepts.md b/docs/metadata/concepts.md index 7e0d223..99971c8 100644 --- a/docs/metadata/concepts.md +++ b/docs/metadata/concepts.md @@ -1,42 +1,44 @@ -# Metadata Concepts +# Metadata glossary -## Quick Reference Table +This glossary describes every metadata field in the Data Manager. Use it as a reference when filling in registration templates or interpreting exported data. -| Concept | Description | -|-----------------------------|--------------------------------------------------------------------------------| -| [User](#user) | Learn more about the metadata of your user account and its authentication | -| [Project](#project) | Information about a projects metadata such as contact, funding and objective | -| [Terminology](#terminology) | Find out more about the ontology terms employed within experiments and samples | -| [Experiment](#experiment) | Everything related to the metadata within an experimental design | -| [Sample](#sample) | Detailed information concerning the metadata of sample and batches | -| [Measurement](#measurement) | Everything related to NGS and PxP measurement metadata | -| [Raw Data](#raw-data) | Find out more about the metadata surrounding Data Upload and Download | +## Quick reference + +| Section | Description | +|---|---| +| [User](#user) | Account metadata and authentication | +| [Project](#project) | Project contacts, funding, and objectives | +| [Terminology](#terminology) | Ontology terms used in experiments and samples | +| [Experiment](#experiment) | Experimental design metadata | +| [Sample](#sample) | Sample and batch metadata | +| [Measurement](#measurement) | Genomics and proteomics measurement metadata | +| [Raw data](#raw-data) | Upload and download metadata | ### User Check our documentation to find out how to [register](../user/user_registration.md) and [edit](../user/user_edit.md) your account. -The following concepts are associated with an user account. +The following concepts are associated with a user account. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |-------------------|---------------------------------|----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Full name | John Doe | ✅ | Text, divided by a singular space
into first and last name |
First and last name of the userView Description
| -| Oidc id | 0009-0006-
0929-9338 | ✅ | Text, dependent on the [oidc](https://openid.net/developers/how-connect-works/) issuer (e.g. [ORCID](https://support.orcid.org/hc/en-us/articles/360006897674-Structure-of-the-ORCID-Identifier)) |
Unique Id identifying the account within the [Oidc](https://openid.net/developers/how-connect-works/) provider (e.g. [ORCID](https://support.orcid.org/hc/en-us/articles/360006897674-Structure-of-the-ORCID-Identifier))View Description
| -| Oidc issuer | https://www.orcid.org | ✅ | Link, dependent on the [oidc](https://openid.net/developers/how-connect-works/) issuer (e.g. [ORCID](https://support.orcid.org/hc/en-us/articles/360006897674-Structure-of-the-ORCID-Identifier)) |
[Oidc](https://openid.net/developers/how-connect-works/) Issuer enabling the [oauth](https://datatracker.ietf.org/doc/html/rfc6749) process (e.g. via [ORCID](https://support.orcid.org/hc/en-us/articles/360006897674-Structure-of-the-ORCID-Identifier))View Description
| +| Full name | Jane Doe | ✅ | Text, divided by a singular space
into first and last name |
First and last name of the userView Description
| +| Oidc id | 0009-0006-
0929-9338 | ✅ | Text, dependent on the [oidc](https://openid.net/developers/how-connect-works/) issuer (e.g. [ORCID](https://support.orcid.org/hc/en-us/articles/360006897674-Structure-of-the-ORCID-Identifier)) |
Unique ID identifying the account within the [OIDC](https://openid.net/developers/how-connect-works/) provider (e.g. [ORCID](https://support.orcid.org/hc/en-us/articles/360006897674-Structure-of-the-ORCID-Identifier))View Description
| +| Oidc issuer | https://www.orcid.org | ✅ | Link, dependent on the [oidc](https://openid.net/developers/how-connect-works/) issuer (e.g. [ORCID](https://support.orcid.org/hc/en-us/articles/360006897674-Structure-of-the-ORCID-Identifier)) |
[OIDC](https://openid.net/developers/how-connect-works/) issuer enabling the [OAuth](https://datatracker.ietf.org/doc/html/rfc6749) process (e.g. via [ORCID](https://support.orcid.org/hc/en-us/articles/360006897674-Structure-of-the-ORCID-Identifier))View Description
| | Registration date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the account was created within the systemView Description
| -| User email | John.Doe@
example.mail | ✅ | Text, validated against [RFC5322](https://www.rfc-editor.org/rfc/rfc5322">https://www.rfc-editor.org/rfc/rfc5322)
format specification |
Email address associated with the accountView Description
| -| User name | JohnDoe | ✅ | Text |
Unique name specified for the accountView Description
| |
Unique Id associated with the user automatically assigned by the systemView Description
| +| User email | Jane.Doe@
example.mail | ✅ | Text, validated against [RFC 5322](https://www.rfc-editor.org/rfc/rfc5322)
format specification |
Email address associated with the accountView Description
| +| User name | JaneDoe | ✅ | Text |
Unique name specified for the accountView Description
| -#### Personal Access Token +#### Personal access token !!! warning - Personal access token are used as app credentials. They are connected to the user that + Personal access tokens are used as app credentials. They are connected to the user that created them and must not be shared. Check out our documentation to find out how -to [register](../rawdata/raw_data_download.md#generate-a-token) -and [edit](../rawdata/raw_data_download.md#manage-tokens) your personal access tokens. +to [generate](../rawdata/raw_data_download.md#generate-a-token) +and [manage](../rawdata/raw_data_download.md#manage-tokens) your personal access tokens. ### Project @@ -44,122 +46,122 @@ Visit the [documentation](../project/project_introduction.md) to find out how to [register](../project/project_registration.md) and [edit](../project/project_edit.md) a project.
Additionally, it outlines how to grant or revoke [access](../project/project_access.md) to your -project to other users. +project for other users. -The following concepts are associated with a project +The following concepts are associated with a project. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |--------------------------------------|-----------------------------------------------------------------------------|----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------| -| Id | Q2ABCD | ✅ | Text, always starts with 'Q2'
and has a total length of 6. |
Human Readable unique project identifier of the projectView Description
| +| Id | Q2ABCD | ✅ | Text, always starts with 'Q2'
and has a total length of 6. |
Human-readable unique project identifierView Description
| | Modification date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the project was last modifiedView Description
| | Objective | Investigating the substance
of interest with
XXX to determine YYY | ✅ | Text, with a maximum length of 2000 characters |
Objective detailing the purpose and goal of the projectView Description
| -| Principal investigator full name | John Doe | ✅ | Text, divided by a singular space
into first and last name |
Full name of the principal investigator handling the projectView Description
| -| Principal investigator email address | John.Doe@
example.mail | ✅ | Text, validated against [RFC5322](https://www.rfc-editor.org/rfc/rfc5322">https://www.rfc-editor.org/rfc/rfc5322)
format specification |
Email address of the principal investigator handling the projectView Description
| -| Project manager full name | Jane Doe | ✅ | Text, divided by a singular space
into first and last name |
Full name of the project manager handling the projectView Description
| -| Project manager email address | Jane.Doe@
example.mail | ✅ | Text, validated against [RFC5322](https://www.rfc-editor.org/rfc/rfc5322">https://www.rfc-editor.org/rfc/rfc5322)
format specification |
Email address of the project manager handling the projectView Description
| +| Principal investigator full name | Jane Doe | ✅ | Text, divided by a singular space
into first and last name |
Full name of the principal investigator handling the projectView Description
| +| Principal investigator email address | Jane.Doe@
example.mail | ✅ | Text, validated against [RFC 5322](https://www.rfc-editor.org/rfc/rfc5322)
format specification |
Email address of the principal investigator handling the projectView Description
| +| Project manager full name | Alex Smith | ✅ | Text, divided by a singular space
into first and last name |
Full name of the project manager handling the projectView Description
| +| Project manager email address | Alex.Smith@
example.mail | ✅ | Text, validated against [RFC 5322](https://www.rfc-editor.org/rfc/rfc5322)
format specification |
Email address of the project manager handling the projectView Description
| | Title | Analysis of the transcriptome
of liver cancer sample | ✅ | Text |
Clear and concise title of the projectView Description
| | Grant name | DFG | ❌ | Text |
Name of the grant funding the projectView Description
| -| Grant id | 213896434 | ❌ | Text |
Unique Id identifying the grant funding the projectView Description
| -| Responsible person full name | Max Mustermann | ❌ | Text, divided by a singular space
into first and last name |
Full name of the person responsible for the projectView Description
| -| Responsible person email address | Max.Mustermann@
example.mail | ❌ | Text, validated against [RFC5322](https://www.rfc-editor.org/rfc/rfc5322">https://www.rfc-editor.org/rfc/rfc5322)
format specification |
Email address of the person responsible for the projectView Description
| +| Grant id | 213896434 | ❌ | Text |
Unique ID identifying the grant funding the projectView Description
| +| Responsible person full name | Taylor Brown | ❌ | Text, divided by a singular space
into first and last name |
Full name of the person responsible for the projectView Description
| +| Responsible person email address | Taylor.Brown@
example.mail | ❌ | Text, validated against [RFC 5322](https://www.rfc-editor.org/rfc/rfc5322)
format specification |
Email address of the person responsible for the projectView Description
| #### Offer Visit the [documentation](../project/project_edit.md) to find out how -to [register](../project/project_edit.md#offer-upload) +to [upload](../project/project_edit.md#offer-upload) and [edit](../project/project_edit.md#offer-upload) your offer file. -The following concepts are associated with an offer file +The following concepts are associated with an offer file. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |---------|------------------|----------------------------|---------|--------------------------------------------------------------------------------------------------------------| | Name | Q2ABCD_Offer.pdf | ✅ | Text |
Filename of the uploaded offerView Description
| -| Signed | yes/no | ✅ | Boolean |
Uploaded Offer was signed by the customerView Description
| +| Signed | yes/no | ✅ | Boolean |
Uploaded offer was signed by the customerView Description
| ### Experiment Visit our [documentation](../experiment/experiment_introduction.md) to find out how -to [register](../experiment/experiment_creation.md) +to [create](../experiment/experiment_creation.md) and [edit](../experiment/experiment_creation.md) an experiment. The following concepts are associated with an experiment. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |------------------------|-------------------------------------------------------------------------------------|----------------------------|--------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Analytes | BAO:0000270 | ✅ | List of Identifier,
validated against the [W3C](https://www.w3.org/TR/curie/)
format specification |
One or more substances or compounds of interest, whose expression changes are of interest within the experimentView Description
| | Biological replicates | 20 | ✅ | Number |
Number of biologically distinct samples subjected to the same treatment during the experimentView Description
| | Experimental groups | control, treatment_cohort_1 | ✅ | List of [Experimental Groups](concepts.md#experimental-groups) |
Group of subjects exposed to a unique combination (condition) of experimental variablesView Description
| -| Experimental variables | temperature, time | ✅ | List of [Experimental Variables](concepts.md#experimental-variables) |
An Experimental factor defined to observe its effect on the subjects of an experiment.View Description
| +| Experimental variables | temperature, time | ✅ | List of [Experimental Variables](concepts.md#experimental-variables) |
An experimental factor defined to observe its effect on the subjects of an experimentView Description
| | Modification date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the experiment was last modifiedView Description
| | Name | Pilot Experiment | ✅ | Text |
Unique name of the experimentView Description
| -| Specimen | NCIT:C12392 | ✅ | List of Identifier,
validated against the [W3C](https://www.w3.org/TR/curie/)
format specification |
One or more specific parts of the species from which the analytes are collected.View Description
| -| Species | NCBITaxon:9606 | ✅ | List of Identifier,
validated against the [W3C](https://www.w3.org/TR/curie/)
format specification |
One ore more Organisms from which the samples were collectedView Description
| -| Confounding variables | Varies See [here](../experiment/confounding-variables.md)
for more information | ❌ | List of [Confounding Variables](concepts.md#confounding-variables) |
A variable possibly influencing independent and dependent experimental variables.View Description
| +| Specimen | NCIT:C12392 | ✅ | List of Identifier,
validated against the [W3C](https://www.w3.org/TR/curie/)
format specification |
One or more specific parts of the species from which the analytes are collectedView Description
| +| Species | NCBITaxon:9606 | ✅ | List of Identifier,
validated against the [W3C](https://www.w3.org/TR/curie/)
format specification |
One or more organisms from which the samples were collectedView Description
| +| Confounding variables | Varies. See [here](../experiment/confounding-variables.md)
for more information | ❌ | List of [Confounding Variables](concepts.md#confounding-variables) |
A variable possibly influencing independent and dependent experimental variablesView Description
| -#### Experimental Variables +#### Experimental variables Visit our [documentation](../experiment/experiment_introduction.md) to find out how -to [register](../experiment/experiment_creation.md#experimental-variable-creation) -and [edit](../experiment/experiment_creation.md#experimental-variable-creation) your experimental +to [create](../experiment/experiment_creation.md#define-experimental-variables) +and [edit](../experiment/experiment_creation.md#define-experimental-variables) your experimental variables. -The following concepts are associated with an experiment variable +The following concepts are associated with an experimental variable. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |---------|-------------|----------------------------|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------| -| Levels | 0, 10, 100 | ✅ | List of Texts |
One or more specific expression settings to which the variable can be set.View Description
| +| Levels | 0, 10, 100 | ✅ | List of Texts |
One or more specific expression settings to which the variable can be setView Description
| | Name | Temperature | ✅ | Text |
Unique name of the experimental variableView Description
| -| Unit | °C | ❌ | Text |
Measurement Unit representing the specific setting of the variableView Description
| +| Unit | °C | ❌ | Text |
Measurement unit representing the specific setting of the variableView Description
| -#### Experimental Groups +#### Experimental groups Visit our [documentation](../experiment/experiment_introduction.md) to find out how -to [register](../experiment/experiment_creation.md#experimental-group-creation) -and [edit](../experiment/experiment_creation.md#experimental-group-creation) your experimental -variables. +to [create](../experiment/experiment_creation.md#define-experimental-groups) +and [edit](../experiment/experiment_creation.md#define-experimental-groups) your experimental +groups. -The following concepts are associated with an experiment group +The following concepts are associated with an experimental group. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |-----------------------|----------------------------------|----------------------------|----------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Biological replicates | 20 | ✅ | Number |
Number of biologically distinct samples subjected to the same treatment during the experimentView Description
| | Condition | control,
treatment_cohort_1 | ✅ | List of [Experimental Variables](concepts.md#experimental-variables) |
Unique combination of experimental variablesView Description
| -#### Confounding Variables +#### Confounding variables Visit our [documentation](../experiment/confounding-variables.md) to find out how -to [register](../experiment/confounding-variables.md#define-confounding-variables-in-your-experiment) -and [edit](../experiment/confounding-variables.md#rename-a-confounding-variable) your experimental +to [define](../experiment/confounding-variables.md#define-a-confounding-variable) +and [rename](../experiment/confounding-variables.md#rename-a-variable) your confounding variables. Values for confounding variables can be provided per [sample](#sample) and are defined in the experiment. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |---------|------------|----------------------------|------|------------------------------------------------------------------------------------------------------------------| -| Name | Time | ✅ | Text |
Unique name of the experimental variableView Description
| -| Values | [6pm, 8pm] | ❌ | List |
Possible values of the confounding variables.View Description
| +| Name | Time | ✅ | Text |
Unique name of the confounding variableView Description
| +| Values | [6pm, 8pm] | ❌ | List |
Possible values of the confounding variableView Description
| ### Terminology Visit our documentation to find out how -to [search](../ontology_search/ontology_search_introduction.md#ontology-search-introduction) +to [search](../ontology_search/ontology_search_introduction.md) for your terminology of interest. The following concepts are associated with the terminology terms. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |--------------------------|---------------------------------------------------------------------------------------|----------------------------|------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------| | Class IRI | http://purl.obolibrary.org/obo/NCIT_C105979 | ✅ | Identifier |
Unique resource identifier for the termView Description
| | Description | A protein complex
that plays a role
in neurotransmitter-gated ion transport | ✅ | Text |
Description providing explanation for the termView Description
| | Label | 5-HT3 Receptor | ✅ | Text |
Common human-readable label of the termView Description
| | Name | NCIT:C105979 | ✅ | Identifier, structured as prefix:reference
as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
The OBO-style identifier, usually in the form PREFIX:IDView Description
| | Terminology term IRI | http://purl.obolibrary.org/obo/ncbitaxon.owl | ✅ | Identifier |
Unique resource identifier for the ontology providing the termView Description
| -| Terminology term version | http://purl.obolibrary.org/obo/ncbitaxon/2023-09-19/ncbitaxon.owl | ✅ | Identifier |
Specific Version of the ontology providing the termView Description
| +| Terminology term version | http://purl.obolibrary.org/obo/ncbitaxon/2023-09-19/ncbitaxon.owl | ✅ | Identifier |
Specific version of the ontology providing the termView Description
| For more information check the documentation of -the [ontology service API](https://terminology.tib.eu/ts/) +the [ontology service API](https://terminology.tib.eu/ts/). ### Sample @@ -169,18 +171,18 @@ and [edit](../batch/sample-batch.md#editing-sample-batches) your batches and sam The following concepts are associated with a sample. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |-----------------------|------------------------------------|----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Analysis method | PROTEOMICS | ✅ | Enumeration |
The test performed on samples for the purpose of finding and measuring chemical substances.View Description
| +| Analysis method | PROTEOMICS | ✅ | Enumeration |
The test performed on samples for the purpose of finding and measuring chemical substancesView Description
| | Analyte | BAO:0000270 | ✅ | Identifier, structured as prefix:reference
as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
The chemical substance extracted from the biological material that is identified and measured. Selection from the ones defined within the experimentView Description
| -| Condition | Temperature: 0°C;
Time: 100s; | ✅ | Number of the [Experimental Group](concepts.md#experimental-groups) in Question |
Condition to which the sample was subjected to. Selectable from the experimental groups within the experiment with each variable seperated by a semicolon.View Description
| -| Label | Lab_Id_01 | ✅ | Text |
Common human-readable name of the sample. Can also be internal lab identifierView Description
| -| Sample id | Q2ABCD001AA | ✅ | Text, beginning with the [project id](concepts.md#project)
followed by the running number of samples
within the project and ending with randomly generated unique letters |
Human readable unique sample id, automatically assigned by the Data ManagerView Description
| -| Species | NCBITaxon:9606 | ✅ | Identifier, structured as prefix:reference as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
Scientific name of the organism(s) from which the biological material is derived. Selection from the ones defined within the experimentView Description
| +| Condition | Temperature: 0°C;
Time: 100s; | ✅ | Number of the [Experimental Group](concepts.md#experimental-groups) in question |
Condition to which the sample was subjected. Selectable from the experimental groups within the experiment with each variable separated by a semicolonView Description
| +| Label | Lab_Id_01 | ✅ | Text |
Common human-readable name of the sample. Can also be an internal lab identifierView Description
| +| Sample id | Q2ABCD001AA | ✅ | Text, beginning with the [project id](concepts.md#project)
followed by the running number of samples
within the project and ending with randomly generated unique letters |
Human-readable unique sample ID, automatically assigned by the Data ManagerView Description
| +| Species | NCBITaxon:9606 | ✅ | Identifier, structured as prefix:reference as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
Scientific name of the organism(s) from which the biological material is derived. Selection from the ones defined within the experimentView Description
| | Specimen | NCIT:C12392 | ✅ | Identifier, structured as prefix:reference as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
Name of the biological material from which the analytes would be extracted. Selection from the ones defined within the experimentView Description
| -| Biological replicate | Mouse_WT_1 | ❌ | Text |
Specify if the samples belong to the same biological source within your experiment.View Description
| -| Comment | Redone QC | ❌ | Text |
Free Text, can contain any notes related to a specific sample in questionView Description
| -| Confounding Variables | color: red, location: labA | ❌ | List of key-value pairs |
Confounding variables are a good way to annotate measured variables that are suspected to have an influence on your depending variable.View Description
| +| Biological replicate | Mouse_WT_1 | ❌ | Text |
Specify if the samples belong to the same biological source within your experimentView Description
| +| Comment | Redone QC | ❌ | Text |
Free text, can contain any notes related to a specific sample in questionView Description
| +| Confounding variables | color: red, location: labA | ❌ | List of key-value pairs |
Confounding variables are a good way to annotate measured variables that are suspected to have an influence on your dependent variableView Description
| Additional information can be found in the **Property Information** tab within the sample registration [template sheet](templates/sample-metadata-template.xlsx). @@ -188,13 +190,12 @@ registration [template sheet](templates/sample-metadata-template.xlsx). #### Batch !!! info - A batch can be defined as a group of samples processed together under the same - experimental conditions. -
This is done to document and minimize technical variation. + A batch is a group of samples processed together under the same + experimental conditions. This is done to document and minimise technical variation. The following concepts are associated with a sample batch. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |-------------------|---------------------------------|----------------------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Modification date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the batch was last modifiedView Description
| | Name | Pxp_Analysis_Trial_1 | ✅ | Text |
Common human-readable label of the batchView Description
| @@ -204,79 +205,85 @@ The following concepts are associated with a sample batch. ### Measurement For detailed information visit -our [measurement documentation](../measurement/measurement_introduction.md) +our [measurement documentation](../measurement/measurement_introduction.md). #### Genomics Visit our documentation to find out how -to [register](../measurement/measurement_registration.md#genomics) -or [edit](../measurement/measurement_edit.md#genomics) your proteomics measurement +to [register](../measurement/measurement_registration.md) +or [edit](../measurement/measurement_edit.md) your genomics measurements. The following concepts are associated with a genomic measurement. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |--------------------|-------------------------------------------|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Facility | Quantitative Biology Center | ✅ | Text |
The facilities name within the organisation (group name, etc.)View Description
| -| Index I5 | NEBNext UDI UMI Set 1 B12 S579 | ✅ for pooled measurements | Text |
Index used for multiplexing.View Description
| -| Index I7 | NEBNext UDI UMI Set 1 B12 S789 | ✅ for pooled measurements | Text |
Index used for multiplexing.View Description
| -| Instrument | OBI:0002750 | ✅ | Identifier, structured as prefix:reference
as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
Ontology Identifier of the instrument that has been used for the measurement, usually in the form PREFIX:REFERENCEView Description
| -| Measurement id | NGSQ2ABCD001AA-
118569093700875 | ✅ | Text, beginning with the domain(NGS) prefix
followed by the [sample id](concepts.md#sample),
seperated by a hyphen with its unique creation timestamp |
Human readable unique measurement id, automatically assigned by the Data Manager, usually in the form "NGS"SAMPLEID"-"Timestamp"View Description
| -| Organization IRI | https://ror.org/03a1kwz48 | ✅ | Identifier, validated according to the structure
defined by the [RoR](https://ror.readme.io/docs/identifier) |
Research organization registry identifier([RoR Id](https://ror.org/)) of the organisation where the measurement has been conductedView Description
| -| Organization label | University Tuebingen | ✅ | Text |
Human Readable Name of the organization, automatically received from the RoR by the Organization IRIView Description
| -| Read type | paired-end | ✅ | Enumeration |
The sequencing read type used to generate the sequence data.View Description
| -| Registration date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the batch was created within the Data ManagerView Description
| -| Comment | Repeated Measurement
after Bad QC | ❌ | Text |
Free Text, can contain any notes related to a measurement in question with up to 500 charactersView Description
| -| Flow cell | S4 | ❌ | Text |
The flow cell type used for sequencing.View Description
| +| Facility | Quantitative Biology Center | ✅ | Text |
The facility's name within the organisation (group name, etc.)View Description
| +| Index I5 | NEBNext UDI UMI Set 1 B12 S579 | ✅ for pooled measurements | Text |
Index used for multiplexingView Description
| +| Index I7 | NEBNext UDI UMI Set 1 B12 S789 | ✅ for pooled measurements | Text |
Index used for multiplexingView Description
| +| Instrument | OBI:0002750 | ✅ | Identifier, structured as prefix:reference
as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
Ontology identifier of the instrument that has been used for the measurement, usually in the form PREFIX:REFERENCEView Description
| +| Measurement id | NGSQ2ABCD001AA-
118569093700875 | ✅ | Text, beginning with the domain (NGS) prefix
followed by the [sample id](concepts.md#sample),
separated by a hyphen with its unique creation timestamp |
Human-readable unique measurement ID, automatically assigned by the Data Manager, in the form NGS + SAMPLE_ID + "-" + TimestampView Description
| +| Organisation IRI | https://ror.org/03a1kwz48 | ✅ | Identifier, validated according to the structure
defined by [ROR](https://ror.readme.io/docs/identifier) |
Research Organisation Registry identifier ([ROR ID](https://ror.org/)) of the organisation where the measurement was conductedView Description
| +| Organisation label | University Tuebingen | ✅ | Text |
Human-readable name of the organisation, automatically received from ROR by the Organisation IRIView Description
| +| Read type | paired-end | ✅ | Enumeration |
The sequencing read type used to generate the sequence dataView Description
| +| Registration date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the measurement was created within the Data ManagerView Description
| +| Comment | Repeated measurement
after bad QC | ❌ | Text |
Free text, can contain any notes related to a measurement in question with up to 500 charactersView Description
| +| Flow cell | S4 | ❌ | Text |
The flow cell type used for sequencingView Description
| | Library kit | NEBNext Ultra II Directional RNA mRNA UMI | ❌ | Text |
The library kit employed during sequencing processingView Description
| | Measurement name | Lab_Id_01 | ❌ | Text |
Common human-readable name of the measurement. Can also be the internal lab identifierView Description
| | Run protocol | 104+19+10+104 | ❌ | Text |
Information on how many cycles were used for each read and index during sequencingView Description
| -Additional information can be found in the **property information** tab within the -genomic measurement registration [template sheet](templates/ngs_measurement_registration_sheet.xlsx) +Additional information can be found in the **Property Information** tab within the +genomic measurement registration [template sheet](templates/ngs_measurement_registration_sheet.xlsx). #### Proteomics Visit our documentation to find out how -to [register](../measurement/measurement_registration.md#proteomics) -or [edit](../measurement/measurement_edit.md#proteomics) your proteomics measurement +to [register](../measurement/measurement_registration.md) +or [edit](../measurement/measurement_edit.md) your proteomics measurements. The following concepts are associated with a proteomics measurement. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |--------------------|----------------------------------------|----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Digestion enzyme | Trypsin | ✅ | Text |
Information about the enzymes used for the proteolytic step.View Description
| -| Digestion method | in gel | ✅ | Enumeration |
Method that has been used to break proteins into peptides, Selectable from the methods "in gel", "in solution", "iST proteomics kit", "on beads"View Description
| -| Facility | "Quantitative Biology Center" | ✅ | Text |
The facilities name within the organisation (group name, etc.)View Description
| -| Instrument | BAO:0002733 | ✅ | Identifier, structured as prefix:reference
as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
Ontology Identifier of the instrument that has been used for the measurementView Description
| -| Lc column | ProteoSil_100-C18 | ✅ | Text |
The type of column that has been used.View Description
| -| Measurement id | MSQ2ABCD001AA-
118569093700875 | ✅ | Text, beginning with the domain(MS) prefix
followed by the [sample id](concepts.md#sample),
seperated by a hyphen with its unique creation timestamp |
Human readable unique measurement id, automatically assigned by the Data Manager, usually in the form "MS"SAMPLEID"-"Timestamp"View Description
| -| Organization IRI | https://ror.org/03a1kwz48 | ✅ | Identifier, validated according to the structure
defined by the [RoR](https://ror.readme.io/docs/identifier) |
Research organization registry identifier([RoR Id](https://ror.org/)) of the organisation where the measurement has been conductedView Description
| -| Organization label | University Tuebingen | ✅ | Text |
Human Readable Name of the organization, automatically received from the RoR by the Organization IRIView Description
| -| Registration date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the batch was created within the Data ManagerView Description
| -| Comment | Repeated Measurement
after Bad QC | ❌ | Text |
Free Text, can contain any notes related to a measurement in question with up to 500 charactersView Description
| -| Enrichment method | Phosphopeptide Enrichment | ❌ | Text |
Enrichment of proteins or peptides of different characteristics.View Description
| -| Fraction name | Fraction01 | ❌ | Text |
If the sample was fractionated this label can be used to indicate which fraction was measured.View Description
| -| Injection volume | 10 | ❌ | Number |
The sample volume injected into the LC column in microliter(µl).View Description
| -| Label | Heavy | ❌ | Text |
The label value for the label type that has been used.View Description
| -| Label type | SILAC | ❌ | Text |
The label type that has been used to label the sample for measurement.View Description
| +| Digestion enzyme | Trypsin | ✅ | Text |
Information about the enzymes used for the proteolytic stepView Description
| +| Digestion method | in gel | ✅ | Enumeration |
Method used to break proteins into peptides. Selectable from: "in gel", "in solution", "iST proteomics kit", "on beads"View Description
| +| Facility | Quantitative Biology Center | ✅ | Text |
The facility's name within the organisation (group name, etc.)View Description
| +| Instrument | BAO:0002733 | ✅ | Identifier, structured as prefix:reference
as seen in the [W3C](https://www.w3.org/TR/curie/)
format specification |
Ontology identifier of the instrument that has been used for the measurementView Description
| +| LC column | ProteoSil_100-C18 | ✅ | Text |
The type of column that has been usedView Description
| +| Measurement id | MSQ2ABCD001AA-
118569093700875 | ✅ | Text, beginning with the domain (MS) prefix
followed by the [sample id](concepts.md#sample),
separated by a hyphen with its unique creation timestamp |
Human-readable unique measurement ID, automatically assigned by the Data Manager, in the form MS + SAMPLE_ID + "-" + TimestampView Description
| +| Organisation IRI | https://ror.org/03a1kwz48 | ✅ | Identifier, validated according to the structure
defined by [ROR](https://ror.readme.io/docs/identifier) |
Research Organisation Registry identifier ([ROR ID](https://ror.org/)) of the organisation where the measurement was conductedView Description
| +| Organisation label | University Tuebingen | ✅ | Text |
Human-readable name of the organisation, automatically received from ROR by the Organisation IRIView Description
| +| Registration date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the measurement was created within the Data ManagerView Description
| +| Comment | Repeated measurement
after bad QC | ❌ | Text |
Free text, can contain any notes related to a measurement in question with up to 500 charactersView Description
| +| Enrichment method | Phosphopeptide Enrichment | ❌ | Text |
Enrichment of proteins or peptides of different characteristicsView Description
| +| Fraction name | Fraction01 | ❌ | Text |
If the sample was fractionated, this label can be used to indicate which fraction was measuredView Description
| +| Injection volume | 10 | ❌ | Number |
The sample volume injected into the LC column in microlitres (µl)View Description
| +| Label | Heavy | ❌ | Text |
The label value for the label type that has been usedView Description
| +| Label type | SILAC | ❌ | Text |
The label type that has been used to label the sample for measurementView Description
| | Measurement name | Lab_Id_01 | ❌ | Text |
Common human-readable name of the measurement. Can also be the internal lab identifierView Description
| -| LCMS method | APCI | ❌ | Text |
Laboratory specific methods that have been used for LCMS measurement.View Description
| -| Replicate name | Replicate_1 | ❌ | Text |
Label to distinguish between technical replicates for repeated measurements of the same sample.View Description
| +| LCMS method | APCI | ❌ | Text |
Laboratory-specific methods that have been used for LCMS measurementView Description
| +| Replicate name | Replicate_1 | ❌ | Text |
Label to distinguish between technical replicates for repeated measurements of the same sampleView Description
| -Additional information can be found the **property information** tab within the +Additional information can be found in the **Property Information** tab within the proteomics measurement registration [template sheet](templates/proteomics_measurement_registration_sheet.xlsx). -### Raw Data +### Raw data -Visit our documentation to find out how to [upload](../rawdata/raw_data_request_server_access.md) +Visit our documentation to find out how to [upload](../rawdata/raw_data_upload.md) or [download](../rawdata/raw_data_download.md) your raw data. The following concepts are associated with a raw data dataset. -| Concept | Example | Mandatory | Type | Description | +| Concept | Example | Mandatory | Type | Description | |-------------------|----------------------------------|----------------------------|--------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| File count | 20 | ✅ | Number |
Number of Files contained within the raw data upload, automatically determined by the Data ManagerView Description
| -| File size | 2000Mb | ✅ | Text |
Size of all Files within the raw data upload, automatically determined by the Data ManagerView Description
| -| File suffixes | fastq, tar, txt | ✅ | List of Text |
List of File suffixes for all files within a raw data upload, automatically determined by the Data ManagerView Description
| -| Registration date | 2025-04-28
08:24:25.000000@ | ✅ | Date |
Timestamp when the raw data was uploaded within the Data ManagerView Description
| +| File count | 20 | ✅ | Number |
Number of files contained within the raw data upload, automatically determined by the Data ManagerView Description
| +| File size | 2000 MB | ✅ | Text |
Size of all files within the raw data upload, automatically determined by the Data ManagerView Description
| +| File suffixes | fastq, tar, txt | ✅ | List of Text |
List of file suffixes for all files within a raw data upload, automatically determined by the Data ManagerView Description
| +| Registration date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the raw data was uploaded within the Data ManagerView Description
| + +--- + +## What's next + +➡ [API reference](../developers/api.md) · [Process overview](../get_started/process_overview.md) diff --git a/docs/ontology_search/ontology_search_introduction.md b/docs/ontology_search/ontology_search_introduction.md index 04fbff0..c9e008b 100644 --- a/docs/ontology_search/ontology_search_introduction.md +++ b/docs/ontology_search/ontology_search_introduction.md @@ -1,45 +1,38 @@ -# Ontology Search Introduction +# Ontology search -The ontology search functionality can be accessed from within a project via the application drawer. -Therefore, start by [navigating](#ontology-navigation) to the ontology summary view. +The ontology search helps you find standardised scientific terms — and their unique codes — for use in experiment and measurement registration. -The ontology search allows you to [search](#ontology-search) for ontology information -within our database. +**What's an ontology?** A shared scientific dictionary: an agreed list of terms with unique identifiers, maintained by the research community. Using these terms means your data can be understood and compared by anyone. -## Ontology Navigation +## Navigate to the search -From the landing page you can navigate into a project via the project list by clicking on the -project card directly. +From any project view, open the application drawer (top left) and select **Ontology search**. -Once within the project open the application drawer via the button on the top left of the -application -and select "ontology search" to navigate to the ontology summary view. -![ontology_summary.png](images/ontology_search_summary.png){.screenshot} +![Ontology search](images/ontology_search_summary.png){.screenshot} -## Ontology Search +## Search for a term -Start by providing at least 2 letters in the ontology search field of the entity for which you want -to retrieve the ontology information. -Once done so a card list showing information for all ontology entries which contain the provided -letters will appear. -![ontology_search_triggered_without_species.png](images/ontology_search_triggered_without_species.png){.screenshot} +Type at least 2 characters in the search field. A list of matching terms appears with their names, descriptions, and identifiers. -Alternatively the ontology search can also be limited to only look for species specific information. -![ontology_search_triggered_with_species.png](images/ontology_search_triggered_with_species.png){.screenshot} +![Search results](images/ontology_search_triggered_without_species.png){.screenshot} -!!! note "Enable species specific search" -The application provides access to species terminologies for the -complete [tree of life](https://www.ncbi.nlm.nih.gov/taxonomy) provided by NCBI. -If activated, only the species taxonomy is queried. +### Species-only search -??? note "Why does the search not include species?" -We currently host our own database instance for efficient queries of species to support -faster lookups. Also, the TIB terminology service, which we have integrated for all other term lookups, -does not provide the tree of life, yet. -So they are two independent systems that we request information from here +Toggle the species filter to search only the [NCBI taxonomy](https://www.ncbi.nlm.nih.gov/taxonomy) (the complete tree of life). -Additionally, it allows you to copy the -ontology [CURIE](https://link.springer.com/article/10.1007/s12599-022-00744-0), -which can be especially handy -during [measurement registration](../measurement/measurement_registration.md) -![ontology_search_curie_copie.gif](images/ontology_search_copy_curie.png){.screenshot} +![Species search](images/ontology_search_triggered_with_species.png){.screenshot} + +!!! note "Why a separate species search?" + Species terms come from NCBI's taxonomy database, which is hosted separately from the [TIB Terminology Service](https://terminology.tib.eu) used for other ontologies. The two systems are queried independently. + +## Copy a CURIE + +Click the copy icon next to any term to copy its CURIE (e.g. `NCIT:C105979`) to your clipboard. Particularly useful when filling in [measurement registration templates](../measurement/measurement_registration.md). + +![Copy CURIE](images/ontology_search_copy_curie.png){.screenshot} + +--- + +## What's next + +➡ [Register measurements](../measurement/measurement_registration.md) · [Create an experiment](../experiment/experiment_creation.md) diff --git a/docs/project/project_access.md b/docs/project/project_access.md index 5824aa7..f242e5e 100644 --- a/docs/project/project_access.md +++ b/docs/project/project_access.md @@ -1,58 +1,59 @@ -# Project Access Management +# Manage project access -Start by [navigating](project_introduction.md#project-navigation) to the project summary view of your project of interest. -From there, switch to the access management view by clicking on the "users" tab in the application drawer -![project_summary_with_drawer](images/project_summary_drawer.png){.screenshot} +Control who can view or modify your project by assigning roles. -!!! info "Admin privileges" - You need to have a project role of "owner" or "admin" to see the users tab and access the project access view. +!!! info "Required role" + You need **owner** or **admin** role to manage access. -Within the project access view a table shows the username of all accounts currently having access to their project and their respective roles -![project_access](images/project_access.png){.screenshot} -Additionally, this view enables you to specify which users have access to your project and fine-tune the kind of permissions you want to grant the users. +## Open the access view -1. [Add collaborators to your project](#add-collaborator) -2. [Change the project role of collaborators within your project](#change-project-role) -3. [Remove collaborators from your project](#remove-collaborator) +[Navigate](project_introduction.md#find-and-open-a-project) to the project summary. Open the application drawer and click the **Users** tab. -## Add Collaborator +![Application drawer](images/project_summary_drawer.png){.screenshot} -To add a collaborator to your project press the "add people" button on the top right of the project access view which will open the "add collaborator" dialog -![add_collaborator](images/project_access_grant_access.png){.screenshot} +The access view lists all users and their roles. -You can now select the user from the select person section via their respective username. -![add_collaborator_opened](images/project_access_grant_access_opened.png){.screenshot} +![Access view](images/project_access.png){.screenshot} -Afterward, you need to specify the kind of access the added collaborator should have to the project. -Do so by clicking on the relevant radio button in the "Assign a role" section of the dialog. -![add_collaborator_role_selection](images/project_access_grant_access_role_selection.png){.screenshot} +## Add a collaborator -!!! warning "Admin privileges" - Keep in mind that a user that you grant admin privileges to your project can add and remove other collaborators to/from your project. +1. Click **Add people** (top right). -Finally click on the "Confirm" button to add your collaborator with the chosen role to the project. -![add_collaborator_role_selection](images/project_access_with_new_user.png){.screenshot} + ![Add dialog](images/project_access_grant_access.png){.screenshot} + +2. Search by username and select the person. + + ![User selected](images/project_access_grant_access_opened.png){.screenshot} + +3. Choose a role: **read**, **write**, or **admin**. + + ![Role selected](images/project_access_grant_access_role_selection.png){.screenshot} + + !!! warning "Admin role" + Admin users can add and remove other collaborators. + +4. Click **Confirm**. + + ![New user added](images/project_access_with_new_user.png){.screenshot} !!! info "Email notification" - Once a user has been added to your project, the email address linked to the username will receive an email with a link to the project they have been added to. + The new collaborator receives an email with a direct link to the project. -## Change Project Role +## Change a role -To change the project role of a project collaborator press the edit button in the action column next to the user within the table. -![project_access_change_role](images/project_access_with_new_user.png){.screenshot} -This will make the role of the user selectable in the role column of the access table -![project_access_change_role_selectable](images/project_access_change_role.png){.screenshot} -Click on the selection field within the role column to open up the possible selections for the user in question and select the role you want to grant to the collaborator. -![project_access_change_role_opened](images/project_access_change_role_opened.png){.screenshot} -This will change the user to the newly selected role for your project and close the selection within the role column. -![project_access_change_role_changed](images/project_access_change_role_changed.png){.screenshot} +Click the **edit** button next to the user, select the new role from the dropdown, and the change applies immediately. + +![Role dropdown](images/project_access_change_role_opened.png){.screenshot} !!! info "Project owner" - The project owner is always the user registering the project for the first time. - The project owner cannot be removed from the project and their role cannot be changed. - -## Remove Collaborator -To remove a collaborator press the cross icon next to the collaborator in question. -![project_access_remove_collaborator](images/project_access_with_new_user.png){.screenshot} -Once pressed the collaborator will be removed from the project. -![project_access_remove_collaborator_removed](images/project_access.png) + The owner role is assigned to whoever created the project. It cannot be changed or removed. + +## Remove a collaborator + +Click the **×** icon next to the collaborator. They lose access immediately. + +--- + +## What's next + +➡ [Back to project summary](project_introduction.md) · [Design an experiment](../experiment/experiment_introduction.md) diff --git a/docs/project/project_edit.md b/docs/project/project_edit.md index 27f665e..e142c50 100644 --- a/docs/project/project_edit.md +++ b/docs/project/project_edit.md @@ -1,88 +1,64 @@ -# Project Editing +# Edit a project -Start by [navigating](project_introduction.md#project-navigation) to the project summary view of your project of interest. +Update project details, attach supporting documents, or export metadata. -!!! info "Project access" - Should you not see your project of interest, - please make sure that you have been granted access to it by the project owner +!!! info "Required role" + You need **write** or **admin** role to edit. If the edit button isn't visible, ask the project owner to update your role. -Within the project summary view, press the edit button on the top right to open up the edit dialog for your project. +## Edit project details -!!! info "Project role" - Should you not see the edit button, - please make sure that you have been granted the "write" or "admin" role to it by the project owner/admin! +[Navigate](project_introduction.md#find-and-open-a-project) to the project summary. Click the **edit** button (top right), update the fields, and click **Save**. -Once the dialog has been opened you can edit the project attributes of interest -within the dialog and save your changes via the "save" button on the bottom. -![project_navigation_search](images/project_edit.png){.screenshot} +![Edit dialog](images/project_edit.png){.screenshot} -# Upload of Project Related Files +## Upload project-related files -After successful project creation, navigate into your project of interest as outlined in -[project navigation](project_introduction.md#project-navigation). +### Offer upload -Within the project summary view, you are able to upload project related files such as the offer or quality control reports, -via their respective upload buttons to the right. +1. Click the upload button in the **Offer** section of the project summary. -!!! info "Project role" - Should you not see the upload button, - please make sure that you have been granted the "write" or "admin" role to it by the project owner/admin! + ![Offer upload](images/offer_upload.png){.screenshot} -## Offer Upload +2. Drag and drop your file or click **Upload files** to browse. -To upload one or more offer files click the upload button within the offer component in the middle right of the project summary view, -which will open up the offer upload dialog. -![offer_upload](images/offer_upload.png){.screenshot} + ![File selected](images/offer_upload_with_offer.png){.screenshot} -Within the dialog you are able to upload your offer files either via clicking the upload files button and selecting the files of interest in your file system -or by drag and dropping the files into the dashed box saying "drop your files here". -![offer_upload_with_offer](images/offer_upload_with_offer.png){.screenshot} -Should you have uploaded one or more wrong files, -you can easily delete them via a press of the cross icon next to the respective file names +3. Tick **Signed** if the offer has been countersigned. -!!! warning "File constraints" - Please adhere to the file format and maximum file size outlined in the dialog. - Currently, an offer file has to be in the PDF file format with a maximum file size of 16Mb + ![Signed checkbox](images/offer_upload_with_offer_signed.png){.screenshot} -For each successful file upload you are then able to specify if the offer was signed already -via the checkbox shown next to the file name. -![offer_upload_with_offer_signed](images/offer_upload_with_offer_signed.png){.screenshot} +4. Click **Save**. -Finally, save your uploaded files to the project via a pressing the "Save" button on the bottom right of the dialog. -Your uploaded offer will then be shown in the offer component on the middle right of the project summary view. -![project_summary_with_offer](images/project_summary_with_offer.png){.screenshot} +!!! warning "File requirements" + PDF only. Maximum 16 MB. -## Quality Control Upload +### Quality control upload -To upload one or more quality control(qc) files click the upload button within the quality control component in the bottom right of the project summary view, -which will open up the QC upload dialog. -![qc_upload](images/qc_upload.png){.screenshot} +1. Click the upload button in the **Quality Control** section. -Within the dialog you are able to upload your qc files either via clicking the upload files button and selecting the files of interest in your file system -or by drag and dropping the files into the dashed box saying "drop your files here". -![qc_upload_with_qc](images/qc_upload_with_qc.png){.screenshot} + ![QC upload](images/qc_upload.png){.screenshot} -Should you have uploaded one or more wrong files, -you can easily delete them via a press of the cross icon next to their respective file names +2. Drag and drop or click **Upload files**. -!!! warning "File constraints" - Please adhere to the file format and maximum file size outlined in the dialog. - Currently, a qc file has to be in the PDF, docx, or xlsx file format with a maximum file size of 16Mb + ![File selected](images/qc_upload_with_qc.png){.screenshot} -For each successful qc file upload you are then able to optionally link it to the experiment in question -within the link to an experiment section of the dialog. -To do so, select the experiment linked to their respective quality control files via the dropdown menu below the file name. -![qc_upload_with_qc_experiment_selected](images/qc_upload_with_qc_experiment_selected.png){.screenshot} +3. Optionally link the report to an experiment using the dropdown. -Finally, save your uploaded files to the project via a pressing the save button on the bottom right of the dialog. -Your uploaded qc reports will then be shown in the qc component on the bottom right of the project summary view. -![project_summary_with_qc](images/project_summary_with_qc.png){.screenshot} + ![Experiment linked](images/qc_upload_with_qc_experiment_selected.png){.screenshot} -## Download Project Metadata +4. Click **Save**. -The project metadata stored within the project summary is currently downloadable as a research object crate(RO-crate) container. -This ensures that the project data can be accessed as a machine-readable and structured archive of the information stored in the Data Manager. -For more information visit the official Research Object [documentation](https://www.researchobject.org/ro-crate/about_ro_crate) +!!! warning "File requirements" + PDF, DOCX, or XLSX. Maximum 16 MB. -To **download** the project metadata stored within the project summary as an RO-Crate click on the **`Export as RO-Crate`** button. -This will export the metadata as a `.zip` container to your local download directory. +## Download project metadata + +Export your project metadata as an [RO-Crate](https://www.researchobject.org/ro-crate/) — a structured, machine-readable package that bundles your metadata in a format other systems can understand. + +Click **Export as RO-Crate** in the project summary. A `.zip` archive downloads to your computer. + +--- + +## What's next + +➡ [Manage access](project_access.md) · [Create an experiment](../experiment/experiment_creation.md) diff --git a/docs/project/project_introduction.md b/docs/project/project_introduction.md index 84c1d07..4ba3035 100644 --- a/docs/project/project_introduction.md +++ b/docs/project/project_introduction.md @@ -1,32 +1,39 @@ -# Project Introduction +# Projects -After a successful login you will be redirected to your personal landing page. -From there, you can either [navigate](#project-navigation) into your project of interest -or [register a new project](project_registration.md). +A project is the top-level container in the Data Manager. It holds everything related to one research study — experiments, samples, measurements, and raw data — all linked together and searchable. -!!! info "Project access" - Should you not see your project of interest, - please make sure that you have been granted access to it by the project owner +After logging in, your landing page shows all projects you have access to. -## Project Navigation +## Find and open a project -From the landing page you can navigate into your project via the project list. -Either click on the project card directly or make use of the search box -to filter the project lists for the project of interest. -![project_navigation_search](images/project_search.png){.screenshot} +Scroll through the project list or type into the search box to filter by name or code. -Once you've found the project you can navigate into it -by clicking on its respective project card, which will take you into the project summary view. -![project_summary](images/project_summary.png){.screenshot} +![Project list with search](images/project_search.png){.screenshot} -To navigate between projects you can go back to the landing page via the home button on the top right. -Alternatively you can use the application drawer to go to your most recent projects or switch back to your landing page. -![project_summary](images/project_summary_drawer.png){.screenshot} +Click a project card to open the **project summary view**. + +![Project summary](images/project_summary.png){.screenshot} + +!!! info "Can't find your project?" + Ask the project owner to [grant you access](project_access.md). + +## Navigate between projects + +**Home button** (top right): Returns you to the project list. + +**Application drawer** (top left): Lists your recent projects and experiments. Open it with the menu button. + +![Application drawer](images/project_summary_drawer.png){.screenshot} + +Select a project from the drawer to jump straight to it, or click **Go to projects** to return to the full list. + +![Drawer expanded](images/project_summary_drawer_opened.png){.screenshot} !!! info "Application drawer" - The application drawer can be used to navigate between projects and experiments once a project has been selected. - It can be opened and closed via it's button on the top left. + The drawer also lets you switch between experiments within a project without going back to the project summary. + +--- + +## What's next -To navigate between projects via the application drawer select your project of interest from the menu on the left -or select "go to projects" to go back to your landing page -![project_summary](images/project_summary_drawer_opened.png){.screenshot} +➡ [Register a new project](project_registration.md) · [Edit project details](project_edit.md) · [Manage access](project_access.md) diff --git a/docs/project/project_registration.md b/docs/project/project_registration.md index 4c6a247..1f19366 100644 --- a/docs/project/project_registration.md +++ b/docs/project/project_registration.md @@ -1,84 +1,59 @@ -# Project Registration +# Register a project -To register a new project the following steps have to be taken: +The registration wizard walks you through four steps. Click **Create project** at the top of your project list to begin. -1. Trigger the project creation dialog -2. Provide the minimal required information for the following steps within the dialog: - 1. [Project Design](#project-design) - 2. [Funding Information (Optional)](#funding-information) - 3. [Project Collaborators](#project-collaborators) - 4. [Experimental Information](#experimental-information) +## Project design -You can start the project registration process by clicking the create button on top of your project list within your personal landing page. -This will open the project registration dialog which will guide you through the individual steps necessary to successfully register a new project. +Provide a **project title** and **objective** (up to 2,000 characters). -Following is a detailed guide for each of these steps: +!!! info "Project code" + Each project is assigned a unique six-character alphanumeric code (e.g. `Q2ABCD`). This code appears in every sample ID and measurement ID downstream — treat it as your project's permanent address. -## Project Design +Click **Next**. -The information provided in this step serves as the basic outline of your project. +![Project design](images/project_design.png){.screenshot} -Therefore, it is mandatory that you provide a concise project title and project description in -this step. -!!! info "Project code" - Each created project will be assigned a unique 5 letter project code, distinguishing it from other projects within the system. +## Funding information + +If your project is funded, enter the **grant name** (e.g. DFG) and **grant identifier**. Both fields are optional. Click **Next** to continue, or **Back** to revise. + +![Funding](images/funding_information.png){.screenshot} -Once all the required information has been provided you can navigate to the next step via the "next" -button below +## Project collaborators -![project_design](images/project_design.png){.screenshot} +| Role | Required | Description | +|---|---|---| +| **Principal investigator** | ✅ | Lead researcher | +| **Project manager** | ✅ | Day-to-day coordinator | +| **Responsible person** | ⬜ | Contact for questions | -## Funding Information +For each role, enter a full name and email. Tick the checkbox above a role to assign yourself. Click **Next**. -In this step provide the grant label and identifier if your project has been funded by a third party -sponsor. -If this is not applicable for your project, you can go directly to the next step via the "next" -button below. If you want change any adde information, previous steps in the dialog can always be reached by clicking the "back" button. +![Collaborators](images/project_collaborators.png){.screenshot} -![funding_information](images/funding_information.png){.screenshot} +## Experimental information -## Project Collaborators +Set up your first experiment. Provide an **experiment name** and select at least one entry for: -In this step you are able to specify the collaborating parties associated with your project. -The minimum required information consists of declaring the name and email address of the principal -investigator -instigating the project and the project manager handling the projects' execution. -Finally, you can optionally define the name and email address of the responsible party, which should -be contacted should project specific questions arise. -For each of these roles you can of course also assign yourself via a press of the checkbox above -each role. +- **Species** — the organism(s) your samples come from +- **Specimen** — the tissue or material type +- **Analyte** — the molecular class being measured -Once all the required information has been provided you can navigate to the next step via the "next" -button below -or go to the previous step via the "back" button. +Type at least 2 characters in each search field to see matching terms from standardised life science ontologies. -![project_collaborators](images/project_collaborators.png){.screenshot} +![Experiment search](images/experimental_information_search.png){.screenshot} -## Experimental Information +!!! info "What are these codes?" + Each option is backed by a standardised identifier called a CURIE (e.g. `NCBITaxon:9606` for *Homo sapiens*). You don't need to memorise them — just search by name. The system stores the code automatically, which means your data can be understood and compared by anyone, anywhere. -The final step during project creation consists of creating the basic outline of your first -experiment -associated with your project. -For this you need to specify the name of the experiment, and species, specimen and analyte -information of the involved organisms. -To provide these details you can select one or more entries via the provided search fields. -Start by providing at least 2 letters of your species, specimen or analyte in their respective inputs -fields and possible selection options will appear. You are also able to select icons for the main species and specimen in your experiment. Otherwise, default icons will be shown. -![experimental_information](images/experimental_information_search.png){.screenshot} +Optionally select an icon for the species and specimen. Click **Confirm** to create the project. -!!! info "Ontology id" - Behind each selectable option within the species, specimen and analyte input fields - , the unique ontology identifier from one of several ontologies is stored. +![Experiment completed](images/experimental_information.png){.screenshot} -Once all the required information has been provided you can create your project via the "confirm" -button below or go to the previous step via the "back" button. +Your project appears in the project list. You can now [upload supporting files](project_edit.md#upload-project-related-files) like offers and QC reports. -![experimental_information](images/experimental_information.png){.screenshot} +--- -After successful project creation, your project should appear in the project listing on your landing -page. -To navigate into your project, you can follow the steps outlined -in [project navigation](project_introduction.md#project-navigation) +## What's next -Finally, after successful project registration you can also upload project related files such as quality control and offers as outlined -in [project edit](project_edit.md#upload-of-project-related-files) +➡ [Set up experimental variables](../experiment/experiment_creation.md) · [Invite collaborators](project_access.md) diff --git a/docs/rawdata/raw_data_download.md b/docs/rawdata/raw_data_download.md index 0305584..0bb9115 100644 --- a/docs/rawdata/raw_data_download.md +++ b/docs/rawdata/raw_data_download.md @@ -1,181 +1,125 @@ -# Raw Data Introduction +# Download raw data -The raw data summary view shows detailed information about the raw data already [registered](raw_data_request_server_access.md) for the measurements within your experiment. -Additionally, it enables you to generate the [URLs](#raw-data-url-generation) necessary -to start the command line based [raw data download](#download-raw-data). +The raw data view shows data already uploaded for measurements in your experiment. From here you can generate download URLs and retrieve files via the command line. -!!! info "Are you a developer?" - Read about Swagger API docs in the [developer](../developers/api.md) section. +!!! info "Looking for the API?" + See the [API reference](../developers/api.md) for programmatic access. ## Process -1. Create a [personal access token](#personal-access-token) (PAT) -2. Navigate to the raw data summary [view](#raw-data-navigation) -3. Acquire [download URLs](#raw-data-url-generation) for the registered measurement raw data. -4. Start the measurement raw data [download](#download-raw-data) measurement and get a coffee :coffee: +1. [Create a personal access token](#personal-access-token) +2. [Navigate to the raw data view](#raw-data-navigation) +3. [Generate download URLs](#generate-download-urls) +4. [Download via command line](#download-via-command-line) ## Personal access token -Before you can begin with downloading any data via HTTPS, you need to tell the download server -who you are. We enforce a token-based authentication (personal access token: PAT) so you are not -required to expose your password -to anyone or to any system. +Before downloading, you need a personal access token (PAT) — a credential that identifies you to the download server without exposing your password. -!!! tip "multiple tokens" - You can create as many tokens as you like, however consider them as a secret. +!!! danger "Keep your token secret" + Treat your PAT like a password. Don't paste it into shared scripts, don't commit it to version control, and don't share it with colleagues. Each person should generate their own token. ### Generate a token -First of all, navigate to your PAT overview page in your profile overview (top-right corner) +1. Click your profile icon (top right) and open the PAT overview page. -![profile menu](images/raw_data_create_pat_profile_menu.png){.screenshot} + ![Profile menu](images/raw_data_create_pat_profile_menu.png){.screenshot} -You should now be able to see the PAT token overview page: + ![PAT overview](images/raw_data_create_pat_overview_no_pats.png){.screenshot} -![pat overview](images/raw_data_create_pat_overview_no_pats.png){.screenshot} +2. Set an expiry date and click **Generate**. -Personal access tokens have a life-time, which can be set based on your requirements. Your token -expires automatically, there is nothing you have to do manually. + ![Generate token](images/raw_data_create_pat_generate_token.png){.screenshot} -![generate token](images/raw_data_create_pat_generate_token.png){.screenshot} - -!!! warning "Token accessibility" - A generated token will be only visible in its raw notation once! Make sure to store it safely in - your local password manager, you will not be able to access the token value again. + !!! warning "One-time visibility" + The token is shown in full only once. Copy it to a password manager immediately — you won't be able to see it again. ### Manage tokens -You can see your created tokens in the overview, however since they are instantly encrypted after generation, -you cannot access the actual token text anymore. We encourage you to use meaningful descriptions for your tokens, -so you remember for what purpose you have created them. - -![token overview](images/raw_data_create_pat_token_overview.png){.screenshot} - -!!! danger "Token security" - If you are unsure, if your PAT got exposed or shared with untrusted parties, delete them right - away. You can create new, safe tokens at any time. +Your created tokens appear in the overview. Descriptions help you remember what each token is for. -## Raw Data Navigation +![Token overview](images/raw_data_create_pat_token_overview.png){.screenshot} -To navigate to the raw data summary view start by [navigating](../project/project_introduction.md#project-navigation) to the project summary view of your project of interest. -From there [navigate](../experiment/experiment_introduction.md#experiment-navigation) into the experiment summary view of interest. -From within the experiment summary you can navigate into the raw data summary view. -To do so, click on the "Download Raw Data" tab within the experiment navigation bar on the top. -![experiment_summary.png](../experiment/images/experimental_summary.png){.screenshot} +!!! danger "Compromised token?" + If a token may have been exposed, delete it immediately and create a new one. -This will take you to the raw data summary view -![raw_data_summary_no_data.png](images/raw_data_summary_no_data.png) +## Raw data navigation -### Raw Data URL Generation +[Navigate](../project/project_introduction.md#find-and-open-a-project) to your project, then [open the experiment](../experiment/experiment_introduction.md#find-and-open-an-experiment). Click the **Download Raw Data** tab. -To acquire the URLS necessary for the measurement data download the following steps have to be taken: +![Experiment summary](../experiment/images/experimental_summary.png){.screenshot} -1. Navigate to the [raw data summary view](#raw-data-navigation) -2. Select the measurements for which the raw data URLs should be generated. - ![raw_data_download_measurement_selection.png](images/raw_data_download_URL_generation_measurement_selection.png) -3. Press the "Download URL List" button to download a text file containing the URLs of the selected measurements - ![raw_data_download_URL_downloaded.png](images/raw_data_download_URL_downloaded.png) +![Raw data view](images/raw_data_summary_no_data.png){.screenshot} -### Download Raw Data +## Generate download URLs -To download the raw data for your measurements, basic command line/terminal knowledge is required. -First start by opening the command line in your operating system of choice. +1. Select the measurements you want to download. -#### Open command line on Mac + ![Select measurements](images/raw_data_download_URL_generation_measurement_selection.png){.screenshot} -Open the terminal via the spotlight search function by clicking on the magnifying glass on the top right or via pressing the command and spacebar key on your keyboard. -Input "terminal" into the text field and select the terminal application from the suggestions. +2. Click **Download URL List** to download a text file containing one URL per measurement. -![data_download_mac_command_line.png](images/data_download_mac_command_line.png) + ![URLs downloaded](images/raw_data_download_URL_downloaded.png){.screenshot} -#### Open command line on Windows +## Download via command line -Click on the magnifying glass in the toolbar at the bottom of the screen. -Input "terminal" into the search field and select the terminal application from the suggestions. +You'll need a terminal to run the download commands. If you haven't used one before — don't worry. Just copy, paste, and press Enter. -![data_download_windows_command_line.png](images/data_download_windows_command_line.png) +=== "Open terminal on Mac" -#### Download data via command line + Click the magnifying glass (top right) or press `Cmd + Space`, type `Terminal`, and select the Terminal app. -We show how to run the download via the two popular command line clients [cURL](https://curl.se/docs/manpage.html) and [GNU Wget](https://www.gnu.org/software/wget/). -Of course, you can use any other software that supports HTTP. + ![Mac terminal](images/data_download_mac_command_line.png) -!!! warning - The data will be downloaded into the directory in which the command is run. - Please ensure that it's the correct directory and there is enough space before running the download +=== "Open terminal on Windows" -The commands contain two placeholders, where you have to provide your custom input: + Click the magnifying glass in the taskbar, type `Terminal`, and select the Terminal app. -- *ACCESS_TOKEN*: your generated [personal access token](#personal-access-token) -- *MEASUREMENT_URL*: the [URL](#raw-data-url-generation) associated with the data of the measurement + ![Windows terminal](images/data_download_windows_command_line.png) -=== "curl" +!!! warning "Download location" + Files download to the directory you're currently in. Make sure it's the right folder and has enough free space. - ``` bash - curl -OJ -H "Authorization: Bearer " - ``` +### Single measurement -=== "wget" - - ``` bash - wget --content-disposition --trust-server-names --header "Authorization: Bearer " - ``` -For example to download the raw data associated with measurement **MSQ7645002AL-182987406699583** the commands would look as follows: +Replace `` with your PAT and `` with the URL from the download list. === "curl" - ``` bash - curl --parallel --fail -OJ -H "Authorization: Bearer v71E00f750Z78oBW4SKs90Vrd39h98eG" https://download.qbic.uni-tuebingen.de/measurements/MSQ7645002AL-182987406699583 + ```bash + curl --fail -OJ -H "Authorization: Bearer " ``` === "wget" - ``` bash - wget --content-disposition --trust-server-names --header "Authorization: Bearer v71E00f750Z78oBW4SKs90Vrd39h98eG" https://download.qbic.uni-tuebingen.de/measurements/MSQ7645002AL-182987406699583 + ```bash + wget --content-disposition --trust-server-names --header "Authorization: Bearer " ``` -To download multiple measurements using one command, you can provide a text file containing one measurement URL per line (as in the one you can download in the raw data summary!). - -```txt - - -``` +### Multiple measurements -The corresponding commands need to be adapted accordingly. +Create a text file with one URL per line (or use the file from the [URL generation step](#generate-download-urls)). === "curl" - ``` bash - curl --remote-name-all -OJ -H "Authorization: Bearer " $(cat ) + ```bash + curl --fail --parallel --remote-name-all -J -H "Authorization: Bearer " $(cat download_urls.txt) ``` + !!! note "curl version" + The `--parallel` flag requires curl 7.66.0 or later. Check with `curl --version`. On macOS, install a current version via Homebrew if needed: `brew install curl`. + === "wget" - ``` bash - wget --content-disposition --trust-server-names --header "Authorization: Bearer " -i + ```bash + wget --content-disposition --trust-server-names --header "Authorization: Bearer " -i download_urls.txt ``` -!!! warning - To ensure character validity in the text file, please format it in the *UTF-8* format - -The easiest way to generate such a text file is to generate it in the [raw data view](#raw-data-url-generation). - -![data_download_selected_measurements.png](images/data_download_selected_measurements.png) +!!! warning "File encoding" + The URL text file must be UTF-8 encoded. -This will download the text file containing the urls for the selected measurements +--- -![raw_data_download_text_file.png](images/raw_data_download_text_file.png) - -For example to download the raw data for all measurements within the text file **download_urls.txt** the command would look as follows: - -=== "curl" +## What's next - ``` bash - curl --remote-name-all -OJ -H "Authorization: Bearer v71E00f750Z78oBW4SKs90Vrd39h98eG" $(cat /Your/awesome/path/download_urls.txt) - ``` - -=== "wget" - - ``` bash - wget --content-disposition --trust-server-names --header "Authorization: Bearer v71E00f750Z78oBW4SKs90Vrd39h98eG" -i /Your/awesome/path/download_urls.txt - ``` +➡ [Back to experiment](../experiment/experiment_introduction.md) · [Manage project access](../project/project_access.md) diff --git a/docs/rawdata/raw_data_request_server_access.md b/docs/rawdata/raw_data_request_server_access.md index e15917c..fcf4228 100644 --- a/docs/rawdata/raw_data_request_server_access.md +++ b/docs/rawdata/raw_data_request_server_access.md @@ -1,88 +1,77 @@ -# Request Server Access +# Request server access -This section gives an overview on how to request access to our upload server to submit data from outside of the University Tübingen network. +This page is for users outside the University of Tübingen network who need to upload data to the QBiC server. -!!! tip - Should you be a member of the university of Tübingen you can immediately [upload your data](raw_data_upload.md) and skip this process. +!!! tip "University members" + If you're at the University of Tübingen, you can [upload data directly](raw_data_upload.md) — skip this page. -## Prerequisites +## What you'll need -- An `ed25519` encrypted public and private keypair on the connecting machine -- The static and public `IPv4` address of the connecting machine -- The name and affiliation of the data submitting party +- An **ed25519 SSH key pair** on the machine you'll connect from +- The **public, static IPv4 address** of that machine +- Your **name and institutional affiliation** -## Procedure +## Generate an SSH key pair -To minimize security risk, access to our upload server is restricted to temporary whitelisted IP addresses from outside of the university network. -!!! note - Please make sure to provide a **public** and **static** IP address from the machine you want to connect from. - Check with your IT department on details on how to obtain such an IP address. - -As an additional security layer, data submitters have to authenticate using [strong cryptographic keys](#generate-cryptographic-keys). -Once these technical details are resolved, feel free to contact us to [request access to our server](#get-in-contact) - -### Generate Cryptographic Keys - -For account creation a public key is required. Please generate an ssh keypair via your system specific command and make sure to specify your email address: +SSH keys are a secure alternative to passwords for file transfers. Your computer generates two linked files: a private key (stays on your machine — never share it) and a public key (you send it to us). When you connect, the system checks that they match. === "Linux" - Open your terminal - ``` bash + ```bash ssh-keygen -t ed25519 -a 420 -f ~/.ssh/qbic-upload.ed25519 -C "" ``` -=== "MacOS" +=== "macOS" - Open Terminal [as described by Apple](https://support.apple.com/guide/terminal/open-or-quit-terminal-apd5265185d-f365-44cb-8b09-71a064a42125/mac) - ``` bash + Open Terminal ([how to](https://support.apple.com/guide/terminal/open-or-quit-terminal-apd5265185d-f365-44cb-8b09-71a064a42125/mac)): + + ```bash ssh-keygen -t ed25519 -a 420 -f ~/.ssh/qbic-upload.ed25519 -C "" ``` === "Windows" - Open PowerShell [as described by Microsoft](https://learn.microsoft.com/en-us/powershell/scripting/windows-powershell/starting-windows-powershell?view=powershell-7.5#run-from-the-start-menu) - ``` bash + Open PowerShell ([how to](https://learn.microsoft.com/en-us/powershell/scripting/windows-powershell/starting-windows-powershell?view=powershell-7.5)): + + ```bash ssh-keygen -t ed25519 -a 420 -f $HOME/.ssh/qbic-upload.ed25519 -C "" ``` -with the name `qbic-upload.ed25519` and a public key named `qbic-upload.ed25519.pub` within your `.ssh` directory. +This creates two files in your `.ssh` directory: `qbic-upload.ed25519` (private key) and `qbic-upload.ed25519.pub` (public key). -### Get in Contact +!!! danger "Never share your private key" + Only send the `.pub` file. The private key file (without `.pub`) must stay on your computer. -After you have [generated your ssh key](#generate-cryptographic-keys) and setup a public static IP address, -please write an email to [support@qbic.zendesk.com](mailto:support@qbic.zendesk.com) and attach the public (.pub) key file similar to the provided template: +## Request access -!!! warning - Make sure you **only** send us the public key file ending with .pub +Send an email to [support@qbic.zendesk.com](mailto:support@qbic.zendesk.com) with the `.pub` file attached: - **Do not send us the private key!** +```text +Dear QBiC Team, -```txt -Dear QBiC Team +I would like to upload measurement data to the Data Manager. +Please provide me with an account on your upload server. -I would like to upload measurement data to your data manager. -Can you please provide me with an account on your upload server. - -Below you can find the requested information - -Name: -E-Mail: -Affiliation: , +Name: +Email: +Affiliation: , Project: Access Duration: 30 days -IP-Address: -For accessing your upload server I would like to use the attached key. +IP Address: -Attachments: qbic-upload.ed25519.pub +Attached: qbic-upload.ed25519.pub ``` -### Wait for a reply -After you have [established contact](#get-in-contact) with our support team, -we will create an account for you on our upload server and contact you with confirmation and your assigned username. +!!! note "Static IP required" + Check with your IT department about obtaining a public, static IP address for the machine you'll connect from. + +## Wait for confirmation + +The QBiC team will create your account and send you a username. Note it down — you'll need it to [connect and upload](raw_data_upload.md). + +--- -!!! note - Please note down the username as this will be the username you will use to connect to our upload server. +## What's next -### Proceed by [uploading data](raw_data_upload.md) +➡ [Upload your data](raw_data_upload.md) diff --git a/docs/rawdata/raw_data_upload.md b/docs/rawdata/raw_data_upload.md index ba89801..48543c1 100644 --- a/docs/rawdata/raw_data_upload.md +++ b/docs/rawdata/raw_data_upload.md @@ -1,277 +1,170 @@ -# Upload Data +# Upload raw data -After creating measurements in the Data Manager, you can upload measured data to our platform. -This section gives an overview on how to upload data to measurements from QBiC's Data Manager. +After registering measurements, upload your instrument output files to the QBiC platform via SFTP. -!!! info - If you are not a member of the University of Tübingen, please [request access as a data submitter](raw_data_request_server_access.md) +!!! info "External collaborators" + Not at the University of Tübingen? [Request server access](raw_data_request_server_access.md) first. ## Prerequisites -The following is required in order to successfully execute the measurement data upload dependent on your affiliation. +=== "University of Tübingen member" -=== "University Tübingen member" - - A connection to the University Of Tübingen network ( - e.g. [using the University VPN](https://uni-tuebingen.de/en/facilities/zentrum-fuer-datenverarbeitung/services/network-services/network-access/remote-access-vpn/)) - - An LDAP account of the University Of Tübingen - - Access to the project of interest - - OpenSSH package or a SFTP **client** software (e.g. [FileZilla](https://filezilla-project.org/download.php?type=client) - or [WinSCP](https://winscp.net)) + - VPN connection to the university network ([setup guide](https://uni-tuebingen.de/en/facilities/zentrum-fuer-datenverarbeitung/services/network-services/network-access/remote-access-vpn/)) + - University LDAP account + - Project access in the Data Manager + - SFTP client ([FileZilla](https://filezilla-project.org/download.php?type=client) or [WinSCP](https://winscp.net)) or the OpenSSH `sftp` command === "External data submitter" - - [Granted access and authorization to our upload server](raw_data_request_server_access.md) - - Access to the project of interest - - OpenSSH package or a SFTP **client** software (e.g. [FileZilla](https://filezilla-project.org/download.php?type=client) - or [WinSCP](https://winscp.net)) -## Process Overview + - [Approved server access](raw_data_request_server_access.md) + - Project access in the Data Manager + - SFTP client or OpenSSH `sftp` command + +## Process overview ```mermaid graph LR - A(Prepare your data - 📝) --> B(Upload your data - ⇧) - B --> D{{Data upload successful?}} - D -- No --> A - D --> F("Well Done - 👍") + A(Prepare data) --> B(Upload via SFTP) + B --> C{Upload OK?} + C -- No --> A + C -- Yes --> D(Done!) ``` -## Prepare your datasets for upload +## Prepare your data -You need to prepare your datasets for us to know to which measurement to attach it. -Uploading data to a measurement is called data registration in the following section. -Folders with a given structure that are moved into the `registration` folder, are automatically -registered in our system. +Each upload needs a folder containing a `metadata.txt` file that maps files to measurement IDs. This file is what links your raw data to the correct measurement record. -### Prepare dataset upload to a singular measurement +!!! warning "No spaces in names" + Folder names and file names must not contain spaces. -For every registration task, the data needs to reside in a distinct folder containing a `metadata.txt` file: +### Single measurement ```text -|- upload-example // folder name is irrelevant - |- metadata.txt // mandatory! - |- file1_1.fastq.gz // all files except for metadata.txt serve as examples - |- file1_2.fastq - |- report.pdf - |- summary.html +upload-example/ +├── metadata.txt +├── file1_R1.fastq.gz +├── file1_R2.fastq.gz +└── report.pdf ``` -!!! warning - Ensure that the uploaded folder name and files do not have a whitespace within their name - -The `metadata.txt` file for such an example would look like this: +The `metadata.txt` maps each file to a measurement ID, separated by a **TAB character** (not spaces): -```bash -MSQTEST001AL-437845761848053 file1_1.fastq.gz -MSQTEST001AL-437845761848053 file1_2.fastq -MSQTEST001AL-437845761848053 report.pdf -MSQTEST001AL-437845761848053 summary.html +``` +MSQTEST001AL-437845761848053 file1_R1.fastq.gz +MSQTEST001AL-437845761848053 file1_R2.fastq.gz +MSQTEST001AL-437845761848053 report.pdf ``` -The example metadata file is provided [here](templates/metadata_same_measurement_example/metadata.txt), -please keep in mind you need to adjust the measurementId and filenames to your specific upload. - -!!! note - Ensure that measurement identifier and filename are separated by a TAB `\t` character and not - by spaces. - - -### Prepare dataset upload to multiple measurements +!!! warning "TAB characters required" + The measurement ID and filename must be separated by a TAB (`\t`), not spaces. They look identical in most editors — use a plain-text editor or export to TSV from a spreadsheet. -You can upload multiple folders to distinct measurements the same way. -Everything at the top level of your created folder is considered. -For uploading folders, specify the name of the folders instead of a file name. -Uploading only specific files from a subdirectory is not supported at the moment. +An example file is provided [here](templates/metadata_same_measurement_example/metadata.txt). Adjust the measurement ID and filenames to match your upload. -To register folders the data needs to reside within an outer folder with the following structure: +### Multiple measurements ```text -|- upload-example // folder name is irrelevant - |- metadata.txt // mandatory! - |- my-registration-batch/ - |-- file1_1.fastq.gz - |-- file1_2.fastq.gz - |- my-registration-batch2/ - |-- file1_1.fastq.gz - |-- file1_2.fastq.gz +upload-example/ +├── metadata.txt +├── batch-1/ +│ ├── file1_R1.fastq.gz +│ └── file1_R2.fastq.gz +└── batch-2/ + ├── file2_R1.fastq.gz + └── file2_R2.fastq.gz ``` -!!! warning - Ensure that the uploaded folder name and files do not have a whitespace within their name - -The folder `upload-example` represents an atomic registration unit and must contain the `metadata.txt` -with information about the measurements identifiers and the folder names to be registered -Basically this means one upload can trigger multiple registrations to distinct measurements. -The `metadata.txt` file for such an example would look like this: - -```bash -MSQTEST001AL-437845761848053 my-registration-batch -MSQTEST002AT-437845764676875 my-registration-batch2 +``` +MSQTEST001AL-437845761848053 batch-1 +MSQTEST002AT-437845764676875 batch-2 ``` -The example metadata file is provided [here](templates/metadata_multiple_measurements_example/metadata.txt), -please keep in mind you need to adjust the measurement Ids and dataset names to your specific upload. - -!!! note - Ensure that measurement identifier and filename are separated by a TAB `\t` character and not - by spaces. - -## Upload your dataset via SFTP Client +An example is provided [here](templates/metadata_multiple_measurements_example/metadata.txt). -Uploading your files to us was never this easy! SFTP is a broadly used file transfer protocol. The -wide-spread use ensures that there exists many client software products that -support uploading files to us. -In this section we will go through the process of connecting to our server -using the [FileZilla](https://filezilla-project.org/download.php?type=client) client as an example. +## Upload via SFTP client -### Install SFTP client -Filezilla provides an up-to-date [documentation](https://wiki.filezilla-project.org/Client_Installation) -on the installation process for each operating system. -We recommend to get in contact with your local IT department should you need assistance or have -chosen a different SFTP client. +We'll use [FileZilla](https://filezilla-project.org/download.php?type=client) as an example. -### Connect and upload your dataset +### Connect to the server -**Open the Site Manager:** You need to add the QBiC's upload server as a site to _FileZilla_ within its site manager. -To open the site manager select it from the menu or press on the highlighted icon. -![An image showing the button leading to the site manager](./images/upload/raw_data_upload_open_site_manager.png){.screenshot} +1. Open the **Site Manager** in FileZilla. -**Add the upload server:** In the _Site Manager_ you can add sites to which you want to connect. For -measurement data registration, select `SFTP - SSH File Transfer Protocol` and -enter `upload.qbic.uni-tuebingen.de` into the `Host` field. -![An image showing the users site manager highlighting the host and connection field](images/upload/raw_data_upload_site_manager_host-fields.png){.screenshot} + ![Site Manager](images/upload/raw_data_upload_open_site_manager.png){.screenshot} -Afterwards, choose the affiliation dependent login method to connect to our upload server. +2. Add a new site with protocol **SFTP** and host `upload.qbic.uni-tuebingen.de`. -=== "University Tübingen member" - Login with your _University of Tübingen_ credentials. Enter your university provided username - into the `User` field. - ![An image showing the users site manager highlighting the user and password field](images/upload/raw_data_upload_site_manager_credential-fields.png){.screenshot} + ![Host fields](images/upload/raw_data_upload_site_manager_host-fields.png){.screenshot} -=== "External data submitter" - Login with your username and key file. Enter your username - into the `User` field and select your private key file from within your local `.ssh` directory by pressing the browse button. - ![An image showing the users site manager highlighting the user field](images/upload/raw_data_upload_site_manager_submitter_credential_fields.png){.screenshot} +3. Enter your credentials: -**Connect to the server:** You can connect to the server by pressing `Connect` in -the _Site Manager_. After connecting to the server, _FileZilla_ shows you the contents of your home directory on the server side. -![An image showing the users home folder. You can see three directories named registration, error and upload.](images/upload/raw_data_upload_remote_filesystem.png){.screenshot} -!!! warning - When you first log in, the server will create some folders. Do not delete these folders! + === "University member" -Once you have prepared your folder, upload it to your user directory on our server. Please do not -upload directly to the registration folder but stage it instead in your user directory. -Once your folder is prepared and uploaded to `upload.qbic.uni-tuebingen.de`, move it to -the `registration` folder. + Enter your university username. -!!! tip - You can easily drag and drop the folder via your mouse - from your local filesystem to our server within filezilla + ![Credentials](images/upload/raw_data_upload_site_manager_credential-fields.png){.screenshot} -Our system will then transfer the folder and proceed with data registration. + === "External submitter" -!!! success - Congratulations you have uploaded your data! + Enter your username and select your private key file from your `.ssh` directory. -Finally, you can view summarized information for your uploaded data within the raw data view of the Data Manager. + ![Key file](images/upload/raw_data_upload_site_manager_submitter_credential_fields.png){.screenshot} -Should your files not appear check the error directory as outlined in the [failed upload section](#handle-failed-uploads) +4. Click **Connect**. -## Upload your dataset via command line + ![Connected](images/upload/raw_data_upload_remote_filesystem.png){.screenshot} -The OpenSSH SFTP program is supported natively by most operating systems. -In this section we will go through the process of connecting to our server -using the [OpenSSH SFTP](https://man.openbsd.org/sftp) program. + !!! warning "Don't delete the default folders" + The server creates `registration`, `error`, and `upload` folders on first login. Do not delete them. -!!! note - This section requires commandline knowledge within your operating system. - We recommend to get in contact with your local IT department should you need assistance - or upload your data via a [dedicated sftp client](#upload-your-dataset-via-sftp-client) +### Upload and register -### Install OpenSSH SFTP +1. Upload your prepared folder to your home directory on the server (not directly to `registration`). +2. Once the upload is complete, move the folder into the `registration` directory. -#### Linux/Mac -Linux and Mac systems do typically have the OpenSSH package containing SFTP pre-installed. -Should this not be the case, try to install the openSSH package with the package manager employed by your system. + !!! tip "Drag and drop" + You can drag and drop between local and remote file panels in FileZilla. -#### Windows -Newer Windows Versions (Windows10 version 1803 and newer and Windows 11) do typically have the OpenSSH package containing SFTP preinstalled. -Should this not be the case, check your system settings to see if the **OpenSSH server feature** is installed within the windows optional features. +3. The system processes your data automatically. -### Connect and upload your dataset +!!! success "Upload complete" + View a summary of your uploaded data in the raw data view of the Data Manager. -!!! tip +If files don't appear, check the `error` directory — see [Handle failed uploads](#handle-failed-uploads). - Check the OpenSSH sftp [manpage](https://man.openbsd.org/sftp) for information on how to use the sftp program. +## Upload via command line -Start by opening the command line within your operating system of choice. -Next navigate to the local working directory containing the dataset you wish to upload. -From within this directory connect to our upload server with the **sftp** command, replacing with your university account credentials -or tbe username granted to you after [requesting access](raw_data_request_server_access.md) to our upload server from outside the university network: +=== "External submitter" -=== "External Data Submitter" - ``` bash - sftp -i @upload.qbic.uni-tuebingen.de + ```bash + sftp -i @upload.qbic.uni-tuebingen.de ``` -=== "University Tübingen Member" - ``` bash - sftp @upload.qbic.uni-tuebingen.de - ``` - -If everything goes well, you'll be connected to your remote home directory within our upload server. -You can check out the directory structure of this directory with the following **ls** command: - -``` bash -ls -ll -``` - -From there navigate to the **Upload** folder on the remote working directory via the **cd** command: - -``` bash -cd upload -``` +=== "University member" -and upload your prepared dataset from your local file system with the **put** command, replacing the `` -with the directory name of your dataset: - -``` bash -put -r -``` - -!!! Info - Keep in mind that the **put** commands uploads the files and folders within the working directory of your local filesystem. - You can check the content of your local working directory cia the **lls** command - ``` bash - lls + ```bash + sftp @upload.qbic.uni-tuebingen.de ``` -After your dataset has been successfully uploaded, you can move it to the **registration** folder via the **rename** command, -replacing the `` with the folder name of your dataset: +After connecting: -``` bash -rename ../registration/ +```bash +ls -l # list remote directory +put -r # upload your folder +rename ../registration/ # trigger registration ``` -Our system will then transfer the folder and proceed with data registration. +!!! tip "Useful SFTP commands" + `lls` lists your local directory. `lpwd` shows your local working directory. `pwd` shows the remote directory. -!!! success - Congratulations you have uploaded your data! +## Handle failed uploads -Finally, you can view summarized information for your uploaded dataset within the raw data view of the Data Manager. +If an upload fails, a folder appears in `/home//error` containing an `error.txt` file and your data. -Should your files not appear check the error directory as outlined in the [failed upload section](#handle-failed-uploads) +![Error folder](images/upload/raw_data_upload_error_folder.png){.screenshot} -## Handle failed uploads +Fix the error described in `error.txt`, then move the folder back to `registration` to retry. -Uploading data to a measurement can fail in certain cases. When an upload fails, a folder is created -in `/home//error`. +--- -!!! info - A failed upload will be contained within a directory named after a universally unique identifier - to ensure data integrity during the registration process within our system. +## What's next -![An image showing the error folder. It contains a directory with an automatically assigned process ID.](./images/upload/raw_data_upload_error_folder.png){.screenshot} -In this new folder, you can find an `error.txt` file describing the error, and the data you tried to upload. -You can then try to fix the error. To retry the upload of the fixed folder, move it to the registration folder again. +➡ [Download raw data](raw_data_download.md) · [Back to experiment](../experiment/experiment_introduction.md) diff --git a/docs/user/password_reset.md b/docs/user/password_reset.md index 32b9b70..eacf973 100644 --- a/docs/user/password_reset.md +++ b/docs/user/password_reset.md @@ -1,44 +1,39 @@ -# Password Reset +# Reset your password -The Data Manager application provides an easy and secure solution -to set a new password should a password associated with a user account be lost. -To reset your password the following steps have to be taken +Forgot your password? You can set a new one in three steps. -1. [Provide the account credentials for which the password should be reset](#trigger-password-reset) -2. [Follow the password reset link you received in your email](#validate-the-password-reset-trigger) -3. [Set a new password for the account according to the password policy](#set-a-new-password-for-the-account) +!!! note "ORCID users" + If you log in via ORCID, you don't have a Data Manager password to reset. Use the ORCID login button instead. -## Trigger Password Reset +## Step 1 — Request a reset link -From the login page you can navigate to a dedicated password reset view via the "Forgot password" -link on the bottom of the login -![login_password_reset](images/password_reset/password_reset.png){.screenshot} +1. On the [login page](https://rdm.qbic.uni-tuebingen.de/login), click **Forgot password**. -You should now be able to see the password reset view: + ![Login page](images/password_reset/password_reset.png){.screenshot} -![password_reset_email](images/password_reset/password_reset_email.png) +2. Enter the email address linked to your account and click **Send**. -Please provide the email address of the account for which the password should be reset -and press the "Send" button. You will receive an email with further instructions. -![password_reset_email_sent](images/password_reset/password_reset_email_sent.png) + ![Email entry](images/password_reset/password_reset_email.png){.screenshot} -## Validate the Password Reset Trigger + ![Confirmation](images/password_reset/password_reset_email_sent.png){.screenshot} -Before the password of the provided account can be reset, -you need to validate that the account indeed belongs to you. -For this you will receive an email shortly after triggering the password reset, containing a unique -password reset link -!!! note "Spam folder" - Please check your spam folder if you didn't receive an email +!!! note "Check your spam folder" + If the email doesn't arrive within a few minutes, check spam or junk. -Click the URL to follow the link, which will lead you to a page which allows you to set a new -password for your account. -![password_reset_new_password](images/password_reset/password_reset_new_password.png) +## Step 2 — Open the reset link -## Set a new password for the account +Click the link in the email. It takes you to a page where you can set a new password. -Please provide a new unique password with at least 12 characters. -Once a valid password is provided and the reset is triggered you will be redirected to a -confirmation screen upon successful reset. -![password_reset_new_password_saved](images/password_reset/password_reset_new_password_saved.png) -From this screen you can switch to the login via the provided instructions. +![New password form](images/password_reset/password_reset_new_password.png){.screenshot} + +## Step 3 — Set a new password + +Enter a new password (at least 12 characters) and confirm. Click the login link on the confirmation screen to sign in. + +![Confirmation](images/password_reset/password_reset_new_password_saved.png){.screenshot} + +--- + +## What's next + +➡ [Log in](https://rdm.qbic.uni-tuebingen.de/login) · [Edit your profile](user_edit.md) diff --git a/docs/user/user_edit.md b/docs/user/user_edit.md index d67957f..639f5c8 100644 --- a/docs/user/user_edit.md +++ b/docs/user/user_edit.md @@ -1,49 +1,51 @@ -# Edit User Information +# Edit your profile -To edit your user information start by logging into the [data-manager](https://rdm.qbic.uni-tuebingen.de/login). -After login, click on your profile icon on the top right to open up the dropdown menu and select the user profile option. -![edit_user_information.png](images/edit_user_information/edit_user_information.png){.screenshot} +Update your username or link your ORCID account. -This will navigate you to your personal profile page showing the information associated with your account: +## Open your profile -1. Username -2. Profile Image -3. Full Name -4. Email Address +1. Log in to the [Data Manager](https://rdm.qbic.uni-tuebingen.de/login). +2. Click your profile icon (top right) and select **User profile**. -![edit_user_information_profile.png](images/edit_user_information/edit_user_information_profile.png){.screenshot} + ![Profile menu](images/edit_user_information/edit_user_information.png){.screenshot} -Within your profile you're able to change your [username](#change-user-name). +Your profile page shows your username, full name, email, and profile image. -## Change User Name +![Profile page](images/edit_user_information/edit_user_information_profile.png){.screenshot} -Within this page, you can change your username by clicking on the blue highlighted "Change Username" text opening up the change username dialog. -![edit_user_information_filled_dialog.png](images/edit_user_information/edit_user_information_filled_dialog.png){.screenshot} +## Change your username -!!! info "Username" - Keep in mind that your username is unique and visible to other users within the system. +1. Click the blue **Change Username** link. +2. Enter your new username. +3. Click **Save**. -Once you're satisfied with your new username, click the "Save" button. -The system will check if the username is unique and not already associated -with another account and if that is the case update your username. -![edit_user_information_changed.png](images/edit_user_information/edit_user_information_changed.png){.screenshot} +The system checks availability. If the name isn't taken, your username updates immediately. -## Link Orcid +![Updated username](images/edit_user_information/edit_user_information_changed.png){.screenshot} -Within this page, you can link your [orcid account](https://www.orcid.org) by clicking on the blue highlighted "link ORCiD account". -![link_orcid_account.png](images/edit_user_information/edit_user_information_link_orcid_account.png){.screenshot} +!!! info "Username visibility" + Your username is visible to other users on the platform. -Clicking on "link ORCiD account" redirects you to the ORCID login page. +## Link ORCID -![orcid_login](images/edit_user_information/link_orcid_login.png){.screenshot} +If you registered with email and want to add ORCID login later: -Login to your ORCID account and grant the Data Manager application to access your publicly available ORCID by pressing the "authorize access" button. +1. On your profile page, click **Link ORCiD account**. -![orcid_registration_authorize_access.png](images/edit_user_information/edit_user_information_orcid_linkage_authorize_access.png){.screenshot} + ![Link ORCID](images/edit_user_information/edit_user_information_link_orcid_account.png){.screenshot} -Should you not have an ORCID account press the "register_now" link -within the ORCID login and follow the provided steps to register a new ORCID account. +2. Sign in to ORCID and click **Authorize access**. -After linking your account, you can see the linked account within your profile page. + ![ORCID login](images/edit_user_information/link_orcid_login.png){.screenshot} -![orcid_registration_authorize_access.png](images/edit_user_information/edit_user_information_orcid_account_linked.png){.screenshot} + ![Authorise](images/edit_user_information/edit_user_information_orcid_linkage_authorize_access.png){.screenshot} + +3. Your linked ORCID appears on your profile page. + + ![ORCID linked](images/edit_user_information/edit_user_information_orcid_account_linked.png){.screenshot} + +--- + +## What's next + +➡ [Register a project](../project/project_registration.md) · [Reset your password](password_reset.md) diff --git a/docs/user/user_registration.md b/docs/user/user_registration.md index e780548..1e53836 100644 --- a/docs/user/user_registration.md +++ b/docs/user/user_registration.md @@ -1,93 +1,77 @@ -# User Registration +# Registration -Before you can begin to manage your projects and access your data within the data-manager, -you need to create an account for the platform. We currently support 2 modes of registration: +You need a free account before you can manage projects or access data. Registration takes about two minutes. -1. [Register an account via ORCID](#register-an-account-via-ORCID) -2. [Register an account via email](#register-an-account-via-email) +Two options: -## Register an account via ORCID +- [Register with ORCID](#register-with-orcid) *(recommended)* +- [Register with email](#register-with-email) -### Login to ORCID +## Register with ORCID -We recommend to register to the Data Manager via an [ORCID](https://orcid.org) account. -This will allow you to log in to the Data Manager application as long as as you keep being logged into your ORCID account, -removing the necessity of a separate login step. - -To do so press on the "login with ORCID" registration card on the bottom of the login page. +ORCID is a free, permanent researcher identifier used by journals, funders, and institutions worldwide. Linking it lets you log in with a single click — no separate password needed. -![login_page_registration](images/user_registration/login_page.png){.screenshot} +!!! tip "No ORCID yet?" + [Create a free account at orcid.org](https://orcid.org/register) — it takes about two minutes. -This will redirect you to the ORCID login page +1. On the [login page](https://rdm.qbic.uni-tuebingen.de/login), click **Login with ORCID**. -![orcid_login](images/user_registration/orcid/orcid_login.png){.screenshot} + ![Login page](images/user_registration/login_page.png){.screenshot} -Login to your ORCID account and grant the Data Manager application to access your publicly available ORCID by pressing the "authorize access" button. +2. Sign in to your ORCID account. -![orcid_registration_authorize_access.png](images/user_registration/orcid/orcid_registration_authorize_access.png){.screenshot} + ![ORCID login](images/user_registration/orcid/orcid_login.png){.screenshot} -Should you not have an ORCID account press the "register_now" link -within the ORCID login and follow the provided steps to register a new ORCID account. +3. Click **Authorize access** to let the Data Manager read your public ORCID profile. -### Finalize Data Manager account + ![ORCID authorisation](images/user_registration/orcid/orcid_registration_authorize_access.png){.screenshot} -If done correctly you will be redirected to the Data Manager application to finalize your account registration by providing us with additional user information. -![orcid_registration_data_manager_registration.png](images/user_registration/orcid/orcid_registration_data_manager_registration.png) +4. Complete your Data Manager profile: -This additional information consists of + - **Full name** — first name and surname + - **Email address** — used for notifications + - **Username** — unique, visible to other users on the platform -1. Your full name consisting of your first name and surname -2. A valid email address of choice which will be associated with your Data Manager account -3. A unique username which identifies you within the Data Manager platform + ![Account completion](images/user_registration/orcid/orcid_registration_data_manager_registration.png){.screenshot} -!!! info "Username" - This username is visible to other users within the platform +5. Click **Register**, then click **Login with ORCID** on the login page to sign in. -Press the register button to register an account within the Data Manager application and navigate to the login page. -Finally, press on the "login with ORCID" registration card on the bottom of the login page to login into the Data Manager application via your ORCID credentials. -![registration page](images/user_registration/login_page.png) +## Register with email -## Register an account via email +1. On the [login page](https://rdm.qbic.uni-tuebingen.de/login), click **Register** (top right). -If you don't want to link your ORCID with the Data Manager application, -you can also register an account with your email address by pressing the register link on the login page or via the "registration" button on the top right. + ![Login page](images/user_registration/login_page.png){.screenshot} -![login_page_registration](images/user_registration/login_page.png) +2. Fill in the form: -You should now be able to see the registration view: + - **Full name** — first name and surname + - **Email address** — used for login and notifications + - **Username** — unique, visible to other users + - **Password** — at least 12 characters -![registration page](images/user_registration/registration_page.png) + ![Registration form](images/user_registration/registration_page.png){.screenshot} -Please provide the required information which consists of: +3. Click **Register**. -1. Your full name consisting of your first name and surname -2. A valid email address of choice which will be associated with your account -3. A unique username which identifies you within the Data Manager platform +### Verify your email -!!! info "Username" - This username is visible to other users within the platform +Before you can log in, confirm your email address by clicking the link in the verification email. -4. A secure password consisting of at least 12 characters +![Verification notice](images/user_registration/email_verification.png){.screenshot} -Once all the information has been provided correctly, -press the "register" button on the bottom to create your account. +!!! note "Check your spam folder" + If the email doesn't arrive within a few minutes, check spam or junk. -### Validate the registered account +After clicking the link, you're redirected to the login page. -Before you can log into the Data Manager platform with your newly created account, -you need to verify the provided email address. -![email_verification](images/user_registration/email_verification.png) +### Log in -After creating the account, you will receive an email containing a validation link. -!!! note "Spam folder" - Please check your spam folder if you didn't receive an email +Enter your email and password on the [login page](https://rdm.qbic.uni-tuebingen.de/login). -Click the link within the email, which -will automatically validate the email address and lead you to the registration page -![login page](images/user_registration/login_page_validated.png) +![Filled login page](images/user_registration/login_page_filled.png){.screenshot} -### Login to the Data Manager +--- -After your account has been verified, you are now able to log in to the Data Manager. -This can be done on the dedicated login page via the provided email address and password of the newly created account. -![login page](images/user_registration/login_page_filled.png) +## What's next + +➡ [Register your first project](../project/project_registration.md) diff --git a/mkdocs.yml b/mkdocs.yml index e845204..62c13ab 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -11,36 +11,35 @@ site_name: Life Science Data Management repo_url: https://github.com/qbicsoftware/research-data-management repo_name: qbicsoftware/research-data-management nav: - - What's new: index.md + - Home: index.md - Get started: - Process Overview: get_started/process_overview.md - - Batch: batch/sample-batch.md - - Developers & Stewards: - - API: developers/api.md - - Metadata Concepts: metadata/concepts.md + - Process overview: get_started/process_overview.md + - User account: + - Registration: user/user_registration.md + - Edit profile: user/user_edit.md + - Password reset: user/password_reset.md + - Project: + - Introduction: project/project_introduction.md + - Registration: project/project_registration.md + - Editing: project/project_edit.md + - Access management: project/project_access.md - Experiment: - Introduction: experiment/experiment_introduction.md - Creation: experiment/experiment_creation.md - - Confounding Variables: experiment/confounding-variables.md + - Confounding variables: experiment/confounding-variables.md + - Sample batch: batch/sample-batch.md - Measurement: - Introduction: measurement/measurement_introduction.md - - Editing: measurement/measurement_edit.md - Registration: measurement/measurement_registration.md - - Ontology: - - Introduction: ontology_search/ontology_search_introduction.md - - Project: - - Introduction: project/project_introduction.md - - Access Management: project/project_access.md - - Editing: project/project_edit.md - - Registration: project/project_registration.md + - Editing: measurement/measurement_edit.md - Raw data: - - Download Data: rawdata/raw_data_download.md - - Upload Data: rawdata/raw_data_upload.md - - Request Access to Server: rawdata/raw_data_request_server_access.md - - User: - - Editing: user/user_edit.md - - Password Reset: user/password_reset.md - - Registration: user/user_registration.md + - Request server access: rawdata/raw_data_request_server_access.md + - Upload data: rawdata/raw_data_upload.md + - Download data: rawdata/raw_data_download.md + - Ontology search: ontology_search/ontology_search_introduction.md + - Reference: + - API: developers/api.md + - Metadata glossary: metadata/concepts.md extra_css: - stylesheets/extra.css extra: From 930710c6c24b535a7f7547fcd6e4236e5e96df28 Mon Sep 17 00:00:00 2001 From: Steffengreiner Date: Thu, 26 Feb 2026 16:46:22 +0100 Subject: [PATCH 2/5] chore: remove proposals folder and add agent configuration files - Remove docs/proposals/ now that all pages have been merged into the live documentation - Add AGENTS.md documenting the multi-agent debate system - Add opencode.json with agent definitions and permissions --- AGENTS.md | 165 ++++++++++++++++++++++++++++++++++++++++++++++++++ opencode.json | 86 ++++++++++++++++++++++++++ 2 files changed, 251 insertions(+) create mode 100644 AGENTS.md create mode 100644 opencode.json diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..966e39a --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,165 @@ +# AGENTS.md + +> This file describes the AI agent system used to generate documentation proposals in this repository. All agents are configured in `opencode.json`. Generated proposals live in `docs/proposals/` and require human review before adoption. + +--- + +## Overview + +Documentation in this repository is generated through a structured multi-agent debate. Rather than having a single AI write documentation in one pass, four specialist agents each contribute their perspective, challenge one another's drafts, and a fifth editorial agent unifies the result into a single coherent voice. + +The goal is documentation that is simultaneously **accurate**, **reproducible**, **accessible**, and **readable** — without any single perspective dominating at the expense of the others. + +--- + +## How It Works + +Every documentation request runs through four rounds: + +**Round 1 — Independent Drafts** +The four subject-matter agents each write a draft independently, without seeing each other's work. This ensures each perspective is genuinely uninfluenced before the debate begins. + +**Round 2 — Cross-Review** +Each agent reviews the other three drafts and returns specific, cited challenges: factual corrections, compliance gaps, accessibility problems, and concrete rewrite proposals. Vague criticism is not accepted — every challenge must include a suggested fix. + +**Round 3 — Consensus Synthesis** +The `docs` orchestrator reconciles all four drafts and the Round 2 challenges into a single consensus document. Where agents disagree, the orchestrator applies a clear precedence order (see below) and records its reasoning in a `## Debate Summary` section at the top of every proposal. + +**Round 4 — Editorial Pass** +The `technical-editor` agent receives the reconciled draft and performs a final pass focused entirely on voice, flow, and readability — without touching any factual content. This is the step that ensures the output reads as one coherent document rather than a committee report. + +The final file is written to `docs/proposals/.md` and marked as a proposal pending human review. + +--- + +## Agent Roles + +### `docs` — Orchestrator +**Model**: claude-opus-4-6 +**Role**: Routes requests, manages the four debate rounds, synthesises consensus, and writes the final proposal to `docs/proposals/`. + +The orchestrator never writes documentation itself during the debate. Its job is coordination, conflict resolution, and final synthesis. It is the only agent with write access, and that access is restricted exclusively to `docs/proposals/`. + +**Conflict precedence order** (applied during Round 3 synthesis): +1. Compliance and FAIR accuracy → `data-steward` challenges take precedence +2. Computational and provenance accuracy → `bioinformatician` challenges take precedence +3. Plain-language accessibility → `lab-personnel` challenges take precedence +4. Structure and platform navigability → `ls-user-guide` challenges take precedence + +--- + +### `data-steward` — Compliance & Metadata Accuracy +**Model**: claude-sonnet-4-6 +**Voice in the debate**: *"Is this FAIR-compliant and correctly annotated?"* + +The data steward is the compliance firewall. It ensures that data and metadata descriptions conform to domain-specific schemas, controlled vocabularies, and Open Science regulations. It holds the line on accuracy — no compliance gap passes through without a citation and a correction. + +**Domain coverage**: +- Genomics: FASTQ, BAM/CRAM, VCF; EGA, SRA, ENA submission standards +- Proteomics: mzML/mzXML, mzIdentML; PRIDE/ProteomeXchange requirements +- Immunopeptidomics: HLA-peptidomics, PSMs, MHC ligandome metadata, IEDB schemas +- Imaging: OME-TIFF, CZI, DICOM, Visium; BioImage Archive and REMBI standards + +**Key standards enforced**: FAIR principles, OBO Foundry, EFO, NCIT, PSI-MS, MIAME, MINSEQE, REMBI, Plan S, Horizon Europe DMP, NIH DMSP + +--- + +### `lab-personnel` — Bench Scientist / PI Perspective +**Model**: claude-sonnet-4-6 +**Voice in the debate**: *"Can a bench scientist actually follow this?"* + +This agent represents the primary documentation consumer: a PI or wet-lab scientist with deep biological expertise but little data management background. It advocates for plain language, short steps, and everyday analogies. If a section would cause a busy scientist to disengage, this agent flags it and proposes a simpler alternative. + +**What it protects against**: unexplained acronyms, jargon-heavy compliance language, walls of text, steps that feel disconnected from daily lab workflow, and documentation that respects the data manager more than the scientist reading it. + +--- + +### `bioinformatician` — Computational Provenance & Pipeline Accuracy +**Model**: claude-sonnet-4-6 +**Voice in the debate**: *"Is the computational provenance chain correctly and reproducibly described?"* + +This agent fills the gap between compliance metadata and computational reality. It ensures that documentation correctly describes how derived data traces back to primary data — including software versions, parameter sets, checksums, workflow execution records, and environment specifications. If a described workflow would be non-reproducible as written, this agent catches it. + +**Domain coverage**: +- Genomics pipelines: FastQC, STAR, BWA-MEM2, GATK, Salmon, Nextflow, Snakemake +- Proteomics pipelines: MaxQuant, MSFragger, Spectronaut, DIA-NN, Perseus, MSstats +- Immunopeptidomics: OptiType, HLA-HD, PSM-level filtering criteria +- Imaging analysis: ilastik, CellProfiler, QuPath, Cellpose + +**Key standards**: FAIR4RS, Workflow RO-Crate, CWL/WDL metadata + +--- + +### `ls-user-guide` — Platform Structure & Navigation +**Model**: claude-haiku-4-5-20251001 +**Voice in the debate**: *"Is this well-structured and does it hold together as a platform guide?"* + +This agent sits at the intersection of the other three. It ensures documentation is grounded in actual platform UI steps and navigation paths, clearly distinguishes required from optional fields, and proposes bridge language when the other three agents are pulling in different directions. It is also the agent most likely to catch structural seams — places where the document stops flowing naturally. + +**What it protects against**: vague platform references, inconsistent field labelling, missing callouts for common mistakes, and logical gaps that would send a user looking elsewhere for help. + +--- + +### `technical-editor` — Voice & Readability +**Model**: claude-sonnet-4-6 +**Role**: Round 4 post-synthesis editorial pass — not a debate participant. + +The technical editor never participates in the debate. It receives only the final reconciled consensus draft and performs one job: making it read like it was written by one thoughtful human, not assembled by a committee. + +**Target register**: Conversational but professional — precise and confident without being stiff, friendly without being patronising. Think Notion's help docs or a well-edited Nature Methods protocol. + +**What it fixes**: +- Tonal whiplash between sections written in different registers +- Passive voice, hedging language, and throat-clearing openers +- AI-characteristic patterns: overly balanced constructions, unnaturally complete enumerations, hollow transitions +- Inconsistent formatting of UI elements, field labels, and callout styles +- Sentence rhythm — deliberately varying length so the text reads as human + +**What it never changes**: factual content, compliance requirements, tool names, standard citations, the Debate Summary, or the proposal warning header. + +--- + +## Output Format + +Every generated file in `docs/proposals/` follows this structure: + +``` +> ⚠️ PROPOSAL — Pending review and approval before adoption. + +## Debate Summary +[Key conflicts between agents and orchestrator tie-break reasoning] + +--- +[Document content] +``` + +The Debate Summary is intentionally preserved in the final output. It gives human reviewers full visibility into where the agents disagreed and why a particular position was chosen — making the review process faster and more informed. + +--- + +## Permissions & Write Access + +| Agent | Write access | Editable paths | +|---|---|---| +| `docs` (orchestrator) | ✅ Yes | `docs/proposals/` only | +| `data-steward` | ❌ No | — | +| `lab-personnel` | ❌ No | — | +| `bioinformatician` | ❌ No | — | +| `ls-user-guide` | ❌ No | — | +| `technical-editor` | ❌ No | — | + +No subagent can write, edit, or overwrite any file in the repository. Only the orchestrator can write, and only to `docs/proposals/`. Existing files are never overwritten — a version suffix (e.g. `-v2`) is appended if a filename already exists. + +--- + +## Extending the System + +**Adding a new agent perspective**: Define the new agent in `opencode.json` with `"write": false, "edit": false`. Update the orchestrator's `permission.task` block to include it, add it to Rounds 1 and 2 in the orchestrator prompt, and document it here. + +**Adding a new omic modality**: Update the `data-steward` and `bioinformatician` prompts in `opencode.json` with the relevant file formats, metadata standards, and pipeline tools. No structural changes to the debate protocol are needed. + +**Changing the editorial register**: Update the `technical-editor` prompt's target register description and reference points. The rest of the pipeline is unaffected. + +--- + +*This file is maintained manually. When the agent configuration in `opencode.json` changes, update this file to reflect the change.* diff --git a/opencode.json b/opencode.json new file mode 100644 index 0000000..b0c9040 --- /dev/null +++ b/opencode.json @@ -0,0 +1,86 @@ +{ + "$schema": "https://opencode.ai/config.json", + "agent": { + + "docs": { + "description": "Primary documentation orchestrator for a life science data management platform. Runs a structured 4-round debate — independent drafts, cross-review, consensus synthesis, and a final editorial pass — before writing a proposal exclusively to docs/proposals/.", + "mode": "primary", + "color": "#D74E09", + "model": "anthropic/claude-opus-4-6", + "prompt": "You are the documentation orchestrator for a life science research data management platform. Your ONLY job is to delegate tasks using the Task tool and then synthesise the final consensus document — never write documentation yourself during the debate rounds.\n\n## Debate Protocol\n\n---\n**ROUND 1 — Independent Drafts**\nDelegate the full documentation request independently (without sharing each other's work) to all four subject-matter subagents:\n- 'data-steward' → compliance-focused, FAIR-accurate draft\n- 'lab-personnel' → plain-language, scientist-friendly draft\n- 'bioinformatician' → computational provenance and pipeline-aware draft\n- 'ls-user-guide' → structured, platform-navigable draft\n\nCollect all four drafts before proceeding.\n\n---\n**ROUND 2 — Cross-Review / Challenge**\nPass all four Round 1 drafts to each of the four subagents with the instruction to review the other three and return:\n1. Factual errors, compliance gaps, or incorrect technical descriptions — cite the specific standard, tool, or reason\n2. Passages too complex or inaccessible for the intended audience\n3. Concrete, specific rewrite proposals for every flagged passage — not just criticism\n\nCollect all four challenge responses before proceeding.\n\n---\n**ROUND 3 — Consensus Synthesis (you)**\nYou synthesise a single consensus documentation draft that:\n- Is factually accurate and FAIR-compliant (data-steward challenges take precedence on compliance)\n- Correctly represents the computational provenance chain (bioinformatician challenges take precedence on pipeline accuracy)\n- Is written in plain, accessible language (lab-personnel challenges take precedence on readability)\n- Is logically structured and platform-navigable (ls-user-guide challenges take precedence on structure)\n- Opens with a '## Debate Summary' section listing the key conflicts between agents and your tie-break reasoning\n\nThis synthesised draft is NOT the final output — pass it to Round 4 before writing any file.\n\n---\n**ROUND 4 — Editorial Pass**\nDelegate your Round 3 synthesis to 'technical-editor' with the instruction to perform a full editorial pass for voice consistency, readability, and naturalness.\n\nCollect the edited draft, then write it as the final output:\n- Prepend '> ⚠️ PROPOSAL — Pending review and approval before adoption.' at the very top\n- Output path: `docs/proposals/.md`\n- Never write to any path outside `docs/proposals/`\n- Never overwrite an existing file — append a version suffix (e.g. `-v2`) if a file already exists\n\n---\n## General Rules\n- Always delegate. Never write documentation yourself during Rounds 1, 2, or 4.\n- Instruct each agent whether they are in MODE A (draft), MODE B (review), or MODE C (editorial pass) at the start of every delegation.\n- In Round 2, always pass all four drafts to each reviewer so they have full context.\n- Do not ask clarifying questions unless the request is completely ambiguous.", + "permission": { + "task": { + "*": "deny", + "data-steward": "allow", + "lab-personnel": "allow", + "bioinformatician": "allow", + "ls-user-guide": "allow", + "technical-editor": "allow" + } + }, + "tools": { + "write": true, + "edit": false + }, + "write_restrictions": { + "allowed_paths": ["docs/proposals/"] + } + }, + + "data-steward": { + "description": "Life science Data Steward specialising in FAIR data management, Open Science compliance, and omic data modalities (genomics, proteomics, immunopeptidomics, imaging). Compliance and metadata accuracy voice in the debate loop.", + "mode": "subagent", + "model": "anthropic/claude-sonnet-4-6", + "prompt": "You are a highly specialised Data Steward in a life science research data management platform. You participate in a structured documentation debate orchestrated by 'docs'. At the start of each invocation, the orchestrator will tell you whether you are in MODE A or MODE B.\n\n**MODE A — Draft**: Produce a technically accurate, compliance-focused documentation draft on the given topic.\n**MODE B — Review**: You will receive drafts from the other three agents (lab-personnel, bioinformatician, ls-user-guide). Review each and return structured challenge feedback.\n\n---\n## Domain Expertise\n\n### Omic Data Modalities\n- **Genomics**: FASTQ, BAM/CRAM, VCF; EGA, SRA, ENA submission standards; reference genome versioning (GRCh38, GRCm39)\n- **Proteomics**: mzML/mzXML, mzIdentML, quantification matrices; PRIDE/ProteomeXchange requirements; sample preparation metadata (digestion protocol, enrichment strategy)\n- **Immunopeptidomics**: HLA-peptidomics datasets, peptide-spectrum matches (PSMs), MHC ligandome metadata, allele typing records, IEDB-compatible annotation schemas\n- **Imaging**: OME-TIFF, CZI, LIF, DICOM, Visium spatial omics; BioImage Archive (BIA) metadata standards; REMBI checklist\n\n### Research Data Lifecycle\nAlways enforce correct linkage between:\n- **Primary data**: Raw, unprocessed instrument output — never modified after acquisition\n- **Derived data**: Processed outputs (count matrices, peptide lists, segmentation masks) — must reference primary data via unambiguous provenance metadata (accession numbers, MD5/SHA256 checksums, persistent DOIs, software version + parameters used)\n\n### Compliance Standards\nFAIR principles, OBO Foundry, EFO, NCIT, PSI-MS, MIAME, MINSEQE, REMBI, Plan S, Horizon Europe DMP requirements, NIH Data Management and Sharing Policy\n\n## Behavioural Rules\n- Be concise and direct. Use numbered, structured feedback.\n- Always cite the relevant standard, schema, or ontology when raising a compliance issue.\n- Never speculate — state exactly what additional information is required if context is insufficient.\n- When reviewing: prioritise compliance and factual accuracy issues above all else.\n- Do NOT write to any files. Return all output as text to the orchestrator.", + "tools": { + "write": false, + "edit": false + } + }, + + "lab-personnel": { + "description": "Represents a bench scientist or PI with deep biological expertise but little data management background. Plain-language accessibility advocate in the debate loop.", + "mode": "subagent", + "model": "anthropic/claude-sonnet-4-6", + "prompt": "You are a bench scientist and principal investigator (PI) with deep biological expertise but little to no data management background. You participate in a structured documentation debate orchestrated by 'docs'. At the start of each invocation, the orchestrator will tell you whether you are in MODE A or MODE B.\n\n**MODE A — Draft**: Produce a plain-language, scientist-friendly documentation draft on the given topic.\n**MODE B — Review**: You will receive drafts from the other three agents (data-steward, bioinformatician, ls-user-guide). Review each and return challenge feedback from the perspective of a busy bench scientist.\n\n---\n## Your Perspective\n- Expert in biology, not in data systems, pipelines, or compliance frameworks\n- Busy and time-pressured — will disengage from documentation that is too long, too technical, or jargon-heavy\n- Cares about: running experiments smoothly, meeting funder/journal requirements with minimum overhead, being able to find data again later, and sharing results with collaborators without friction\n\n## When Drafting (Mode A)\n- Plain English. Short sentences. No unexplained jargon or acronyms.\n- Small numbered steps — never a wall of text.\n- Everyday analogies where helpful (e.g. 'metadata is like the label on a sample tube — it tells anyone who picks it up exactly what's inside, who prepared it, and when')\n- Briefly explain *why* a step matters in terms the scientist cares about (reproducibility, publishability, finding the data in 2 years)\n- One concept at a time\n\n## When Reviewing (Mode B)\n- Flag every passage that would confuse or lose a bench scientist unfamiliar with data management\n- Identify unexplained acronyms, jargon, compliance language, or computational terminology\n- Propose specific, simpler rewrite alternatives — don't just flag, fix\n- Note steps that feel unnecessary, overwhelming, or disconnected from daily lab workflow\n- Be constructive and specific\n\n## Behavioural Rules\n- Acknowledge compliance and computational requirements briefly and move on — do not lecture\n- Do NOT write to any files. Return all output as text to the orchestrator.", + "tools": { + "write": false, + "edit": false + } + }, + + "bioinformatician": { + "description": "Computational biologist and data engineer specialising in omics analysis pipelines, data provenance, and the linkage between primary and derived data in pre-clinical life science research. Computational accuracy and provenance voice in the debate loop.", + "mode": "subagent", + "model": "anthropic/claude-sonnet-4-6", + "prompt": "You are a computational biologist and data engineer working in a pre-clinical life science research environment. You participate in a structured documentation debate orchestrated by 'docs'. At the start of each invocation, the orchestrator will tell you whether you are in MODE A or MODE B.\n\n**MODE A — Draft**: Produce a computationally accurate, provenance-focused documentation draft on the given topic.\n**MODE B — Review**: You will receive drafts from the other three agents (data-steward, lab-personnel, ls-user-guide). Review each and return challenge feedback focused on computational correctness, pipeline descriptions, and data provenance accuracy.\n\n---\n## Domain Expertise\n\n### Analysis Pipelines & Tools\n- **Genomics**: Read QC (FastQC, MultiQC), alignment (STAR, BWA-MEM2, minimap2), variant calling (GATK, DeepVariant), quantification (Salmon, featureCounts), workflow managers (Nextflow, Snakemake)\n- **Proteomics**: Database search engines (MaxQuant, MSFragger, Spectronaut, DIA-NN), post-processing (Perseus, MSstats), FDR thresholds and their documentation\n- **Immunopeptidomics**: HLA typing tools (OptiType, HLA-HD), peptide prediction and filtering pipelines, PSM-level filtering criteria that must be recorded\n- **Imaging**: Segmentation tools (ilastik, CellProfiler, QuPath, Cellpose), registration workflows, channel metadata requirements\n\n### Data Provenance & Linkage\n- Software name, version, and exact parameter sets used at each processing step must be recorded\n- Intermediate files and their checksums (MD5/SHA256) should be registered where they represent stable analysis checkpoints\n- Workflow execution records (e.g. Nextflow run reports, Snakemake logs) are provenance artefacts and should be archived alongside outputs\n- Container or environment specifications (Docker image digest, conda environment YAML) are required for computational reproducibility\n\n### Reproducibility Standards\nFAIR principles as applied to software and workflows, Workflow RO-Crate, CWL/WDL metadata, FAIR4RS principles\n\n## When Drafting (Mode A)\n- Be precise about what computational metadata must be recorded at each step\n- Describe provenance linkage clearly — what connects a derived output back to its primary input\n- Keep language accessible where possible — not all readers are bioinformaticians, but accuracy is non-negotiable\n- Flag where documentation of a pipeline step is commonly neglected in practice\n\n## When Reviewing (Mode B)\n- Flag computational inaccuracies, missing pipeline steps, or incorrect tool/format descriptions\n- Identify where provenance linkage is described vaguely or incompletely\n- Flag where a described workflow would be non-reproducible as written\n- Propose specific, concrete corrections — cite tool documentation or community standards where relevant\n- Flag passages that oversimplify computational steps to the point of being misleading\n\n## Behavioural Rules\n- Balance precision with accessibility — flag over-simplifications but also flag unnecessary complexity\n- Do NOT write to any files. Return all output as text to the orchestrator.", + "tools": { + "write": false, + "edit": false + } + }, + + "ls-user-guide": { + "description": "Life science platform documentation specialist. Produces structured, platform-navigable guides and bridges compliance, computational, and plain-language requirements within the debate loop.", + "mode": "subagent", + "model": "anthropic/claude-haiku-4-5-20251001", + "prompt": "You are a technical documentation specialist for a life science research data management platform. You participate in a structured documentation debate orchestrated by 'docs'. At the start of each invocation, the orchestrator will tell you whether you are in MODE A or MODE B.\n\n**MODE A — Draft**: Produce a structured, platform-oriented documentation draft on the given topic.\n**MODE B — Review**: You will receive drafts from the other three agents (data-steward, lab-personnel, bioinformatician). Review each and return challenge feedback focused on structure, platform usability, and bridging the three perspectives into coherent documentation.\n\n---\n## Your Focus\nYou sit at the intersection of all three other voices — your job is to produce documentation that is:\n- **Structured**: clear headings, logical reading flow, consistent formatting throughout\n- **Platform-oriented**: grounded in actual platform UI steps, field names, and navigation paths\n- **Balanced**: accurate enough to satisfy compliance and computational requirements, simple enough for a bench scientist to follow without support\n\n## When Drafting (Mode A)\n- Logical step-by-step structure with descriptive headings\n- Reference specific platform actions (e.g. 'Navigate to Register Dataset → select your Assay Type → fill in the Sample Source field')\n- Brief, one-sentence explanations of *why* each field or step matters\n- Accessible language — assume a competent scientist, not a data engineer or compliance officer\n- Clearly distinguish required fields from optional ones (e.g. use ✅ Required / ⬜ Optional labels)\n- Include brief callout boxes for common mistakes or frequently missed fields\n\n## When Reviewing (Mode B)\n- Check that the structure is logical and navigable for a first-time platform user\n- Flag gaps where platform-specific guidance is missing or described too abstractly\n- Identify conflicts between the data-steward's compliance requirements, the bioinformatician's provenance requirements, and the lab-personnel's plain-language needs — propose bridge language that satisfies all three\n- Suggest concrete formatting improvements (tables for field descriptions, numbered steps for workflows, callout boxes for warnings)\n- Flag anything that would require a user to leave the documentation to understand what to do\n\n## Behavioural Rules\n- Do not adjudicate compliance or computational disputes — flag them clearly for the orchestrator to resolve\n- Do NOT write to any files. Return all output as text to the orchestrator.", + "tools": { + "write": false, + "edit": false + } + }, + + "technical-editor": { + "description": "Post-synthesis editorial agent. Takes the orchestrator's consensus draft and performs a final pass for voice consistency, naturalness, and readability — eliminating AI-isms, tonal whiplash, and structural seams without altering factual content.", + "mode": "subagent", + "model": "anthropic/claude-sonnet-4-6", + "prompt": "You are a professional technical editor specialising in life science platform documentation. You are the last agent in the pipeline — you receive a fully fact-checked, compliance-reviewed consensus draft from the orchestrator and your sole job is to make it read like it was written by one thoughtful human, not assembled by a committee.\n\nThe orchestrator will invoke you in MODE C only.\n\n**MODE C — Editorial Pass**: Receive the consensus draft and return a polished final version.\n\n---\n## Your Target Register\n**Conversational but professional — not overbearing.**\nThink of the best documentation you've read: it respects your intelligence, gets to the point, and doesn't make you feel like you're reading a compliance manual or a chatbot output. It sounds like a knowledgeable colleague wrote it on a good day.\n\nConcrete reference points:\n- Notion's help docs: clear, friendly, never condescending\n- Stripe's developer docs: precise, confident, no filler\n- A well-edited Nature Methods protocol: authoritative but human\n\n---\n## What You Fix\n\n### Voice & Tone\n- Unify tonal register across all sections — no jarring shifts from casual to formal and back\n- Remove hedging language that adds no information ('it is important to note that', 'please be aware that', 'it should be mentioned')\n- Replace passive voice with active where it reads more naturally ('the file must be uploaded by the user' → 'upload the file')\n- Cut throat-clearing openers ('This section will explain...', 'In this guide, we will walk you through...')\n- Remove hollow affirmations and filler transitions ('Great!', 'Now that we've covered X, let's move on to Y')\n\n### AI-ism Elimination\n- Rewrite any sentence that sounds like it was generated rather than written — overly balanced constructions, suspiciously even lists, unnaturally complete enumerations\n- Break up sentences that try to do too much at once\n- Vary sentence length deliberately — a mix of short punchy sentences and longer explanatory ones reads as human\n- Replace generic verbs with specific ones ('perform the action' → 'click Register', 'utilise' → 'use')\n\n### Structural Seams\n- Smooth transitions between sections so the document flows as a single piece, not a patchwork\n- Ensure section headings are parallel in grammatical form and consistent in specificity\n- Check that the opening of each section connects logically to what came before\n- Verify the document has a coherent narrative arc — a reader should always know where they are and what comes next\n\n### Formatting Consistency\n- Standardise field labels, button names, and UI references to a consistent format throughout (e.g. always bold for UI elements: **Register Dataset**)\n- Ensure ✅ Required / ⬜ Optional labels are applied consistently wherever fields are listed\n- Check that callout boxes, notes, and warnings follow a consistent style\n\n---\n## What You Do NOT Change\n- Factual content, compliance requirements, or technical specifications — these have been validated upstream\n- The '## Debate Summary' section — leave this untouched\n- The proposal warning at the top — leave this untouched\n- Field names, tool names, standard names, accession formats — do not paraphrase these\n\n---\n## Output\nReturn the complete edited document as clean text. Do not add commentary, explanations, or a summary of your changes — just return the polished document.\nDo NOT write to any files. Return all output as text to the orchestrator.", + "tools": { + "write": false, + "edit": false + } + } + + } +} From 992cdfafe858f0afec470fcef156b40a02c45f92 Mon Sep 17 00:00:00 2001 From: Steffengreiner Date: Tue, 10 Mar 2026 17:20:22 +0100 Subject: [PATCH 3/5] docs: revise What's Next sections, add release notes, consolidate redundancy, and apply nav quick wins - Enforce single forward-facing link in What's Next across all 20 doc pages - Replace concepts.md What's Next with See Also (reference page pattern) - Add inline cross-references: password/profile, collaborator tips, search tips, confounding variable tips, TIB/NCBI info, metadata glossary ref - Replace process lists with mermaid diagrams in measurement_registration and raw_data_download - Consolidate ROR Organisation ID into single note above tabbed sections in measurement_registration - Replace repeated ontology search instructions with tip admonitions linking to canonical page - Update index.md with release notes v1.12.0-v1.12.5 including PI/PM swap action item - mkdocs.yml: wrap Sample batch in Samples section, promote Metadata glossary to top-level, rename Reference to Developer reference, rename Editing to Edit, enable navigation.sections --- docs/batch/sample-batch.md | 2 +- docs/developers/api.md | 4 +- docs/experiment/confounding-variables.md | 2 +- docs/experiment/experiment_creation.md | 10 +++- docs/experiment/experiment_introduction.md | 2 +- docs/index.md | 56 +++++++++++++++---- docs/measurement/measurement_edit.md | 5 +- docs/measurement/measurement_introduction.md | 2 +- docs/measurement/measurement_registration.md | 15 +++-- docs/metadata/concepts.md | 6 +- .../ontology_search_introduction.md | 7 ++- docs/project/project_access.md | 2 +- docs/project/project_edit.md | 2 +- docs/project/project_introduction.md | 4 +- docs/project/project_registration.md | 8 ++- docs/rawdata/raw_data_download.md | 12 ++-- docs/rawdata/raw_data_upload.md | 5 +- docs/user/password_reset.md | 4 +- docs/user/user_edit.md | 4 +- mkdocs.yml | 12 ++-- 20 files changed, 117 insertions(+), 47 deletions(-) diff --git a/docs/batch/sample-batch.md b/docs/batch/sample-batch.md index cd431f2..0087d72 100644 --- a/docs/batch/sample-batch.md +++ b/docs/batch/sample-batch.md @@ -77,4 +77,4 @@ Click **Download sample metadata** to export all registered sample metadata as a ## What's next -➡ [Register measurements](../measurement/measurement_registration.md) · [Back to experiment](../experiment/experiment_introduction.md) +➡ [Register measurements](../measurement/measurement_registration.md) diff --git a/docs/developers/api.md b/docs/developers/api.md index 04d1a46..7ea00d7 100644 --- a/docs/developers/api.md +++ b/docs/developers/api.md @@ -11,10 +11,12 @@ Interactive API documentation is available via Swagger UI: **[Open Swagger UI](https://download.qbic.uni-tuebingen.de/swagger-ui/index.html)** +> Not familiar with the metadata fields? See the [Metadata glossary](../metadata/concepts.md). + Authentication uses the same [personal access token](../rawdata/raw_data_download.md#personal-access-token) as the command-line download. --- ## What's next -➡ [Download raw data](../rawdata/raw_data_download.md) · [Metadata glossary](../metadata/concepts.md) +➡ [Download raw data](../rawdata/raw_data_download.md) diff --git a/docs/experiment/confounding-variables.md b/docs/experiment/confounding-variables.md index 30b79a7..37486e9 100644 --- a/docs/experiment/confounding-variables.md +++ b/docs/experiment/confounding-variables.md @@ -29,4 +29,4 @@ Open the confounding variables editing controls, enter the new name, and save. E ## What's next -➡ [Register sample batches](../batch/sample-batch.md) · [Back to experiment overview](experiment_introduction.md) +➡ [Register sample batches](../batch/sample-batch.md) diff --git a/docs/experiment/experiment_creation.md b/docs/experiment/experiment_creation.md index 2d48fe6..b6a62f5 100644 --- a/docs/experiment/experiment_creation.md +++ b/docs/experiment/experiment_creation.md @@ -10,12 +10,15 @@ In the dialog, provide: - **Experiment name** — unique within the project, easy to identify -- **Species** — the organism(s) your samples come from (type at least 2 characters to search) +- **Species** — the organism(s) your samples come from - **Specimen** — the biological material type (e.g. blood, liver tissue) - **Analyte** — the molecular class being measured (e.g. protein, mRNA) You can select multiple entries per field. Optionally choose an icon for the species and specimen. +!!! tip "Searching for terms" + Type at least 2 characters in any search field to see matching terms from standardised scientific databases. Pick from the list — it makes your data easier to find and share later. See the [ontology search guide](../ontology_search/ontology_search_introduction.md) for tips on finding the right term. + ![Search active](images/create_experiment_search.png){.screenshot} Click **Add**. @@ -67,8 +70,11 @@ Click **Add** to save. !!! warning "Groups are locked once samples are registered" Remove all registered samples before editing groups. +!!! tip "Track confounding factors" + Before registering samples, you can [define confounding variables](confounding-variables.md) — factors like batch effects, operator, or reagent lot that you didn't control but want to account for during analysis. + --- ## What's next -➡ [Register sample batches](../batch/sample-batch.md) · [Define confounding variables](confounding-variables.md) +➡ [Register sample batches](../batch/sample-batch.md) diff --git a/docs/experiment/experiment_introduction.md b/docs/experiment/experiment_introduction.md index c65bdd2..76fceae 100644 --- a/docs/experiment/experiment_introduction.md +++ b/docs/experiment/experiment_introduction.md @@ -25,4 +25,4 @@ This opens the **experiment summary view**. ## What's next -➡ [Create a new experiment](experiment_creation.md) · [Define confounding variables](confounding-variables.md) · [Register sample batches](../batch/sample-batch.md) · [View measurements](../measurement/measurement_introduction.md) +➡ [Create a new experiment](experiment_creation.md) diff --git a/docs/index.md b/docs/index.md index cbbfd90..f53a3b3 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,22 +6,54 @@ Not sure where to begin? Head to the **[Get started guide](get_started/process_o --- + ## What's new -### Data submission from outside the University of Tübingen network +### Duplicate condition detection and improved measurement grid -
October 10th, 2025
+
March 9, 2026
-- **External data submission:** Collaborators outside the university network can now upload measurement data. See how to [request server access](rawdata/raw_data_request_server_access.md) and [upload your data](rawdata/raw_data_upload.md). -- **ORCID linking:** Connect your ORCID account to an existing Data Manager profile at any time from your [user profile](user/user_edit.md#link-orcid). +- **Duplicate condition warning:** The system now alerts you when you try to add a condition that already exists in your experiment — catching silent metadata errors before they reach your samples. +- **Improved measurement view:** The measurement table uses a new flexible grid layout, making it easier to search, sort, and browse large sets of measurements. +- **Bug fix — PI and PM roles were swapped:** A bug in project creation caused the Principal Investigator and Project Manager fields to be written to the wrong roles. This is now corrected. **If you created a project between v1.12.0 and v1.12.4, open your project settings and verify that PI and PM are assigned to the right people.** Audit log entries from the affected period cannot be retroactively corrected — document any discrepancies in your Data Management Plan. + +📌 Full [release notes (v1.12.5)](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.12.5) on GitHub. --- ## Update history +### Selective downloads, experimental variables, and raw data filtering + +
November 11, 2025
+ +📌 Full [release notes (v1.12.0)](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.12.0) on GitHub. + +Highlights: + +- **Selective raw data download:** Choose which measurements to download instead of retrieving everything at once. +- **Experimental variables:** Define the conditions you're testing (drug concentration, temperature, time point) and the groups that structure your sample metadata. +- **Raw data filtering:** Filter registered datasets by properties using the new filter grid. URL export now includes only the datasets matching your current filter. + +### Fixes and improvements (v1.12.1–v1.12.4) + +
November 2025 – February 2026
+ +- **v1.12.1** (Nov 17, 2025): Fixed Data Privacy and Legal Notice display. Raw dataset search now includes measurement ID for more precise filtering. +- **v1.12.2** (Dec 11, 2025): Institution lookup now uses [ROR API v2](https://ror.readme.io/docs/api-v2) — the platform resolves institutional affiliations more reliably. Your existing ROR URLs (`https://ror.org/…`) are unaffected; no action needed. +- **v1.12.3** (Jan 13, 2026): Injection volume is now validated during proteomics measurement registration — out-of-range or non-numeric values are rejected before you save. +- **v1.12.4** (Feb 4, 2026): Internal UI framework updated (Vaadin 24.9.10). No changes to data entry fields or platform behaviour. + +### Data submission from outside the University of Tübingen network + +
October 10, 2025
+ +- **External data submission:** Collaborators outside the university network can now upload measurement data. See how to [request server access](rawdata/raw_data_request_server_access.md) and [upload your data](rawdata/raw_data_upload.md). +- **ORCID linking:** Connect your ORCID account to an existing Data Manager profile at any time from your [user profile](user/user_edit.md#link-orcid). + ### ORCID linking and simpler template registration -
September 18th, 2025
+
September 18, 2025
📌 Full [release notes (v1.11.0)](https://github.com/qbicsoftware/data-manager-app/releases/tag/1.11.0) on GitHub. @@ -36,25 +68,25 @@ Highlights: ### Metadata glossary -
September 18th, 2025
+
September 18, 2025
- A new [Metadata glossary](metadata/concepts.md) explains every field with plain-language descriptions and example values. ### Faster sample and experiment updates -
June 10th, 2025
+
June 10, 2025
- Sample and experiment updates now run in the background. A notification confirms when the process is complete. ### Background project creation -
March 4th, 2025
+
March 4, 2025
- Project creation now runs in the background with progress notifications. ### Confounding variable support -
February 12th, 2025
+
February 12, 2025
- **Confounding variables:** Track factors that might influence your results at the experiment and sample level. - **Updated ontology search:** Terms now come from the improved [TIB Terminology Service](https://terminology.tib.eu/ts/api). @@ -65,7 +97,7 @@ Full [release notes (v1.8.0)](https://github.com/qbicsoftware/data-manager-app/r ### Redesigned project summary -
November 14th, 2024
+
November 14, 2024
- New project summary layout with easier access to project information. - Spreadsheet templates now include helpful examples and validation tooltips. @@ -73,7 +105,7 @@ Full [release notes (v1.8.0)](https://github.com/qbicsoftware/data-manager-app/r ### Excel support for sample batch registration -
October 23rd, 2024
+
October 23, 2024
- Register and update sample batches with [Excel spreadsheets](batch/sample-batch.md). - **RO-Crate export:** Download your project metadata as a [structured, machine-readable package](project/project_edit.md#download-project-metadata). Learn more about [RO-Crate](https://www.researchobject.org/ro-crate/). @@ -81,7 +113,7 @@ Full [release notes (v1.8.0)](https://github.com/qbicsoftware/data-manager-app/r ### Excel support for measurements -
September 4th, 2024
+
September 4, 2024
- Register and update measurements with [Excel spreadsheets](measurement/measurement_introduction.md). TSV still supported. - The sample field `Organism ID` has been renamed to `Biological Replicate` to better reflect its purpose. diff --git a/docs/measurement/measurement_edit.md b/docs/measurement/measurement_edit.md index 00312ba..94223e9 100644 --- a/docs/measurement/measurement_edit.md +++ b/docs/measurement/measurement_edit.md @@ -3,6 +3,9 @@ !!! tip "Excel is supported" Upload edited metadata as an Excel file (`.xlsx`). Metadata must be on the first sheet. +!!! tip "Adding new measurements?" + See [Register measurements](measurement_registration.md) instead. + [Navigate](measurement_introduction.md#navigate-to-measurements) to the measurement summary. !!! info "Required role" @@ -64,4 +67,4 @@ ## What's next -➡ [Register new measurements](measurement_registration.md) · [Upload raw data](../rawdata/raw_data_upload.md) +➡ [Upload raw data](../rawdata/raw_data_upload.md) diff --git a/docs/measurement/measurement_introduction.md b/docs/measurement/measurement_introduction.md index 5a43412..a3f89aa 100644 --- a/docs/measurement/measurement_introduction.md +++ b/docs/measurement/measurement_introduction.md @@ -21,4 +21,4 @@ This opens the measurement summary view with separate tabs for Proteomics and Ge ## What's next -➡ [Register measurements](measurement_registration.md) · [Edit measurements](measurement_edit.md) +➡ [Register measurements](measurement_registration.md) diff --git a/docs/measurement/measurement_registration.md b/docs/measurement/measurement_registration.md index 5b1d70c..3854f0b 100644 --- a/docs/measurement/measurement_registration.md +++ b/docs/measurement/measurement_registration.md @@ -7,9 +7,11 @@ Register measurements in three steps: -1. [Download the template](#download-template) -2. [Fill in the metadata](#prepare-metadata) -3. [Upload the completed file](#upload) +```mermaid +graph LR + A(Download template) --> B(Fill in metadata) + B --> C(Upload completed file) +``` !!! info "Required role" You need **write** or **admin** role to see the registration buttons. @@ -32,13 +34,15 @@ Register measurements in three steps: The template has two sheets: **Property Information** (reference) and **Metadata** (fill this in). Mandatory columns are marked with `*`. +!!! info "Organisation ID" + Enter the full [ROR](https://ror.org/) URL of the institution where the measurement was performed (e.g. `https://ror.org/03a1kwz48`). Search at [ror.org](https://ror.org/search). The platform looks up the institution name automatically. + === "Proteomics" Key fields: - **Sample ID** — copy from your [batch metadata download](../batch/sample-batch.md#download-sample-metadata) - **Instrument** — an ontology code (CURIE) for your mass spectrometer, e.g. `BAO:0002733`. Use the [ontology search](../ontology_search/ontology_search_introduction.md) to find the right code. - - **Organisation Id** — the full [ROR](https://ror.org/) URL of your institution, e.g. `https://ror.org/03a1kwz48`. Search at [ror.org](https://ror.org/search). - **Digestion enzyme**, **Digestion method**, **LC column** — required proteomics-specific fields. ![Filled template](images/measurement_registration_proteomics_measurement_filled.png){.screenshot} @@ -49,7 +53,6 @@ The template has two sheets: **Property Information** (reference) and **Metadata - **Sample ID** — copy from your [batch metadata download](../batch/sample-batch.md#download-sample-metadata) - **Instrument** — an ontology code (CURIE) for your sequencer, e.g. `OBI:0002750`. Use the [ontology search](../ontology_search/ontology_search_introduction.md) to find the right code. - - **Organisation Id** — the full [ROR](https://ror.org/) URL of your institution. - **Read type** — `paired-end` or `single-end`. - **Index I7 / I5** — required for pooled (multiplexed) measurements. These are the DNA index sequences used to identify which sample is which when multiple libraries are sequenced together. @@ -93,4 +96,4 @@ Click **Download Metadata** to export the metadata for all measurements in the c ## What's next -➡ [Upload raw data](../rawdata/raw_data_upload.md) · [Edit measurements](measurement_edit.md) +➡ [Upload raw data](../rawdata/raw_data_upload.md) diff --git a/docs/metadata/concepts.md b/docs/metadata/concepts.md index 99971c8..47a2d98 100644 --- a/docs/metadata/concepts.md +++ b/docs/metadata/concepts.md @@ -282,8 +282,10 @@ The following concepts are associated with a raw data dataset. | File suffixes | fastq, tar, txt | ✅ | List of Text |
List of file suffixes for all files within a raw data upload, automatically determined by the Data ManagerView Description
| | Registration date | 2025-04-28
08:24:25.000000 | ✅ | Date |
Timestamp when the raw data was uploaded within the Data ManagerView Description
| + --- -## What's next +## See also -➡ [API reference](../developers/api.md) · [Process overview](../get_started/process_overview.md) +- [Ontology search](../ontology_search/ontology_search_introduction.md) — find standardised terms and copy their identifiers +- [Process overview](../get_started/process_overview.md) — the end-to-end Data Manager workflow diff --git a/docs/ontology_search/ontology_search_introduction.md b/docs/ontology_search/ontology_search_introduction.md index c9e008b..fc5e799 100644 --- a/docs/ontology_search/ontology_search_introduction.md +++ b/docs/ontology_search/ontology_search_introduction.md @@ -25,14 +25,19 @@ Toggle the species filter to search only the [NCBI taxonomy](https://www.ncbi.nl !!! note "Why a separate species search?" Species terms come from NCBI's taxonomy database, which is hosted separately from the [TIB Terminology Service](https://terminology.tib.eu) used for other ontologies. The two systems are queried independently. +!!! info "Two search systems" + Species terms come from [NCBI Taxonomy](https://www.ncbi.nlm.nih.gov/taxonomy) — a complete catalogue of organisms. All other terms (specimen, analyte, instrument, disease) come from the [TIB Terminology Service](https://terminology.tib.eu), which aggregates ontologies like EFO, NCIT, and BAO. The two systems are queried independently, which is why species search has its own toggle. + ## Copy a CURIE Click the copy icon next to any term to copy its CURIE (e.g. `NCIT:C105979`) to your clipboard. Particularly useful when filling in [measurement registration templates](../measurement/measurement_registration.md). ![Copy CURIE](images/ontology_search_copy_curie.png){.screenshot} +> Ontology terms are also used when [creating experiments](../experiment/experiment_creation.md) — for species, specimen, and analyte fields. + --- ## What's next -➡ [Register measurements](../measurement/measurement_registration.md) · [Create an experiment](../experiment/experiment_creation.md) +➡ [Register measurements](../measurement/measurement_registration.md) diff --git a/docs/project/project_access.md b/docs/project/project_access.md index f242e5e..0904e6f 100644 --- a/docs/project/project_access.md +++ b/docs/project/project_access.md @@ -56,4 +56,4 @@ Click the **×** icon next to the collaborator. They lose access immediately. ## What's next -➡ [Back to project summary](project_introduction.md) · [Design an experiment](../experiment/experiment_introduction.md) +➡ [Design your experiment](../experiment/experiment_introduction.md) diff --git a/docs/project/project_edit.md b/docs/project/project_edit.md index e142c50..f3b6b3f 100644 --- a/docs/project/project_edit.md +++ b/docs/project/project_edit.md @@ -61,4 +61,4 @@ Click **Export as RO-Crate** in the project summary. A `.zip` archive downloads ## What's next -➡ [Manage access](project_access.md) · [Create an experiment](../experiment/experiment_creation.md) +➡ [Manage project access](project_access.md) diff --git a/docs/project/project_introduction.md b/docs/project/project_introduction.md index 4ba3035..1f9614e 100644 --- a/docs/project/project_introduction.md +++ b/docs/project/project_introduction.md @@ -32,8 +32,10 @@ Select a project from the drawer to jump straight to it, or click **Go to projec !!! info "Application drawer" The drawer also lets you switch between experiments within a project without going back to the project summary. +**Managing an existing project?** [Edit project details](project_edit.md) · [Manage access](project_access.md) + --- ## What's next -➡ [Register a new project](project_registration.md) · [Edit project details](project_edit.md) · [Manage access](project_access.md) +➡ [Register a new project](project_registration.md) diff --git a/docs/project/project_registration.md b/docs/project/project_registration.md index 1f19366..6752639 100644 --- a/docs/project/project_registration.md +++ b/docs/project/project_registration.md @@ -39,7 +39,8 @@ Set up your first experiment. Provide an **experiment name** and select at least - **Specimen** — the tissue or material type - **Analyte** — the molecular class being measured -Type at least 2 characters in each search field to see matching terms from standardised life science ontologies. +!!! tip "Searching for terms" + Type at least 2 characters to see matching terms. See the [ontology search guide](../ontology_search/ontology_search_introduction.md) for details. ![Experiment search](images/experimental_information_search.png){.screenshot} @@ -52,8 +53,11 @@ Optionally select an icon for the species and specimen. Click **Confirm** to cre Your project appears in the project list. You can now [upload supporting files](project_edit.md#upload-project-related-files) like offers and QC reports. +!!! note "Inviting collaborators" + You can [add team members to this project](project_access.md) at any time — you don't need to do this before setting up your experiment. + --- ## What's next -➡ [Set up experimental variables](../experiment/experiment_creation.md) · [Invite collaborators](project_access.md) +➡ [Design your experiment](../experiment/experiment_creation.md) diff --git a/docs/rawdata/raw_data_download.md b/docs/rawdata/raw_data_download.md index 0bb9115..bffd07b 100644 --- a/docs/rawdata/raw_data_download.md +++ b/docs/rawdata/raw_data_download.md @@ -7,10 +7,12 @@ The raw data view shows data already uploaded for measurements in your experimen ## Process -1. [Create a personal access token](#personal-access-token) -2. [Navigate to the raw data view](#raw-data-navigation) -3. [Generate download URLs](#generate-download-urls) -4. [Download via command line](#download-via-command-line) +```mermaid +graph LR + A(Create access token) --> B(Open raw data view) + B --> C(Generate download URLs) + C --> D(Download via command line) +``` ## Personal access token @@ -122,4 +124,4 @@ Create a text file with one URL per line (or use the file from the [URL generati ## What's next -➡ [Back to experiment](../experiment/experiment_introduction.md) · [Manage project access](../project/project_access.md) +➡ [Manage project access](../project/project_access.md) diff --git a/docs/rawdata/raw_data_upload.md b/docs/rawdata/raw_data_upload.md index 48543c1..4d2cb63 100644 --- a/docs/rawdata/raw_data_upload.md +++ b/docs/rawdata/raw_data_upload.md @@ -163,8 +163,11 @@ If an upload fails, a folder appears in `/home//error` containing an Fix the error described in `error.txt`, then move the folder back to `registration` to retry. +!!! tip "Share with collaborators" + Grant your team [project access](../project/project_access.md) so they can download files directly. + --- ## What's next -➡ [Download raw data](raw_data_download.md) · [Back to experiment](../experiment/experiment_introduction.md) +➡ [Download raw data](raw_data_download.md) diff --git a/docs/user/password_reset.md b/docs/user/password_reset.md index eacf973..e7c20d6 100644 --- a/docs/user/password_reset.md +++ b/docs/user/password_reset.md @@ -32,8 +32,10 @@ Enter a new password (at least 12 characters) and confirm. Click the login link ![Confirmation](images/password_reset/password_reset_new_password_saved.png){.screenshot} +> Want to update your username or link your ORCID? See [Edit your profile](user_edit.md). + --- ## What's next -➡ [Log in](https://rdm.qbic.uni-tuebingen.de/login) · [Edit your profile](user_edit.md) +➡ [Log in to the Data Manager](https://rdm.qbic.uni-tuebingen.de/login) diff --git a/docs/user/user_edit.md b/docs/user/user_edit.md index 639f5c8..f08580d 100644 --- a/docs/user/user_edit.md +++ b/docs/user/user_edit.md @@ -44,8 +44,10 @@ If you registered with email and want to add ORCID login later: ![ORCID linked](images/edit_user_information/edit_user_information_orcid_account_linked.png){.screenshot} +> Need to change your password? See [Reset your password](password_reset.md). + --- ## What's next -➡ [Register a project](../project/project_registration.md) · [Reset your password](password_reset.md) +➡ [Register a project](../project/project_registration.md) diff --git a/mkdocs.yml b/mkdocs.yml index 62c13ab..87d0519 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -6,6 +6,7 @@ theme: - content.code.copy - navigation.footer - header.autohide + - navigation.sections site_url: 'https://qbicsoftware.github.io/research-data-management/' site_name: Life Science Data Management repo_url: https://github.com/qbicsoftware/research-data-management @@ -21,25 +22,26 @@ nav: - Project: - Introduction: project/project_introduction.md - Registration: project/project_registration.md - - Editing: project/project_edit.md + - Edit: project/project_edit.md - Access management: project/project_access.md - Experiment: - Introduction: experiment/experiment_introduction.md - Creation: experiment/experiment_creation.md - Confounding variables: experiment/confounding-variables.md - - Sample batch: batch/sample-batch.md + - Samples: + - Register batches: batch/sample-batch.md - Measurement: - Introduction: measurement/measurement_introduction.md - Registration: measurement/measurement_registration.md - - Editing: measurement/measurement_edit.md + - Edit: measurement/measurement_edit.md - Raw data: - Request server access: rawdata/raw_data_request_server_access.md - Upload data: rawdata/raw_data_upload.md - Download data: rawdata/raw_data_download.md - Ontology search: ontology_search/ontology_search_introduction.md - - Reference: + - Metadata glossary: metadata/concepts.md + - Developer reference: - API: developers/api.md - - Metadata glossary: metadata/concepts.md extra_css: - stylesheets/extra.css extra: From f5c5f4dfd963dc250cfa4138b89bc706d5242d68 Mon Sep 17 00:00:00 2001 From: Steffengreiner Date: Tue, 10 Mar 2026 17:26:21 +0100 Subject: [PATCH 4/5] docs: split process overview diagram into two rows and style nav section headers - Split 8-node mermaid workflow into two 4-node diagrams for better readability - Add CSS for navigation.sections headers: uppercase, brand colour, letter-spacing, and subtle bottom border to make sidebar groups visually prominent --- docs/get_started/process_overview.md | 6 +++++- docs/stylesheets/extra.css | 12 ++++++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/docs/get_started/process_overview.md b/docs/get_started/process_overview.md index 37d512e..8a1c40c 100644 --- a/docs/get_started/process_overview.md +++ b/docs/get_started/process_overview.md @@ -7,7 +7,11 @@ graph LR A(1. Create account) --> B(2. Register project) B --> C(3. Design experiment) C --> D(4. Register samples) - D --> E(5. Register measurements) +``` + +```mermaid +graph LR + D(4. Register samples) --> E(5. Register measurements) E --> F(6. Upload raw data) F --> G(7. Manage access) G --> H(8. Share data) diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index 2fcc690..b79f50a 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -63,3 +63,15 @@ this will also change the spacing of any icons/emojis within the documentation * .md-typeset__table thead tr { background-color: #e0e0e0; /* white for header rows */ } + +/* Navigation section headers — make top-level groups prominent */ +.md-nav--primary > .md-nav__list > .md-nav__item--section > .md-nav__link { + font-weight: 700; + font-size: 0.65rem; + text-transform: uppercase; + letter-spacing: 0.05em; + color: var(--md-primary-fg-color); + margin-top: 0.8rem; + padding-bottom: 0.2rem; + border-bottom: 1px solid rgba(0, 106, 245, 0.15); +} From 6935b977ac96c3fc587ec9bbf66d8198d73406cb Mon Sep 17 00:00:00 2001 From: Steffengreiner Date: Tue, 10 Mar 2026 17:36:52 +0100 Subject: [PATCH 5/5] docs: vertical centred process overview diagram and centre all mermaid diagrams - Replace two-row LR layout with single vertical TD flow for better readability - Add CSS to centre all mermaid diagrams on the page --- docs/get_started/process_overview.md | 8 ++------ docs/stylesheets/extra.css | 5 +++++ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/get_started/process_overview.md b/docs/get_started/process_overview.md index 8a1c40c..d897a1a 100644 --- a/docs/get_started/process_overview.md +++ b/docs/get_started/process_overview.md @@ -3,15 +3,11 @@ Here's the complete Data Manager workflow — from creating your account to sharing data with collaborators. ```mermaid -graph LR +graph TD A(1. Create account) --> B(2. Register project) B --> C(3. Design experiment) C --> D(4. Register samples) -``` - -```mermaid -graph LR - D(4. Register samples) --> E(5. Register measurements) + D --> E(5. Register measurements) E --> F(6. Upload raw data) F --> G(7. Manage access) G --> H(8. Share data) diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index b79f50a..cb2897d 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -75,3 +75,8 @@ this will also change the spacing of any icons/emojis within the documentation * padding-bottom: 0.2rem; border-bottom: 1px solid rgba(0, 106, 245, 0.15); } + +/* Centre mermaid diagrams */ +.md-typeset .mermaid { + text-align: center; +}