You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,20 +14,20 @@
14
14
15
15
1. <b>Data Processing</b>: Large datasets undergo preprocessing and [normalization](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/preprocessing/normalization.ipynb) (if the user opts to) and are segmented into training, testing, and validation sets.
16
16
17
-
2. <b>Features Extraction</b>: A model is trained on feature subsets in a batch-wise process, so all features and samples are utilized in the feature selection process. Then, the top-k features are selected to train the final model, using a feature score based on the model's coefficients/weights or [SHAP analayis](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/analysis/shap_analysis/shap_heatmap.ipynb).
17
+
2. <b>Features Extraction</b>: A model is trained on feature subsets in a batch-wise process, so all features and samples are utilised in the feature selection process. Then, the top-k features are selected to train the final model, using a feature score based on the model's coefficients/weights or [SHAP analayis](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/analysis/shap_analysis/shap_heatmap.ipynb).
18
18
19
19
3. <b>Training</b>: A Deep Neural Network (DNN) is trained on the training dataset. The validation dataset is used to validate the model at each epoch, and early stopping is performed if applicable. Also, a [batch correction](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/preprocessing/batch_correction.ipynb) method is available to correct batch effects during training in the pipeline.
20
20
21
-
4. <b>Evaluation & Downstream Analysis</b>: The trained model is evaluated using the test dataset by calculating metrics such as precision, recall, f1-score, and accuracy. Various visualizations, such as ROC curve of class annotation, feature rank plots, heatmap of top genes per class, [DGE analysis](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/analysis/differential_gene_expression/dge.ipynb), and [gene recall curves](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/analysis/gene_recall_curve/gene_recall_curve.ipynb), are generated.
21
+
4. <b>Evaluation & Downstream Analysis</b>: The trained model is evaluated using the test dataset by calculating metrics such as precision, recall, f1-score, and accuracy. Various visualisations, such as ROC curve of class annotation, feature rank plots, heatmap of top genes per class, [DGE analysis](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/analysis/differential_gene_expression/dge.ipynb), and [gene recall curves](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/analysis/gene_recall_curve/gene_recall_curve.ipynb), are generated.
22
22
23
-
**The below flowchart also explains the major steps of the scaLR platform.**
23
+
**The flowchart below also explains the major steps of the scaLR platform.**
24
24
25
25

26
26
27
27
## Pre-requisites and installation scaLR
28
28
29
29
30
-
- ScaLR can be installed using git or pip. It is tested in Python 3.10 and it is recommended to use that environment.
30
+
- ScaLR can be installed using git or pip. It is tested in Python 3.10, and it is recommended to use that environment.
31
31
32
32
```
33
33
conda create -n scaLR_env python=3.10
@@ -46,7 +46,7 @@ pip install -r requirements.txt
46
46
```
47
47
pip install pyscaLR
48
48
```
49
-
**Note:** If the user wants to run the entire pipeline via installing pip pyscalr, they should clone/download these files(`pipeline.py` and `config.yaml`) from the git repository.
49
+
**Note:** If the user wants to run the entire pipeline by installing pip pyscalr, they should clone/download these files(`pipeline.py` and `config.yaml`) from the git repository.
50
50
51
51
## Input data format
52
52
- Currently the pipeline expects all datasets in [anndata](https://anndata.readthedocs.io/en/latest/tutorials/notebooks/getting-started.html) formats (`.h5ad` files only).
@@ -60,7 +60,7 @@ pip install pyscaLR
60
60
61
61
1. It is necessary that the user modify the configuration file, and each stage of the pipeline is available inside the config folder [config.yml] as per your requirements. Simply omit/comment out stages of the pipeline you do not wish to run.
62
62
2. Refer **config.yml** & **it's detailed config**[README](https://github.com/infocusp/scaLR/blob/main/config/README.md) file on how to use different parameters and files.
63
-
3. Then use the `pipeline.py` file to run the entire pipeline according to your configurations. This file takes as argument the path to config (`-c | --config`), along with optional flags to log all parts of the pipelines (`-l | --log`) and to analyze memory usage (`-m | --memoryprofiler`).
63
+
3. Then use the `pipeline.py` file to run the entire pipeline according to your configurations. This file takes as argument the path to config (`-c | --config`), along with optional flags to log all parts of the pipelines (`-l | --log`) and to analyse memory usage (`-m | --memoryprofiler`).
64
64
5.`python pipeline.py --config /path/to/config.yaml -l -m` to run the scaLR.
65
65
66
66
## Example configs
@@ -315,13 +315,13 @@ Detailed tutorials have been made on how to use some pipeline functionalities as
315
315
- **Normalization** [](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/preprocessing/normalization.ipynb)
316
316
- **Batch correction** [](https://colab.research.google.com/github/infocusp/scaLR/blob/main/tutorials/preprocessing/batch_correction.ipynb)
317
317
318
-
- **An example of jupyter notebook to [run scaLR in local machine](https://github.com/infocusp/scaLR/blob/main/tutorials/pipeline/scalr_pipeline_local_run.ipynb)**.
318
+
- **An example of Jupyter notebook to [run scaLR in local machine](https://github.com/infocusp/scaLR/blob/main/tutorials/pipeline/scalr_pipeline_local_run.ipynb)**.
319
319
320
320
## Experiment output structure
321
321
- **pipeline.py**:
322
322
The main script that performs an end-to-end run.
323
323
- `exp_dir`: root experiment directory for the storage of all step outputs of the platform specified in the config.
324
-
- `config.yml`: copy of config file to reproduce the user-defined experiment.
324
+
- `config.yml`: copy of the config file to reproduce the user-defined experiment.
325
325
326
326
- **data_ingestion**:
327
327
Reads the data and splits it into Train/Validation/Test sets for the pipeline. Then, it performs sample-wise normalization on the data.
@@ -346,25 +346,25 @@ Trains a final model based on `train_datapath` and `val_datapath` in config.
346
346
- `model`
347
347
- `logs`: directory containing Tensorboard Logs for the training of the model.
348
348
- `checkpoints`: directory containing model weights checkpointed at every interval specified in config.
349
-
- `best_model`: the best model checkpoint contains information to use model for inference/resume training.
349
+
- `best_model`: the best model checkpoint contains information to use the model for inference/resume training.
350
350
- `model_config.yaml`: config file containing model parameters.
351
-
- `mappings.json`: contains mapping of class_names to class_ids used by model during training.
351
+
- `mappings.json`: contains the mapping of class_names to class_ids used by the model during training.
352
352
- `model.pt`: contains model weights.
353
353
354
354
- **eval_and_analysis**:
355
-
Performs evaluation of best model trained on user-defined metrics on the test set. Also performs various downstream tasks.
355
+
Performs evaluation of the best model trained on user-defined metrics on the test set. Also performs various downstream tasks.
356
356
- `exp_dir`
357
357
- `analysis`
358
-
- `classification_report.csv`: contains classification report showing Precision, Recall, F1, and accuracy metrics for each class on the test set.
358
+
- `classification_report.csv`: contains a classification report showing Precision, Recall, F1, and accuracy metrics for each class on the test set.
359
359
- `gene_analysis`
360
360
- `score_matrix.csv`: score of the final model, for each feature and class. shape: n_classes X n_features.
361
361
- `top_features.json`: a file containing a list of selected top features/biomarkers.
362
362
- `test_samples/full_samples`
363
363
- `heatmaps`
364
-
- `class_name.svg`: heatmap for top genes of a particular class w.r.t those genes association in other classes. E.g., B.svg, C.svg, etc.
364
+
- `class_name.svg`: heatmap for top genes of a particular class w.r.t. those genes' association in other classes. E.g., B.svg, C.svg, etc.
365
365
- `roc_auc.svg`: contains ROC-AUC plot for all classes.
- `gene_recall_curve_info.json`: contains reference genes list which are present in top_K ranked genes per class for each model.
367
+
- `gene_recall_curve_info.json`: contains the reference genes list, which are present in the top_K ranked genes per class for each model.
368
368
- `pseudobulk_dge_result`
369
369
- `pbkDGE_celltype_factor_categories_0_vs_factor_categories_1.csv`: contains Pseudobulk DGE results between selected factor categories for a celltype.
370
370
- `pbkDGE_celltype_factor_categories_0_vs_factor_categories_1.svg`: volcano plot of Log2Foldchange vs -log10(p-value) of genes.
@@ -374,5 +374,5 @@ Performs evaluation of best model trained on user-defined metrics on the test se
374
374
375
375
## Citation
376
376
377
-
Jogani Saiyam, Anand Santosh Pol, Mayur Prajapati, Amit Samal, Kriti Bhatia, Jayendra Parmar, Urvik Patel, Falak Shah, Nisarg Vyas, and Saurabh Gupta. "scaLR: a low-resource deep neural network-based platform for single cell analysis and biomarker discovery." bioRxiv (2024): 2024-09.
377
+
Jogani, S., Pol, A. S., Prajapati, M., Samal, A., Bhatia, K., Parmar, J., ... & Gupta, S. (2025). scaLR: a low-resource deep neural network-based platform for single cell analysis and biomarker discovery. Briefings in Bioinformatics, 26(3), bbaf243.
0 commit comments