Merge pull request #349 from SpatialHackathon/docs

Update Docs

Author: Jieran-S

.github/workflows/build_mkdocs.yml

@@ -19,6 +19,6 @@ jobs:
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
-          pip install mkdocs
+          pip install -r docs/requirements.txt
       - name: Build and deploy MkDocs
         run: mkdocs gh-deploy --force

CONTRIBUTING.md

@@ -8,31 +8,34 @@ SpaceHack 2023 is an open source project and we love to receive contributions fr
 
 # Getting started
 
-## Creating new modules
-
- 1. First create or claim a **GitHub issue** from the [SpaceHack issue board.](https://github.com/SpatialHackathon/SpaceHack2023/issues) that describes the module you want to implement and add the appropriate **tags**, and **assign** the task to yourself.
- 2. Add **metadata** to our metadata [spreadsheet](https://docs.google.com/spreadsheets/d/1QCeAF4yQG4bhZSGPQwwVBj_XF7ADY_2mK5xivAIfHsc/edit). Please fill in as much as you can as metadata is helpful! If you feel the need, please also add new columns or add additional notes. The metadata should be added to the appropriate tabs:
-    - [datasets](https://docs.google.com/spreadsheets/d/1QCeAF4yQG4bhZSGPQwwVBj_XF7ADY_2mK5xivAIfHsc/edit#gid=1453488771)
-    - [computational methods](https://docs.google.com/spreadsheets/d/1QCeAF4yQG4bhZSGPQwwVBj_XF7ADY_2mK5xivAIfHsc/edit#gid=0)
-    - [evaluation metrics](https://docs.google.com/spreadsheets/d/1QCeAF4yQG4bhZSGPQwwVBj_XF7ADY_2mK5xivAIfHsc/edit#gid=4776337)
-    - [simulations and technical evaluation](https://docs.google.com/spreadsheets/d/1QCeAF4yQG4bhZSGPQwwVBj_XF7ADY_2mK5xivAIfHsc/edit#gid=640974611)
- 3. Now you are ready to create a new git **[branch](https://learngitbranching.js.org/)**. Try to give your new branch an intuitive prefix such as `data_...`, `method_...` or `metric_..`. You can create a new branch in several ways: (i) [create a branch directly from the issue board](https://docs.github.com/en/issues/tracking-your-work-with-issues/creating-a-branch-for-an-issue) and then `git checkout` that branch, or (ii) via the command line:
-```
-# clone the template repository
-git clone https://github.com/SpatialHackathon/SpaceHack2023.git
-# create and switch to a new branch for your e.g. method "X"
-git branch method_x_naveedishaque # try to make the branch name unique!
-git checkout method_x_naveedishaque
-# link the branch to the issue via the issue board: https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue
-```
- 4. Modify the files, filenames, and code in `template/`, referring to the examples in the `data`, `method`, or `metric` subfolder. If your method requires a specific type or preprocessing, please reach out to the organisers!
- 5. Test. We are currently working on validators and automatic testing scripts... but this is tricky. Reach out to Niklas Muller-Botticher when you are ready to test!
- 6. Create a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request?tool=cli)
- 7. Code review (by whom?) and merge your contributed module into the GitHub main branch!
+Our workflow is set up to allow everyone to contribute "modules" in their preferred programming language (.. as long as that is either R or Python). A module can either be a dataset, a computational method, or an evaluation metric.
+![Workflow](img/workflow.svg)
+
+This repository contains some templates and examples of how to implement your module so that it interfaces seamlessly with other modules in the workflow. For example, if you want to implement a new method, you do not need to worry about input data or evaluation metrics as long as you follow the template for reading input and writing output - if you correctly adhere to the input and output guidelines, you should be able to interface with our default data modules and default evaluation metrics modules. The default modules are:
+
+ - data: LIBD Visium DLPFC dataset (4 samples, each with 3 replicates)
+ - methods: BayesSpace and SpaGCN
+ - evaluation metrics: ARI and V_measure
+
+ ## How to contribute a module
+
+Module contribution will be managed via GitHub. The steps to contribute a module are:
+
+1. Fork (or if you are part of the SpaceHack community branch) the latest version of the [SpaceHack repository](https://github.com/SpatialHackathon/SpaceHack2023)
+
+2. Make a copy of a [template]({{ repo_branch_url }}/templates/) depending on whether you are implementing a data, method, or metric module. You can have a look at existing modules if you are unsure what to do.
+
+3. Modify the files, filenames, and code in your copied template and move it to the correct module directory.
+
+4. Test. Before you make a pull request make sure that you can run the default modules. If it is a dataset, try to run e.g. SpaGCN or BayesSpace (or any other implemented method). If you implemented a method, try to run it on a dataset, and then test your output with an existing method.
+
+5. Create a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request?tool=cli).
+
+6. Wait for code review and your module being merged into the workflow.
 
 Easy!
 
-# How to report a bug
+## How to report a bug
 
 When filing an issue, make sure to answer these five questions:
 
@@ -42,12 +45,11 @@ When filing an issue, make sure to answer these five questions:
 4. What did you expect to see?
 5. What did you see instead?
 
-### Feature requests
+## Feature requests
 
 If you find yourself wishing for a feature or modules that don't exist in SpaceHack 2023, you are probably not alone. There are bound to be others out there with similar needs. Many of the features that SpaceHack 2023 has today have been added because our users saw the need. Open an issue on our issues list on GitHub that describes the feature you would like to see, why you need it, and how it should work.
 
-
-# Code review process
+## Code review process
 
 The core team looks at Pull Requests on an ad-hoc basis. 
 After feedback has been given we expect responses within four weeks. After four weeks we may close the pull request if it isn't showing any activity.

README.md

@@ -1,100 +1,114 @@
-# SACCELERATOR - a flexible framework for applying spatially aware clustering methods
-
-Spatial omics have transformed tissue architecture and cellular heterogeneity analysis by integrating molecular data with spatial localization. In spatially resolved transcriptomics, identifying spatial domains is critical for analysis of anatomical regions within heterogeneous datasets and understanding tissue function. Since 2020, more than 50 spatially aware clustering methods have been developed for this task. However, the reliability of existing benchmarks is undermined by their narrow focus on Visium and brain tissue datasets, as well as the dependence on questionable ground truth annotations. Here, we implemented a consensus framework that surpasses traditional benchmarking practices.
-
-Our framework comprises a community-driven benchmark-like platform that streamlines data formatting, method integration, and metric evaluation while accommodating new methods and datasets. Currently, the platform includes 22 spatially aware clustering methods across 15 datasets spanning 9 technologies and diverse tissue types. The benchmark approach uncovered significant limitations in generalizability and reproducibility where methods that perform well on healthy tissues often falter on cancer samples. We also found that anatomical labels commonly used as ground truths are often biased, potentially error-prone, and in some cases, unsuitable for benchmarking efforts.
-
-In light of these issues, we adopt a flexible expert-in-the-loop consensus-driven approach. This goes beyond traditional ensemble/consensus methods, and allows researchers to interact with intermediate results to determine which tools should be used to generate a consensus. We believe that the inclusion of an expert-in-the-loop is critical to ensure that the computational analysis matches the biological question at hand, and we believe that when the focus of the analysis is to uncover novel biological discoveries, tissue experts are accessible more often than not.
-
-# General setup
-
-This framework has established (and allows users to contribute)  "modules" in their preferred programming language (.. as long as that is either R or Python). A module is a set of scripts set up something in one of the following categories: a dataset, a computational method, or an evaluation metric. Interfaces between each category enable seamless integration of new data, methods, or metrics, thus enabling an extensible and community-driven framework. 
-
-![image](https://github.com/user-attachments/assets/ed55184d-d43f-4546-bee5-7b12e9ff8154)
-
-## Modules
-
-This repository contains some templates and examples of how to implement your module so that it interfaces seamlessly with other modules in the workflow. For example, if you want to implement a new method, you do not need to worry about input data or evaluation metrics as long as you follow the template for reading input and writing output - if you correctly adhere to the input and output guidelines, you should be able to interface with our default data modules and default evaluation metrics modules. 
-
-The existing modules are:
- - data (currently 28)
-   - LIBD Visium DLPFC dataset (4 samples, each with 3 replicates)
-   - SEA_AD_data
-   - STARmap-2018-mouse-cortex
-   - STARmap_plus
-   - abc_atlas_wmb_thalamus
-   - cosmx_liver
-   - cosmx_lung
-   - her2st-breast-cancer
-   - locus_coeruleus
-   - merfish_devheart
-   - mouse_brain_sagittal_anterior
-   - mouse_brain_sagittal_posterior
-   - mouse_kidney_coronal
-   - osmfish_Ssp
-   - pachter_simulation
-   - slideseq2_olfactory_bulb
-   -  sotip_simulation
-   - spatialDLPFC
-   - stereoseq_developing_Drosophila_embryos_larvae
-   - stereoseq_liver
-   - stereoseq_mouse_embryo
-   - stereoseq_olfactory_bulb
-   - visium_breast_cancer_SEDR
-   - visium_chicken_heart
-   - visium_hd_cancer_colon
-   - xenium-breast-cancer
-   - xenium-mouse-brain-SergioSalas
- - methods (currently 24):
-   - BANKSY
-   - BayesSpace
-   - CellCharter
-   - DRSC
-   - DeepST
-   - Giotto
-   - GraphST
-   - SCAN-IT
-   - SC_MEB
-   - SEDR
-   - SOTIP
-   - STAGATE
-   - SpaceFlow
-   - SpiceMix
-   - bass
-   - conST
-   - maple
-   - meringue
-   - precast
-   - scanpy
-   - seurat
-   - spaGCN
-   - spatialGE
-   - stardust
- - evaluation metrics (currently 17)
-   - ARI
-   - CHAOS
-   - Calinski-Harabasz
-   - Completeness
-   - Davies-Bouldin
-   - Entropy
-   - FMI
-   - Homogeneity
-   - LISI
-   - MCC
-   - NMI
-   - PAS
-   - SpatialARI
-   - V_measure
-   - cluster-specific-silhouette
-   - domain-specific-f1
-   - jaccard
-
-# Contributing and Code of Conduct
-
-Read our [Contributing Guide](CONTRIBUTING.md) and [Code of Conduct](CODE_OF_CONDUCT.md).
-
-# Contributors
-
+# SACCELERATOR - a flexible framework for applying spatially aware clustering methods
+
+Spatial omics have transformed tissue architecture and cellular heterogeneity analysis by integrating molecular data with spatial localization. In spatially resolved transcriptomics, identifying spatial domains is critical for analysis of anatomical regions within heterogeneous datasets and understanding tissue function. Since 2020, more than 50 spatially aware clustering methods have been developed for this task. However, the reliability of existing benchmarks is undermined by their narrow focus on Visium and brain tissue datasets, as well as the dependence on questionable ground truth annotations. Here, we implemented a consensus framework that surpasses traditional benchmarking practices.
+
+Our framework comprises a community-driven benchmark-like platform that streamlines data formatting, method integration, and metric evaluation while accommodating new methods and datasets. Currently, the platform includes 22 spatially aware clustering methods across 15 datasets spanning 9 technologies and diverse tissue types. The benchmark approach uncovered significant limitations in generalizability and reproducibility where methods that perform well on healthy tissues often falter on cancer samples. We also found that anatomical labels commonly used as ground truths are often biased, potentially error-prone, and in some cases, unsuitable for benchmarking efforts.
+
+In light of these issues, we adopt a flexible expert-in-the-loop consensus-driven approach. This goes beyond traditional ensemble/consensus methods, and allows researchers to interact with intermediate results to determine which tools should be used to generate a consensus. We believe that the inclusion of an expert-in-the-loop is critical to ensure that the computational analysis matches the biological question at hand, and we believe that when the focus of the analysis is to uncover novel biological discoveries, tissue experts are accessible more often than not.
+
+# General setup
+
+This framework has established (and allows users to contribute) "modules" in their preferred programming language (.. as long as that is either R or Python). A module is a set of scripts set up something in one of the following categories: a dataset, a computational method, or an evaluation metric. Interfaces between each category enable seamless integration of new data, methods, or metrics, thus enabling an extensible and community-driven framework. 
+
+![image](https://github.com/user-attachments/assets/ed55184d-d43f-4546-bee5-7b12e9ff8154)
+
+## Modules
+
+This repository contains some templates and examples of how to implement your module so that it interfaces seamlessly with other modules in the workflow. For example, if you want to implement a new method, you do not need to worry about input data or evaluation metrics as long as you follow the template for reading input and writing output - if you correctly adhere to the input and output guidelines, you should be able to interface with our default data modules and default evaluation metrics modules. 
+
+The existing modules are:
+ - data (currently 28)
+   - LIBD Visium DLPFC dataset (4 samples, each with 3 replicates)
+   - SEA_AD_data
+   - STARmap-2018-mouse-cortex
+   - STARmap_plus
+   - abc_atlas_wmb_thalamus
+   - cosmx_liver
+   - cosmx_lung
+   - her2st-breast-cancer
+   - locus_coeruleus
+   - merfish_devheart
+   - mouse_brain_sagittal_anterior
+   - mouse_brain_sagittal_posterior
+   - mouse_kidney_coronal
+   - osmfish_Ssp
+   - pachter_simulation
+   - slideseq2_olfactory_bulb
+   - sotip_simulation
+   - spatialDLPFC
+   - stereoseq_developing_Drosophila_embryos_larvae
+   - stereoseq_liver
+   - stereoseq_mouse_embryo
+   - stereoseq_olfactory_bulb
+   - visium_breast_cancer_SEDR
+   - visium_chicken_heart
+   - visium_hd_cancer_colon
+   - xenium-breast-cancer
+   - xenium-mouse-brain-SergioSalas
+ - methods (currently 24):
+   - BANKSY
+   - BayesSpace
+   - CellCharter
+   - DRSC
+   - DeepST
+   - Giotto
+   - GraphST
+   - SCAN-IT
+   - SC_MEB
+   - SEDR
+   - SOTIP
+   - STAGATE
+   - SpaceFlow
+   - SpiceMix
+   - bass
+   - conST
+   - maple
+   - meringue
+   - precast
+   - scanpy
+   - seurat
+   - spaGCN
+   - spatialGE
+   - stardust
+ - evaluation metrics (currently 17)
+   - ARI
+   - CHAOS
+   - Calinski-Harabasz
+   - Completeness
+   - Davies-Bouldin
+   - Entropy
+   - FMI
+   - Homogeneity
+   - LISI
+   - MCC
+   - NMI
+   - PAS
+   - SpatialARI
+   - V_measure
+   - cluster-specific-silhouette
+   - domain-specific-f1
+   - jaccard
+
+
+# Contributions
+
+Our workflow is set up to allow everyone to contribute "modules", whether it is a dataset, a computational method, or an evaluation metric.
+![image](https://github.com/SpatialHackathon/SpaceHack2023/assets/114547/7c002916-0a90-4fe7-8745-489313bc0192)
+
+This repository contains some templates and examples of how to implement your module so that it interfaces seamlessly with other modules in the workflow. 
+Please refer to our 
+[Contribution guide](https://spatialhackathon.github.io/SpaceHack2023/contributing/) and the 
+[Module documentation](https://spatialhackathon.github.io/SpaceHack2023/modules/) 
+for more details.
+
+
+## Contributing and Code of Conduct
+
+Read our [Contributing Guide](CONTRIBUTING.md) and [Code of Conduct](CODE_OF_CONDUCT.md).
+
+
+## Contributors
+
 <!-- readme: contributors -start -->
 <table>
 	<tbody>
@@ -248,14 +262,14 @@ Read our [Contributing Guide](CONTRIBUTING.md) and [Code of Conduct](CODE_OF_CON
 		</tr>
 	<tbody>
 </table>
-<!-- readme: contributors -end -->
-
-# Citation
-
-We are close to releasing a preprint. Until then, please cite us as follows:
-
-> SpaceHack 2.0. Participants. SpaceHack 2.0: an expert in the loop consensus driven framework for spatially aware clustering [Computer software]. https://github.com/SpatialHackathon/SpaceHack2023
-
-# License
-
-We have adopted the "MIT No Attribution" (MIT-0) License. It is currently attributed to the "SpaceHack organizers", but please also make sure to add your name to your contributions. More on MIT-0 [here](https://github.com/aws/mit-0)
+<!-- readme: contributors -end -->
+
+# Citation
+
+We are close to releasing a preprint. Until then, please cite us as follows:
+
+> SpaceHack 2.0. Participants. SpaceHack 2.0: an expert in the loop consensus driven framework for spatially aware clustering [Computer software]. https://github.com/SpatialHackathon/SpaceHack2023
+
+# License
+
+We have adopted the "MIT No Attribution" (MIT-0) License. It is currently attributed to the "SpaceHack organizers", but please also make sure to add your name to your contributions. More on MIT-0 [here](https://github.com/aws/mit-0)