Add imaging studies for all TCGA Pancan separated by modality#2260
Add imaging studies for all TCGA Pancan separated by modality#2260rmadupuri wants to merge 5 commits into
Conversation
| ```bash | ||
| for f in $(cut -f2 ~/Downloads/idc_ohif.tsv | gsort | uniq | grep tcga_ | grep -v Filters | grep -v coad | grep -v read); do (head -1 coadread_tcga_pan_can_atlas_2018/data_resource_patient.txt; cut -f1,2,4 ~/Downloads/idc_ohif.tsv | tail -n +9 | grep $f | cut -f1,3 | awk -vFS='\t' -vOFS='\t' '{$1=substr($1,0,12); $3="https://viewer.imaging.datacommons.cancer.gov/viewer/"$2; $2="IDC_OHIF_V2"; print $0}' | gsort -k1,1 | uniq | rev | uniq -f2 | rev; ) > ${f/tcga_/}_tcga_pan_can_atlas_2018/*data_resource*patient*; done | ||
| ``` | ||
| **Note**: While a single study may contain multiple imaging modalities, data was extracted at the study level and then split by modality for cBioPortal integration. This modality-level organization allows users to selectively access specific imaging types (e.g., CT scans vs. H&E slides) for each patient. |
There was a problem hiding this comment.
The URL patterns you provide at the bottom of this document are at the study level, so this paragraph is confusing.
| 1. Downloading TCGA imaging metadata from IDC using the idc-index package | ||
| 2. Linking patients to their imaging studies via OHIF and SLIM viewer URLs | ||
|
|
||
| **Script**: [generate_imaging_resources.py](https://github.com/cBioPortal/datahub-study-curation-tools/tree/master/generate_imaging_resources) |
There was a problem hiding this comment.
The script referenced uses custom code to generate viewer URLs. I recommend you use this function in IDCClient: https://idc-index.readthedocs.io/en/latest/api/idc_index.html#idc_index.index.IDCClient.get_viewer_URL. Note that your URLs point to the legacy OHIF Viewer v2, which will be deprecated. You should instead use OHIF Viewer v3. The function will generate OHIF v3 viewer links for radiology studies.
| ### Generate Resource Files | ||
|
|
||
| Patient-level resource files are generated for each TCGA Pan Cancer study by: | ||
| 1. Downloading TCGA imaging metadata from IDC using the idc-index package |
There was a problem hiding this comment.
You should keep track of the version of IDC data you use for generating those links. You can get it with https://idc-index.readthedocs.io/en/latest/api/idc_index.html#idc_index.index.IDCClient.get_idc_version. It would also be best if you updated your links after each IDC release. Although data removal is rare, it can happen, and would lead to broken viewer links.
|
I added custom Parquet to enable mapping between GDC and IDC in this PR ImagingDataCommons/idc-index-data#111, the resulting file is available here for the latest release: https://storage.googleapis.com/idc-index-data-artifacts/current/release_artifacts/gdc_idc_mapping.parquet and in https://storage.googleapis.com/idc-index-data-artifacts/23.6.1/release_artifacts/gdc_idc_mapping.parquet for the versioned release. I can modify CORS to allow your dev/prod URL to access the file. I think this might be the most optimal approach for integration. We can customize the content of the Parquet for cBioPortal as needed. |
|
Thanks @fedorov, will look into it. |
4 participants
What?
Enhancement to the previous PR #2015
Details on how the data is generated:
https://github.com/cBioPortal/datahub/blob/2615fc6c90fb0e30994d43a09c13d1f2abb81122/docs/tcga_pan_can_atlas/ohif-viewer.md
Available Imaging Modalities:
Testing:
H&E Slide using SLIM viewer:
CT Scan using OHIF viewer:
