Discussion: meetings/2026-05-12

[Alishamsnia]

@jeromelecoq @farznaj

Hey, I started some preliminary analysis for single-session ROI quality control for Calcium imaging data.

You can find the complete pipeline explanation, along with the related figures for one example session, here:

QC_report.pdf

[Alishamsnia]

This is the pipeline for this preliminary analysis:
https://github.com/Alishamsnia/Allen_OpenScope_Ophys_QC

[jeromelecoq]

Maybe @Dedalus9 you want to try this repo out?
Perhaps it could be adapted as we discussed to look at AWS raw data.

[Dedalus9]

Hi @jeromelecoq, I will try it out. I'll design a notebook that can also load AWS raw data. I think it would be helpful, also, to setup the notebook such that it's clear to other users how to validate metrics. I would be curious to get your feedback. Here's how I'm conceiving it. It will be divided into stages of progressive complexity. The first stage will involve individual sessions; the second stage will involve a sample of good and bad sessions; the third stage will involve all sessions. For reproducibility, each stage will include code for generating a figure that can be used to assess the quality of the metric. For the final stage, the user will be required to design their code to adhere to some design pattern that can be scaled and easily maintained.

I think, for evaluating metrics, it would be helpful to have examples of bad data. Would you be able to provide access to this if it's already been excluded from the datasets? A good quality control metric, for instance, should be able to distinguish good from bad data. For reproducibility, we could include code in the second and third stages that classifies good from bad data based on that metric.

[jeromelecoq]

That makes sense to me.
We typically did not upload the very bad data. But you can see in the session table sheet on Google Sheets the notes from each experiments for issues we flagged.

For 2p, there are planes with issues that we flagged individually. If you look at the qc json in the folder, you should be able to detect those.

I would suggest you start and we can go from there ?

[maierav]

One idea might be to be explicit about our 2-stage QC approach:

1. Initial QC (Allen Institute-internal): sessions that did not meet the inclusion criteria got rejected entirely

2. In-paper QC analyses for the released data: an overview of the variance in data quality using various metrics (thus allowing anybody to apply further, more restrictive criteria if they are so inclined).

We also might want to decide if the paper describes the .nwb, AWS, or both. Maybe just focus on the .nwb, and mention that the AWS is also available?

[jeromelecoq]

I think we were planning to describe both in the paper with a stronger focus on NWB. Both the NWBs and the raw data are actually hosted by the same AWS open data program in the end.

[jeromelecoq]

from the current manuscript., this is the current metrics we extract from the raw data to check data quality.

1. Image saturation: Initial frames were inspected to ensure fewer than 1000 saturated pixels and adequate use of the detector dynamic range.
2. Photobleaching: Baseline fluorescence at the beginning and end of the session was compared; sessions with >20% signal loss were excluded.
3. Targeting accuracy: Imaging locations were registered to ISI-derived maps to confirm correct visual area targeting.
4. Z-axis stability: Mean images from the first and last 5 minutes were compared to a post hoc z-stack (±30 µm, 0.75 µm steps) to estimate drift. Sessions with >10 µm drift were excluded.
5. Animal well-being: Behavioral videos were reviewed for signs of stress (e.g., excessive secretion, orbital tightening, abnormal posture). Animals exhibiting sustained stress responses were removed from the experiment.
6. Temporal synchronization: Alignment across all recorded data streams was verified.
7. System integrity: Data streams were assessed for hardware or software failures affecting data quality.
8. Motion artifacts: Residual motion was evaluated after motion correction.
9. Interictal activity: Full-field fluorescence traces (first 10,000 frames) were analyzed for abnormal transient events. Sessions with potential interictal activity were manually reviewed and excluded if necessary (Steinmetz et al. 2017).

[jeromelecoq]

access to S3 is introduced here : https://docs.google.com/presentation/d/16pukgoNJ8y2fFwfE9ErGTLTxn81HCSLickkPncqPWb0/edit?slide=id.g3de82949ca6_2_176#slide=id.g3de82949ca6_2_176

[jeromelecoq]

Here is an example qc file generated by the 2p processing pipeline : https://open.quiltdata.com/b/aind-open-data/tree/multiplane-ophys_846289_2026-04-13_11-49-19_processed_2026-04-14_09-14-44/quality_control.json

[jeromelecoq]

@Alishamsnia, I went through your document. This is a great start!
Probably a few more metadata would be helpful for each session to be in the output for correlation purposes:

• Plane depth
• Session number (how many times was imaging attempted on this mouse)
• I would load the quality_control.json that is listed above and add those numbers in.
• Perhaps the session type (sequence, motion...) would be helpful to see if there is a systematic trends associated with the session type (for example motion artifacts) as it would impact any downstream analysis.
• The quality_control.json should list existing plane with already flagged issues. That might be helpful for everyone to review.
• Motion correction traces and derived metrics off.

I am sure there a number of additional SNR metrics that could be build upon for event detection or such.

[farznaj]

@jeromelecoq

I talked to my students (Ali and Davis) about QC for mesoscope data.

We came up with this detailed plan for QC. Please read it and let us know if we should proceed with developing this script.

They will develop a script that gives the following:

Plane level:

1. FOV with all overlayed ROIs

• Title: Number of ROIs

2. Distribution of ROI sizes
3. Distribution of soma/dendrite probability

• Note: we can also add a metric for how round/elongated the ROI shape is.

Within each plane, then we go through individual neurons and show the following:
Neuron level:
For each neuron show:

1. FOV, with that specific neuron marked

• Title: ROI size and soma/dendrite probability

2. show DFF

• Title: DFF drift, computed as early vs late median DFF

3. Show OASIS noise distribution

• Title: is it gaussian, show R^2

4. Average calcium transient

• get spikes from OASIS, and compute average calcium transients, across spikes

5. Stimulus-aligned DFF average across trials

• Title: AUC in a 500ms after stimulus
• NOTE: this is with the assumption that the neuron is stimulus responsive [which may not be true for all brain areas]

@Alishamsnia @davisgrubin

[jeromelecoq]

This makes sense to me It would be nice to consider cross session alignement too. So perhaps the FOV view could be displayed on a row for the planes in the same area and depth ? What do you think? *Jérôme*

…

On Wed, May 20, 2026 at 1:00 PM Farzaneh Najafi ***@***.***> wrote: @jeromelecoq <https://github.com/jeromelecoq> I talked to my students (Ali and Davis) about QC for mesoscope data. We came up with this detailed plan for QC. Please read it and let us know if we should proceed with developing this script. They will develop a script that gives the following: *Plane level:* 1. FOV with all overlayed ROIs - Title: Number of ROIs 2. Distribution of ROI sizes 3. Distribution of soma/dendrite probability - Note: we can also add a metric for how round/elongated the ROI shape is. Within each plane, then we go through individual neurons and show the following: *Neuron level:* For each neuron show: 1. FOV, with that specific neuron marked - Title: ROI size and soma/dendrite probability 2. show DFF - Title: DFF drift, computed as early vs late median DFF 3. Show OASIS noise distribution - Title: is it gaussian, show R^2 4. Average calcium transient - get spikes from OASIS, and compute average calcium transients, across spikes 5. Stimulus-aligned DFF average across trials - Title: AUC in a 500ms after stimulus - NOTE: this is with the assumption that the neuron is stimulus responsive [which may not be true for all brain areas] @Alishamsnia <https://github.com/Alishamsnia> @davisgrubin <https://github.com/davisgrubin> — Reply to this email directly, view it on GitHub <#153 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AATAHT3YDB2VRYFCT5ZYH4T43YFF5AVCNFSM6AAAAACY24MJGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMOJZGUYTSOI> . You are receiving this because you were mentioned.Message ID: <AllenNeuralDynamics/openscope-community-predictive-processing/repo-discussions/153/comments/16995199 @github.com>

[farznaj]

Just to confirm , by session you mean each plane? So we have 8 sessions in a day , 4 per area. Correct ? I’ll ask ali and Davis to start working on the script. On Wed, May 20, 2026 at 10:26 PM Jérôme Lecoq ***@***.***> wrote:

…

This makes sense to me It would be nice to consider cross session alignement too. So perhaps the FOV view could be displayed on a row for the planes in the same area and depth ? What do you think? *Jérôme* On Wed, May 20, 2026 at 1:00 PM Farzaneh Najafi ***@***.***> wrote: > @jeromelecoq <https://github.com/jeromelecoq> > > I talked to my students (Ali and Davis) about QC for mesoscope data. > > We came up with this detailed plan for QC. Please read it and let us know > if we should proceed with developing this script. > > They will develop a script that gives the following: > > *Plane level:* > > 1. FOV with all overlayed ROIs > > > - Title: Number of ROIs > > > 2. Distribution of ROI sizes > 3. Distribution of soma/dendrite probability > > > - Note: we can also add a metric for how round/elongated the ROI shape > is. > > Within each plane, then we go through individual neurons and show the > following: > *Neuron level:* > For each neuron show: > > 1. FOV, with that specific neuron marked > > > - Title: ROI size and soma/dendrite probability > > > 2. show DFF > > > - Title: DFF drift, computed as early vs late median DFF > > > 3. Show OASIS noise distribution > > > - Title: is it gaussian, show R^2 > > > 4. Average calcium transient > > > - get spikes from OASIS, and compute average calcium transients, > across spikes > > > 5. Stimulus-aligned DFF average across trials > > > - Title: AUC in a 500ms after stimulus > - NOTE: this is with the assumption that the neuron is stimulus > responsive [which may not be true for all brain areas] > > @Alishamsnia <https://github.com/Alishamsnia> @davisgrubin > <https://github.com/davisgrubin> > > — > Reply to this email directly, view it on GitHub > < #153 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AATAHT3YDB2VRYFCT5ZYH4T43YFF5AVCNFSM6AAAAACY24MJGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMOJZGUYTSOI> > . > You are receiving this because you were mentioned.Message ID: > <AllenNeuralDynamics/openscope-community-predictive-processing/repo-discussions/153/comments/16995199 > @github.com> > — Reply to this email directly, view it on GitHub <#153 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADTJ7QX3AHWQKHLHO5A27KT43ZSPBAVCNFSM6AAAAACY24MJGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMOJZHA2DCNI> . You are receiving this because you were mentioned.Message ID: <AllenNeuralDynamics/openscope-community-predictive-processing/repo-discussions/153/comments/16998415 @github.com>

[jeromelecoq]

I mean a session is a recording day. Each mesoscope set of 8 imaging planes is recorded 8 times with cell matching *Jérôme* On Wed, May 20, 2026 at 8:33 PM Farzaneh Najafi ***@***.***> wrote:

…

Just to confirm , by session you mean each plane? So we have 8 sessions in a day , 4 per area. Correct ? I’ll ask ali and Davis to start working on the script. On Wed, May 20, 2026 at 10:26 PM Jérôme Lecoq ***@***.***> wrote: > This makes sense to me > It would be nice to consider cross session alignement too. So perhaps the > FOV view could be displayed on a row for the planes in the same area and > depth ? > > What do you think? > > > *Jérôme* > > On Wed, May 20, 2026 at 1:00 PM Farzaneh Najafi ***@***.***> > wrote: > > > @jeromelecoq <https://github.com/jeromelecoq> > > > > I talked to my students (Ali and Davis) about QC for mesoscope data. > > > > We came up with this detailed plan for QC. Please read it and let us > know > > if we should proceed with developing this script. > > > > They will develop a script that gives the following: > > > > *Plane level:* > > > > 1. FOV with all overlayed ROIs > > > > > > - Title: Number of ROIs > > > > > > 2. Distribution of ROI sizes > > 3. Distribution of soma/dendrite probability > > > > > > - Note: we can also add a metric for how round/elongated the ROI shape > > is. > > > > Within each plane, then we go through individual neurons and show the > > following: > > *Neuron level:* > > For each neuron show: > > > > 1. FOV, with that specific neuron marked > > > > > > - Title: ROI size and soma/dendrite probability > > > > > > 2. show DFF > > > > > > - Title: DFF drift, computed as early vs late median DFF > > > > > > 3. Show OASIS noise distribution > > > > > > - Title: is it gaussian, show R^2 > > > > > > 4. Average calcium transient > > > > > > - get spikes from OASIS, and compute average calcium transients, > > across spikes > > > > > > 5. Stimulus-aligned DFF average across trials > > > > > > - Title: AUC in a 500ms after stimulus > > - NOTE: this is with the assumption that the neuron is stimulus > > responsive [which may not be true for all brain areas] > > > > @Alishamsnia <https://github.com/Alishamsnia> @davisgrubin > > <https://github.com/davisgrubin> > > > > — > > Reply to this email directly, view it on GitHub > > < > #153 (comment)>, > > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AATAHT3YDB2VRYFCT5ZYH4T43YFF5AVCNFSM6AAAAACY24MJGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMOJZGUYTSOI> > > > . > > You are receiving this because you were mentioned.Message ID: > > > <AllenNeuralDynamics/openscope-community-predictive-processing/repo-discussions/153/comments/16995199 > > > @github.com> > > > > — > Reply to this email directly, view it on GitHub > < #153 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/ADTJ7QX3AHWQKHLHO5A27KT43ZSPBAVCNFSM6AAAAACY24MJGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMOJZHA2DCNI> > . > You are receiving this because you were mentioned.Message ID: > <AllenNeuralDynamics/openscope-community-predictive-processing/repo-discussions/153/comments/16998415 > @github.com> > — Reply to this email directly, view it on GitHub <#153 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AATAHT5XPBTK7FUZYBUAOVT43Z2JTAVCNFSM6AAAAACY24MJGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMOJZHA4DONA> . You are receiving this because you were mentioned.Message ID: <AllenNeuralDynamics/openscope-community-predictive-processing/repo-discussions/153/comments/16998874 @github.com>

[Dedalus9]

Here's the link to my GitHub: https://github.com/Dedalus9/openscope-ophys-qc.git . It builds on code originally developed by @Alishamsnia, is designed for others to use, and loads data from AWS. The repository includes a notebook for loading single or multiple ophys sessions.

I am currently extending this pipeline to generate quality control figures, including metrics to track movement artifacts across sessions. I could also replicate some of the figures suggested by @farznaj as an additional validation check.

@jeromelecoq I'd be happy to discuss the results next Tuesday if that works.

[jeromelecoq]

Hi Nicholas, It is fine by me if you want to present next week. Should we add you to the schedule?

[Dedalus9]

Yes, do add me to the schedule.

[jeromelecoq]

Ok. I will dedicate the session to 2p data QC/analysis. If some folks in this thread want to contribute (in addition to Nicholas), please feel free to bring slides.

[farznaj]

Sounds good. Is the plan still that my students also work on QC? Asking so we avoid doing overlapping work and can optimize our efforts.

…

On Wed, May 27, 2026 at 9:21 AM Nicholas Adam Rodriguez < ***@***.***> wrote: Here's the link to my GitHub: https://github.com/Dedalus9/openscope-ophys-qc.git . I put together the GitHub for others to use. In it, there is a notebook for loading single or multiple sessions. I am currently working on quality control figures, extending this notebook. I am currently working on generating quality control figures, e.g., figures to track movement artifacts across sessions. I could also replicate some of the figures suggested by @farznaj <https://github.com/farznaj> to provide an additional check. @jeromelecoq <https://github.com/jeromelecoq> Perhaps I could discuss my results next Tuesday? — Reply to this email directly, view it on GitHub <#153?email_source=notifications&email_token=ADTJ7QVSOFU7FH3MC72QJJT443TXPA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZQG43DANJQUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-17076050>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADTJ7QR3DH2SY6U6A7APJZD443TXPAVCNFSM6AAAAACY24MJGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMBXGYYDKMA> . You are receiving this because you were mentioned.Message ID: <AllenNeuralDynamics/openscope-community-predictive-processing/repo-discussions/153/comments/17076050 @github.com>

[Dedalus9]

Yes, I will just not do any of what you listed in this comment: #153 (comment)

They are also welcome to make use of the GitHub repository I put together if they find it helpful.

Will they finish their QC analysis by next Tuesday? I'd like to present my results then if that's alright with @jeromelecoq. I could also present my analysis the week thereafter so to keep the topic of ophys quality control all on one day.

[farznaj]

great.
@davisgrubin see above

[jeromelecoq]

Hi Nicholas, It is fine by me if you want to present next week. Should we add you to the schedule?

[davisgrubin]

I should have some time later today to start putting together the figures, will update tomorrow afternoon if I think I'll have results to present Tuesday 6/2 if that sounds good?

[Dedalus9]

Yes, that's fine. Yes, you could add me to the schedule.

[davisgrubin]

Hi @Dedalus9 , I will take a look at your notebook and see what has already been done and what figures @farznaj recommended in her previous comment and see what the overlap / difference is. I was planning on doing those for next week's meeting as well, so probably good to make sure we aren't duplicating work?

[Dedalus9]

What were you planning on doing for next week's meeting? Also, I haven't included any QC figures of movement artifacts in the repo I shared #153 (comment). I'll include them later today. I was planning at looking at temporal synchronization, system integrity, motion artifacts, and interictal activity. If I have the bandwidth, I will also look at cross-session FOV alignment.

With respect to the plan mentioned here, #153 (comment), is the intention to assess whether signals are interpretable and when they are meaningful?

@Alishamsnia What are you planning on doing?

Perhaps we can make a checklist in this thread so none of our work is overlapping.

[davisgrubin]

I was planning on creating the figures that @farznaj mentioned here: #153 (comment)

[Dedalus9]

In that case, our QC analyses won't overlap.

[jeromelecoq]

Ok done *Jérôme*

…

On Fri, May 29, 2026 at 12:49 PM Nicholas Adam Rodriguez < ***@***.***> wrote: Yes, do add me to the schedule. — Reply to this email directly, view it on GitHub <#153?email_source=notifications&email_token=AATAHT3S7ORFFEGR4F6OQ2D45HSULA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRGA4TKMBRUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-17109501>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AATAHT2UYALEE26ZBHIIMQ345HSULAVCNFSM6AAAAACY24MJGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMJQHE2TAMI> . You are receiving this because you were mentioned.Message ID: <AllenNeuralDynamics/openscope-community-predictive-processing/repo-discussions/153/comments/17109501 @github.com>

[farznaj]

@Dedalus9 @davisgrubin
to decide on the SNR metric,

nicholas, could you share what you already have?

[Dedalus9]

I share my updated GitHub repository here: #156 (comment)

I think it would be better to have a dedicated thread for discussing the validation of this dataset.