[Feature request] Conversion to NWB

[tuanpham96]

Since many neuroscience data are being converted into NWB format, either directly or via extension (e.g. suite2p output, DLC output, ...).

I don't use ArControl personally but a person in the lab uses it. After skimming the .aconf file and the outputs .txt/.mat files, I recognize some similarity between the behavior state framework with BEADL from a naive understanding. There's already an existing extension to convert BEADL output to NWB called ndx-beadl. Though it was developed for BEADL, it seems that the extension may be used more generally, based on ndx-beadl README.

The extension ndx_beadl defines a collection of interlinked table data structures for storing behavioral tasks and data. While the extension has been designed with BEADL in mind, the data structures are general and are intended to be useful even without BEADL.

I want to ask whether there's any plan to convert output of ArControl to NWB, maybe using ndx-beadl as a starting point.

Thanks!

[chenxinfeng4]

Thanks for your interest in ArControl. The standardization of neuroscience data remains a challenge. Thanks for your mention about the NWB project, which was great at my short glance. I will try it.

The BEADL has many common with ArControl. They both share the state machine concept and use XML to describe the behavioral tasks. However, the ArControl is a software and hardware combined solution, I wonder how BEADL software will work on Arduino board. That may take time for BEADL to grow stronger. I hope the BEADL will open source soon.

I will try to convert the behavior data to the NWB format. Since I'm a newbie in NWB, please check whether the code work for you. Would you like to send a piece of demo ArControl data and NWB data file (behavior data embedded) in my e-mail? chenxinfeng@pku.edu.cn

[chenxinfeng4]

Conversion to NWB, done. See the (https://github.com/chenxinfeng4/ArControl-convert2-nwb) for detail.

pip install -U git+https://github.com/chenxinfeng4/ArControl-convert2-nwb
python -m arcontrol2nwb "Arcontrol/data/A/B/2022-1113-224711.txt"

[tuanpham96]

Sorry I had other things on my plate and haven't been able to get back to you.

Thank you so much for taking the time to develop that! I'll play around with that and tell you how it goes. I will send you a sample data from my labmate who uses ArControl over email.

I'll play around and make more code/data-relevant comments by raising issues on that repository. But 2 notes I'd have from a quick glance:

• The variable names are not very intuitive for people new to the dataset/experiments to look at, maybe switch swap description and name?
  • Of course, this runs into issue of name conflict because sometimes there could be multiple Go Cues events in different components or states. Not sure how to solve this.
  • On the same note, how easy is it that users can include metadata information about each state inside ArControl itself. For example, in addition to comments about what each of them is, maybe even a preferred variable label in output?
• This is more general, but do you think it is possible or even a good idea to somehow include information about the task structure, which may be derived from the .aconf files?
  • As in, the connection between the different timeseries objects, i.e. how C1S1 is designed to lead to C1S2
  • This might/might not be helpful for (1) future references of how the experiments were designed, especially when people leave and (2) people who new to the datasets either because of new collaboration or the datasets are available on public open repositories like dandi

[chenxinfeng4]

It's a good habit to give every INPUT/OUTPUT/STATE a specific & unique name. But, not all people like do that. Actually in my experments, I just pick some important INPUT/OUTPUT/STATE to name, and left others blank.

Maybe you can use Go Cues to inversely search proper STATE id C1S1. I'm considering to append event_name (Go Cue) aside event (C1S1) in next release. However, that would be too redundancy.
That will be great if I can find the way to include metadata information.

[chenxinfeng4]

This is more general, but do you think it is possible or even a good idea to somehow include information about the task structure, which may be derived from the .aconf files?

Convert .aconf to general task structure is possible but not fascinating. There's no standard task structure to refer. And some dynamic feature of ArControl task cannot be easily standaridized. I want to ArControl Designer is simple and efficient. Try to get familiar with ArControl Designer.

[chenxinfeng4]

future references of how the experiments were designed, especially when people leave

The .aconf can be shared/transmition with other computers. (But remember to open&re-save it in first transmition before you can use it). I think people may draw some simple manuscripts to explicate how the task is aranged.

people who new to the datasets either because of new collaboration or the datasets are available on public open repositories like dandi

ArControl output is raw hardware event data. It cann't automaticlly tell you per stimulus time histogram in the go trial, go & hit, go & miss, nogo trials. You need to calculate and append these infomation to the nbw file on your own, so that the new to the datasets will get your point.