Search within date range

[Akseli-Ilmanen]

Hi,

Currently, one can search for multiple sessions using the @to@ by looping through session numbers.

A cool feature would it be if I can specify a date-range, and then download/upload folders with a date within that range. For example, I could specify 20250306@TO@20250309, and get all folders from below except the last one.

Ideally, this does not require me to have folders for every date range. E.g. I could specify all dates in March, and it would return all the folders in that month even if I did not do recordings every day.

Thanks!

Best
Akseli

[JoeZiminski]

Hi @Akseli-Ilmanen, thanks for this suggestion, this is a really nice idea! Maybe we can have a @DATETO@ tag for this purpose. I'll leave some notes here on the implementation to come back to ASAP but it might take a month or so (am currently on leave). Of course any contributions are very welcome so please feel free to raise a PR for this issue. Thanks for the idea!

Notes on implementation:

Currently there are two cases for the tags e.g. @TO@ or @*@, for folder creation, restricting transfers, or both. The @TO@ tag operates directly on a passed list of names by expanding the list to include all subjects / sessions within the range. This works for both creating folders and transferring folders. In the @DATETO@ range, (I think?) it makes sense that this is only for restricting transfers, as I doubt it would be required to create a long set of folders between two dates. This makes it similar to the wildcard @*@ tag in terms of implementation i.e. it should be used only for data transfers.

The wildcard searches happen in this function. This just does a straight string replace of @*@ to * for globing based on filenames. I think the easiest implementation will be to expand this function to handle the @DATETO@ case as well. Rather than playing around with the lower-level search functions, here the <number>@DATETO@<number> could be replaced with date-*. This would grab all names with the date- key, and then names not within range stripped from the resulting list of names using pythons datetime functions and the datashuttle utility get_values_from_bids_formatted_name. There would have to be a fair amount of checking (e.g. that date does not already appear in the search) and would need to handle @TIMETO@ and @DATETIMETO@ also but that should be fairly straightforward.

[Akseli-Ilmanen]

Hi, @JoeZiminski thanks for your reply. Yes, this sound great. I am fairly new to coding, so I think I won't be able raise the PR myself. Yes, the function would only make sense for data transfer, not for folder creation.

Thanks! :)

[Planeshiffhter]

Hi @JoeZiminski, I'd like to start working on this. Could you suggest which files need to be changed instead of folder.py so I can better understand the issue?

[Diya910]

Hi @JoeZiminski, I have made changes to the function according to my understanding. I have also tested the function locally. Would love to dive deeper into how I can improve the code. Your reviews and guidance on the PR will be very helpful.
Thankyou

[SKM2227229725]

I’m interested in helping with the @Dateto@ functionality improvements or related enhancements.

I’ll start by reviewing PR #510 and the search_for_wildcards implementation to understand the current approach.
If there are remaining gaps or follow-up tasks needed for v1.0.0, I’d be happy to help.

[JoeZiminski]

Hi @SKM2227229725, thanks for looking into this, this has already been completed in #510, but please let me know if there are other issues you would be intereted in working on. On GitHub, on the right hand side there is a panel 'Development' which contains PRs that reference this issue, usually they complete the linked issue.