-
Notifications
You must be signed in to change notification settings - Fork 0
adding a util function to calculate decaying reward-rate #65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
ed48236
d9cd2b0
4dabaa6
57f79de
ea1d60b
32922f3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| import numpy as np | ||
| from scipy.signal import fftconvolve | ||
|
|
||
| def reward_rate(event_times, | ||
| t_start=None, | ||
| t_end=None, | ||
| dt=1/20, # matching photometry frame rate | ||
| tau=25.0, # to be optimised | ||
| kernel="exp", # exp/hyperbolic | ||
| hyper_p=1.0, # only for hyperbolic | ||
| normalize_kernel=True): | ||
| """ | ||
| Compute a reward-rate timeseries by convolving discrete reward events with a decay kernel. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| event_times : (N,) array-like | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should be renamed reward_time if this is specific to reward_rate-- if we just want to convolve this to any type of rate, we should rename this function instead.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good point. when/if aversive component is integrated, we can do the same excersize to calculate punishment rate, and I was picturing it.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i would make it event_rate instead. it should be general, so long that the tau, hyper_p are not tuned for reward, but for the trial length
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. according to your comment above, the latter seems correct. can you provide documentation for the tau? or can you add a new issue to describe a function to map RT --> tau values? would be nice to add to this PR but we can keep this PR as it is also
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I haven't worked on optimizing tau using RT. I will require a lot of dataset and should be separated from this as it's biology (not a part of util-prep). |
||
| 1-D array of reward timestamps [s]. | ||
| t_start, t_end : float, optional | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. would there be a reason to change t_start/t_end to a different time? is there value including this as an input?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For the current usage, not really, but if you have multiple sessions in a recording and want to partially calculate RewRate, this is handy. A good option to have as default=None.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. keep this as it is-- i think we are going to consider multisession analysis in a different capsule. https://github.com/AllenNeuralDynamics/aind-dynamic-foraging-multisession-analysis |
||
| Output time range [s]. Defaults: t_start=0, t_end=max(event_times). | ||
| dt : float, default 0.001 | ||
| Time step of the output signal [s]. | ||
| tau : float, default 1.0 | ||
| Time constant of the kernel [s]. | ||
| kernel : {"exp", "hyperbolic"}, default "exp" | ||
| * "exp": k(t) = exp(-t / tau) (classic EWMA) | ||
| * "hyperbolic": k(t) = (1 + t / tau)^{-hyper_p} (hyperbolic discounting) | ||
| hyper_p : float, default 1.0 | ||
| Power *p* of the hyperbolic kernel. Ignored if kernel="exp". | ||
| normalize_kernel : bool, default True | ||
| If True, scale the kernel so its integral equals 1. | ||
|
|
||
| Returns | ||
| ------- | ||
| t : (T,) ndarray | ||
| Time axis [s]. | ||
| r_rate : (T,) ndarray | ||
| Reward rate (events · s⁻¹). | ||
|
|
||
| Notes | ||
| ----- | ||
| * Kernel is truncated at 5 × tau to keep computation fast. | ||
| * fftconvolve gives O(N log N) performance. | ||
| """ | ||
|
|
||
| event_times = np.asarray(event_times, dtype=float) | ||
| if event_times.ndim != 1: | ||
| raise ValueError("event_times must be a 1-D array of timestamps.") | ||
|
|
||
| # ------------------------------------------------------------------ | ||
| # Time axis: start at 0 so the series is zero until the first reward | ||
| # ------------------------------------------------------------------ | ||
| if t_start is None: | ||
| t_start = 0.0 | ||
| if t_end is None: | ||
| t_end = event_times.max() | ||
| t = np.arange(t_start, t_end + dt, dt) | ||
|
|
||
| # ------------------------------------------------------------------ | ||
| # Binary event series aligned to the timeline | ||
| # ------------------------------------------------------------------ | ||
| event_series = np.zeros_like(t) | ||
| idx = np.searchsorted(t, event_times - 1e-12) | ||
| idx = idx[(idx >= 0) & (idx < len(t))] | ||
| event_series[idx] = 1.0 | ||
|
|
||
| # ------------------------------------------------------------------ | ||
| # Build the decay kernel | ||
| # ------------------------------------------------------------------ | ||
| t_kernel = np.arange(0, 5 * tau, dt) | ||
|
|
||
| if kernel == "exp": | ||
| k = np.exp(-t_kernel / tau) | ||
|
|
||
| elif kernel == "hyperbolic": | ||
| if hyper_p <= 0: | ||
| raise ValueError("hyper_p must be > 0 for a valid hyperbolic kernel.") | ||
| k = (1.0 + t_kernel / tau) ** (-hyper_p) | ||
|
|
||
| else: | ||
| raise ValueError("kernel must be 'exp' or 'hyperbolic'.") | ||
|
|
||
| if normalize_kernel: | ||
| k /= k.sum() * dt # area = 1 | ||
|
|
||
| # ------------------------------------------------------------------ | ||
| # Convolution (truncate to match timeline length) | ||
| # ------------------------------------------------------------------ | ||
| r_rate = fftconvolve(event_series, k, mode="full")[:len(t)] | ||
|
|
||
| return t, r_rate | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
move these comments to the comment block below, so folks can see this when they run 'help' with the function