Bugfixes and Updates (LARGE)

--KEY THINGS--
* `nigeLab.libs.DiskData` re-work
  + Now, clear reason for three obj.type_ categories:
      - 'MatFile' :  Files that should not grow or shrink once they have been allocated on the disk (such as from the already-known number of samples in the record)
     - 'Hybrid': "Vector" style file that can be expanded along columns. This should be used for data streams such as RawData or DigIO.
  - 'Event': "Matrix" style file that can be expanded along rows. This should be used for things like 'Spikes' or 'ScoredEvents'.
  + Split `subsref` and `subsasgn` into more modular design to make it clearer what logic is in those methods.
  + Initialization for all three types of input arguments (single, two inputs, or three inputs) should all work now (previously, it didn't seem the single-input case worked).
  + Now no longer creates a matfile (.diskfile_); instead, uses hdf5 handling for everything.
* Bug with doRawExtraction should be resolved
  + May need to re-run extraction for STREAMS (it was incorrectly using 8-bit values that resulted in empty streams which otherwise contained data)

-----------------------------
* `nigeLab.defaults`
  + Add flag to skip UI Prompt for videos in `+defaults/Videos`
  + Update `+defaults/doActions` so each struct field that corresponds to a `doMethod` has a third field: `batch`
     - This field is checked for example if it is expected that many important Blocks may need to have a particular field (such as .Videos) but that not every Block in a Tank is expected to have it. Instead of throwing an error, `checkActionIsValid` will return a flag as false if the .Status of .Fields listed in .batch are not completed.

* `nigeLab.libs`
  + Update .empty() methods of `.VidStreamsType` and `.VideosFieldType`
  + `nigeLab.libs.VidStreamsType` and `nigeLab.libs.VideosFieldType` should now initialize correctly.
     - Unfortunately, once they started initializing correctly, it caused the _Obj.mat files to expand in size by an order of magnitude. Re-worked how data pointers are handled for these classes, so they should no longer be as inefficient

* `nigeLab.Block`
  + Update a bunch of the Block methods that referenced deprecated parameters fields (`VideoPars` and `SortPars`).
  + Remove deprecated methods:
     - `nigeLab.Block/getFileType()`
     - `nigeLab.Block/getFieldType()`
     - `nigeLab.Block/getFieldTypeIndex()`
  + Corresponding methods are implemented in `nigeLab.nigelObj`
  + `nigelab.Block/doVidInfoExtraction` is now working more-or-less as intended.
  + `nigeLab.Block/getStatus()` this method is a mess and needs to be moved to `nigelObj` soon. Basically, we want it to behave the same way when we call it at all levels otherwise it gets too confusing.
       - For now, just put some shitty code in there to make it work for the time-being.

* `nigeLab.Tank`
  + Add `varargin` to `nigeLab.Tank/runFun()` method, which allows it to supply input

Additions
* `nigeLab.nigelObj/getFieldInfo()` and `nigeLab.libs.nigelButtonArray`
  + Were supposed to work in combination to make mouse-over and clickable things happen for DashBoard but I think it's too messy and didn't want to waste time on it. Regardless, they exist (and that's why).
* Added `.empty` methods to a few other Classes so their validation doesn't screw up other working classes.

Update

Update

* Bugfix again on doRawExtraction
* Add parsing from Buccelli & Murphy 2019 FSM to Intan RHS header parse

Update

* Fix initEvents
* Add RHS parsing for intan2Block for modified FSM thing
* Fix doEventDetection and doEventHeaderExtraction

Author: m053m716

+nigeLab/+defaults/Block.m

@@ -136,7 +136,7 @@
    pars.Delimiter 'Stream.mat'];
 TAG.Videos = ... % Videos: behavioral videos
    [pars.Delimiter '%s', ...
-    pars.Delimiter '%g.%s']; % "Video_Left-A_0.mp4" "Video_Left-A_1.mp4"
+    pars.Delimiter '%s.%s']; % "Video_Left-A_0.mp4" "Video_Left-A_1.mp4"
 
 Fields =  { ...
    'Raw';            % 1  - hard-coded for extraction

+nigeLab/+defaults/Event.m

@@ -33,9 +33,9 @@
 %    'Complete';        % 16)
 %    };        
 pars.Name = {...    % Example B (RHD)
-   'trialrunning';    % 1) Trial running "HIGH" events
-   'beambreak';       % 2) Beam break events
-   'nosepoke';        % 3) Nose-poke beam break
+   'trial-running';    % 1) Trial running "HIGH" events
+   'beam-break';       % 2) Beam break events
+   'nose-poke';        % 3) Nose-poke beam break
    'Reach';           % 4) Reach scored onset
    'Grasp';           % 5) Grasp scored onset
    'Support';         % 6) Support scored onset
@@ -138,18 +138,37 @@
 %    'Falling';   % 11)
 %    'Level';     % 12) (skip 13-16 because not 'auto' field)
 %    };
+% pars.EventSource = {... % Example A (RHS)
+%    'Channels';   % 1) 'Stim' is associated with channels
+%    'Streams';    % 2) 
+%    'Streams';    % 3)
+%    'Streams';    % 4)
+%    'Streams';    % 5)
+%    'Streams';    % 6)
+%    'Streams';    % 7)
+%    'Streams';    % 8)
+%    'Streams';    % 9)
+%    'Streams';    % 10)
+%    'Streams';    % 11)
+%    'Streams';    % 12) (skip 13-16 because not 'auto' field)
+%    };
 pars.TrialDetectionInfo = struct(... % For sync using LED (Example B)
    'Field','DigIO',...
-   'Name','trialrunning',...
+   'Name','trial-running',...
    'Source',[],...
-   'Debounce',0.250,...
-   'Threshold',0.5,...
+   'Debounce',0.100,...% Used in parsing other 'auto' events as well
+   'Threshold',0.5,... % Used in parsing other 'auto' events as well
    'Type','Rising');
 pars.EventDetectionType = {... % Example B (RHD)
    'Rising';    % 1)
    'Rising';    % 2) 
    'Rising';    % 3) (skip 4-7 because not 'auto' fields)
    };
+pars.EventSource = {...
+   'Streams';    % 1)
+   'Streams';    % 2) 
+   'Streams';    % 3) (skip 4-7 because not 'auto' fields)
+   };
 
 %% Error parsing (do not change)
 % Check that number of elements of Name matches that of Fields

+nigeLab/+defaults/Queue.m

@@ -25,7 +25,8 @@
 pars.Remote.RecRoot = 'P:/Recorded_Data'; 
 pars.Remote.SaveRoot = 'P:/Processed_Data';
 
-pars.ClusterList = {'CPLMJS'; 'CPLMJS2'; 'CPLMJS3'};
+% pars.ClusterList = {'CPLMJS';'CPLMJS2';'CPLMJS3'};
+pars.ClusterList = {'CPLMJS';'CPLMJS2'};
 pars.NWorkerMinMax = [1,1]; % Min & Max # workers to assign to a job
 pars.WaitTimeSec = 1; % Time to wait between checking for new cluster
 pars.InitTimeSec = 5; % Time to wait when initializing cluster

+nigeLab/+defaults/Video.m

@@ -80,6 +80,7 @@
 pars = struct;
 pars.HasVideo = true;
 pars.HasVidStreams = true;
+pars.UseVideoPromptOnEmpty = false;
 
 % % % -- For Video Scoring -- % % %
 
@@ -160,8 +161,8 @@
 % to toggle it.
 
 pars.VidFilePath    = { ... % "Includes" for where videos might be. Stops after first non-empty path.
-   'K:\Rat\Video\BilateralReach\Murphy'; 
-   'K:\Rat\Video\BilateralReach\RC';
+...   'K:\Rat\Video\BilateralReach\Murphy'; 
+...   'K:\Rat\Video\BilateralReach\RC';
    'K:\Rat\Video\Audio Discrimination Task\post-surg'
    };
 
@@ -172,12 +173,14 @@
                            '*_0.MP4','First Video Only';...
                            '*.MP4;*.mp4;*.avi','Video Files (*.mp4,*.avi)';...
                            '*.*','All Files (*.*)'};  
-pars.SelectionUITitle = 'No Videos Found - Select VIDEO associated with BLOCK';
-
 pars.FileExt = '.MP4';
-% For DynamicVars expressions, see 'Metadata parsing' below
-pars.DynamicVars = {'$AnimalID','$Year','$Month','$Day','$SessionID','$View','$MovieID'}; % KUMC: "Murphy"
-% pars.DynamicVars = {'$AnimalID','$Year','$Month','$Day','&MovieID'}; % KUMC: "RC"
+% For DynamicVars expressions, see 'Metadata parsing' below; main
+% difference here is that ALL variables (if there are enough tokens) are
+% included as metadata variables; '$' vs '~' only denotes whether to use
+% that particular variable in figuring out other videos belonging to a
+% given recording.
+pars.DynamicVars = {'$AnimalID','$Year','$Month','$Day','$SessionID','~View','~MovieID'}; % KUMC: "Murphy"
+% pars.DynamicVars = {'$AnimalID','$Year','$Month','$Day','~MovieID'}; % KUMC: "RC"
 pars.MovieIndexVar = 'MovieID'; % KUMC: "RC" (and in general)
 
 % Information about video scoring
@@ -197,14 +200,10 @@
 pars.Alignment_FS = struct('TDT',125,'RHD',100,'RHS',100);
 
 %% Less-likely to change these parameters
-% Paths information
-pars.File = [];
-
 % Metadata parsing
 pars.Delimiter = '_'; % Break filename "variables" by this
 pars.IncludeChar = '$'; % Include data from these variables
 pars.ExcludeChar = '~'; % Exclude data from these variables
-pars.Meta = [];
 
 pars.ValueShortcutFcn = @nigeLab.workflow.defaultVideoScoringShortcutFcn;
 pars.VideoScoringStringsFcn = @nigeLab.workflow.defaultVideoScoringStrings;
@@ -219,17 +218,35 @@
    (~isempty(pars.VidStreamName));
 
 if pars.HasVidStreams
+   dyVar = cellfun(@(x)x(2:end),pars.DynamicVars,'UniformOutput',false);
+   if ismember('Key',dyVar)
+      error(['nigeLab:' mfilename ':BadConfig'],...
+         '[DEFAULTS/VIDEO]: Invalid use of reserved meta variable (''Key'')');
+   end
+   
+   % Check that the CameraSourceVar and MovieIndexVar were added to `pars`
+   if ~isfield(pars,'CameraSourceVar')
+      error(['nigeLab:' mfilename ':BadConfig'],...
+         ['CameraSourceVar parameter field is missing\n' ...
+         '->\t(can be left as [] if unwanted)']);
+   end
+   if ~isfield(pars,'MovieIndexVar')
+      error(['nigeLab:' mfilename ':BadConfig'],...
+         ['MovieIndexVar parameter field is missing\n' ...
+         '->\t(can be left as [] if unwanted)']);
+   end
+   
    n = numel(pars.VidStreamName);
    if ~isempty(pars.CameraSourceVar)
       pars.VidStreamField = cell(n,1);
       pars.VidStreamFieldType = cell(n,1);
 
       % Check that the CameraSourceVar is good
-      dyVar = cellfun(@(x)x(2:end),pars.DynamicVars,'UniformOutput',false);
       idx = ismember(dyVar,pars.CameraSourceVar);
       if sum(idx) ~= 1
-         error('CameraSourceVar (%s) is not a member of pars.DynamicVars (%s)',...
-            pars.CameraSourceVar,dyVar);
+         error(['nigeLab:' mfilename ':BadConfig'],...
+            'CameraSourceVar (%s) is not a member of pars.DynamicVars',...
+            pars.CameraSourceVar);
       end
 
       % For each Cell corresponding to a different subset of markings,

+nigeLab/+defaults/doActions.m

@@ -18,16 +18,16 @@
 %% Set "dependencies" for checking that stages are valid for `doMethods`
 pars = struct;
 pars.doAutoClustering = doAction({'Spikes'},true);
-pars.doBehaviorSync= doAction({'Video'});
-pars.doEventDetection = doAction();
-pars.doEventHeaderExtraction = doAction({'Video'});
+pars.doBehaviorSync= doAction({},false,{'Raw','Video'});
+pars.doEventDetection = doAction({'Raw'},true);
+pars.doEventHeaderExtraction = doAction({},false,{'Raw','Video'});
 pars.doLFPExtraction = doAction({'Raw'},true);
 pars.doRawExtraction = doAction({},true);
 pars.doReReference = doAction({'Filt'},true);
 pars.doSD = doAction({'CAR'},true);
 pars.doUnitFilter = doAction({'Raw'},true);
-pars.doVidInfoExtraction = doAction({'Video'});
-pars.doVidSyncExtraction = doAction({'Video'});
+pars.doVidInfoExtraction = doAction({},true,{});
+pars.doVidSyncExtraction = doAction({},false,{'Raw','Video'});
 
 %% Parse output
 if nargin < 1
@@ -44,7 +44,7 @@
 end
 
 % Helper function to make `doAction` param struct
-   function doStruct = doAction(required,en)
+   function doStruct = doAction(required,en,to_check_on_batch_run)
       %DOACTION  Helper function to create doAction struct
       %
       %  doStruct = doAction({'field1',...,'fieldk'}); Sets required fields
@@ -58,9 +58,20 @@
          en = false;
       end
 
+      if nargin < 3
+         to_check_on_batch_run = {};
+      end
+      
+      if ~isempty(to_check_on_batch_run) && ~isempty(required)
+         error(['nigeLab:' mfilename ':BadConfig'],...
+            '[DOACTIONS]: If .batch has members, .required should be empty.');
+      end
+      
       doStruct = struct;
       doStruct.required = required;
       doStruct.enabled = en;
+      doStruct.batch = to_check_on_batch_run;
+      
    end
 
 end