Activity recognition

[sthoduka]

Develop a node to use a ML model to recognize activities from video

• Use this model with the pretrained weights for the charades dataset

• Class labels for the Charades dataset can be found in the file 'Charades_v1_classes.txt' here

The node should:

1. subscribe to a camera topic (for rgb images)
2. accumulate N frames (for this model it looks like they use 64 frames)
3. transform them as expected by the model (resized to 224x224 and reshaped to 3x64x224x224)
4. classify the sequence of frames
5. Repeat for next sequence of frames

[ElNandes]

Hi,
I am getting this error when i try to run the test file that you wrote before running this i had just installed opencv using "pip3 install opencv-python"

kinetic_user@HP-Ubuntu:/kinetic/pytorch-i3d$ python3 test.py
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    import torchvision.transforms as transforms
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torchvision/__init__.py", line 3, in <module>
    from torchvision import models
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torchvision/models/__init__.py", line 12, in <module>
    from . import detection
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torchvision/models/detection/__init__.py", line 1, in <module>
    from .faster_rcnn import *
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torchvision/models/detection/faster_rcnn.py", line 14, in <module>
    from .roi_heads import RoIHeads
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torchvision/models/detection/roi_heads.py", line 210, in <module>
    @torch.jit.script
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/__init__.py", line 1290, in script
    fn = torch._C._jit_script_compile(qualified_name, ast, _rcb, get_default_args(obj))
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/_recursive.py", line 568, in try_compile_fn
    return torch.jit.script(fn, _rcb=rcb)
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/__init__.py", line 1290, in script
    fn = torch._C._jit_script_compile(qualified_name, ast, _rcb, get_default_args(obj))
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/__init__.py", line 2030, in _get_overloads
    compiled_fns.append(_compile_function_with_overload(overload_fn, qual_name, obj))
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/__init__.py", line 2010, in _compile_function_with_overload
    overload_signature = torch.jit.annotations.get_signature(overload_fn, None, None, inspect.ismethod(overload_fn))
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/annotations.py", line 79, in get_signature
    signature = parse_type_line(type_line, rcb, loc)
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/annotations.py", line 165, in parse_type_line
    arg_types = [ann_to_type(ann, loc) for ann in arg_ann]
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/annotations.py", line 165, in <listcomp>
    arg_types = [ann_to_type(ann, loc) for ann in arg_ann]
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/annotations.py", line 303, in ann_to_type
    the_type = try_ann_to_type(ann, loc)
  File "/home/kinetic_user/.local/lib/python3.5/site-packages/torch/jit/annotations.py", line 296, in try_ann_to_type
    the_type = torch._C._resolve_type_from_object(ann, loc, fake_rcb)
TypeError: _resolve_type_from_object(): incompatible function arguments. The following argument types are supported:
    1. (arg0: object, arg1: torch._C._jit_tree_views.SourceRange, arg2: Callable[[str], function]) -> torch._C.Type

Invoked with: typing.Union[int, NoneType], None, <function try_ann_to_type.<locals>.fake_rcb at 0x7f61650e9b70>

[sthoduka]

try using torchvision 0.5.0 or 0.7.0 pip3 install torchvision==0.5
see: pytorch/vision#2132