Skip to content

tsm.md

Latest commit

 

History

History
111 lines (72 loc) · 3.52 KB

tsm.md

File metadata and controls

111 lines (72 loc) · 3.52 KB

简体中文 | English

TSM

Contents

Introduction

Temporal Shift Module (TSM) is a popular model that attracts more attention at present. The method of moving through channels greatly improves the utilization ability of temporal information without increasing any additional number of parameters and calculation amount. Moreover, due to its lightweight and efficient characteristics, it is very suitable for industrial landing.


This code implemented single RGB stream of TSM networks. Backbone is ResNet-50.

Please refer to the ICCV 2019 paper for details TSM: Temporal Shift Module for Efficient Video Understanding

Data

Please refer to K400 data download and preparation k400 data preparation

Please refer to UCF101 data download and preparation ucf101 data preparation

Train

download pretrain-model

wget https://videotag.bj.bcebos.com/PaddleVideo/PretrainModel/ResNet50_pretrain.pdparams

and add path to MODEL.framework.backbone.pretrained in config file as：

MODEL:
    framework: "Recognizer2D"
    backbone:
        name: "ResNet"
        pretrained: your weight path

Start training

You can start training with different dataset using different config file. For UCF-101 dataset, we use 4 cards to train:

python -B -m paddle.distributed.launch --gpus="0,1,2,3"  --log_dir=log_tsm  main.py  --validate -c configs/recognition/tsm/tsm.yaml

  • Args -c is used to specify config file.

  • For finetune please download our trained model TSM.pdparams，and specify file path with --weights.

  • For the config file usage，please refer to config.

Test

python3 main.py --test -c configs/recognition/tsm/tsm.yaml -w output/TSM/TSM_best.pdparams
  • Download the published model TSM.pdparams, then you need to set the --weights for model testing

Accuracy on Kinetics400:

seg_num target_size Top-1
8 224 0.7086

Inference

export inference model

To get model architecture file TSM.pdmodel and parameters file TSM.pdiparams, use:

python3 tools/export_model.py -c configs/recognition/tsm/tsm.yaml \
                              -p output/TSM/TSM_best.pdparams \
                              -o inference/TSM

infer

python3 tools/predict.py --video_file data/example.avi \
                         --model_file inference/TSM/TSM.pdmodel \
                         --params_file inference/TSM/TSM.pdiparams \
                         --use_gpu=True \
                         --use_tensorrt=False

Reference