Created by Yizhak Ben-Shabat (Itzik)1, 2, Oren Shrout1 and Stephen Gould2
This is the code for the 3DInAction paper for 3D point cloud action recognition. It allows to train, test and evaluate the tasks of per-frame and per-clip action classification.
Please follow the installation instructions below.
The code was tested with python 3.8.16 torch 1.10.1 and CUDA 11.3.
sh
conda create -n tpatches_env python=3.8.16
conda activate tpatches_env
conda install pip #for using pip commands in the conda environments
# Install with instructions from https://pytorch.org/get-started/locally/
# Below is instructions for installation of long term support
conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge
#Then install all other requirements
pip install -r requirements.txt
Then compile the pointnet2 modules for FPS:
cd ./models
python setup.py install
We use wandb
for tracking our experiments (losses accuracy etc.)
so you may need to either set up an account or comment out the logging code lines.
We evaluate on three datasets:
- DFAUST (~3.4GB)
- IKEA ASM (~117GB)
- MSR-Action3D FPS (200MB)
Download the datasets, extract the .zip
file and update the dataset_path
in the .config
file under DATA
.
When using these datasets, make sure to cite their papers DFAUST, IKEA ASM, MSR-Action3D.
To train, test and evaluate with the default settings run
sh run_experiment.sh
For a customized model, edit the .config
file.
Examples for different configurations are available in the configs
directory.
This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 893465. We also thank the NVIDIA Academic Hardware Grant Program for providing high-speed A5000 GPU
This paper was accepted to CVPR 2024 (the citation will be updated once CVF makes the paper public).
If you find our work useful in your research, please cite our paper:
{
@article{benshabat2023tpatches,
title={3DInAction: Understanding Human Actions in 3D Point Clouds},
author={Ben-Shabat, Yizhak and Shrout, Oren and Gould, Stephen},
journal={arXiv preprint arXiv:2303.06346},
year={2023}
}
See LICENSE file.