HOI-Learning-List

Some recent (2015-now) Human-Object Interaction Learing studies. If you find any errors or problems, please feel free to comment.

Dataset

• HAKE (CVPR2020) [Website] [Paper]

• Ambiguous-HOI (CVPR2020) [Website] [Paper]

• HICO-DET (WACV2018) [Website] [Paper]

• HCVRD (AAAI2018) [Website] [Paper]

• V-COCO (May 2015) [Website] [Paper]

• HICO (ICCV2015) [Website] [Paper]

• OpenImage [Website] [Paper]

• PIC [Website]

More...

Method

HOI Recognition: Image-based, to recognize all the HOIs in one image.

• PaStaNet (CVPR2020) [Code] [Data] [Paper]

• Pairwise (ECCV2018) [Paper]

• Attentional Pooling for Action Recognition (NIPS2017) [Code] [Paper]

• Learning Models for Actions and Person-Object Interactions with Transfer to Question Answering (ECCV2016) [Code] [Paper]

• Contextual Action Recognition with R*CNN (ICCV2015) [Code] [Paper]

• HOCNN (ICCV2015) [Code] [Paper]

More...

HOI Detection: Instance-based, to detect the human-object pairs and classify the interactions.

• PaStaNet (CVPR2020) [Code] [Data] [Paper]

• DJ-RN (CVPR2020) [Code] [Paper]

• Cascaded Human-Object Interaction Recognition (CVPR2020) [Code] [Paper]

• PPDM (CVPR2020) [Code] [Paper]

• IP-Net (CVPR2020) [Code] [Paper]

• VSGNet (CVPR2020) [Code] [Paper]

• HOID (CVPR2020) [Paper]

• PMFNet(ICCV2019) [Code] [Paper]

• No-Frills (ICCV2019) [Code] [Paper]

• Analogy (ICCV2019) [Code] [Paper]

• RPNN (ICCV2019) [Paper]

• Deep Contextual Attention for Human-Object Interaction Detection (ICCV2019) [Paper]

• Interactiveness (CVPR2019) [Code] [Paper]

• GPNN (ECCV2018) [Code] [Paper]

• iCAN (BMVC2018) [Code] [Paper]

• InteractNet (CVPR2018) [Paper]

• Scaling Human-Object Interaction Recognition through Zero-Shot Learning (WACV2018) [Paper]

• HO-RCNN (WACV2018) [Code] [Paper]

• VS-GATs (Mar 2020) [Paper]

• Classifying All Interacting Pairs in a Single Shot (Jan 2020) [Paper]

• Deep Contextual Attention for Human-Object Interaction Detection (Oct 2019) [Paper]

More...

Result

HICO-DET:

1) Detector: COCO pre-trained

Method	Pub	Full(def)	Rare(def)	None-Rare(def)	Full(ko)	Rare(ko)	None-Rare(ko)
Shen et al.	WACV2018	6.46	4.24	7.12	-	-	-
HO-RCNN	WACV2018	7.81	5.37	8.54	10.41	8.94	10.85
InteractNet	CVPR2018	9.94	7.16	10.77	-	-	-
GPNN	ECCV2018	13.11	9.34	14.23	-	-	-
Xu et. al	ICCV2019	14.70	13.26	15.13	-	-	-
iCAN	BMVC2018	14.84	10.45	16.15	16.26	11.33	17.73
Wang et. al.	ICCV2019	16.24	11.16	17.75	17.73	12.78	19.21
Functional (suppl)	AAAI2020	16.96	11.73	18.52	-	-	-
Interactiveness	CVPR2019	17.03	13.42	18.11	19.17	15.51	20.26
No-Frills	ICCV2019	17.18	12.17	18.68	-	-	-
RPNN	ICCV2019	17.35	12.78	18.71	-	-	-
PMFNet	ICCV2019	17.46	15.65	18.00	20.34	17.47	21.20
Interactiveness-optimized	CVPR2019	17.54	13.80	18.65	19.75	15.70	20.96
HOID	-	17.85	12.85	19.34	-	-	-
Julia et al.	ICCV2019	19.40	14.60	20.90	-	-	-
IP-Net	CVPR2020	19.56	12.79	21.58	22.05	15.77	23.92
VS-GATs	arXiv	19.66	15.79	20.81	-	-	-
VSGNet	CVPR2020	19.80	16.05	20.91	-	-	-
DJ-RN	CVPR2020	21.34	18.53	22.18	23.69	20.64	24.60

2) Detector: pre-trained on COCO, fine-tuned on HICO-DET train set (with GT human-object pair boxes)

Method	Pub	Full(def)	Rare(def)	None-Rare(def)	Full(ko)	Rare(ko)	None-Rare(ko)
PPDM (one-stage, paper)	CVPR2020	21.10	14.46	23.09	-	-	-
PPDM (one-stage, github-hourglass104)	CVPR2020	21.73/21.94	13.78/13.97	24.10/24.32	24.58/24.81	16.65/17.09	26.84/27.12
Functional	AAAI2020	21.96	16.43	23.62	-	-	-

3) Enhanced with HAKE:

Method	Pub	Full(def)	Rare(def)	None-Rare(def)	Full(ko)	Rare(ko)	None-Rare(ko)
iCAN	BMVC2018	14.84	10.45	16.15	16.26	11.33	17.73
iCAN+HAKE-HICO-DET	CVPR2020	19.61	17.29	20.30	22.10	20.46	22.59
Interactiveness	CVPR2019	17.03	13.42	18.11	19.17	15.51	20.26
Interactiveness+HAKE-HICO-DET	CVPR2020	22.12	20.19	22.69	24.06	22.19	24.62
Interactiveness+HAKE-Large	CVPR2020	22.66	21.17	23.09	24.53	23.00	24.99

Ambiguous-HOI

Detector: COCO pre-trained

Method	mAP
iCAN	8.14
Interactiveness	8.22
Julia et al.(reproduced)	9.72
DJ-RN	10.37

V-COCO: Scenario1

1) Detector: COCO pre-trained

Method	Pub	AP(role)
Gupta et al.	arXiv	31.8
InteractNet	CVPR2018	40.0
GPNN	ECCV2018	44.0
iCAN	BMVC2018	45.3
Xu et. al	CVPR2019	45.9
Wang et. al.	ICCV2019	47.3
Interactiveness	CVPR2019	47.8
Zhou et. al.	CVPR2020	48.9
Interactiveness-optimized	CVPR2019	49.0
IP-Net	CVPR2020	51.0
VSGNet	CVPR2020	51.8
PMFNet	ICCV2019	52.0

2) Enhanced with HAKE:

Method	Pub	AP(role)
iCAN	CVPR2019	45.3
iCAN+HAKE-Large (transfer learning)	CVPR2020	49.2
Interactiveness	CVPR2019	47.8
Interactiveness+HAKE-Large (transfer learning)	CVPR2020	51.0

HICO

1) Default

Method	mAP
R*CNN	28.5
Girdhar et.al.	34.6
Mallya et.al.	36.1
Pairwise	39.9

2) Enhanced with HAKE:

Method	mAP
Mallya et.al.	36.1
Mallya et.al.+HAKE-HICO	45.0
Pairwise	39.9
Pairwise+HAKE-HICO	45.9
Pairwise+HAKE-Large	46.3