Learning Unified Representations for Multi-Resolution Face Recognition

This repository is the pytorch implementation of Learning Unified Representations for Multi-Resolution Face Recognition (accepted by BMVC 2023).

Branch-to-Trunk network (BTNet) is a representation learning method for multi-resolution face recognition. It consists of a trunk network (TNet), namely a unified encoder, and multiple branch networks (BNets), namely resolution adapters. As per the input, a resolution-specific BNet is used and the output are implanted as feature maps in the feature pyramid of TNet, at a layer with the same resolution.

Our code is based on InsightFace , CurricularFace and QMUL-SurvFace-Open-Set-Identification.

Main Requirements

• PyTorch (torch>=1.6.0)
• tensorboard
• easydict
• mxnet
• sklearn
• matplotlib
• opencv-python
• scikit-image

Prepare Datasets

• MS1MV3 (93k IDs, 5.2M images)
• IJB-C (3,531 IDs, 130k images)
• QMUL-SurvFace (15,573 IDs, 463.5k images)

Training

To train BTNet in the paper with 4 GPUs, run the following commands:

1. To train the trunk network:

# Resolution 112
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train_multi_reso.py configs/ms1mv3_r50_reso112

2. To train the branch networks:

# Resolution 28
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train_multi_reso_distill.py configs/ms1mv3_r50_reso28
# Resolution 14
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train_multi_reso_distill.py configs/ms1mv3_r50_reso14
# Resolution 7
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train_multi_reso_distill.py configs/ms1mv3_r50_reso7

3. (Optional) To finetune the trunk network on QMUL-SurvFace:

# Resolution 112
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train_multi_reso.py configs/surv_r50_reso112

4. (Optional) To finetune the branch networks on QMUL-SurvFace:

# Resolution 28
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train_multi_reso_distill.py configs/surv_r50_reso28
# Resolution 14
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train_multi_reso_distill.py configs/surv_r50_reso14
# Resolution 7
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train_multi_reso_distill.py configs/surv_r50_reso7

Evaluation

1. Multi-resolution identity matching (pairs <id1, id2>):

To evaluate BTNet on face verification benchmarks, run:

cd eval
# LFW, CALFW, CPLFW
python verification_multi_reso.py --target 'lfw,calfw,cplfw' --weight1 /path/to/model1 --weight2 /path/to/model2 --reso1 id1_resolution --reso2 id2_resolution --model_reso1 model1_input_resolution --model_reso2 model2_input_resolution --upsample1 0 --upsample2 0
# CFP-FF, CFP-FP, AgeDB_30
python verification_multi_reso.py --target 'cfp_ff,cfp_fp,agedb_30' --weight1 /path/to/model1 --weight2 /path/to/model2 --reso1 id1_resolution --reso2 id2_resolution --model_reso1 model1_input_resolution --model_reso2 model2_input_resolution --upsample1 0 --upsample2 0

To evaluate BTNet on QMUL-SurvFace 1:N face identification, run:

python eval_survface_1n_multi_reso.py --meta_path /path/to/QMUL-SurvFace/dataset --weight112 /path/to/TNet --weight28 /path/to/BNet28 --weight14 /path/to/BNet14 --weight7 /path/to/BNet7 --reso_indicator 'max' --branch_select 'ceil'

2. Multi-resolution feature aggregation (template <img1, img2>):

To evaluate BTNet on IJB-C 1:1 face verification, run:

python eval_ijbc_11_multi_reso.py --model-prefix1 /path/to/model1/ --model-prefix2 /path/to/model2 --reso1 img1_resolution --reso2 img2_resolution --model_reso1 model1_input_resolution --model_reso2 model2_input_resolution --upsample1 0 --upsample2 0

To evaluate BTNet on IJB-C 1:N face identification, run:

python eval_ijbc_1n_multi_reso.py --model-prefix1 /path/to/model1/ --model-prefix2 /path/to/model2 --reso1 img1_resolution --reso2 img2_resolution --model_reso1 model1_input_resolution --model_reso2 model2_input_resolution --upsample1 0 --upsample2 0

Pre-trained Models

You can download BTNet pretrained on MS1MV3 here:

• Baidu Yun Pan: lnu4

You can download BTNet finetuned on QMUL-SurvFace here:

• Baidu Yun Pan: 5z75

##Results BTNet achieves the following performances:

(a) Face verification benchmarks and IJB-C

Test setting	Face verification benchmarks	IJB-C 1:1 verification (1e-4)	IJB-C 1:N identification (0.1)
112&7	86.10	88.17	83.93
112&14	94.08	93.97	91.87
112&28	96.65	95.62	94.33
7&7	77.78	35.47	27.70
14&14	90.90	82.08	76.65
28&28	96.27	94.50	92.89
112&112	97.25	96.06	94.85

(b) QUML-SurvFace

Method	AUC	TPIR20(0.3)	TPIR20(0.2)	TPIR20(0.1)	TPIR20(0.01)
avg.+floor	32.6	27.9	23.4	16.5	1.4
avg.+near	34.6	30.3	25.7	18.9	1.5
avg.+ceil	35.4	31.1	26.8	20.3	2.2
min+floor	32.3	27.6	23.2	16.1	1.4
min+near	34.0	29.6	25.0	18.0	1.4
min+ceil	35.3	31.0	26.6	19.9	2.0
max+floor	33.6	29.1	24.5	17.6	1.3
max+near	35.2	31.0	26.4	19.6	1.7
max+ceil	35.4	31.2	26.9	20.6	2.5

Contributing

We appreciate all contributions to improve the multi-resolution face recognition model zoo of BTNet.

All content in this repository is licensed under the MIT license.