This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge column damage detection submitted to the IC-SHM Challenge 2021. The semantic segmentation framework used in this paper leverages importance sampling, semantic mask, and multi-scale test time augmentation to achieve a 0.836 IoU for scene component segmentation and a 0.467 IoU for concrete damage segmentation on the Tokaido Dataset. The framework was implemented on MMSegmentation using Python.

• HRNet
• Swin
• ResNest

• PSPNet
• UperNet
• OCRNet

The following table reports IoUs for structural component segmentation.

The following table reports IoUs for damage segmentation of pure texture images.

The following table reports IoUs for damage segmentation of real scene images.

The code is developed under the following configurations.

• Hardware: >= 2 GPUs for training, >= 1 GPU for testing. The script supports sbatch training and testing on computer clusters.
• Software:
  • System: Ubuntu 16.04.3 LTS
  • CUDA >= 10.1
• Dependencies:
  • Conda: This is optional, but we suggest using conda to configure the environment.
  • Pytorch >= 1.6.0
  • MMCV
  • MMSeg
  • OpenCV >= 4.5.0
  • tqdm

1. Install conda and create a conda environment

   $ conda create -n open-mmlab
   $ source activate open-mmlab
   $ conda install pytorch=1.6.0 torchvision cudatoolkit=10.1 -c pytorch

2. Install mmcv-full

   $ pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.6.0/index.html

3. Install mmsegmentation

   $ pip install git+https://github.com/open-mmlab/mmsegmentation.git

4. Install other dependencies

   $ pip install opencv, tqdm, numpy, scipy

5. Download the Tokaido dataset from IC-SHM Challenge 2021.

1. Example single model training using multiple GPUs

   $ python3 -m torch.distributed.launch --nproc_per_node=2 --nnodes=2 --master_port=$RANDOM ./apis/train_damage_real.py \
     --nw hrnet \
     --cp $CHECKPOINT_DIR \
     --dr $DATA_ROOT \
     --conf $MODEL_CONFIG \
     --bs 16 \
     --train_split $TRAIN_SPLIT_PATH \
     --val_split $VAL_SPLIT_PATH \
     --width 1920 \
     --height 1080 \
     --distributed \
     --iter 100000 \
     --log_iter 10000 \
     --eval_iter 10000 \
     --checkpoint_iter 10000 \
     --multi_loss \
     --ohem \
     --job_name dmg

2. Example shell script for preparing the whole dataset and train all models for the whole pipeline.

   $ ./scripts/main_training_script.sh

1. Eval one model

   $ python3 ./test/test.py \
     --nw hrnet \
     --task single \
     --cp $CONFIG_PATH \
     --dr $DATA_ROOT \
     --split_csv $RAW_CSV_PATH \
     --save_path $OUTPOUT_DIR \
     --img_dir $INPUT_IMG_DIR \
     --ann_dir $INPUT_GT_DIR \
     --split $TEST_SPLIT_PATH \
     --type cmp \
     --width 640 \
     --height 360

2. Example shell script for testing the whole pipeline and generate the output using the IC-SHM Challenge format.

   $ ./scripts/main_testing_script.sh

3. Visualization (Add the --cmp flag when visualizing components.)

   $ ./modules/viz_label.py \
     --input $SEG_DIR
     --output $OUTPUT_DIR
     --raw_input $IMG_DIR
     --cmp 

If you find the code useful, please cite the following paper.

Liu, J., Wei, Y., Chen, B. & Noh, H. (2023). A hierarchical semantic segmentation framework for computer vision-based bridge damage detection. SMART STRUCTURES AND SYSTEMS 31(4):325-334