Prompt_Learning_Paper_List

Review and Survey Paper

• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing, Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig, [Paper] [Project] [Github]

• Reasoning with Language Model Prompting: A Survey, Shuofei Qiao1,2∗, Yixin Ou1,2∗, Ningyu Zhang1,2†, Xiang Chen1,2, Yunzhi Yao,Shumin Deng4, Chuanqi Tan3, Fei Huang3, Huajun Chen, [Paper] [Github]

Year 2024

• MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba, Masakazu Yoshimura, Teruaki Hayashi, Yota Maeda [Paper]

• UniCell: Universal Cell Nucleus Classification via Prompt Learning, Junjia Huang, Haofeng Li, Xiang Wan, Guanbin Li [Paper] [Code]

• Spatio-temporal Prompting Network for Robust Video Feature Extraction, Guanxiong Sun, Chi Wang, Zhaoyu Zhang, Jiankang Deng, Stefanos Zafeiriou, Yang Hua [Paper] [Code]

• Revisiting the Power of Prompt for Visual Tuning, Yuzhu Wang, Lechao Cheng, Chaowei Fang, Dingwen Zhang, Manni Duan, Meng Wang [Paper]

• Bai, Yang, et al. "Sentence-level prompts benefit composed image retrieval." arXiv preprint arXiv:2310.05473 (2023). [Paper] [Code]

• Memory-Efficient Prompt Tuning for Incremental Histopathology Classification, Yu Zhu1,2*, Kang Li1*†, Lequan Yu3, Pheng-Ann Heng [Paper]

• [AAAI-2024] Concept-Guided Prompt Learning for Generalization in Vision-Language Models, Yi Zhang1,2, Ce Zhang3, Ke Yu2, Yushun Tang2, Zhihai He [Paper]

• Liu, Haotian, et al. "Visual instruction tuning." arXiv preprint arXiv:2304.08485 (2023). [Paper]

Year 2023

• Regressor-Segmenter Mutual Prompt Learning for Crowd Counting, Mingyue Guo, Li Yuan, Zhaoyi Yan, Binghui Chen, Yaowei Wang, Qixiang Ye, [Paper]

• Image Super-Resolution with Text Prompt Diffusion, Zheng Chen, Yulun Zhang, Jinjin Gu, Xin Yuan, Linghe Kong, Guihai Chen, Xiaokang Yang [Paper] [Code]

• [arXiv-2023] T-Rex: Counting by Visual Prompting, [Paper] [Code]

• [arXiv-2023] Visual In-Context Prompting, Feng Li, Qing Jiang, Hao Zhang, Tianhe Ren, Shilong Liu, Xueyan Zou, Huaizhe Xu, Hongyang Li, Chunyuan Li, Jianwei Yang, Lei Zhang, Jianfeng Gao [Paper] [Code]

• [arXiv-2023] Active Prompt Learning in Vision Language Models, [Paper]

• [arXiv-2023] Adversarial Prompt Tuning for Vision-Language Models, [Paper]

• [arXiv-2023] FreeKD: Knowledge Distillation via Semantic Frequency Prompt, [Paper]

• [ICCV-2023] Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Models, [Paper]

• Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editing, [Paper]

• Tuning Multi-mode Token-level Prompt Alignment across Modalities, Dongsheng Wang, Miaoge Li, Xinyang Liu, MingSheng Xu, Bo Chen [Paper] [Code]

• Prompt Learning for Action Recognition, Xijun Wang*, Ruiqi Xian*, Tianrui Guan, Dinesh Manocha [Paper]

• Fine-Grained Visual Prompting, Lingfeng Yang, Yueze Wang, Xiang Li, Xinlong Wang, Jian Yang [Paper]

• ProTeCt: Prompt Tuning for Hierarchical Consistency, Tz-Ying Wu, Chih-Hui Ho, Nuno Vasconcelos [Paper]

• Explicit Visual Prompting for Universal Foreground Segmentations, Weihuang Liu, Xi Shen, Chi-Man Pun, and Xiaodong Cun [Paper] [Code]

• LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-Tailed Multi-Label Visual Recognition, Peng Xia1, 2, 4 Di Xu5 Lie Ju1, 2, 3, * Ming Hu1, 2, 3 Jun Chen6 Zongyuan Ge [Paper] [Code]

• Deeply Coupled Cross-Modal Prompt Learning, [Paper] [Code]

• Do We Really Need a Large Number of Visual Prompts? [Paper]

• Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization, (ACL 2023) Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, Jimmy Ba and Amjad Almahairi. [Paper] [Code]

• Transfer Visual Prompt Generator across LLMs, Ao Zhang Hao Fei Yuan Yao Wei Ji Li Li Zhiyuan Liu Tat-Seng Chua [Paper] [Code]

• MVP-SEG: Multi-View Prompt Learning for Open-Vocabulary Semantic Segmentation, Jie Guo Qimeng Wang Yan Gao Xiaolong Jiang Xu Tang Yao Hu Baochang Zhang [Paper]

• Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models, Yaohua Zha, Jinpeng Wang, Tao Dai,*, Bin Chen, Zhi Wang, and Shu-Tao Xia [Paper] [Code]

• Chain of Thought Prompt Tuning for Vision-Language Models, Jiaxin Ge Hongyin Luo Siyuan Qian Yulu Gan Jie Fu [Paper]

• Progressive Visual Prompt Learning with Contrastive Feature Re-formation, Chen Xu, Haocheng Shen Fengyuan Shi Boheng Chen Yixuan Liao Xiaoxin Chen Limin Wang [Paper]

• Towards Robust Prompts on Vision-Language Models， Jindong Gu, Ahmad Beirami, Xuezhi Wang, Alex Beutel, Philip Torr, and Yao Qin [Paper]

Year 2022

• Guo, Jiaxian, et al. "From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models." arXiv preprint arXiv:2212.10846 (2022). [Paper] [Code]

• Conditional Prompt Learning for Vision-Language Models,Kaiyang Zhou, Jingkang Yang, Chen Change Loy, Ziwei Liu, CVPR2022 [Paper] [Code]

• Learning to Prompt for Vision-Language Models,Kaiyang Zhou, Jingkang Yang, Chen Change Loy, Ziwei Liu, IJCV2022 [Paper] [Code]

• Visual Prompt Tuning,Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, Ser-Nam Lim, ECCV2022 [Paper] [Code]

• Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models,Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, Chaowei Xiao [Paper] [Project] [Code]

• LPT: LONG-TAILED PROMPT TUNING FOR IMAGE CLASSIFICATION,Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo [Paper]

• MAPLE: MULTI-MODAL PROMPT LEARNING,Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, Fahad Shahbaz Khan [Paper] [Code]

• Texts as Images in Prompt Tuning for Multi-Label Image Recognition,Zixian Guo, Bowen Dong, Zhilong Ji, Jinfeng Bai, Yiwen Guo, Wangmeng Zuo [Paper] [Code]

• DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations,Ximeng Sun, Ping Hu, Kate Saenko, NIPS202 [Paper]

• CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning,James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla ,Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, Zsolt Kira1 [Paper]

• Unleashing the Power of Visual Prompting At the Pixel Level,Junyang Wu, Xianhang Li, Chen Wei, Huiyu Wang, Alan Yuille, Yuyin Zhou, Cihang Xie [Paper] [Code]

• PIVOT: Prompting for Video Continual Learning, Andrés Villa, Juan León Alcázar, Motasem Alfarra, Kumail Alhamoud, Julio Hurtado, Fabian Caba Heilbron, Alvaro Soto, Bernard Ghanem [Paper]

• Multitask Vision-Language Prompt Tuning, Sheng Shen†* Shijia Yang†∗ Tianjun Zhang†∗ Bohan Zhai [Paper] [Code]

• Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models, Chengcheng Ma et al. [Paper] [Code]

• Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination, Yue Yang et al. [Paper] [Code]

• A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models, [Paper] [Code]

• CLIP-Adapter: Better Vision-Language Models with Feature Adapters, [Paper] [Code]

• Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models, [Paper]

Year 2021

• CPT: COLORFUL PROMPT TUNING FOR PRE-TRAINED VISION-LANGUAGE MODELS, Yuan Yao, Ao Zhang, Zhengyan Zhang, Zhiyuan Liu, Tat-Seng Chua, Maosong Sun [Paper] [Code]

Year 2020

Year 2019