Literature for Deep Learning Interpretability

Papers

Year	Conf.	Paper	Citation	Code
2020	CVPR	Explaining Knowledge Distillation by Quantifying the Knowledge	0
2020	ICLR	Knowledge Isomorphism between Neural Networks	0
2020	ICLR	Interpretable Complex-Valued Neural Networks for Privacy Protection	2
2019	AI	Explanation in artificial intelligence: Insights from the social sciences	552
2019	NMI	Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead	192
2019	NeurIPS	This looks like that: deep learning for interpretable image recognition	55	Pytorch
2019	NeurIPS	A benchmark for interpretability methods in deep neural networks	14
2019	NeurIPS	Full-gradient representation for neural network visualization	3
2019	NeurIPS	On the (In) fidelity and Sensitivity of Explanations	4
2019	NeurIPS	Towards Automatic Concept-based Explanations	9	T.F.
2019	NeurIPS	CXPlain: Causal explanations for model interpretation under uncertainty	4
2019	CVPR	Interpreting CNNs via Decision Trees	71
2019	CVPR	From Recognition to Cognition: Visual Commonsense Reasoning	74	Pytorch
2019	CVPR	Attention branch network: Learning of attention mechanism for visual explanation	28
2019	CVPR	Interpretable and fine-grained visual explanations for convolutional neural networks	10
2019	CVPR	Learning to Explain with Complemental Examples	10
2019	CVPR	Revealing Scenes by Inverting Structure from Motion Reconstructions	12	T.F.
2019	CVPR	Multimodal Explanations by Predicting Counterfactuality in Videos	3
2019	CVPR	Visualizing the Resilience of Deep Convolutional Network Interpretations	1
2019	ICCV	U-CAM: Visual Explanation using Uncertainty based Class Activation Maps	9
2019	ICCV	Towards Interpretable Face Recognition	7
2019	ICCV	Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded	14
2019	ICCV	Understanding Deep Networks via Extremal Perturbations and Smooth Masks	9	Pytorch	✔️
2019	ICCV	Explaining Neural Networks Semantically and Quantitatively	5
2019	ICLR	Hierarchical interpretations for neural network predictions	24	Pytorch
2019	ICLR	How Important Is a Neuron?	10
2019	ICLR	Gan dissection: Visualizing and understanding generative adversarial networks	54	PyTorch	✔️
2019	ICLR	Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks	10
2018	ICML	Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples	64	Pytorch
2019	ICML	Towards A Deep and Unified Understanding of Deep Neural Models in NLP	10	Pytorch
2019	ICAIS	Interpreting black box predictions using fisher kernels	16
2019	ACMFAT	Explaining explanations in AI	92
2019	AAAI	Interpretation of neural networks is fragile	100	T.F.
2019	AAAI	Classifier-agnostic saliency map extraction	7
2019	AAAI	Can You Explain That? Lucid Explanations Help Human-AI Collaborative Image Retrieval	1
2019	AAAIW	Unsupervised Learning of Neural Networks to Explain Neural Networks	10
2019	AAAIW	Network Transplanting	4
2019	CSUR	A Survey of Methods for Explaining Black Box Models	518
2019	JVCIR	Interpretable convolutional neural networks via feedforward design	28	Keras
2019	ExplainAI	The (Un)reliability of saliency methods	128
2019	ACL	Attention is not Explanation	88
2019	EMNLP	Attention is not not Explanation	22
2019	arxiv	Attention Interpretability Across NLP Tasks	6
2019	arxiv	Interpretable CNNs	2
2018	ICLR	Towards better understanding of gradient-based attribution methods for deep neural networks	160
2018	ICLR	Learning how to explain neural networks: PatternNet and PatternAttribution	113
2018	ICLR	On the importance of single directions for generalization	109	Pytorch
2018	ICLR	Detecting statistical interactions from neural network weights	45	Pytorch
2018	ICLR	Interpretable counting for visual question answering	27	Pytorch
2018	CVPR	Interpretable Convolutional Neural Networks	213
2018	CVPR	Tell me where to look: Guided attention inference network	110	Chainer
2018	CVPR	Multimodal Explanations: Justifying Decisions and Pointing to the Evidence	106	Caffe
2018	CVPR	Transparency by design: Closing the gap between performance and interpretability in visual reasoning	70	Pytorch
2018	CVPR	Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks	48
2018	CVPR	What have we learned from deep representations for action recognition?	25
2018	CVPR	Learning to Act Properly: Predicting and Explaining Affordances from Images	23
2018	CVPR	Teaching Categories to Human Learners with Visual Explanations	19	Pytorch
2018	CVPR	What do deep networks like to see?	17
2018	CVPR	Interpret Neural Networks by Identifying Critical Data Routing Paths	10	T.F.
2018	ECCV	Deep clustering for unsupervised learning of visual features	289	Pytorch
2018	ECCV	Explainable neural computation via stack neural module networks	43	T.F.
2018	ECCV	Grounding visual explanations	40
2018	ECCV	Textual explanations for self-driving vehicles	47
2018	ECCV	Interpretable basis decomposition for visual explanation	40	Pytorch
2018	ECCV	Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases	23
2018	ECCV	Vqa-e: Explaining, elaborating, and enhancing your answers for visual questions	17
2018	ECCV	Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance	14	Pytorch
2018	ECCV	Diverse feature visualizations reveal invariances in early layers of deep neural networks	8	T.F.
2018	ECCV	ExplainGAN: Model Explanation via Decision Boundary Crossing Transformations	2
2018	ICML	Interpretability beyond feature attribution: Quantitative testing with concept activation vectors	168	T.F.
2018	ICML	Learning to explain: An information-theoretic perspective on model interpretation	94
2018	ACL	Did the Model Understand the Question?	42	T.F.
2018	FITEE	Visual interpretability for deep learning: a survey	208
2018	NeurIPS	Sanity Checks for Saliency Maps	195
2018	NeurIPS	Explanations based on the missing: Towards contrastive explanations with pertinent negatives	53	T.F.
2018	NeurIPS	Towards robust interpretability with self-explaining neural networks	101	Pytorch
2018	NeurIPS	Attacks meet interpretability: Attribute-steered detection of adversarial samples	42
2018	NeurIPS	DeepPINK: reproducible feature selection in deep neural networks	22	Keras
2018	NeurIPS	Representer point selection for explaining deep neural networks	20	T.F.
2018	NeurIPS Workshop	Interpretable convolutional filters with sincNet	30
2018	AAAI	Anchors: High-precision model-agnostic explanations	290
2018	AAAI	Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients	146	T.F.
2018	AAAI	Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions	87	T.F.
2018	AAAI	Interpreting CNN Knowledge via an Explanatory Graph	70	Matlab
2018	AAAI	Examining CNN Representations with respect to Dataset Bias	33
2018	WACV	Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks	127
2018	IJCV	Top-down neural attention by excitation backprop	297
2018	TPAMI	Interpreting deep visual representations via network dissection	73
2018	DSP	Methods for interpreting and understanding deep neural networks	617
2018	Access	Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI)	279
2018	JAIR	Learning Explanatory Rules from Noisy Data	127	T.F.
2018	MIPRO	Explainable artificial intelligence: A survey	92
2018	BMVC	Rise: Randomized input sampling for explanation of black-box models	78	Keras	✔️
2018	arxiv	Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation	25
2018	arxiv	Manipulating and measuring model interpretability	106
2018	arxiv	How convolutional neural network see the world-A survey of convolutional neural network visualization methods	36
2018	arxiv	Revisiting the importance of individual units in cnns via ablation	33
2018	arxiv	Computationally Efficient Measures of Internal Neuron Importance	1
2017	ICML	Understanding Black-box Predictions via Influence Functions	641	Pytorch
2017	ICML	Axiomatic attribution for deep networks	623	Keras	✔️
2017	ICML	Learning Important Features Through Propagating Activation Differences	531
2017	ICLR	Visualizing deep neural network decisions: Prediction difference analysis	241	Caffe
2017	ICLR	Exploring LOTS in Deep Neural Networks	26
2017	NeurIPS	A Unified Approach to Interpreting Model Predictions	1036		✔️
2017	NeurIPS	Real time image saliency for black box classifiers	141	Pytorch
2017	NeurIPS	SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability	131
2017	CVPR	Mining Object Parts from CNNs via Active Question-Answering	17
2017	CVPR	Network dissection: Quantifying interpretability of deep visual representations	463
2017	CVPR	Improving Interpretability of Deep Neural Networks with Semantic Information	52
2017	CVPR	MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network	109	Torch
2017	CVPR	Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering	509
2017	CVPR	Knowing when to look: Adaptive attention via a visual sentinel for image captioning	553	Torch
2017	CVPRW	Interpretable 3d human action analysis with temporal convolutional networks	140
2017	ICCV	Grad-cam: Visual explanations from deep networks via gradient-based localization	1980	Pytorch
2017	ICCV	Interpretable Explanations of Black Boxes by Meaningful Perturbation	371	Pytorch
2017	ICCV	Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention	101
2017	ICCV	Understanding and comparing deep neural networks for age and gender classification	52
2017	ICCV	Learning to disambiguate by asking discriminative questions	11
2017	IJCAI	Right for the right reasons: Training differentiable models by constraining their explanations	125
2017	IJCAI	Understanding and improving convolutional neural networks via concatenated rectified linear units	250	Caffe
2017	AAAI	Growing Interpretable Part Graphs on ConvNets via Multi-Shot Learning	31	Matlab
2017	ACL	Visualizing and Understanding Neural Machine Translation	69
2017	EMNLP	A causal framework for explaining the predictions of black-box sequence-to-sequence models	74
2017	CVPR Workshop	Looking under the hood: Deep neural network visualization to interpret whole-slide image analysis outcomes for colorectal polyps	19
2017	survey	Interpretability of deep learning models: a survey of results	70
2017	arxiv	SmoothGrad: removing noise by adding noise	289
2017	arxiv	Interpretable & explorable approximations of black box models	97
2017	arxiv	Distilling a neural network into a soft decision tree	162	Pytorch
2017	arxiv	Towards interpretable deep neural networks by leveraging adversarial examples	54
2017	arxiv	Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models	319		✔️
2017	arxiv	Contextual Explanation Networks	32	Pytorch
2017	arxiv	Challenges for transparency	75
2017	ACMSOPP	Deepxplore: Automated whitebox testing of deep learning systems	384
2017	CEURW	What does explainable AI really mean? A new conceptualization of perspectives	94
2017	TVCG	ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models	141
2016	NeurIPS	Synthesizing the preferred inputs for neurons in neural networks via deep generator networks	286	Caffe
2016	NeurIPS	Understanding the effective receptive field in deep convolutional neural networks	391
2016	CVPR	Inverting Visual Representations with Convolutional Networks	306
2016	CVPR	Visualizing and Understanding Deep Texture Representations	90
2016	CVPR	Analyzing Classifiers: Fisher Vectors and Deep Neural Networks	100
2016	ECCV	Generating Visual Explanations	273	Caffe
2016	ECCV	Design of kernels in convolutional neural networks for image classification	13
2016	ICML	Understanding and improving convolutional neural networks via concatenated rectified linear units	250
2016	ICML	Visualizing and comparing AlexNet and VGG using deconvolutional layers	35
2016	EMNLP	Rationalizing Neural Predictions	288	Pytorch
2016	IJCV	Visualizing deep convolutional neural networks using natural pre-images	254	Matlab
2016	IJCV	Visualizing Object Detection Features	25	Caffe
2016	KDD	Why should i trust you?: Explaining the predictions of any classifier	3358	GitHub	✔️
2016	TVCG	Visualizing the hidden activity of artificial neural networks	152
2016	TVCG	Towards better analysis of deep convolutional neural networks	225
2016	NAACL	Visualizing and understanding neural models in nlp	306	Torch
2016	arxiv	Understanding neural networks through representation erasure)	155
2016	arxiv	Grad-CAM: Why did you say that?	119
2016	arxiv	Investigating the influence of noise and distractors on the interpretation of neural networks	31
2016	arxiv	Attentive Explanations: Justifying Decisions and Pointing to the Evidence	47
2016	arxiv	The Mythos of Model Interpretability	1177
2016	arxiv	Multifaceted feature visualization: Uncovering the different types of features learned by each neuron in deep neural networks	144
2015	ICLR	Guided Backprop: Striving for Simplicity: The All Convolutional Net	2080	Pytorch	⭐
2015	CVPR	Understanding deep image representations by inverting them	1046	Matlab
2015	ICCV	Understanding deep features with computer-generated imagery	103	Caffe
2015	ICML Workshop	Understanding Neural Networks Through Deep Visualization	1114	T.F.
2015	AAS	Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model	331
2015	PLOS	On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation	1029		⭐
2014	ECCV	Visualizing and Understanding Convolutional Networks	9168	Pytorch	⭐
2014	ICLR	Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps	2399	Pytorch	⭐
2013	ICCV	Hoggles: Visualizing object detection features	293

* Paper list sorted by number of citation

Tutorials and Talks

Year	Conf.	Title
2020	CVPR	Interpretable Machine Learning for Computer Vision
		Zeynep Akata: Modeling Conceptual Understanding in Image Reference Games	video, slide
		Bolei Zhou: Exploring and Exploiting Interpretable Semantics in GANs	video, slide	✔️
		Ruth C. Fong: Understanding Deep Neural Networks	video, slide	✔️
		Christopher Olah: Introduction to Circuits in CNNs.	video, slide
2019	CVPR	Interpretable Machine Learning for Computer Vision
2019	Valse	Bolei Zhou, Quanshi Zhang: 20190116-03 可解释性AI专题	video	✔️
2019	-	Hung-Yi Lee, Explainable ML	video
2018	CVPR	Interpretable Machine Learning for Computer Vision

Libraries & Tools

• Captum: Model Interpretability for PyTorch
• TorchRay: Understanding Deep Networks via Extremal Perturbations and Smooth Masks