Collections-of-Visual-Storytelling-StoryNLP

This repository aims to collect the articles and codes for the Visual Storytelling (VIST) task. VIST is a vision-and-language task. It aims to summarize the idea of a photo stream and tells a story about it (in natural language). Be careful about its difference from the "storytelling with data", which is more related to data visualization.

Visual Storytelling Task Website ←

More introduction and examples are shown in the site.

Dataset Website ←

Google Drive archived (313GB in total). If you have no access to google drive, maybe you should contact authors engaging in this task in your area for help.

Useful Tool:

To evaluate the generated story, metrics like BLEU, CIDEr, METEOR, ROUGE are most common used.

Evaluation Metrics Implementation: vist eval.It works with vist api

The task also needs human evaluation, usually in prespective of "Relevance", "Expressiveness" and "Concreteness", taking AREL(No Metrics Are Perfect, ACL 18) as a referebce.

• Please feel free to pull requests or open an issue to add papers.

---

2021

Title	Venue	Type	Code	Star
Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual Storytelling	AAAI21	KG
[Imagine,Reason and Write: Visual Storytelling with Graph Knowledge and Relational Reasoning]	AAAI21	KG+SG

2020

Title	Venue	Type	Code	Star
Knowledge-Enriched Visual Storytelling	AAAI20	KG	Pytorch1.3	17+
What Makes A Good Story? Designing Composite Rewards for Visual Storytelling	AAAI20	RL
Hide-and-Tell: Learning to Bridge Photo Streams for Visual Storytelling	AAAI20	En\Decoder
Storytelling from an Image Stream Using Scene Graphs	AAAI20	SceneGraph
Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling	ACMMM20	Few-Shot

2019

Title	Venue	Type	Code	Star
Visual Story Post-Editing	ACL19	Post-Editing	Dataset	10+
Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation	AAAI19	RL
Hierarchical Photo-Scene Encoder for Album Storytelling	AAAI19	En\Decoder
Informative Visual Storytelling with Cross-modal Rules	ACMMM19	ATT	Pytorch 0.3	3
Knowledgeable Storyteller: A Commonsense-Driven Generative Model for Visual Storytelling	IJCAI19	KG	No Source	7+
Emotion Reinforced Visual Storytelling	ICMR19	RL	Other
Dixit: Interactive Visual Storytelling via Term Manipulation	WWW19	Other
A Hierarchical Approach for Visual Storytelling Using Image Description	ICIDS19	En\Decoder
The Steep Road to Happily Ever After: An Analysis of Current Visual Storytelling Models	NAACL19 StoryNLP workshop	Analysis
Character-Centric Storytelling	NAACL19 StoryNLP workshop	Other
"My Way of Telling a Story": Persona based Grounded Story Generation	NAACL19 StoryNLP workshop	Other
Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication	ASLP19	Topic
Visual Storytelling via Predicting Anchor Word Embeddings in the Stories	ICCV19 workshop	Other
Incorporating Textual Evidence in Visual Storytelling	DSNNLG19	ATT

2018

Title	Venue	Type	Code	Star
No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling	ACL18	RL	Pytorch 0.3	117+
Show, Reward and Tell: Automatic Generation of Narrative Paragraph From Photo Stream by Adversarial Training	AAAI18	RL
Using Inter-Sentence Diverse Beam Search to Reduce Redundancy in Visual Storytelling	NAACL18 StoryNLP workshop	BS
Adversarial Learning for Visual Storytelling with Sense Group Partition	ACCV18	RL
GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation	NAACL18 StoryNLP workshop	En\Decoder	Pytorch1.0	34+
Contextualize, Show and Tell: A Neural Visual Storyteller	NAACL18 StoryNLP workshop	En\Decoder	Tensorflow1.0	21+
A Pipeline for Creative Visual Storytelling	NAACL18 StoryNLP workshop	Other
Stories for Images-in-Sequence by using Visualand Narrative Components	ICTI18	En\Decoder	Tensorflow1.6	34+

2016-2017

Title	Venue	Type	Code	Star
Let Your Photos Talk Generating Narrative Paragraph for Photo Stream	AAAI17	ATT
Learning Deep Contextual Attention Network for Narrative Photo Stream Captioning	ACM MM17 workshop	FasterRCNN+ATT
Hierarchically-Attentive RNN for Album Summarization and Storytelling	EMNLP17	En\Decoder	Python	26+
Sort Story: Sorting Jumbled Images and Captions into Stories	EMNLP16	Other