updated readme and licenses

README.md

@@ -2,9 +2,9 @@
 
 The datasets and the evaluations for the adapted data-driven dialogue models are described in detail in the papers:
 
-    Bahar Irfan, Mehdi Hellou, Alexandre Mazel, and Tony Belpaeme (2020), "Challenges of a Real-World HRI Study with Non-Native English Speakers: Can Personalisation Save the Day?", Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI), DOI: 10.1145/3371382.3378278.
+ * Bahar Irfan, Mehdi Hellou, Alexandre Mazel, Tony Belpaeme (2020), "Challenges of a Real-World HRI Study with Non-Native English Speakers: Can Personalisation Save the Day?", Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI), DOI: 10.1145/3371382.3378278.
 
-    Bahar Irfan and Tony Belpaeme (under review), "Coffee with a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions", Frontiers in Robotics and AI.
+ * Bahar Irfan and Tony Belpaeme (under review), "Coffee with a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions", Frontiers in Robotics and AI.
 
 Please cite both papers if you are using the datasets; cite the first paper for generic and personalised barista robots based on the datasets; cite the second paper if you are referring to the data-driven dialogue evaluations on the datasets.
 
@@ -20,7 +20,7 @@ Personalised Barista with Preferences Information Dataset (*barista-personalised
 
 *all_labels* folder contains datasets with all possible correst responses (each response is separated with "|") for each customer phrase, exactly corresponding (in terms of customer orders, phrases and names in order) to the Barista Datasets. 
 
-*ParlAI_format* folder contains the Barista Datasets in ParlAI format (https://parl.ai/docs/tutorial_task.html#quickstart-adding-a-new-dataset). The original Barista Datasets are in FbDialogTeacher format (the previous format of datasets in ParlAI).
+*ParlAI_format* folder contains the Barista Datasets in ParlAI (Miller et al., 2017) [https://parl.ai/docs/tutorial_task.html#quickstart-adding-a-new-dataset](format). The original Barista Datasets are in FbDialogTeacher format (the previous format of datasets in ParlAI).
 
 *info* folder contains the proportion of phrases containing customer name or preferences (*personal(ised)*), phrases containing order item (*order*), remaining (*other*) phrases, and phrases from the Barista Dataset (*Only Barista* (B7)) within all the utterances (*Utterance count*) of the barista (bot) for each task and dataset.
 
@@ -30,7 +30,7 @@ For detailed descriptions of the datasets, see the README files in the correspon
 
 ## Data-Driven Dialogue Models
 
-The Barista datasets were evaluated with the state-of-the-art data-driven dialogue models: Supervised Embeddings (Dodge et al., 2015; Bordes et al., 2016), Sequence-to-Sequence (Sutskever et al., 2015), End-to-End Memory Networks (Sukhbaatar et al., 2016), Generative Profile Memory Networks (Zhang et al., 2018), Key-Value Memory Networks (Miller et al., 2016; Zhang et al., 2018) and Split Memory Networks (Joshi et al., 2017). We adapted the code from ParlAI (https://github.com/facebookresearch/ParlAI) for Sequence-to-Sequence, Generative Profile Memory Networks and Key-Value Memory Networks models and the code from Joshi et al., 2017 (https://github.com/chaitjo/personalized-dialog) for End-to-End Memory Networks, Split Memory Networks and Supervised Embeddings. The adapted code is provided here (in *parlai_internal* for former methods, in *baselines* for latter methods) for reproducibility of the evaluations on the Barista Datasets.
+The Barista datasets were evaluated with the state-of-the-art data-driven dialogue models: Supervised Embeddings (Dodge et al., 2015; Bordes et al., 2016), Sequence-to-Sequence (Sutskever et al., 2015), End-to-End Memory Networks (Sukhbaatar et al., 2016; Bordes et al., 2016), Generative Profile Memory Networks (Zhang et al., 2018), Key-Value Memory Networks (Miller et al., 2016; Zhang et al., 2018) and Split Memory Networks (Joshi et al., 2017). We adapted the implementations from [https://github.com/facebookresearch/ParlAI](ParlAI) for Sequence-to-Sequence, Generative Profile Memory Networks and Key-Value Memory Networks models and the implementation from [https://github.com/chaitjo/personalized-dialog](Joshi et al., 2017) for End-to-End Memory Networks, Split Memory Networks and Supervised Embeddings. The adapted code is provided here (in *parlai_internal* for former methods, in *baselines* for latter methods) for reproducibility of the evaluations on the Barista Datasets.
 
 Clone the repository (or download the *barista-datasets-and-models* release).
 
@@ -72,7 +72,9 @@ hops: in End-to-End Memory Networks, Split Memory Networks, Key-Value Memory Net
 
 Within the bash shell, run the following commands to train and test the baselines. All tasks for the dataset will be trained/tested (i.e., 1 to 7 for Barista Dataset, and 0 to 8 for Personalised Barista Datasets).
 
-### End-to-End Memory Networks
+### End-to-End Memory Networks: Best performing model for Personalised Task-Oriented Dialogue
+
+Adapted implementation from: https://github.com/chaitjo/personalized-dialog
 
 Train:
 
@@ -84,6 +86,8 @@ Test:
 
 ### Split Memory Networks
 
+Adapted implementation from: https://github.com/chaitjo/personalized-dialog
+
 Train:
 
     $ cd /app/hrinlp/baselines/MemN2N-split-memory/;bash bin/train_all.sh [dataset_name] [task_name] [hops]
@@ -95,6 +99,8 @@ Test:
 
 ### Supervised Embeddings
 
+Adapted implementation from: https://github.com/chaitjo/personalized-dialog
+
 Train:
 
     $ cd /app/hrinlp/baselines/supervised-embedding/;bash bin/train_all.sh [dataset_name] [task_name]
@@ -103,7 +109,9 @@ Test:
 
     $ cd /app/hrinlp/baselines/supervised-embedding/;bash bin/test_all.sh [dataset_name] [task_name]
 
-### Sequence-to-Sequence
+### Sequence-to-Sequence: Best performing model for Generic Task-Oriented Dialogue
+
+Adapted implementation from: https://github.com/facebookresearch/ParlAI/tree/master/projects/convai2/baselines/seq2seq
 
 Train:
 
@@ -115,6 +123,8 @@ Test: (Used 1 for batch size and number of threads in the paper)
 
 ### Generative Profile Memory Networks
 
+Adapted implementation from: https://github.com/facebookresearch/ParlAI/tree/6a76a555ea84b06e2914cdea4c56a46a5f495821/projects/personachat
+
 Train:
 
     $ cd /app/ParlAI/;bash parlai_internal/agents/profilememory/train_all.sh [dataset_name] [task_name]
@@ -125,6 +135,8 @@ Test: (Used 1 for batch size and number of threads in the paper)
 
 ### Key-Value Memory Networks
 
+Adapted implementation from: https://github.com/facebookresearch/ParlAI/tree/master/projects/convai2/baselines/kvmemnn
+
 Train: (18 threads were used in the experiments)
 
     $ cd /app/ParlAI/;bash parlai_internal/agents/kvmemnn/train_all.sh [dataset_name] [task_name] [hops] [numthreads]
@@ -139,7 +151,7 @@ Models with other hyperparameters (as described in the Irfan and Belpaeme paper)
 
 ## Trained Models
 
-*barista-datasets-and-trained-models* release contains the Barista Datasets, adapted code for models and the best trained models corresponding to the reported results (in Irfan et al., under review) for each baseline on the Barista Datasets. Profile Memory and Seq2Seq models have only the last task of the datasets (i.e., task 7 in Barista or task 8 in Personalised Barista Datasets), because the trained models are very large in size. Key-Value Memory Network was only trained with one hop, due to the vast amount of time required for training.
+*barista-datasets-and-trained-models* release contains the Barista Datasets, adapted code for models and the best trained models corresponding to the reported results (in Irfan & Belpaeme, under review) for each baseline on the Barista Datasets. Profile Memory and Seq2Seq models have only the last task of the datasets (i.e., task 7 in Barista or task 8 in Personalised Barista Datasets), because the trained models are very large in size. Key-Value Memory Network was only trained with one hop, due to the vast amount of time required for training. *performance_results* contains the per-response accuracy of each trained model on the Barista Datasets.
 
 ## License
 
@@ -149,3 +161,15 @@ The Barista datasets are released under a Creative Commons Attribution-NonCommer
 
 For any information or for requesting Barista Datasets with different order items, customer names, or larger dataset size, contact Bahar Irfan: bahar.irfan (at) plymouth (dot) ac (dot) uk.
 
+## References
+
+ * Alexander H. Miller, Will Feng, Adam Fisch, Jiasen Lu, Dhruv Batra, Antoine Bordes, Devi Parikh, Jason Weston (2017), "ParlAI: A Dialog Research Software Platform", [https://arxiv.org/abs/1705.06476](arXiv:1705.06476)
+ * Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander Miller, Arthur Szlam, Jason Weston (2015), "Evaluating prerequisite qualities for learning end-to-end dialog systems", [https://arxiv.org/abs/1511.06931](arXiv:1511.06931) 
+ * Antoine Bordes, Y-Lan Boureau, Jason Weston (2016), "Learning End-to-End Goal-Oriented Dialog", [https://arxiv.org/abs/1605.07683](arXiv:1605.07683)
+ * Ilya Sutskever, Oriol Vinyals, Quoc V. Le (2015), "Sequence to Sequence Learning with Neural Networks", [https://arxiv.org/abs/1409.3215](arXiv:1409.3215)
+ * Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus (2015), "End-To-End Memory Networks", [https://arxiv.org/abs/1503.08895](arXiv:1503.08895)
+ * Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, Jason Weston (2018), "Personalizing Dialogue Agents: I have a dog, do you have pets too?", [https://arxiv.org/abs/1801.07243](arXiv:1801.07243)
+ * Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, Jason Weston (2016), "Key-Value Memory Networks for Directly Reading Documents", [https://arxiv.org/abs/1606.03126](arXiv:1606.03126)
+ * Chaitanya K. Joshi, Fei Mi, Boi Faltings (2017), "Personalization in Goal-Oriented Dialog", [https://arxiv.org/abs/1706.07503](arXiv:1706.07503)
+
+