awesome-nlp

A curated list of resources dedicated to Natural Language Processing

Maintainers - Keon, Martin, Nirant, Dhruv

Please read the contribution guidelines before contributing. Please feel free to create pull requests.

Contents

• Research Summaries and Trends
• Tutorials
  • Reading Content
  • Videos and Courses
  • Books
• Libraries
  • Node.js
  • Python
  • C++
  • Java
  • Kotlin
  • Scala
  • R
  • Clojure
  • Ruby
  • Rust
• Services
• Annotation Tools
• Datasets
• NLP in Korean
• NLP in Arabic
• NLP in Chinese
• NLP in German
• NLP in Spanish
• NLP in Indic Languages
• NLP in Thai
• NLP in Vietnamese
• Other Languages
• Credits

Research Summaries and Trends

• NLP-Overview is an up-to-date overview of deep learning techniques applied to NLP, including theory, implementations, applications, and state-of-the-art results. This is a great Deep NLP Introduction for researchers.
• NLP-Progress tracks the progress in Natural Language Processing, including the datasets and the current state-of-the-art for the most common NLP tasks
• NLP's ImageNet moment has arrived
• ACL 2018 Highlights: Understanding Representation and Evaluation in More Challenging Settings
• Four deep learning trends from ACL 2017. Part One: Linguistic Structure and Word Embeddings
• Four deep learning trends from ACL 2017. Part Two: Interpretability and Attention
• Highlights of EMNLP 2017: Exciting Datasets, Return of the Clusters, and More!
• Deep Learning for Natural Language Processing (NLP): Advancements & Trends
• Survey of the State of the Art in Natural Language Generation

Tutorials

Back to Top

Reading Content

General Machine Learning

• Jason's Machine Learning 101 from Google's Senior Creative Engineer explains Machine Learning for engineer's and executives alike
• a16z AI Playbook is a great link to forward to your managers or content for your presentations
• Machine Learning Blog by Brian McFee
• Ruder's Blog by Sebastian Ruder for commentary on the best of NLP Research

Introductions and Guides to NLP

• Ultimate Guide to Understand & Implement Natural Language Processing
• Introduction to NLP at Hackernoon is for people who suck at math - in their own words
• NLP Tutorial by Vik Paruchari
• Natural Language Processing: An Introduction by Oxford
• Deep Learning for NLP with Pytorch
• Hands-On NLTK Tutorial - The hands-on NLTK tutorial in the form of Jupyter notebooks

Blogs and Newsletters

• Blog: Deep Learning, NLP, and Representations
• Blog: The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) and The Illustrated Transformer
• Blog: Natural Language Processing by Hal Daumé III
• Tutorials by Radim Řehůřek on using Python and gensim to process language corpora
• arXiv: Natural Language Processing (Almost) from Scratch
• Karpathy's The Unreasonable Effectiveness of Recurrent Neural Networks

Videos and Online Courses

Deep Learning and NLP

Word embeddings, RNNs, LSTMs and CNNs for Natural Language Processing | Back to Top

• Udacity's Intro to Artificial Intelligence course which touches upon NLP as well
• Udacity's Deep Learning using Tensorflow which covers a section on using deep learning for NLP tasks (covering Word2Vec, RNN's and LSTMs)
• Oxford's Deep Natural Language Processing has videos, lecture slides and reading material
• Stanford's Deep Learning for Natural Language Processing (cs224-n) by Richard Socher and Christopher Manning
• Coursera's Natural Language Processing by National Research University Higher School of Economics
• Carnegie Mellon University's Neural Networks for NLP by Language Technology Institute there

Classical NLP

Bayesian, statistics and Linguistics approaches for Natural Language Processing | Back to Top

• Natural Language Processing by Prof. Mike Collins at Columbia
• Statistical Machine Translation - a Machine Translation course with great assignments and slides
• NLTK with Python 3 for Natural Language Processing by Harrison Kinsley(sentdex). Good tutorials with NLTK code implementation
• Computational Linguistics I by Jordan Boyd-Graber, Lectures from University of Maryland
• Deep NLP Course by Yandex Data School, covering important ideas from text embedding to machine translation including sequence modeling, language models and so on.

Books

• Speech and Language Processing by Prof. Dan Jurafsy
• Text Mining in R
• Natural Language Processing with Python

Libraries

Back to Top

• Node.js and Javascript - Node.js Libaries for NLP | Back to Top

  • Twitter-text - A JavaScript implementation of Twitter's text processing library
  • Knwl.js - A Natural Language Processor in JS
  • Retext - Extensible system for analyzing and manipulating natural language
  • NLP Compromise - Natural Language processing in the browser
  • Natural - general natural language facilities for node
  • Poplar - A web-based annotation tool for natural language processing (NLP)

• Python - Python NLP Libraries | Back to Top

  • TextBlob - Providing a consistent API for diving into common natural language processing (NLP) tasks. Stands on the giant shoulders of Natural Language Toolkit (NLTK) and Pattern, and plays nicely with both 👍
  • spaCy - Industrial strength NLP with Python and Cython 👍
    • textacy - Higher level NLP built on spaCy
  • gensim - Python library to conduct unsupervised semantic modelling from plain text 👍
  • scattertext - Python library to produce d3 visualizations of how language differs between corpora
  • AllenNLP - An NLP research library, built on PyTorch, for developing state-of-the-art deep learning models on a wide variety of linguistic tasks.
  • PyTorch-NLP - NLP research toolkit designed to support rapid prototyping with better data loaders, word vector loaders, neural network layer representations, common NLP metrics such as BLEU
  • Rosetta - Text processing tools and wrappers (e.g. Vowpal Wabbit)
  • PyNLPl - Python Natural Language Processing Library. General purpose NLP library for Python. Also contains some specific modules for parsing common NLP formats, most notably for FoLiA, but also ARPA language models, Moses phrasetables, GIZA++ alignments.
  • jPTDP - A toolkit for joint part-of-speech (POS) tagging and dependency parsing. jPTDP provides pre-trained models for 40+ languages.
  • BigARTM - a fast library for topic modelling
  • Snips NLU - A production ready library for intent parsing
  • Chazutsu - A library for downloading&parsing standard NLP research datasets
  • Word Forms - Word forms can accurately generate all possible forms of an English word
  • Multilingual Latent Dirichlet Allocation (LDA) - A multilingual and extensible document clustering pipeline

• C++ - C++ Libraries | Back to Top

  • MIT Information Extraction Toolkit - C, C++, and Python tools for named entity recognition and relation extraction
  • CRF++ - Open source implementation of Conditional Random Fields (CRFs) for segmenting/labeling sequential data & other Natural Language Processing tasks.
  • CRFsuite - CRFsuite is an implementation of Conditional Random Fields (CRFs) for labeling sequential data.
  • BLLIP Parser - BLLIP Natural Language Parser (also known as the Charniak-Johnson parser)
  • colibri-core - C++ library, command line tools, and Python binding for extracting and working with basic linguistic constructions such as n-grams and skipgrams in a quick and memory-efficient way.
  • ucto - Unicode-aware regular-expression based tokenizer for various languages. Tool and C++ library. Supports FoLiA format.
  • libfolia - C++ library for the FoLiA format
  • frog - Memory-based NLP suite developed for Dutch: PoS tagger, lemmatiser, dependency parser, NER, shallow parser, morphological analyzer.
  • MeTA - MeTA : ModErn Text Analysis is a C++ Data Sciences Toolkit that facilitates mining big text data.
  • Mecab (Japanese)
  • Moses
  • StarSpace - a library from Facebook for creating embeddings of word-level, paragraph-level, document-level and for text classification

• Java - Java NLP Libraries | Back to Top

  • Stanford NLP
  • OpenNLP
  • ClearNLP
  • Word2vec in Java
  • ReVerb Web-Scale Open Information Extraction
  • OpenRegex An efficient and flexible token-based regular expression language and engine.
  • CogcompNLP - Core libraries developed in the U of Illinois' Cognitive Computation Group.
  • MALLET - MAchine Learning for LanguagE Toolkit - package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.
  • RDRPOSTagger - A robust POS tagging toolkit available (in both Java & Python) together with pre-trained models for 40+ languages.

• Kotlin - Kotlin NLP Libraries | Back to Top

  • Lingua A language detection library for Kotlin and Java, suitable for long and short text alike

• Scala - Scala NLP Libraries | Back to Top

  • Saul - Library for developing NLP systems, including built in modules like SRL, POS, etc.
  • ATR4S - Toolkit with state-of-the-art automatic term recognition methods.
  • tm - Implementation of topic modeling based on regularized multilingual PLSA.
  • word2vec-scala - Scala interface to word2vec model; includes operations on vectors like word-distance and word-analogy.
  • Epic - Epic is a high performance statistical parser written in Scala, along with a framework for building complex structured prediction models.

• R - R NLP Libraries | Back to Top

  • text2vec - Fast vectorization, topic modeling, distances and GloVe word embeddings in R.
  • wordVectors - An R package for creating and exploring word2vec and other word embedding models
  • RMallet - R package to interface with the Java machine learning tool MALLET
  • dfr-browser - Creates d3 visualizations for browsing topic models of text in a web browser.
  • dfrtopics - R package for exploring topic models of text.
  • sentiment_classifier - Sentiment Classification using Word Sense Disambiguation and WordNet Reader
  • jProcessing - Japanese Natural Langauge Processing Libraries, with Japanese sentiment classification

• Clojure | Back to Top

  • Clojure-openNLP - Natural Language Processing in Clojure (opennlp)
  • Infections-clj - Rails-like inflection library for Clojure and ClojureScript
  • postagga - A library to parse natural language in Clojure and ClojureScript

• Ruby | Back to Top

  • Kevin Dias's A collection of Natural Language Processing (NLP) Ruby libraries, tools and software
  • Practical Natural Language Processing done in Ruby

• Rust

  • whatlang — Natural language recognition library based on trigrams
  • snips-nlu-rs - A production ready library for intent parsing

Services

NLP as API with higher level functionality such as NER, Topic tagging and so on | Back to Top

• Wit-ai - Natural Language Interface for apps and devices
• IBM Watson's Natural Language Understanding - API and Github demo
• Amazon Comprehend - NLP and ML suite covers most common tasks like NER, tagging, and sentiment analysis
• Google Cloud Natural Language API - Syntax Analysis, NER, Sentiment Analysis, and Content tagging in atleast 9 languages include English and Chinese (Simplified and Traditional).
• ParallelDots - High level Text Analysis API Service ranging from Sentiment Analysis to Intent Analysis
• Microsoft Cognitive Service
• TextRazor
• Rosette

Annotation Tools

• GATE - General Architecture and Text Engineering is 15+ years old, free and open source
• Anafora is free and open source, web-based raw text annotation tool
• brat - brat rapid annotation tool is an online environment for collaborative text annotation
• tagotag, costs $
• prodigy is an annotation tool powered by active learning, costs $
• LightTag - Hosted and managed text annotation tool for teams, costs $

Techniques

Text Embeddings

Back to Top

Text embeddings allow deep learning to be effective on smaller datasets. These are often first inputs to a deep learning archiectures and most popular way of transfer learning in NLP. Embeddings are simply vectors or a more generically, real valued representations of strings. Word embeddings are considered a great starting point for most deep NLP tasks.

The most popular names in word embeddings are word2vec by Google (Mikolov) and GloVe by Stanford (Pennington, Socher and Manning). fastText seems to be a fairly popular for multi-lingual sub-word embeddings.

Word Embeddings

Back to Top

Embedding	Paper	Organisation	gensim- Training Support	Blogs
word2vec	Official Implementation, T.Mikolove et al. 2013. Distributed Representations of Words and Phrases and their Compositionality. pdf	Google	Yes ✔️	Visual explanations by colah at Deep Learning, NLP, and Representations; gensim's Making Sense of word2vec
GloVe	Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. pdf	Stanford	No ❎	Morning Paper on GloVe by acoyler
fastText	Official Implementation, T. Mikolov et al. 2017. Enriching Word Vectors with Subword Information. pdf	Facebook	Yes ✔️	Fasttext: Under the Hood

Notes for Beginners:

• Thumb Rule: fastText >> GloVe > word2vec
• You can find pre-trained fasttext Vectors in several languages
• If you are interested in the logic and intuition behind word2vec and GloVe: The Amazing Power of Word Vectors and introduce the topics well
• arXiv: Bag of Tricks for Efficient Text Classification, and arXiv: FastText.zip: Compressing text classification models were released as part of fasttext

Sentence and Language Model Based Word Embeddings

Back to Top

• ElMo from Deep Contextualized Word Represenations - PyTorch implmentation - TF Implementation
• ULimFit aka Universal Language Model Fine-tuning for Text Classification by Jeremy Howard and Sebastian Ruder
• InferSent from Supervised Learning of Universal Sentence Representations from Natural Language Inference Data by facebook
• CoVe from Learned in Translation: Contextualized Word Vectors
• Pargraph vectors from Distributed Representations of Sentences and Documents. See doc2vec tutorial at gensim
• sense2vec - on word sense disambiguation
• Skip Thought Vectors - word representation method
• Adaptive skip-gram - similar approach, with adaptive properties
• Sequence to Sequence Learning - word vectors for machine translation

Question Answering and Knowledge Extraction

Back to Top

• DrQA: Open Domain Question Answering by facebook on Wikipedia data
• DocQA: Simple and Effective Multi-Paragraph Reading Comprehension by AllenAI
• Markov Logic Networks for Natural Language Question Answering
• Template-Based Information Extraction without the Templates
• Relation extraction with matrix factorization and universal schemas
• Privee: An Architecture for Automatically Analyzing Web Privacy Policies
• Teaching Machines to Read and Comprehend - DeepMind paper
• Relation Extraction with Matrix Factorization and Universal Schemas
• Towards a Formal Distributional Semantics: Simulating Logical Calculi with Tensors
• Presentation slides for MLN tutorial
• Presentation slides for QA applications of MLNs
• Presentation slides

Datasets

Back to Top

• nlp-datasets great collection of nlp datasets

Multilingual NLP Frameworks

Back to Top

• UDPipe is a trainable pipeline for tokenizing, tagging, lemmatizing and parsing Universal Treebanks and other CoNLL-U files. Primarily written in C++, offers a fast and reliable solution for multilingual NLP processing.
• NLP-Cube : Natural Language Processing Pipeline - Sentence Splitting, Tokenization, Lemmatization, Part-of-speech Tagging and Dependency Parsing. New platform, written in Python with Dynet 2.0. Offers standalone (CLI/Python bindings) and server functionality (REST API).

NLP in Korean

Back to Top

Libraries

• KoNLPy - Python package for Korean natural language processing.
• Mecab (Korean) - C++ library for Korean NLP
• KoalaNLP - Scala library for Korean Natural Language Processing.
• KoNLP - R package for Korean Natural language processing

Blogs and Tutorials

• dsindex's blog
• Kangwon University's NLP course in Korean

Datasets

• KAIST Corpus- A corpus from the Korea Advanced Institute of Science and Technology in Korean.
• Naver Sentiment Movie Corpus in Korean
• Chosun Ilbo archive - dataset in Korean from one of the major newspapers in South Korea, the Chosun Ilbo.

NLP in Arabic

Back to Top

Libraries

• goarabic- Go package for Arabic text processing
• jsastem - Javascript for Arabic stemming
• PyArabic - Python libraries for Arabic

Datasets

• Multidomain Datasets - Largest Available Multi-Domain Resources for Arabic Sentiment Analysis
• LABR - LArge Arabic Book Reviews dataset
• Arabic Stopwords - A list of Arabic stopwords from various resources

NLP in Chinese

Back to Top

Libraries

• jieba - Python package for Words Segmentation Utilities in Chinese
• SnowNLP - Python package for Chinese NLP
• FudanNLP- Java library for Chinese text processing

NLP in German

• German-NLP - Curated list of open-access/open-source/off-the-shelf resources and tools developed with a particular focus on German

NLP in Spanish

Back to Top

Corpora

• Columbian Political Speeches
• Copenhagen Treebank
• Reuters Corpora RCV2
• Spanish Billion words corpus with Word2Vec embeddings

NLP in Indic languages

Back to Top

Hindi

Corpora and Treebanks

• Hindi Dependency Treebank - A multi-representational multi-layered treebank for Hindi and Urdu
• Universal Dependencies Treebank in Hindi
  • Parallel Universal Dependencies Treebank in Hindi - A smaller part of the above-mentioned treebank.

NLP in Thai

Back to Top

Libraries

• PyThaiNLP - Thai NLP in Python Package
• JTCC- A character cluster library in Java
• CutKum - Word segmentation with deep learning in TensorFlow
• Thai Language Toolkit - Based on a paper by Wirote Aroonmanakun in 2002 with included dataset
• SynThai- Word segmentation and POS tagging using deep learning in Python

Corpora

• Inter-BEST - A text corpus with 5 million words with word segmentation
• Prime Minister 29- Dataset containing speeches of the current Prime Minister of Thailand

NLP in Vietnamese

Back to Top

NLP in Danish

• Named Entity Recognition for Danish

Libraries

• underthesea - Vietnamese NLP Toolkit
• vn.vitk - A Vietnamese Text Processing Toolkit
• VnCoreNLP - A Vietnamese natural language processing toolkit

Corpora

• Vietnamese treebank - 10,000 sentences for the constituency parsing task
• BKTreeBank - a Vietnamese Dependency Treebank
• UD_Vietnamese - Vietnamese Universal Dependency Treebank
• VIVOS - a free Vietnamese speech corpus consisting of 15 hours of recording speech by AILab
• VNTQcorpus(big).txt - 1.75 million sentences in news

Other Languages

• Russian: pymorphy2 - a good pos-tagger for Russian
• Asian Languages: Thai, Lao, Chinese, Japanese, and Korean ICU Tokenizer implementation in ElasticSearch
• Ancient Languages: CLTK: The Classical Language Toolkit is a Python library and collection of texts for doing NLP in ancient languages
• Dutch: python-frog - Python binding to Frog, an NLP suite for Dutch. (pos tagging, lemmatisation, dependency parsing, NER)
• Hebrew: NLPH_Resources - A collection of papers, corpora and linguistic resources for NLP in Hebrew

Credits

Awesome NLP was seeded with curated content from the lot of repositories, some of which are listed below | Back to Top

• ai-reading-list
• nlp-reading-group
• awesome-spanish-nlp
• jjangsangy's awesome-nlp
• awesome-machine-learning
• DL4NLP

Back to Top