I came across this list a while ago, and it has been hugely useful to me, which is why I’ve augmented it and put it up here. It contains an (almost) exhaustive list of libraries, packages, tools, implementations of tasks, tutorials, current state of the art models and established baselines.
The version that I saw first also had extensive video lectures, tutorials and article/paper references, which I cannot seem to find, but when I do, it will go under the first resource list. Enjoy!
Implementations:
Libraries:
NLP Python Libraries:
C++ Libraries:
Java Libraries:
Scala Libraries:
R Libraries:
Clojure:
Ruby:
Rust:
Services:
Articles:
Word Vectors:
Resources about word vectors, aka word embeddings, and distributed representations for words. Word vectors are numeric representations of words that are often used as input to deep learning systems. This process is sometimes called pretraining.
Efficient Estimation of Word Representations in Vector Space [Distributed Representations of Words and Phrases and their Compositionality] (http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf) Mikolov et al. 2013. Generate word and phrase vectors. Performs well on word similarity and analogy task and includes Word2Vec source code Subsamples frequent words. (i.e. frequent words like "the" are skipped periodically to speed things up and improve vector for less frequently used words Word2Vec tutorial in TensorFlow
Deep Learning, NLP, and Representations Chris Olah (2014) Blog post explaining word2vec.
GloVe: Global vectors for word representation Pennington, Socher, Manning. 2014. Creates word vectors and relates word2vec to matrix factorizations. Evalutaion section led to controversy by Yoav Goldberg Glove source code and training data
Thought Vectors:
Thought vectors are numeric representations for sentences, paragraphs, and documents. The following papers are listed in order of date published, each one replaces the last as the state of the art in sentiment analysis.
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank Socher et al. 2013. Introduces Recursive Neural Tensor Network. Uses a parse tree.
Distributed Representations of Sentences and Documents Le, Mikolov. 2014. Introduces Paragraph Vector. Concatenates and averages pretrained, fixed word vectors to create vectors for sentences, paragraphs and documents. Also known as paragraph2vec. Doesn't use a parse tree. Implemented in gensim. See doc2vec tutorial
Deep Recursive Neural Networks for Compositionality in Language Irsoy & Cardie. 2014. Uses Deep Recursive Neural Networks. Uses a parse tree.
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks Tai et al. 2015 Introduces Tree LSTM. Uses a parse tree.
Semi-supervised Sequence Learning Dai, Le 2015 "With pretraining, we are able to train long short term memory recurrent networks up to a few hundred timesteps, thereby achieving strong performance in many text classification tasks, such as IMDB, DBpedia and 20 Newsgroups."
Machine Translation:
Neural Machine Translation by jointly learning to align and translate Bahdanau, Cho 2014. "comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation." Implements attention mechanism. English to French Demo
Sequence to Sequence Learning with Neural Networks Sutskever, Vinyals, Le 2014. (nips presentation). Uses LSTM RNNs to generate translations. " Our main result is that on an English to French translation task from the WMT’14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8" seq2seq tutorial in
Single Exchange Dialogs:
A Neural Network Approach toContext-Sensitive Generation of Conversational Responses Sordoni 2015. Generates responses to tweets. Uses Recurrent Neural Network Language Model (RLM) architecture of (Mikolov et al., 2010). source code: RNNLM Toolkit
Neural Responding Machine for Short-Text Conversation Shang et al. 2015 Uses Neural Responding Machine. Trained on Weibo dataset. Achieves one round conversations with 75% appropriate responses.
A Neural Conversation Model Vinyals, Le 2015. Uses LSTM RNNs to generate conversational responses. Uses seq2seq framework. Seq2Seq was originally designed for machine transation and it "translates" a single sentence, up to around 79 words, to a single sentence response, and has no memory of previous dialog exchanges. Used in Google Smart Reply feature for Inbox
Memory and Attention Models (from DL4NLP):
Reasoning, Attention and Memory RAM workshop at NIPS 2015. slides included
Memory Networks Weston et. al 2014, and End-To-End Memory Networks Sukhbaatar et. al 2015. Memory networks are implemented in MemNN. Attempts to solve task of reason attention and memory. Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks Weston 2015. Classifies QA tasks like single factoid, yes/no etc. Extends memory networks. Evaluating prerequisite qualities for learning end to end dialog systems Dodge et. al 2015. Tests Memory Networks on 4 tasks including reddit dialog task. See Jason Weston lecture on MemNN
Neural Turing Machines Graves et al. 2014.
Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets Joulin, Mikolov 2015. Stack RNN source code and blog post
General Natural Language Processing:
Named Entity Recognition:
Neural Networks:
Supplementary Materials:
Blogs:
Credits:
Tutorials and Courses:
Videos:
Deep Learning for NLP:
Deep Natural Language Processing +lecture slides and course description for the Deep Natural Language Processing course offered in Hilary Term 2017 at the University of Oxford. Stanford CS 224D: Deep Learning for NLP class Class by Richard Socher. 2016 content was updated to make use of Tensorflow. Lecture slides and reading materials for 2016 class here. Videos for 2016 class here. Note that there are some lecture videos missing for 2016 (lecture 9, and lectures 12 onwards). All videos for 2015 class here
Udacity Deep Learning Deep Learning course on Udacity (using Tensorflow) which covers a section on using deep learning for NLP tasks. This section covers how to implement Word2Vec, RNN's and LSTMs.
A Primer on Neural Network Models for Natural Language Processing Yoav Goldberg. October 2015. No new info, 75 page summary of state of the art.