C++ implementation for Neural Network-based NLP, such as LSTM machine translation!
This project ONLY requires a template library for linear algebra, Eigen (http://eigen.tuxfamily.org/index.php?title=Main_Page)
Please note that this project started just for fun as my hobby, but sometimes it can be used to develop state-of-the-art models!
The LSTM implemented in this project employs a variant of the major LSTM's gate computation where previous cell states are used to compute input/output gates. See [1, 2] for the simplified version of the LSTM implemented here.
[1] http://arxiv.org/abs/1410.4615
[2] http://nlp.stanford.edu/pubs/tai-socher-manning-acl2015.pdf
BlackOut [3, 4] is an approximation method to softmax classification learning with the large number of classes.
[3] http://arxiv.org/abs/1511.06909
[4] https://github.com/IntelLabs/rnnlm
Layer Normalization [5] is a normalization method for deep neural networks and it can be easily applied to recurrent neural networks, such as LSTMs.
[5] http://arxiv.org/abs/1607.06450
-
select your appropriate compiler (now, Linux or Mac OSX)
CXX=g++ # for linux
CXX=clang-omp++ # for Mac OSX Yosemite (suggested by xuanchien@github) -
modify the line in Makefile to use Eigen
EIGEN_LOCATION=$$HOME/local/eigen_new #Change this line to use Eigen -
run the command "make"
-
./run the command "n3lp", and then the seq2seq model training starts (currently)
Feel free to tell me (hassy@logos.t.u-tokyo.ac.jp) if you are using N3LP or have any questions!
-
Neural Machine Translation with Source-Side Latent Graph Parsing (Hashimoto and Tsuruoka, EMNLP, 2017)
Paper: https://arxiv.org/abs/1702.02265
Project page: http://www.logos.t.u-tokyo.ac.jp/~hassy/publications/arxiv2017latent -
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks (Hashimoto et al., EMNLP, 2017)
Paper: https://arxiv.org/abs/1611.01587
Project page: http://www.logos.t.u-tokyo.ac.jp/~hassy/publications/arxiv2016jmt -
Learning to Parse and Translate Improves Neural Machine Translation (Eriguchi et al., ACL, 2017)
Paper: https://arxiv.org/abs/1702.03525 -
Tree-to-Sequence Attentional Neural Machine Translation (Eriguchi et al., ACL, 2016)
Paper: http://www.logos.t.u-tokyo.ac.jp/~eriguchi/paper/ACL2016/ACL2016.pdf
Code: https://github.com/tempra28/tree2seq -
The UT-AKY/KAY systems at WAT 2016 (Eriguchi et al., WAT, 2016; Hashimoto et al., WAT, 2016)
Paper (UT-AKY): https://aclweb.org/anthology/W/W16/W16-4617.pdf
Paper (UT-KAY): http://www.logos.t.u-tokyo.ac.jp/~hassy/publications/wat2016/paper_kay.pdf
- Kazuma Hashimoto - Mainly developing this project
- Akiko Eriguchi - Developing practical applications
MIT licence