Publications

Highlights

The compositionality of neural networks: integrating symbolism and connectionism.

The compositionality of neural networks has been a hot topic recently, on which no consensus exists. In this paper, we argue that this in part stems from the fact that there is little consensus on what it means for a network to be compositional, and we present a set of five tests that provide a bridge between the linguistic and philosophical knowledge about compositionality and deep neural networks. We use our tests to evaluate a recurrent model, a convolution-based model and a Transformer, and provide an in depth analysis of the results.

Dieuwke Hupkes, Verna Dankers, Mathijs Mul and Elia Bruni

preprint

Learning compositionally through attentive guidance

While neural network models have been successfully applied to domains that require substantial generalisation skills, recent studies have implied that they struggle when solving the task they are trained on requires inferring its underlying compositional structure. In this paper, we introduce Attentive Guidance, a mechanism to direct a sequence to sequence model equipped with attention to find more compositional solutions.

Dieuwke Hupkes, Anand Singh, Kris Korrel, German Kruszewski and Elia Bruni

paper

 

Full List

The compositionality of neural networks: integrating symbolism and connectionism.
Dieuwke Hupkes, Verna Dankers, Mathijs Mul and Elia Bruni   •  

Mastering emergent language: learning to guide in simulated navigation
Mathijs Mul, Diane Bouchacourt and Elia Bruni   •  

Assessing incrementality in sequence-to-sequence models
Dennis Ulmer, Dieuwke Hupkes and Eia Bruni   •   Repl4NLP,  ACL 2019

Assessing incrementality in sequence-to-sequence models
Dennis Ulmer, Dieuwke Hupkes and Elia Bruni   •   Repl4NLP,  ACL 2019

Transcoding compositionally: using attention to find more generalizable solutions
Kris Korrel, Dieuwke Hupkes, Verna Dankers and Elia Bruni   •   BlackboxNLP,  ACL 2019

On the Realization of Compositionality in Neural Networks
Joris Baan, Jana Leible, Mitja Nikolaus, David Rau, Dennis Ulmer, Tim Baumgärtner, Dieuwke Hupkes and Elia Bruni   •   BlackboxNLP,  ACL 2019

The Fast and the Flexible: training neural networks to learn to follow instructions from small data
Rezka Leonandya, Elia Bruni, Dieuwke Hupkes and German Kruszewski   •   IWCS 2019

Learning compositionally through attentive guidance
Dieuwke Hupkes, Anand Singh, Kris Korrel, German Kruszewski and Elia Bruni   •   CICLing 2019

Do language models understand anything? On the ability of LSTMs to understand negative polarity items
Jaap Jumelet and Dieuwke Hupkes   •   BlackboxNLP,  EMNLP 2018