Capsule networks are a sophisticated neural network architecture originally proposed in the domain of computer vision. Motivated by some of the main problems with convolutional neural networks for image classification, capsule layers allow low-level features about an image to be effectively combined into higher-level information. In linguistic terms, we might call this compositionality. Given the fact that compositionality exists in natural language, we might wonder whether this kind of architecture could be useful for NLP tasks. Three recent papers have shown that capsule networks do in fact perform well on text classification tasks. One of the other big takeaways from these papers is that the capsule representation allows for effective transfer learning.
Last semester in school, I worked on a project where we attempted to build word embeddings from the Voynich manuscript and use them to learn something about the linguistic structure (or lack thereof) within the text. You can check out our codebase or our paper. The reason why this methodology is exciting is that building word embeddings is completely unsupervised. This makes it a very appealing methodology for decipherment-like tasks. If done properly, we should be able to visualize the word embedding space and understand something about the relationship between words and characters in the manuscript!
I’ve decided to share a literature review blog post about Learning to Transduce with Unbounded Memory (Grefenstette et al., 2016). I’ve found that blog posts summarizing literature have been helpful to me in the past, and after having written this review for a class, I thought it might be useful to other people to share it on my blog.
Ragnarok is a well known part of the Norse mythological canon presented by Snorri Sturluson. But what happens after the world is destroyed? Gylfaginning actually tells us, and I’ve translated the corresponding section below. There’s a lot that might be said about how this passage bears on the authenticity of the Ragnarok story, but I won’t go into that. It’s an interesting story that I think is more fun just to read for yourself.
I’ve integrated MathJax onto my blog so that I have the capability to display equations in LaTeX. For example, here is a description of the update rule for gradient descent:
In order to practice my Old Norse, I decided to write up a thorough translation of a short excerpt from the Icelandic Saga of Mary. I’ve given the original text underneath, which comes from A New Introduction to Old Norse edited by Anthony Faulkes. Please let me know if you have any corrections or suggestions! If I post more translations here, I will try to choose longer and more interesting passages.
Verbum antiquum amo. Ut animis patiar lego qui sunt distincti tempore meo. Per literas sententias discessorum sentire possumus.
This summer, I had the opportunity to do NLP research at the Boston College Language Learning Lab (L3) under the guidance of Josh Hartshorne and Sven Dietz. My work at L3 focused on two main tasks: fluency assessment and native language inference from second-language text. These problems were cool both because of their practical importance (for teaching languages among other applications), and because I got to experiment with interesting methods to attack them like convolution, dependency parsing, and kernel classifiers.
This is my personal blog. For more information about me or what I work on, check the links in the nav bar.