Browsing by Subject "finite-state capacity"

Sort by: Order: Results:

Now showing items 1-1 of 1
  • Yli-Jyrä, Anssi (Linköping University Electronic Press, 2019)
    NEALT Proceedings Series
    Deep neural networks (DNN) and linguistic rules are currently the opposite ends in the scale for NLP technologies. Until recently, it has not been known how to combine these technologies most effectively. Therefore, the technologies have been the object of almost disjoint research communities. In this presentation, I first recall that both Constraint Grammar (CG) and vanilla RNNs have finite-state properties. Then I relate CG to Google’s Transformer architecture (with two kinds of attention) and argue that there are significant similarities between these two seemingly unrelated architectures.