TALK.md — Neural Turing Machines: Coverage, Q&A, and Citations

Press & Blog Coverage

  1. MIT Technology Review (2014) — Will Knight covered NTM under the headline “This Computer Is the Brain a Deep-Learning Pioneer Has Been Dreaming Of,” noting that DeepMind’s approach of coupling neural networks with external memory represented a step toward machines that can learn algorithms rather than just patterns. (Article URL has since been reorganised by MIT Technology Review.)

  2. DeepMind Blog (2014) — DeepMind published a blog post announcing the NTM architecture alongside the arXiv paper, describing it as “a neural network architecture that takes inspiration from both models of biological working memory and the design of digital computers.”

  3. Two Minute Papers (YouTube) — The popular AI explainer channel covered NTM in a video titled “Neural Turing Machines — Computers That Learn Like Humans,” highlighting the differentiable memory and the copy-task generalisation results.

  4. Hacker News (October 2014) — The paper reached the front page of Hacker News, generating extensive discussion about whether NTMs truly “learn algorithms” or merely learn smooth approximations, and comparisons to symbolic AI approaches.

  5. Reddit r/MachineLearning — Discussed extensively, with practitioners noting the difficulty of training NTMs (instability, sensitivity to hyperparameters) and the significance of the generalisation results.

Interview-Style Q&A

Q1: What is the key difference between an NTM and a standard LSTM?

A: A standard LSTM stores all its “memory” in the hidden state vector — a fixed-size representation that must encode everything the network needs to remember. An NTM separates the controller (the “CPU”) from the memory matrix (the “RAM”), giving the network an explicit, addressable external memory that it can read from and write to via attentional heads. This separation means memory capacity scales with the size of the memory matrix (N×M) without increasing the number of controller parameters, and the network can learn to use memory slots like registers in a computer.

Q2: Why is the memory access called “blurry” or “soft”?

A: In a real Turing machine or digital computer, the read/write head accesses exactly one memory location at a time — a hard, discrete operation. NTMs instead use normalised weightings over all N locations, so each read blends together (via a convex combination) the contents of multiple memory rows, and each write modifies multiple rows to varying degrees. This “softness” makes every operation differentiable, enabling gradient-descent training. The sharpening parameter γ lets the network approach hard addressing when needed by concentrating the weighting on a single location.

Q3: How does NTM generalise to sequences longer than it was trained on?

A: The copy task experiments trained on sequences of length 1–20, but the NTM could copy sequences of length 100+. This works because the NTM learns an algorithm (write each input to a sequential memory slot, then read them back in order) rather than memorising specific input-output mappings. The algorithm is length-invariant — the write head simply iterates through memory locations, and the read head follows the same pattern. A standard LSTM, by contrast, tends to encode the whole sequence in its hidden state and degrades when the sequence exceeds the lengths it was trained on.

Q4: What was the relationship between NTMs and the later Differentiable Neural Computers (DNCs)?

A: DNCs (Graves et al., Nature 2016) are the successor to NTMs. The key improvement is the addressing mechanism: DNCs replace the NTM’s somewhat opaque location-based addressing (interpolation + circular convolution + sharpening) with a more interpretable mechanism based on “usage” tracking — the controller can allocate free memory slots, deallocate them after use, and track temporal ordering of writes. DNCs also added a temporal linkage matrix that records which memory location was written after which, enabling sequential retrieval. DNCs demonstrated performance on more complex tasks like question answering on the bAbI dataset.

Q5: What are the main practical limitations of NTMs?

A: (1) Training instability — NTMs are notoriously difficult to train; small hyperparameter changes can prevent convergence. (2) Slow training — the sequential read/write operations make each forward/backward pass expensive. (3) The architecture was eventually superseded by attention-based transformers (Vaswani et al. 2017), which achieve similar content-based “soft lookup” functionality more efficiently using the attention mechanism over key-value pairs, without the complex location-based addressing machinery. (4) The tasks NTMs were demonstrated on (copy, sort, associative recall) are simple algorithmic benchmarks, not real-world applications.

Common Misconceptions

  1. “NTMs are Turing machines.” No — they are analogous to Turing machines. Real Turing machines use discrete, hard memory access and infinite tape. NTMs use differentiable, soft (weighted) access and finite memory. The name is an analogy, not a literal equivalence. The paper itself says “Unlike a Turing machine, an NTM is a differentiable computer.”

  2. “NTMs were the first neural networks with external memory.” Not quite — earlier work by Das et al. (1992) used recurrent networks with external stack memory, and the Neural Network Turing Machine concept has roots in the 1990s. NTMs were the first to make the full read/write/addressing pipeline differentiable end-to-end and demonstrate algorithmic generalisation.

  3. “NTMs can learn any algorithm.” Theoretically, RNNs are Turing-complete (Siegelmann & Sontag 1995), and NTMs extend RNNs, so they have the capacity to simulate arbitrary procedures. But “having capacity” is very different from “being able to learn from data.” The paper demonstrates only simple algorithms (copy, sort, recall). Learning complex programs remains an open problem.

  4. “The copy task is trivial.” For short sequences, yes — any RNN can copy. The key result is generalisation: the NTM trained on length ≤20 copies length 100 perfectly, while LSTM fails catastrophically. This is evidence the NTM learned the algorithm of copying, not just a pattern.

Real Citations

  • Graves, A., Wayne, G., & Danihelka, I. (2014). Neural Turing Machines. arXiv:1410.5401.
  • Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., et al. (2016). Hybrid computing using a neural network with dynamic external memory. Nature, 538, 471–476. (DNC paper)
  • Weston, J., Chopra, S., & Bordes, A. (2014). Memory Networks. arXiv:1410.3916.
  • Sukhbaatar, S., Weston, J., & Fergus, R. (2015). End-to-End Memory Networks. NeurIPS 2015. arXiv:1503.08895.
  • Siegelmann, H. T., & Sontag, E. D. (1995). On the computational power of neural nets. Journal of Computer and System Sciences, 50(1), 132–150.
  • Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780.
  • Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv:1409.0473. (Related attention work)
  • Vaswani, A., et al. (2017). Attention Is All You Need. NeurIPS 2017. (Transformers as a successor paradigm using content-based attention)