Jakub Pachocki has indeed done great work on the self-learning Dota video game player.

I like it how neuroscientists are now learning from machine learning engineers.

Didn’t Sam say he’ll be safe at Big Sur if everything goes to hell due to AI? Perhaps that wasn’t exactly what the board wanted to hear.

It may not be as flashy as NLP or CV …

In the sense that you can use it to predict the stock market, it’s pretty flashy.

However, pages 6 and 13 describe tricks for making loops:

One way to bypass this limitation is to exploit the autoregressive inference procedure. Since the model is called iteratively at inference time, this effectively provides an “outer-loop” that can enable a certain kind of sequential computation, where the sequential state is encoded into the prior context. This is exactly what scratchpads enable.

The RASP conjecture provides a natural way to understand why scratchpads (Nye et al., 2021; Wei et al., 2022) can be helpful: scratchpads can simplify the next-token prediction task, making it amenable to a short RASP-L program. One especially common type of simplification is when a scratchpad is used to “unroll” a loop, turning a next-token problem that requires n sequential steps into n next-token problems that are each only one step. The intuition here is that Transformers can only update their internal state in very restricted ways —given by the structure of attention—but they can update their external state (i.e. context) in much more powerful ways. This helps explain why parity does not have a RASP-L program, but addition with index hints does. Both tasks require some form of sequential iteration, but in the case of addition, the iteration’s state is external: it can be decoded from the input context itself.

Page 6 of the paper https://arxiv.org/pdf/2310.16028.pdf has the intuition behind the RASP programming language:

A key characteristic of RASP is that it only allows parallelizable operations, because
Transformers are an inherently parallel model of computation. This makes performing inherently-sequential computation, such as iterating through each input symbol and updating an internal state, tricky if not impossible to write in RASP. This is why loops are not allowed in RASP: a Transformer has only constant depth, and cannot directly simulate an arbitrary number of loop iterations.