A few weeks back I wrote about AGI. About what's actually needed to get there. I said two things. Continual learning, and what I called the context problem.

The context problem isn't about making windows bigger. It's about removing context as a bottleneck altogether. The way humans don't hit a wall at 200,000 tokens and forget the start of a conversation.

A team from Nanyang Tech, Fudan, Shanghai Jiao Tong and a few other universities just dropped a paper that pokes at exactly this. With sixty-four numbers.

That's not a metaphor by the way. The actual answer they're proposing is an 8x8 matrix. Sixty-four cells.. smaller than a sudoku puzzle.