AI Doesn't Learn People Yet

Over the past year, I've spent a lot of time thinking about what happens after the first interaction.

Not how these systems perform on a single task. That part is already impressive — models can answer questions, generate code, explain concepts, and in many cases do it better than most tools we've had before.

What I keep coming back to is simpler.

What happens next?

If you use most AI systems today, even the better ones, you'll notice a pattern. The first interaction is usually strong. By the third or fourth, you're repeating yourself — clarifying context, restating preferences, nudging the system back toward what you actually meant.

Nothing is obviously broken. It just doesn't seem to be accumulating anything about you.

I ran into this in a place where it was hard to overlook.

I've been using AI to help my son work through early math. Basic addition and subtraction — nothing where the concepts are complicated, which meant the feedback loop was fast and the sessions were short. Individually, each one looked fine.

After a few days, something felt off in a way that took a while to name.

He kept making the same kind of mistake in subtraction problems that involved borrowing. Not random slips — he'd get the easier problems right. It was that specific step, repeatedly, across several sessions. The system would explain it again if prompted. It would generate more practice. But the fact that this had been happening over and over didn't seem to register anywhere. Each session started more or less from scratch.

We were having the same conversation, slightly rephrased, multiple times.

The gap wasn't knowledge, and it wasn't really memory in the narrow sense — most systems can surface prior context if you structure things right. What was missing was something harder to name.

None of it was turning into a picture of him.

There's a kind of adjustment that happens when you spend time helping someone — a student, a colleague, anyone you're trying to work through something with. You start to carry a sense of where they are that isn't just recall. You notice the same hesitation showing up again. You remember what framing didn't work. Sometimes you change your approach before they've said anything, because something earlier told you to.

You don't wait to be asked the same question again. You get there first.

Most AI systems don't do that. Even with access to history, they're still largely operating exchange by exchange. What's happened before can inform the current response, but it doesn't reliably turn into something that carries forward and quietly changes what comes next.

The system feels capable, but not adaptive. It responds well. It doesn't learn you.

There's been real progress on agents that can act — call tools, complete workflows, iterate. That's meaningful. But doing things well isn't the same as understanding the person you're doing them for, and those have been moving at different speeds.

You feel it in small friction. Repeating preferences that shouldn't need repeating. Re-explaining constraints that haven't changed. Gradually adjusting how you use the system to work around it, rather than the other way around. After a while, it starts to feel less like something that's learning, and more like something that needs to be nudged back into the right state each time.

In my last post, I wrote about how memory in these systems is really an interpretation problem — deciding what something means and whether it should matter beyond the moment.

But even getting that right doesn't fully close this gap.

You can store the right things and still end up in the same place — slightly different versions of the same interaction, repeating on a loop, because nothing that was stored is changing what happens next. The distance between "having access to what occurred" and "being shaped by it" turns out to be significant.

I've been experimenting with a small system that tries to close that distance in a very narrow setting. It doesn't try to answer everything. It just tracks a learner over time, watches where things break, and tries to let that actually affect what it does next rather than just how it explains something in the moment.

Even in that constrained setup, something shifts. The same explanations stop resurfacing. The questions start to line up better with what's already happened. It starts to feel less like a series of independent exchanges and more like something that's slowly forming a picture.

I'm not sure what to call it. Personalization usually means something shallower — preferences, prompt adjustments, maybe retrieval layered in. That helps at the surface. It doesn't quite get at the idea of a system that's maintaining a working model of a specific person and letting that model change what it decides to do, not just what it says.

That part still feels mostly unsolved.

If these systems are going to move past answering questions and completing tasks, that's probably where the work is — not making models smarter in the general sense, but making them better at holding a picture of a specific person over time and actually being changed by it.

What that looks like at scale, or whether the current architectures can even support it cleanly, I'm genuinely not sure.

But the gap is real. And right now, most systems are on the wrong side of it.

They interact well.

They don't really learn people yet.