Recently, I found myself in a rather curious conversation with someone who competes in speed puzzling. There were a few others in the group who loved puzzles too, though not quite to the extent of racing against a clock for them. And as she went about explaining how she solved them, something rather fascinating began to unfold.

It wasn’t merely that the others couldn’t quite follow her method. She seemed genuinely puzzled that they couldn’t see what was so patently obvious to her. And, delightfully, she wasn’t having much success explaining it either! One could almost see the two sides making an earnest attempt to meet somewhere in the middle, each becoming a little more bewildered by the other’s bewilderment.

I could almost see their minds being wired differently. Give them the same puzzle, the same pieces, the same picture on the box — and somehow they would not be looking at quite the same thing. What had become instinctive to one was still a succession of deliberate decisions to another. The peculiar thing about expertise is that sometimes even the expert cannot tell you what it is that they have learnt to see.

That conversation has stayed with me. Rather unexpectedly, it has also been making me think about AI agents, and how on earth they perceive the software we build.

Read more: Discoverable Is Not Callable

What does an agent actually see?

We have developed this convenient habit of personifying agents. They see our applications, read our documentation, understand a product, decide what to do next. It makes for wonderfully convenient conversations. But it also quietly suggests that there is another person sitting before the computer, only one with a prodigious appetite for manuals and a curious tolerance for error messages.

For us humans, the physical world offers a rather generous common ground. Photons strike our eyes. Sound waves arrive at our ears. We touch things, turn them around, grow familiar with how they behave. My speed-puzzling acquaintance and the people listening to her may have interpreted the world quite differently, but at least they had much the same physical world to begin with.

An agent built around a large language model does not have that particular luxury. There is fascinating work being done on world models, embodied intelligence and multimodal systems, of course. And agents can already inspect screenshots, listen to audio and interact with browsers. But the information that reaches the model has travelled through an altogether different apparatus. The photons to our eyes and the sound waves to our ears become, for the model, digital representations — tokens, encoded images, structured tool outputs, documents, and a great many bytes that somebody, somewhere, decided were worth exposing.

That is an analogy about the means of perception, not an assertion that a language model experiences the world as we do. But it leads to a rather extraordinary thought.

The very sentence I am writing in this blog post might become part of how an agent constructs its understanding of the world in which my software exists. If it encounters this page in a search, a retrieval result or its working context, these words are no longer merely my explanation to another human. They are part of the material from which it must infer what is out there.

A README written in haste. A command-line flag described in a help screen. An old webpage that was never retired. A schema with an unfortunately named field. For the agent, all of these can be evidence of what a product is, what it does, and perhaps what it is supposed to do.

We spend extraordinary amounts of ingenuity making software intelligible to human senses. Colour, hierarchy, affordances, empty states, the precise placement of a button, sometimes the little animation that reassures you that the button actually did something. Yet our newest user may encounter the product through none of those things, or through a rather impoverished translation of them.

And therein lies a peculiar problem. An agent can know an astonishing amount about your product, and still have very little idea how to use it.

Knowing is not doing

For much of computing, software had two fairly distinct audiences. Humans interacted with interfaces. Machines interacted with APIs. We designed one for exploration and judgement, the other for execution. It wasn’t an immaculate division, but it served us rather well.

Agents sit awkwardly between the two. They read like humans, invoke things like software, and compose like programmers. They can infer an intention from an imprecise request, work out which capabilities might be useful and, if we have been thoughtful enough to make those capabilities reachable, try putting them to work.

Suppose you ask an agent to check whether the API you’ve just implemented still conforms to its specification. It may correctly identify a testing tool, explain precisely why it is suitable, and give you a rather convincing account of how the verification ought to be performed.

What would your next sentence be?

Probably: Great! Do it.

And here the apparent intelligence of the recommendation collides rather unceremoniously with the reality of the product. Perhaps the agent can invoke the tests, inspect the results and continue. Perhaps it discovers that the product has a beautiful dashboard, a splendid collection of feature pages, and absolutely no interface it can reliably operate from where it is working.

In which case, the answer becomes something along the lines of: You’ll need to do that part yourself.

Quite a fall from grace.

This is what I mean by the difference between discoverability and callability. Being known to the agent is not the same as being operable by it. An agent might recommend software it cannot use; conversely, it might make extraordinary use of an unglamorous tool whose capabilities happen to be within reach.

I don’t think this requires us to invent a grand theory about how language models rank products, or claim that they secretly prefer CLIs over graphical applications. The pressure is more prosaic. A recommendation that can be acted upon leads to an outcome. One that cannot often leads to more work for the human. Over enough interactions, that difference becomes consequential.

And callability is hardly a synonym for the command line. It might arrive through a CLI, an API, an MCP server, or even a browser that the agent can operate. Each comes with its own little assortment of virtues and miseries. What matters is that the capability is reachable, its behaviour understandable, and its failures something the agent can recognise and recover from.

The software no longer ends at the software

There is another consequence here, one that I suspect will take us builders a little longer to internalise.

Documentation traditionally sat downstream of the product. We built the thing. Then we described the thing. Marketing sat further downstream still, explaining why somebody should care about the thing. The interface, meanwhile, had the distinguished responsibility of actually letting somebody do the very thing!

That arrangement becomes considerably less tidy when an agent consumes all of these as evidence for deciding what to do. Its model of the product may be assembled from documentation, examples, API schemas, CLI help, repository instructions, skills, error messages, and occasionally a webpage that somebody forgot existed.

A stale sentence can now behave a little like a product bug. An ambiguous parameter can behave like a broken affordance. A well-written error can become a recovery path. A clear example can become something close to executable understanding. The boundary between product, documentation and distribution starts getting strangely porous.

This does not mean that writing a more beautiful README magically makes a product agent-ready. The agent still needs permissions, reliable execution, observable results and some means of knowing that the action actually worked. And an incorrect example can mislead it with an enthusiasm that would make a confident human look positively restrained.

Nevertheless, these things around the product are no longer merely about the product. They increasingly participate in its operation. For a user whose experience is mediated through language and tools, the distinction is rather difficult to maintain.

Building for a different pair of eyes

I don’t think this means abandoning the graphical software we have spent decades learning to build. Humans still need exploration, direct manipulation, shared state and the comfort of being able to see what is happening. There will remain good reasons to design a workflow explicitly, and even better reasons not to let an autonomous caller loose upon every consequential operation.

But the existence of this different user changes what it means for a capability to be well designed. It is no longer enough to ask whether somebody can find the right button. We must also ask whether the agent can discover the capability, understand its purpose, invoke it within the right boundaries, inspect the outcome and, when things go awry, find its way back.

Some of that work belongs in executable interfaces. Some belongs in schemas, descriptions and documentation. Some belongs in the deliberately unromantic business of permissions, errors and verification. Curiously, the pieces we have long treated as supporting cast are now auditioning for rather more important roles.

For a long time, a product’s website was largely its shop window. We described what the software could do, persuaded somebody that it was worth their attention, and ushered them towards the actual experience. That experience lived elsewhere — inside an application, behind a login, or after an installation. Documentation sat nearby, rather like a manual one might consult when things became sufficiently troublesome.

What an odd arrangement this becomes when the visitor is an agent.

It might encounter your product through a search result, a README, an API specification, some forgotten documentation, or a rather enthusiastic blog post written three years ago. From these fragments, it must construct an understanding of what your software is, what it is capable of, and whether any of those capabilities might help with the task at hand.

And this is where something rather fundamental changes.

Your product’s presence on the web is no longer merely describing the product. It is increasingly participating in the experience of the product itself.

A beautifully designed application can be almost invisible to an agent, while an obscure command mentioned in a well-written document might become the means through which it accomplishes something extraordinary. Conversely, describing an impressive capability without providing any means of invoking it leaves the agent with a rather peculiar predicament — it knows what ought to be possible, but cannot actually do it.

Neither better documentation nor another machine-readable interface solves this in isolation. The agent must be able to discover the capability, understand why it exists, find a way to invoke it, and make sense of what happens afterwards. The web presence, the executable interface, the error messages, the examples — all of them now participate in what we used to call the product experience.

We have spent decades perfecting the experience inside our software, while treating much of what surrounds it as supporting material. It would be rather amusing if the next great shift in product design demanded that we pay equal attention to everything outside it.

I keep returning to that conversation with the speed puzzler. What struck me was not merely that she solved puzzles faster, but that the possibilities available to her seemed almost invisible to those beside her. And for all her expertise, she could not simply lend them her way of seeing.

We software builders have a curious advantage here. We get to design not merely the puzzle, but also the means through which somebody encounters it.

Perhaps that, then, is our new obligation. Not simply to make our products discoverable by AI, nor merely to expose a few commands for it to call, but to build software that an entirely different kind of intelligence can meaningfully experience — discover, understand, operate, and recover from when things go wrong.

And that experience may begin long before the AI agent ever opens our application.

It may begin with the first sentence it reads about us.