Recently I’ve been thinking quite a lot about information availability for agents, and the fact that this is a very difficult and potentially irresoluble problem. Let me try to explain myself.

I talked before about a mental model on differentiating between the LLM models and the tools that access those models. I think that now that’s clearer as we are using more and more agents. We understand that we can use Claude Code with different models (like Sonnet or Opus) that change the capacity of the agent, but not its capabilities. The same Claude Code can read and write files, access the internet, communicate with MCP servers or even launch smaller agents. But a bigger model is capable of greater orchestration, for longer and more complex tasks¹.

But as we are developing our specific agents to perform our tasks, we need to change our focus from the model itself to the other part of the equation, the different tools that feed information to the model.

Ok, let’s use an example

To illustrate, let’s say that we want to create an agent that helps us with our email flow. Everyone receives too much email these days, and it’s difficult to be sure that we are ignoring the superfluous and attending to the important.

Perfect, we can design an agent that connects to our email inbox and categorise our emails into three priorities: P1, P2 and P3:

• P1 are the most important ones, emails that are important to answer quickly
• P2 emails should be read, and perhaps answered, but they can be done later.
• P3 are the emails that are safe to ignore

Connecting our agent to our inbox is not very difficult, we can program our own agent, and use a library to do so, using credentials. There are also available MCP servers². But we need to be more precise about what we mean

Which ones are the important emails? Which ones are the ones safe to ignore?

It’s all quite personal… We can easily discard most of the newsletter that we receive. Most are just marketing notifications that are not that interesting. Except that sometimes, for some offers, we may be. Perhaps you are not actively looking for new running shoes, but if the brand you use have a 30% discount or better, you’ll consider it. Perhaps in that case ideally the agent should move those emails to P2, instead of P3.

P1 emails are probably easier to recognise. If the email is part of an active thread, it’s a very safe P1, and if they are addressed specifically to you perhaps is also a P1… or maybe they are a cold calling that you want to ignore (P3), or at least maintain as a P2.

All of this is not an easy question for a model. The model only knows its training (the general information the model was created with) and the context that we are providing. Just the training is too broad and ambiguous.

For AI models, everything that’s not in the training needs to be in the context

We need to add enough context to the model call to get a good result. It is actually more important to have a good context than a very capable model! Our task of categorising each email doesn’t require a big model³. A good prompt, for sure, but also more contextual information that can help the model to make its job.

For the email categorisation, some elements that can be provide context:

• My address book, so I treat preferentially the people there. Even better if I have some extra info like “Mom” (P1) or “John, car seller” (P2)
• Calendar events. If an email is related to an event, is important to push it up the queue⁴
• A collection of previously selected emails, with their priorities, that can act as examples⁵
• Emails detailing purchases in the last year, in case that there’s some offer that it may be related, as a way of pushing those 1% marketing emails that you may be actually interested in.⁶
• And so on..

Each individual element is not necessarily either difficult or complex. But the accumulation of all of them are. Adding context to a task makes it more expensive, as the number of tokens grow. And there’s a limit on the context window that the model can use.

The design of the tools that the agent use is also important. A generic tool will likely retrieve too much information and provide little focus on the task you want the agent to do. This is the problem with many MCP servers. They are very flexible and allow many operations, like allowing to read email, send email, categorise email, delete, etc. But the same flexibility means that they are not focused for a task in particular, and provide extra context for your task that it’s, effectively, noise. A very common problem with MCPs is have problems when activating multiple of them at the same time.

There are some strategies that can be done, like summarise input information to avoid long verbose texts (in our example, summarise the purchase and example emails). Make the tools smart to allow searching and retrieving information (add a tool to search the address book, over handling always all the information).

(By the way, I discuss a similar agent accessing your email and go over problems, tools, etc in Chapter 14 of the 3rd edition of the Python Automation Cookbook. The raw code is available in GitHub, but the important discussion is on the book, if you’re interested)

More info, more problems

But as the tasks grow and agents become more capable, this problem multiplies. The you try to deal with agents in a work environment, you start realising that you need information from many sources:

• The ticketing system
• Company wiki
• GitHub repos
• Emails
• A lot of conversations happen in a chat system like Slack or Teams.
• HR applications to be able to pull information like days off
• Other services related to other tasks like logging, metrics, alerts, etc
• …

And you know what’s the worst? A lot of this stuff is not even new… We have been dealing with pretty bad systems integration since forever. Even tasks that doesn’t require any AI to be able to be done like coordinating your email and chat with your PTO is still highly manual, requiring to manually set an out-of-office message.

To be able to fully fulfil the promise of AI Agents and be able to achieve really complex tasks, they require a level of integration with information from so many sources that is going to be difficult. Not in a technological sense, each system may provide an API or some way to obtain information. But information, by its own nature, is fragmented. Is duplicated, it has typos, different formats, errors, is contradictory, it’s full of noise or it’s not even captured digitally⁷

And, most importantly, is mixed altogether. The same person will treat very differently an email from their mother than one from their boss. The information to integrate in each case is completely different, and what’s noise in one case is relevant for the other. Having an agent managing a business trip or a holiday with your family requires the agent to behave very differently.

A recent example. I had been discussing a particular problem with a coworker, and I kept researching, so I asked ChatGPT about it… which then returned me links to a conversation that I had previously in Teams, because the integration was enabled. That’s not what I wanted! I wanted the model to research, likely searching in the Internet about the problem, not to feed me back the conversation I already had.

Because in the Real World™️ it’s pretty normal to start a conversation with your boss in Slack, make a call, receive an email, update a ticket, and comment the result later in a WhatsApp group⁸. And it makes sense in your head⁹, but the data treatment of it is a big challenge.¹⁰

I think that there may be an irresoluble complexity at the core of integration, which can present limits to all of those problems.

I don’t know if models will advance so much that they will be able to compile all this fragmented data to the point that they are able to make sense of everything. But for sure to be able to get there we will need to generate a huge effort in data infrastructure to capture and present the relevant information at the relevant time. Which I think is going to be a huge challenge for AI adoption and success.

And given that the “information era” only created a tremendous amount of collected data that requires a massive amount of (human) effort into transforming it in something useful, I’m not entirely sure if it will be possible, or the laws of entropy will prevail.

1. And they are more expensive ↩︎
2. Which is probably not the right answer, I will elaborate later ↩︎
3. A small model also makes this economically feasible. If you try to analyse each email using Fable, your wallet is going to suffer ↩︎
4. This opens the possibility also to use the agent to write events in the calendar, based on emails ↩︎
5. This may provide a feedback loop to the agent, but it’s not as easy as it sounds… It may take a while until is properly fine-tuned ↩︎
6. This may lead to the trap of “you are interested in washing machine as you purchased one last week”, though ↩︎
7. I use a physical notebook to write my own things. I highly recommend it ↩︎
8. There’s the argument that, for personal information, most information is likely in your phone. Meaning that it may be easier to retrieve and integrate. While fragmented, it’s probably less so than a corporate environment dealing with thousands of people, many with very different requirements. ↩︎
9. And then we are surprised when we cannot find exactly when we read a particular comment ↩︎
10. I am not even commenting the privacy and security implications. Grating a highly personal level of access to an AI company to your data is scary at multiple levels. We are already creeped out by targeted ads and joke about “delete my browser history”. ↩︎