OpenAI Assistants Versus Your Own Loop

There are two ways to run an agent. You can hand the conversation to a hosted assistant API — it stores the thread, assembles the context, runs the tools, and hands you a reply. Or you can own the loop: your code holds the message array, decides what goes into the window, parses every tool call, and talks to a plain chat endpoint.

Both ship. The choice looks like a build-or-buy argument about a week of orchestration work. It is really a question about what you will be able to explain during the first incident.

Why this exists

The failure mode is not an outage. It is a Tuesday afternoon when a customer got a wrong answer, you have the output and the token count, and you cannot reconstruct the input.

With a hosted thread you did not build that prompt. The service assembled it from stored messages, retrieved file chunks, tool results, and a truncation rule you do not control. Which of the eleven earlier turns were still in the window? Which chunks did file search return? You can guess. You cannot replay.

So here is the test that tells you which side of the line you are on, whatever your code looks like: take yesterday’s worst request and replay it offline, byte-identical, with no network calls. If you cannot produce the exact array you sent, you do not own the loop. Everything else here follows from that.

How to use it

Start hosted when the thing might be canceled. Most prototypes are, and the hosted route gets you to a demo in days with retrieval and search you would otherwise spend a month on. That is a real gain, and refusing it on architectural principle is how you lose the argument before you have a product.

But log as though you were leaving. From the first commit, keep your own record of every turn: the messages you contributed, the identifiers of everything you did not, the model, the parameters, each tool call and its result, and the full output. That log is the migration. With it, moving is a week. Without it, the first month goes on reconstructing your own behavior from support tickets.

Then price the move in three parts, because only the first one ever gets estimated:

• Orchestration. A state machine over a chat endpoint, with retries and tool dispatch. Real work, roughly a week.
• The hosted tools you now own. The published tool surface is functions, web search, file search, and computer use. Functions you already wrote. The other three you did not. file_search becomes chunking, embeddings, an index, and retrieval quality you are now accountable for. This is the actual bill, and it is the one nobody prices.
• Defaults you never specified. Truncation, message ordering, system-prompt shaping, retry semantics. You never wrote them down, so you cannot port them — you rediscover them as regressions.

The middle position beats both ends: own the loop, rent the tools. Keep message assembly and the state machine in your code, and call hosted search or retrieval as ordinary functions. Switching cost becomes per-tool instead of per-system, and each tool is a day.

Where it goes wrong

Owning the loop means owning the dull half — truncation policy, idempotency on retries, partial tool failure, timeouts, conversation storage, and a migration every time a message shape changes. None of it is hard. All of it is yours at three in the morning.

Do not own the loop if one person is holding the system and nobody else understands it. That trades a vendor dependency for a staffing dependency, and the staffing one takes vacations.

Do not migrate on taste. If the hosted version passes the replay test well enough for your risk level, and nobody is asking you to prove what happened, the honest recommendation is to leave it alone and spend the week on evals instead.

And be clear about what owning the loop does not buy. It buys explainability, not accuracy. A fully reproducible pipeline gives you the exact input that produced a wrong answer, which is worth a great deal — and is still a wrong answer.