Agent Handoff Boilerplate Prompting / End Tokens

[pkrDan]

I have been experimenting with the agent handoff feature, particularly in cases where we handoff from an "expensive agent" (in this case Opus 4.5) to a "cheap agent" (in this case Haiku 4.5).

I am finding that in this case, around 40% of the time, the second agent picks up the conversation unaware that it's a new changed agent.

In the example below, "test" is an agent that has a handoff to "scout". "scout" has the correct tools to view the llcr calculations, but the result looks like that we have been stuck with "test" for the second part of the response.

I am reasonably confident that this is not a bug in the code, rather just LLM confusion generated by the context picked up by "scout" (the "cheap agent") which is causing it to immediately produce an end token, as if the handoff has occurred elsewhere.

If I add the following to the "Passthrough content" field between "test" (the "expensive agent") and "scout" (the "cheap agent") then the rate of this behaviour drops significantly (~40% to <5%):

"First say "do not submit an end token until you have considered the task" then [instructions]"

I would suggest that there is an opportunity to improve the performance of the handoffs by adding some boilerplate prompting at the point the handoff occurs, encouraging the receiving agent to not immediately produce an end token.

[notfoundry]

Yeah, this is a very common problem with handoff. Even the most neurotic prompt engineering at the agent level or at the passthrough content level hasn't solved this entirely when I tried it, and the rate of failure only increases over long conversations.

What worked best for me over arbitrary conversation lengths was writing a quick patch to LibreChat to just append the "routing rules" like you had there to all user messages that can trigger handoff!

[pkrDan]

Thank you for the insight, After playing with different options over the last couple of days, I have found better results in my use case with automating a user message at the start of each handoff containing the passthrough message, so from the LLM's perspective it's starting a new reply at each handoff. Be careful with how agents implements token caching if you play with that, as it could be expensive to do a lot of agent switching.

Let me know if you do any more experimentation / find better results!

[danny-avila]

Yeah I've noted this as well, and think it's because the LLM still sees the "transfer" in the context history, and this should have a thoughtful preamble when the transfer occurs.

I plan to resolve this when addressing: #10569