cors-chat: fix llama-server rejecting every follow-up message - #313
Merged
Conversation
The Copy Markdown button made the conversation bar's three buttons squeeze the heading to a sliver on narrow screens, wrapping the meta line word-by-word. Ellipsize the meta line like the title, and on small screens wrap the bar so the heading gets its own full-width row with the buttons below it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WLaDaQJjq7ymoHdXh9KvNx
llama-server's /v1/responses parser only recognizes an assistant input item when it carries type: "message", and rejects the request with "Cannot determine type of 'item'" otherwise — which broke every follow-up message. OpenAI's canonical output-message shape includes the field too, so this is a no-op for endpoints that already worked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WLaDaQJjq7ymoHdXh9KvNx
Sending a second message in any conversation against llama.cpp's
llama-server failed with the error "Cannot determine type of 'item'",
while the first message in a conversation worked fine.
The cause is in how conversation history was replayed. Each request to
/v1/responses sends the full conversation as the `input` array, and
assistant turns were serialized as:
{ "role": "assistant", "content": [{ "type": "output_text", ... }] }
with no `type` field on the item itself. OpenAI's Responses API accepts
this "easy input message" shape for any role, but llama-server's
Responses-to-chat-completions converter (tools/server/server-chat.cpp)
is stricter: its lenient no-`type` branch only matches roles user,
system and developer, and its assistant branch requires both
role == "assistant" AND type == "message". An assistant item without
`type: "message"` therefore matches no branch and falls through to:
throw std::invalid_argument("Cannot determine type of 'item'");
That is why only follow-ups failed: the first request contains no
assistant history, every later request does.
Fix: include `type: "message"` on assistant items in responseInput().
This matches OpenAI's canonical output-message shape, so endpoints that
already worked are unaffected, and llama-server now accepts multi-turn
conversations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UjzUyC8mGYGDt5h5taGsMH
Contributor
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
simonw
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This one is a bit confusing because I pushed the fix in the wrong branch.