Skip to content

fix(deepseek,litellm): show the max-output notice when a streamed reply hits the length limit - #1499

Open
pcbeingused333 wants to merge 1 commit into
SciSharp:masterfrom
pcbeingused333:fix/deepseek-litellm-stream-length
Open

pcbeingused333 wants to merge 1 commit into
SciSharp:masterfrom
pcbeingused333:fix/deepseek-litellm-stream-length

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

No linked issue (sibling of #1488, #1495 and #1497: found by comparing the streaming and non-streaming finish handling of the DeepSeek and LiteLLM providers).

When a streamed DeepSeek or LiteLLM response ends with finish_reason=length, GetChatCompletionsStreamingAsync took the generic else if (choice.FinishReason.HasValue) branch and returned the truncated text as if it were a complete answer, with no warning. The non-streaming paths (GetChatCompletions, GetChatCompletionsAsync) of the same providers already log a warning and return "AI response exceeded max output length" for the same case.

Fix: in each plugin, ProviderHelper.ResolveStreamedContent maps the finish reason to the final content: Length returns the same notice as the non-streaming path (new MaxTokensMessage constant), anything else keeps the collected text. The streaming branch also logs the same warning as the non-streaming ones.

Test: DeepSeekAIStreamedContentTests and LiteLLMStreamedContentTests cover Stop (text and empty) and Length (text and empty). The Length cases fail with the old behaviour (the helper does not exist on main; the old branch returned the collected text). Adds project references to the two plugins in BotSharp.Core.UnitTests.

This change was prepared with the assistance of an AI coding tool (Claude Code); the author is responsible for the submission.

…ly hits the length limit

The streaming path returned the truncated text as a normal answer when
finish_reason was length; the non-streaming paths already return the
"AI response exceeded max output length" notice.
@qodo-code-review

Copy link
Copy Markdown
Contributor

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant