Repository navigation
fix(deepseek,litellm): show the max-output notice when a streamed reply hits the length limit - #1499
Open
pcbeingused333 wants to merge 1 commit into
Open
pcbeingused333 wants to merge 1 commit into
pcbeingused333 wants to merge 1 commit into
Conversation
…ly hits the length limit The streaming path returned the truncated text as a normal answer when finish_reason was length; the non-streaming paths already return the "AI response exceeded max output length" notice.
Contributor
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No linked issue (sibling of #1488, #1495 and #1497: found by comparing the streaming and non-streaming finish handling of the DeepSeek and LiteLLM providers).
When a streamed DeepSeek or LiteLLM response ends with
finish_reason=length,GetChatCompletionsStreamingAsynctook the genericelse if (choice.FinishReason.HasValue)branch and returned the truncated text as if it were a complete answer, with no warning. The non-streaming paths (GetChatCompletions,GetChatCompletionsAsync) of the same providers already log a warning and return"AI response exceeded max output length"for the same case.Fix: in each plugin,
ProviderHelper.ResolveStreamedContentmaps the finish reason to the final content:Lengthreturns the same notice as the non-streaming path (newMaxTokensMessageconstant), anything else keeps the collected text. The streaming branch also logs the same warning as the non-streaming ones.Test:
DeepSeekAIStreamedContentTestsandLiteLLMStreamedContentTestscover Stop (text and empty) and Length (text and empty). The Length cases fail with the old behaviour (the helper does not exist on main; the old branch returned the collected text). Adds project references to the two plugins inBotSharp.Core.UnitTests.This change was prepared with the assistance of an AI coding tool (Claude Code); the author is responsible for the submission.