.Net: Update StepwisePlannerConfig to Improve MaxPromptTokens Calculation and Add Tests for Edge Cases #2850
+80
−13
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shawncal
added
.NET
github-actions
bot
changed the title
hario90
approved these changes
Sep 16, 2023
dotnet/src/Extensions/Planning.StepwisePlanner/StepwisePlannerConfig.cs
Outdated
Show resolved
Hide resolved
dotnet/src/IntegrationTests/Planning/StepwisePlanner/StepwisePlannerTests.cs
Show resolved
Hide resolved
lemillermicrosoft
force-pushed
the
915_stepwise_tokens
branch
from
fd1531f to
d6626ca
Compare
lemillermicrosoft
added
the
PR: breaking change
lemillermicrosoft
force-pushed
the
915_stepwise_tokens
branch
from
ac8e223 to
5d6c3a7
Compare
This commit adjusts the token management in the StepwisePlanner by introducing separate limits for prompt and completion tokens. It also improves the trimming process by specifying the number of steps removed and handling cases where the chat history is too long for completion.
Add two new tests to StepwisePlannerTests.cs: one for checking if the plan execution fails when there are too many functions, and another for verifying that the plan execution succeeds when there are almost too many functions. This helps ensure the planner can handle different scenarios with a large number of functions.
- Replace hardcoded "Result not found" message with a constant `NoFinalAnswerFoundMessage` - Update comment for `MaxTokens` in `StepwisePlannerConfig` to clarify that it includes both prompt and completion tokens
- Changed test methods to use async Task instead of async void - Renamed OpenApiSkillExecutionParameters to OpenApiPluginExecutionParameters - Updated test method names to include "Async" suffix
lemillermicrosoft
force-pushed
the
915_stepwise_tokens
branch
from
5d6c3a7 to
926c0d7
Compare
shawncal
approved these changes
Sep 21, 2023
SOE-YoungS
pushed a commit
to SOE-YoungS/semantic-kernel
that referenced
this pull request
Nov 1, 2023
…tion and Add Tests for Edge Cases (microsoft#2850) This pull request updates the StepwisePlannerConfig class in the Planning.StepwisePlanner project to improve the calculation of the MaxPromptTokens property, ensuring that the maximum number of iterations allowed in a plan is properly taken into account. Additionally, it introduces a new configuration property for the maximum number of tokens allowed in a completion request and adds two new tests to cover edge cases where the number of functions is too large or almost too large. Changes: - Modified the MaxPromptTokens property in the StepwisePlannerConfig class to use the MaxTokensRatio for calculating the maximum number of prompt tokens. - Updated the summary comment for the MaxPromptTokens property to reflect the new calculation method. - Add a new test `ExecutePlanFailsWithTooManyFunctions` to check if the execution fails when there are too many functions. - Add a new test `ExecutePlanSucceedsWithAlmostTooManyFunctions` to check if the execution succeeds when there are almost too many functions. - Update `StepwisePlannerConfig` to include a new property `MaxTokensRatio` for controlling the ratio of tokens allocated to the completion request. - Update `StepwisePlanner` to use the new `MaxCompletionTokens` property for setting the maximum number of tokens allowed in a completion request. - Modify the trimming logic in `StepwisePlanner` to account for the new token handling configuration and throw an exception if the chat history is too long. - Update the `TrimMessageFormat` constant in `StepwisePlanner` to include the number of steps removed in the message. - No other files or classes were affected by these changes. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request updates the StepwisePlannerConfig class in the Planning.StepwisePlanner project to improve the calculation of the MaxPromptTokens property, ensuring that the maximum number of iterations allowed in a plan is properly taken into account. Additionally, it introduces a new configuration property for the maximum number of tokens allowed in a completion request and adds two new tests to cover edge cases where the number of functions is too large or almost too large.
Changes:
ExecutePlanFailsWithTooManyFunctionsto check if the execution fails when there are too many functions.ExecutePlanSucceedsWithAlmostTooManyFunctionsto check if the execution succeeds when there are almost too many functions.StepwisePlannerConfigto include a new propertyMaxTokensRatiofor controlling the ratio of tokens allocated to the completion request.StepwisePlannerto use the newMaxCompletionTokensproperty for setting the maximum number of tokens allowed in a completion request.StepwisePlannerto account for the new token handling configuration and throw an exception if the chat history is too long.TrimMessageFormatconstant inStepwisePlannerto include the number of steps removed in the message.Contribution Checklist