Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug fix] Reset Budget Job #9329

Merged
merged 5 commits into from
Mar 18, 2025
Merged

[Bug fix] Reset Budget Job #9329

merged 5 commits into from
Mar 18, 2025

Conversation

ishaan-jaff
Copy link
Contributor

@ishaan-jaff ishaan-jaff commented Mar 18, 2025

[Bug fix] Reset Budget Job

Fixes #8364

Details of the issue

  • LiteLLM users reported budgets not being reset for keys, users, teams
  • When budget was being reset we set item.spend=0 and update item.budget_reset_ai and return the item
  • Before returning the item we checked isinstance(result, LiteLLM_TeamTable) if resetting the budget of a team, but the result was of type prisma.LiteLLM_TeamTable and hence returned None
  • Line of code with the issue return result if isinstance(result, LiteLLM_TeamTable) else None

5 Whys Analysis for this issue

Why was the budget not being reset for keys, users, and teams?
→ The function responsible for resetting budgets was returning None instead of the updated item.

Why was the function returning None?
→ The check isinstance(result, LiteLLM_TeamTable) failed because result was of type prisma.LiteLLM_TeamTable.

Why was there a mismatch in types (LiteLLM_TeamTable vs. prisma.LiteLLM_TeamTable)?
→ The code expected LiteLLM_TeamTable, but prisma.LiteLLM_TeamTable was returned due to how Prisma ORM generated the models.

Why was this type mismatch not caught earlier?
→ There were no end-to-end (E2E) tests validating the cron job execution, so the issue was not detected during testing on ci/cd. We only had unit tests around this

Why were there no E2E tests for this scenario?
→ The initial focus was on unit tests + adding better error logging rather than full workflow validation, leading to a gap in coverage. We have now added a test to validate full workflow coverage for keys, users, teams

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • I have added a screenshot of my new test passing locally
  • My PR passes all unit tests on (make test-unit)[https://docs.litellm.ai/docs/extras/contributing_code]
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix
✅ Test

Changes

Sorry, something went wrong.

Copy link

vercel bot commented Mar 18, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 18, 2025 2:55am

Copy link

codecov bot commented Mar 18, 2025

Codecov Report

Attention: Patch coverage is 83.33333% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/proxy/utils.py 60.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@ishaan-jaff
Copy link
Contributor Author

to clarify, the patch coverage is 83.3%, you can see code cov is counting my formatting tool as a code change and stating that line was not tested

Screenshot 2025-03-17 at 9 35 20 PM

@ishaan-jaff ishaan-jaff merged commit 5400615 into main Mar 18, 2025
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: team budget not reseting
1 participant