RateLimiter.withCost combinator + Rate Limiter composition tests

[hsubra89]

Type

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update

Description

This patch updates the RateLimiter module to include another constructor which includes a sub-constructor which allows us to specify a cost while maintaining composability with other RateLimiter instances. This enables the use of this module to rate limit for the following cases :

• Accessing resources with an "API Credit" limit over a time window.

• With composition, accounting for multiple "credit" limits. ( eg: Github API )

• Accounting for different "costs" per api per provider within a single credit limit.

  • Many providers have different API costs for different endpoints within their ecosystem.

Usage Example

import { Effect, RateLimiter } from "effect";
import { compose } from "effect/Function";

const program = Effect.scoped(
  Effect.gen(function* ($) {
    // Create a rate limiter that has an hourly limit of 1000 credits
    const rateLimiter = yield* $(RateLimiter.make(1000, "1 hours"));
    // Query API costs 1 credit per call ( 1 is the default cost )
    const queryAPIRL = compose(rateLimiter, RateLimiter.withCost(1));
    // Mutation API costs 5 credits per call
    const mutationAPIRL = compose(rateLimiter, RateLimiter.withCost(5));
    // ...
    // Use the pre-defined rate limiters
    yield* $(queryAPIRL(Effect.log("Sample Query")));
    yield* $(mutationAPIRL(Effect.log("Sample Mutation")));

    // Or set a cost on-the-fly
    const history = yield* $(
      rateLimiter(Effect.log("Another query with a different cost")).pipe(
        RateLimiter.withCost(3)
      )
    );
  })
);

[changeset-bot]

🦋 Changeset detected

Latest commit: 9a30aae

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 15 packages

Name	Type
effect	Patch
@effect/cli	Patch
@effect/experimental	Patch
@effect/opentelemetry	Patch
@effect/platform-browser	Patch
@effect/platform-bun	Patch
@effect/platform-node-shared	Patch
@effect/platform-node	Patch
@effect/platform	Patch
@effect/printer-ansi	Patch
@effect/printer	Patch
@effect/rpc-http	Patch
@effect/rpc	Patch
@effect/schema	Patch
@effect/typeclass	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[IMax153]

@hsubra89 - apologies for the back-and-forth with the RateLimiter. We discovered the updated implementation was utilizing the fixed-window algorithm for token replenishment instead of the token-bucket algorithm.

The fix is in #2097. Please update this MR once the bugfix is merged.

[hsubra89]

@hsubra89 - apologies for the back-and-forth with the RateLimiter. We discovered the updated implementation was utilizing the fixed-window algorithm for token replenishment instead of the token-bucket algorithm.

The fix is in #2097. Please update this MR once the bugfix is merged.

No problem. I did notice that when the implementation was changed but didn't look into it too deeply.

[hsubra89]

Depends on #2097

[tim-smart]

I think we should merge this first, as the other PR will need to add more tests to make sure cost is well supported.

changes made