Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: improve retry logic for getting the graph subgraph #615

Merged
merged 6 commits into from
Jan 14, 2025

Conversation

janb87
Copy link
Contributor

@janb87 janb87 commented Jan 14, 2025

Summary by Sourcery

Improve the retry logic when fetching graph subgraphs. Introduce a new graphSubgraphs method in the SDK to handle retries and fetch middleware subgraphs. Update the CLI to use the new method.

New Features:

  • Add a graphSubgraphs method to the SDK to retry fetching middleware subgraphs.

Tests:

  • Update tests to cover the new retry logic and the graphSubgraphs method.

Verified

This commit was signed with the committer’s verified signature.
ljharb Jordan Harband

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Copy link
Contributor

sourcery-ai bot commented Jan 14, 2025

Reviewer's Guide by Sourcery

This pull request improves the retry logic for fetching graph subgraphs by implementing a dedicated retry mechanism and removing the noCache parameter from several queries.

Sequence diagram for improved graph subgraph fetching

Loading
sequenceDiagram
    participant Client
    participant SettlemintClient
    participant GraphQLClient
    participant Server

    Client->>SettlemintClient: graphSubgraphs(uniqueName, noCache)
    SettlemintClient->>GraphQLClient: request(getGraphMiddlewareSubgraphs)
    GraphQLClient->>Server: POST /graphql
    alt Success
        Server-->>GraphQLClient: Return data
        GraphQLClient-->>SettlemintClient: Return middleware
        SettlemintClient-->>Client: Return MiddlewareWithSubgraphs
    else Error (5xx, 429, 408)
        Server-->>GraphQLClient: Error response
        Note over GraphQLClient: Retry with exponential backoff
        GraphQLClient->>Server: Retry POST /graphql
    end

Class diagram for updated middleware types

Loading
classDiagram
    class SettlemintClient {
        +middleware: MiddlewareAPI
    }
    class MiddlewareAPI {
        +list(applicationUniqueName: string): Promise~Middleware[]~
        +read(middlewareUniqueName: string): Promise~Middleware~
        +graphSubgraphs(middlewareUniqueName: string, noCache?: boolean): Promise~MiddlewareWithSubgraphs~
        +create(args: CreateMiddlewareArgs): Promise~Middleware~
        +restart(middlewareUniqueName: string): Promise~Middleware~
    }
    class Middleware {
        +uniqueName: string
        +specVersion: string
    }
    class MiddlewareWithSubgraphs {
        +uniqueName: string
        +specVersion: string
        +subgraphs: Subgraph[]
    }
    class Subgraph {
        +name: string
        +graphqlQueryEndpoint: Endpoint
    }

    SettlemintClient --> MiddlewareAPI
    MiddlewareAPI --> Middleware
    MiddlewareAPI --> MiddlewareWithSubgraphs
    MiddlewareWithSubgraphs --> Subgraph

Flow diagram for retry logic

Loading
flowchart TD
    A[Start Request] --> B{First Attempt}
    B -->|Success| C[Return Response]
    B -->|Error| D{Check Error Type}
    D -->|5xx/429/408| E[Calculate Retry Delay]
    D -->|Other Error| F[Return Error]
    E --> G{Max Retries?}
    G -->|Yes| H[Retry Request]
    G -->|No| I[Throw Error]
    H -->|Success| C
    H -->|Error| D

File-Level Changes

Change Details Files
Retry logic for fetching middleware subgraphs
  • Added new query getGraphMiddlewareSubgraphs to fetch middleware subgraphs with retry logic.
  • Added retryWhenFailed utility function to handle retries with exponential backoff.
  • Added fetchWithRetry utility function for retrying HTTP requests.
  • Added graphqlFetchWithRetry utility function for retrying GraphQL requests.
  • Updated getGraphEndpoint to use retryWhenFailed and the new query.
  • Updated connect command to use the updated getGraphEndpoint.
  • Updated graph create command to use the updated getGraphEndpoint.
  • Updated subgraph deploy command to use the updated getGraphEndpoint.
  • Removed unnecessary noCache parameter from existing queries.
  • Added graphSubgraphs method to the Settlemint client.
sdk/js/src/graphql/middleware.ts
sdk/js/src/settlemint.ts
sdk/cli/src/utils/get-cluster-service-endpoint.ts
sdk/utils/src/retry.ts
sdk/utils/src/index.ts
sdk/utils/src/http/graphql-fetch-with-retry.ts
sdk/utils/src/http/fetch-with-retry.ts
sdk/utils/src/http.ts
sdk/cli/src/commands/connect.ts
sdk/cli/src/commands/platform/middleware/graph/create.ts
sdk/cli/src/commands/smart-contract-set/subgraph/deploy.ts
Code generation updates
  • Updated codegenBlockscout to use graphqlFetchWithRetry.
  • Updated testGqlEndpoint to use graphqlFetchWithRetry.
sdk/cli/src/commands/codegen/codegen-blockscout.ts
sdk/cli/src/commands/codegen/test-gql-endpoint.ts
Schema updates
  • Added AddOn enum to the schema.
  • Updated UseCasePricing type to include addOns instead of codeStudioEnabled.
  • Updated getUseCasePricing query to accept addOns as an argument.
sdk/js/schema.graphql
sdk/js/src/helpers/graphql-env.d.ts
Documentation and example updates
  • Added documentation and examples for findMonoRepoRoot and findMonoRepoPackages.
  • Added example usage for retryWhenFailed.
sdk/utils/src/filesystem/mono-repo.ts
sdk/utils/src/retry.ts

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time. You can also use
    this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions github-actions bot added the fix label Jan 14, 2025
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @janb87 - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟢 General issues: all looks good
  • 🟢 Security: all looks good
  • 🟡 Review instructions: 2 issues found
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link

github-actions bot commented Jan 14, 2025

📦 Packages

Package NPM Docker
SDK Cli @settlemint/sdk-cli@1.0.0-pr32572152
SDK The Graph @settlemint/sdk-thegraph@1.0.0-pr32572152
SDK Portal @settlemint/sdk-portal@1.0.0-pr32572152
SDK Hasura @settlemint/sdk-hasura@1.0.0-pr32572152
SDK JS @settlemint/sdk-js@1.0.0-pr32572152
SDK Utils @settlemint/sdk-utils@1.0.0-pr32572152
SDK Next @settlemint/sdk-next@1.0.0-pr32572152
SDK Minio @settlemint/sdk-minio@1.0.0-pr32572152
SDK IPFS @settlemint/sdk-ipfs@1.0.0-pr32572152
SDK Blockscout @settlemint/sdk-blockscout@1.0.0-pr32572152

@janb87 janb87 added the e2e label Jan 14, 2025
@janb87 janb87 enabled auto-merge (squash) January 14, 2025 13:13
@janb87 janb87 merged commit d552bed into main Jan 14, 2025
2 checks passed
@janb87 janb87 deleted the jan/eng-2179-subgraph-deploy-errors-out branch January 14, 2025 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants