Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update indexstorage interface to reduce roundtrips #1838

Merged
merged 1 commit into from Nov 27, 2023

Conversation

bobcallaway
Copy link
Member

This changes the interface for indexstorage in two ways:

  1. Adds a Shutdown() method to clean up resources upon server shutdown
  2. Takes []string on both read and write paths

The redis implementation now pipelines requests into a single roundtrip between Rekor and Redis on both the read and write path (where possible).

Copy link

codecov bot commented Nov 20, 2023

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (69f95e9) 66.33% compared to head (e1902cb) 66.30%.

Files Patch % Lines
pkg/api/api.go 0.00% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1838      +/-   ##
==========================================
- Coverage   66.33%   66.30%   -0.03%     
==========================================
  Files          92       92              
  Lines        9212     9213       +1     
==========================================
- Hits         6111     6109       -2     
- Misses       2354     2356       +2     
- Partials      747      748       +1     
Flag Coverage Δ
e2etests 47.72% <0.00%> (-0.01%) ⬇️
unittests 47.04% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

cpanato
cpanato previously approved these changes Nov 20, 2023
Copy link
Member

@cpanato cpanato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

pkg/api/index.go Outdated
if err != nil {
return handleRekorAPIError(params, http.StatusInternalServerError, fmt.Errorf("index storage error: %w", err), indexStorageUnexpectedResult)
sha := strings.ToLower(util.PrefixSHA(params.Query.Hash))
if queryOperator == "or" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "and" need to be handled too? I think the lookups would be the same for or vs and, it seems like NewCollection just processes them different I think

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's handled in the else clause on line 52. The way the Collector was implemented is that you have to pass it in the queries for each input parameter separately, where as OR is just a concatenation.

@@ -26,14 +26,14 @@ import (
)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test to confirm read/write works with multiple values?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The e2e tests already cover this

Signed-off-by: Bob Callaway <bcallaway@google.com>
@bobcallaway bobcallaway merged commit a7501a6 into sigstore:main Nov 27, 2023
14 checks passed
@github-actions github-actions bot added this to the v1.2.2 milestone Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants