Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DynamoDB integration #18

Draft
wants to merge 7 commits into
base: cachedbInterface
Choose a base branch
from
Draft

DynamoDB integration #18

wants to merge 7 commits into from

Conversation

chofnar
Copy link
Owner

@chofnar chofnar commented Jul 21, 2022

What type of PR is this?

enhancement

Which issue does this PR fix:

Part of project-zot#564

What does this PR do / Why do we need it:

Added AWS DynamoDB driver.

Will this break upgrades or downgrades?

If a client uses DynamoDB and decides to downgrade, they must make sure that the database is migrated to BoltDB to avoid data loss.

Does this PR introduce any user-facing change?:

Adds the possibility to use AWS DynamoDB instead of BoltDB


Problems

  • need to specify protocol in front of dynamodb endpoint (known issue)

Other comments
Inspiration from https://github.com/distribution/distribution

To do

  • check that the example config actually works
  • add coverage tests
  • rewrite some (if not all) cache tests for dynamodb

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Added proper endpoint handling, logging, tests
Implemented more functions, added more tests, updated errors
Added aws databases documentation
Added DB driver and factories, need to implement
Implemented Factory and Driver interfaces for dynamodb
Refactor cache.go and changed some stuff
Rewrote some stuff, TODO: fix PutBlob
Fixed dynamoDB functions
Refactor boltDB to use new interface
Fixed gotcha
Lint
Point S3 to use DynamoDB database instead of boltdb
Reverted s3 use only boltdb
Added more dynamodb tests, fixed errors lint
Added database factory tests, removed Scan and DescribeTable functions
Added loading of db parameters from either config or env
Renamed packages and moved files around to avoid cyclic import
Added more coverage tests
Removed env var usage for config
Added benchmark tests

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
@chofnar chofnar force-pushed the cachedbInterface branch 11 times, most recently from 699a9ef to 60d3d2b Compare August 1, 2022 11:22
@chofnar chofnar force-pushed the cachedbInterface branch 4 times, most recently from a5ab8ec to 44e6374 Compare August 8, 2022 13:21
@chofnar chofnar force-pushed the cachedbInterface branch 12 times, most recently from 25d70ce to 1f5d536 Compare October 13, 2022 14:35
@chofnar chofnar force-pushed the cachedbInterface branch 9 times, most recently from fc8b2d0 to bfa7174 Compare October 24, 2022 07:55
@chofnar chofnar force-pushed the cachedbInterface branch 3 times, most recently from 31df3c8 to 9df5c07 Compare October 28, 2022 13:25
@chofnar chofnar force-pushed the cachedbInterface branch 6 times, most recently from f1cc6c1 to 7ad8818 Compare November 2, 2022 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant