Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature(cache): dynamodb implementation #66

Closed
wants to merge 3 commits into from

Conversation

chofnar
Copy link
Owner

@chofnar chofnar commented Nov 2, 2022

What type of PR is this?

enhancement

Which issue does this PR fix:

Part of project-zot#564

What does this PR do / Why do we need it:

Added AWS DynamoDB driver.

Will this break upgrades or downgrades?

If a client uses DynamoDB and decides to downgrade, they must make sure that the database is migrated to BoltDB to avoid data loss.

New validations that might break stuff:
local storage driver, but explicit RemoteCache=true, this is not valid, so error ou
remote storage driver, but no "cacheDriver" config section, error out

Defaults:
local storage driver -> RemoteCache=false
remote storage driver (s3) -> RemoteCache=true

Does this PR introduce any user-facing change?:

Adds the possibility to use AWS DynamoDB instead of BoltDB

Problems

  • need to specify protocol in front of dynamodb endpoint (known issue)

To do

  • fix lint problems
  • check that the example config actually works
  • remove comment from controller.go line 461
  • fix misleading example config (local storage with remote cache, is invalid)
  • rerun benchmarks
  • add negative tests
  • add coverage tests
  • rewrite some (if not all) cache tests for dynamodb

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

peusebiu and others added 3 commits November 2, 2022 13:05
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
…rface (project-zot#667)

Moved boltdb to a driver implementation for such interface
Added CreateCacheDatabaseDriver in controller
Fixed default directory creation (boltDB will only create the file, not the dir
Added coverage tests
Added example config for boltdb
Re-added caching on subpaths, rewrote CreateCacheDatabaseDriver
Fix tests
Made cacheDriver argument mandatory for NewImageStore, added more validation, added defaults
Moved cache interface to own file, removed useRelPaths from config
Got rid of cache config, refactored
Moved cache to own package and folder
Renamed + removed cache factory to backend, replaced CloudCache to RemoteCache
Moved storage constants back to storage package
moved cache interface and factory to storage package, changed remoteCache defaulting

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants