Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy load boto client when using datadogpy for metrics. #558

Merged
merged 2 commits into from
Mar 18, 2025

Conversation

purple4reina
Copy link
Contributor

What does this PR do?

Lazy load boto3 and botocore libraries to use only when required.

Motivation

Faster cold starts when using datadogpy for sending metrics.

Testing Guidelines

Additional Notes

Original method for creating boto3 client

import boto3
boto3.client('kms')

But boto3.client code is just (see https://github.com/boto/boto3/blob/2b18684afe102cfd28f9ddc873d7862209d1da69/boto3/__init__.py#L92)

return _get_default_session().client(*args, **kwargs)

where _get_default_session() is defined as (see https://github.com/boto/boto3/blob/2b18684afe102cfd28f9ddc873d7862209d1da69/boto3/__init__.py#L34)

from boto3.session import Session
DEFAULT_SESSION = Session(**kwargs)

The client method on a Session is just (see https://github.com/boto/boto3/blob/2b18684afe102cfd28f9ddc873d7862209d1da69/boto3/session.py#L215)

return self._session.create_client(*args, **kwargs)

Where self._session is an instance of botocore.session.Session (using botocore.session.get_session see https://github.com/boto/boto3/blob/2b18684afe102cfd28f9ddc873d7862209d1da69/boto3/session.py#L62)

Long story short, we can avoid importing all of boto3!!

Types of Changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog
  • This PR passes the integration tests (ask a Datadog member to run the tests)

Sorry, something went wrong.

@purple4reina purple4reina force-pushed the rey.abolofia/lazy-boto branch from cc91977 to 6943a60 Compare March 17, 2025 21:09
@purple4reina purple4reina force-pushed the rey.abolofia/lazy-boto branch from 6943a60 to 3db64ea Compare March 18, 2025 17:30
@purple4reina purple4reina marked this pull request as ready for review March 18, 2025 17:59
@purple4reina purple4reina requested a review from a team as a code owner March 18, 2025 17:59
@purple4reina
Copy link
Contributor Author

I tested this by setting a DD_API_KEY_SECRET_ARN and then calling get_api_key directly. It was able to properly use botocore to fetch the api key.

@purple4reina purple4reina force-pushed the rey.abolofia/lazy-boto branch from 72a6537 to f8f7d9a Compare March 18, 2025 18:11
@purple4reina purple4reina merged commit 8398da0 into main Mar 18, 2025
60 checks passed
@purple4reina purple4reina deleted the rey.abolofia/lazy-boto branch March 18, 2025 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants