Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/resource: fix gs:// fetches in GCE without a service account #1363

Merged
merged 1 commit into from May 10, 2022
Merged

Conversation

bgilbert
Copy link
Contributor

@bgilbert bgilbert commented May 9, 2022

When running in GCE, we assumed that we should always perform authenticated GS fetches. However, these can fail if the VM is not associated with a service account, even if the object being fetched is publicly readable:

error while reading content from ...: metadata: GCE metadata "instance/service-accounts/default/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.read_only" not defined

Query the VM's service account scopes first, and if that query fails (presumably because there is no service account), fall back to anonymous access.

When running in GCE, we assumed that we should always perform
authenticated GS fetches.  However, these can fail if the VM is not
associated with a service account, even if the object being fetched is
publicly readable:

    error while reading content from ...: metadata: GCE metadata "instance/service-accounts/default/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.read_only" not defined

Query the VM's service account scopes first, and if that query fails
(presumably because there is no service account), fall back to anonymous
access.
@cgwalters
Copy link
Member

I assume there's an externally-reported bug here, or did you find this yourself? Are you working on #1362 ?

@cgwalters
Copy link
Member

This seems sane to me, but should we (also/instead) tell anyone who is using gs:// for unauthenticated access to instead access via https://? What's the advantage of going via gs:// ?

@bgilbert
Copy link
Contributor Author

bgilbert commented May 9, 2022

In the course of working on coreos/fedora-coreos-config#1717, I realized that we don't have kola tests for GCS, and while adding those I discovered that of course the feature doesn't completely work. I'm not working on #1362 per se, but note that that issue only covers running these tests in upstream CI; the FCOS release test suite will run them regardless. (Which is too late to prevent a bad Ignition release, but better than nothing.)

The advantage of gs:// is authenticated access, but it's legal for configs to use it for unauthenticated access as well, and configs might want to do so for consistency. (s3:// behaves similarly.) And in the unauthenticated case, we shouldn't fail provisioning solely because the instance was started without credentials.

Copy link
Member

@sohankunkerkar sohankunkerkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested this using gs://ignition-test-fixtures/resources/anonymous
LGTM

@bgilbert bgilbert merged commit 801a2ae into coreos:main May 10, 2022
@bgilbert bgilbert deleted the gs branch May 10, 2022 16:55
bgilbert added a commit to coreos/coreos-assembler that referenced this pull request May 16, 2022
We want to be able to test that anonymous fetches S3 -> EC2 and
GCS -> GCE work without instance credentials.  This has been a
problem in the past: coreos/ignition#1363
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants