Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: move custom image cdn url generator implementation to adapter #38715

Merged
merged 2 commits into from Nov 30, 2023

Conversation

pieh
Copy link
Contributor

@pieh pieh commented Nov 28, 2023

Description

This builds on top of #38685 to add pluggable way of providing custom image cdn url generator implementation by adapters, so platform specific endpoint is not hardcoded in platform agnostic package

Documentation

Tests

Related Issues

FRA-95

@gatsbot gatsbot bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Nov 28, 2023
@pieh pieh added topic: media Related to gatsby-plugin-image, or general image/media processing topics topic: adapters Related to Gatsby Adapters and removed status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer labels Nov 28, 2023
Comment on lines +31 to 36
{
title: `remote-file (SSR, Page Query)`,
pagePath: `/routes/ssr/remote-file/`,
fileCDN: false,
placeholders: false,
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some context here why fileCDN and placeholders are not checked - simply they don't exactly work.

fileCDN explanation:
Because fileCDN is not supported - it would try to run FILE_CDN job "locally" in lambda - however there is a problem with running actual IMAGE_CDN and FILE_CDN jobs (this happens when there is no handling at request time) - the "worker" responsible for handling those jobs is failing to load - this happens locally as well in gatsby serve, so this is not adapter or Netlify issue - it's a framework issue. However even if we fix worker loading - it still actually wouldn't exactly "just work" on Netlify as there is no straight forward way to add additional static assets to deployment after initial build and deployment finished - this likely could be supported via proxy lambda or something like that, but that would need additional work to properly work.

placeholders explanation:
linkfs we use in

// Alias the cache dir paths to the temp dir
const lfs = link(fs, rewrites) as typeof import("fs")
// linkfs doesn't pass across the `native` prop, which graceful-fs needs
for (const key in lfs) {
if (Object.hasOwnProperty.call(fs[key], `native`)) {
lfs[key].native = fs[key].native
}
}
const dbPath = path.join(TEMP_CACHE_DIR, `data`, `datastore`)
// 'promises' is not initially linked within the 'linkfs'
// package, and is needed by underlying Gatsby code (the
// @graphql-tools/code-file-loader)
lfs.promises = link(fs.promises, rewrites)
// Gatsby uses this instead of fs if present
// eslint-disable-next-line no-underscore-dangle
global._fsWrapper = lfs

doesn't seem to provide implementation for new lfs.WriteStream() which is used to generate placeholders (it does provide implementation for fs.createWriteStream so it should be possible to fix/patch it up, but didn't want to do it in this PR)

// For array of cropFocus values, append them as comma-separated string
params.append(`crop`, cropFocus.join(`,`))
} else {
params.append(`crop`, cropFocus)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we want to add any kind of comment here that we are using a field that is supported but not documented with crop (position is documented but both work with the api).

Copy link
Contributor Author

@pieh pieh Nov 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it just alias and we could just use position instead of crop to use documented one?

(I just moved implementation as-is from gatsby-plugin-utils alternative url generator, so I have no context on this part)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's a good point. I was trying to mirror using the same parameters, but I suppose we can just change it.

@kathmbeck kathmbeck marked this pull request as ready for review November 30, 2023 14:45
@kathmbeck kathmbeck merged commit d31412a into alternate-image-url Nov 30, 2023
30 of 32 checks passed
@kathmbeck kathmbeck deleted the adapters-image-cdn branch November 30, 2023 14:45
pieh added a commit that referenced this pull request Dec 15, 2023
* alternate image url construction

* try using image cdn in e2e site

* Update netlify.toml

* have separate check for dispatching image and file service

* fix tests?

* try to use images from deploy (so we can avoid using ones hosted externally)

* replicate prod-runtime imagecdn tests in adapters

* fix import

* adjusting remote-file tests

* adjusting remote-file tests 2

* cleanup/test

* assert naturalWidth/height in image-cdn tests (both adapters and production-runtime)

* remove unused

* don't use path prefix for alternate image cdn url

* _gatsby/file is prefixed

* feat: move custom image cdn url generator implementation to adapter (#38715)

* feat: move custom image cdn url generator implementation to adapter

* provide public types for custom image cdn url generator function signature and individual arguments

* use position/cover

* update comment

* update docs

* chore: types/jsdocs shuffle

* apply suggestion from #38685

* remove docs from feature branch

* feat: provide custom FILE_CDN url generator from adapter (#38735)

* feat: provide file cdn from adapters

* update test

* fix tests

* use edge function for non-image File CDN

* why edge function was not deployed?

* bump netlify-cli

* ?

* bump node for adapters tests

* add execa to dev deps

* cleanup

* some jsdocs updates

* add note that generated urls ideally are relative, but can be absolute as well

* feat: allow adding remote file allowed url patterns (#38719)

* feat: move custom image cdn url generator implementation to adapter

* provide public types for custom image cdn url generator function signature and individual arguments

* feat: allow adding image cdn allowed url patterns

* Module.createRequireFromPath doesn't exist anymore in Node 18, and because package requires at least that version we remove it

* fix contentful source image url

* fix wordpress source image url

* rename ImageCdnAllowed to RemoteFileAllowed as it's not just for image cdn

* compare allowed remote urls in netlify.toml with ones generated by gatsby

* url testing in filecdn

* jsdocs

* print warnings for netlify.toml about missing remote_images patterns

* test if any existing pattern in netlify.toml allow needed remote url instead of just string comparison

* chore: update adapter README about imageCDN

* use correct remote_images for adapters e2e site

---------

Co-authored-by: Michal Piechowiak <misiek.piechowiak@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: adapters Related to Gatsby Adapters topic: media Related to gatsby-plugin-image, or general image/media processing topics
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants