serverless-storage-demo

Deploy a simple serverless workload on Cloud Run using Terraform.

This demo repository is based on the "Build a Serverless App with Cloud Run that Creates PDF Files" Qwiklabs quest.

Architecture

Our infrastructure is backed by the following Google Cloud services:

• Compute Engine
  • Cloud Run
• Cloud Storage
  • Storage Notifications
• Cloud Pub/Sub

The front-end service exposes a public HTTP interface to the user. New files are ingested into the upload Cloud Bucket, which notifies the ingest service through the upload Pub/Sub topic.

Successful uploads are saved to the process Cloud Bucket, and can kick off additional workloads through the ingest topic. Further processing notifies consumers using the process topic.

Source

Packages

Re-usable components, maintained in / (root) and /packages.

Package name	Description
serverless-storage-demo	Root-level workspace containing development dependencies & scripts
ingest	Cloud Storage file ingest service
ingest-pdf	Document to PDF converter
front-end	User facing web service

Setup

Ensure tools are installed if required:

Language	Version	Tools
Node.JS	v14	nvm: https://github.com/nvm-sh/nvm Version is locked in .nvmrc

Ensure version of Node.JS is correct:

# If using external version manager
nvm current
nvm install && nvm use

node --version

Install dependencies:

# If pnpm not installed
npm i -g pnpm@6.32.4

pnpm install

Checks

Run lint checks on a package locally:

# Define package
export PACKAGE=ingest

cd packages/${PACKAGE} && pnpm lint

Run unit tests on a package locally:

# Define package
export PACKAGE=ingest

cd packages/${PACKAGE} && pnpm test

Usage

Running a package locally:

# Define package
export PACKAGE=ingest

cd packages/${PACKAGE} && pnpm start

A helper script is available to spin up a local dev environment:

./scripts/localdev.sh

Building a Docker image:

# Define package and push location
export PACKAGE=ingest
export PROJECT=andrejus-web

docker buildx build . \
    --file packages/${PACKAGE}/Dockerfile \
    --tag eu.gcr.io/${PROJECT}/${PACKAGE}:latest \
    --push

A helper script is available to build and push all images:

./scripts/build-push-all.sh

Terraform Module

Instances of this repository can be spun up using sample below.

Example when used in e.g. a terraspace stack: screencast

module "demo" {
    source = "git::https://github.com/andrejusk/serverless-storage-demo.git//terraform/module?ref=master"

    # Service definitions
    project = "andrejus-web"
    service = "serverless-demo"

    # Service location definitions
    gcs_location = "EU"
    region       = "europe-west2"

    # Service revision definitions
    frontend_image  = "eu.gcr.io/andrejus-web/srvls-demo-frontend:latest"
    ingest_image    = "eu.gcr.io/andrejus-web/srvls-demo-ingest:latest"
    ingestpdf_image = "eu.gcr.io/andrejus-web/srvls-demo-ingestpdf:latest"
}