Implement firebase-functions-test wrapV2 #131

TheIronDev · 2022-04-12T20:07:39Z

Description

Implement wrap function for v2 CloudFuntions.

Code sample

const cloudEventInstance = {..._getDefaultCloudEvent(), data};
const handler = (cloudEvent) => ({cloudEvent});
const cloudFn = storage.onObjectMetadataUpdated('bucket', handler);
expect(cloudFn(cloudEventInstance).cloudEvent).equal(cloudEventInstance);

TheIronDev · 2022-04-13T01:28:49Z

Requesting a review from @inlined

src/v2.ts

This commit makes some minor updates to resolve pr comments. I still need to go back and clean up the createCloudEvent usecase. Stay tuned!

This commit implements the createMockCloudEvent. This simplifies the story of testing the wrapped V2 function.

inlined

Unfortunately this SDK is much larger scope to build. The primary reason for this repo isn't because customers can't create mock event envelopes (formerly event contexts). Most of their functions won't even rely on the envelope. The challenge they face is coming up with realistic event data.

For each event type we need a full event where all fields that are generally present have some well known value and, to a best effort, the data is cohesive (e.g. if the customer specifies a bucket, it is the bucket for both the event envelope and the event data).

I have half a mind to suggest that we should actually accept a CloudEvent<T> | T and just add an event wrapper if none of a certain number of fields are actually defined. Or we can just tell everyone that they need to call with {data: whatIActuallyCareAbout} all the time.

spec/v2.spec.ts

inlined · 2022-04-26T03:21:13Z

spec/v2.spec.ts

+      });
+    });
+
+    describe('callable functions', () => {


Honestly I have mixed feelings about even supporting wrap for functions that don't have an eventTrigger. We don't do any parsing into native SDK types nor do we fill in any default values. Maybe we should just tell developers to use <theirFn>.run? Thoughts @taeold & @mbleigh?

If we do want to support wrap I wonder if we should be coming up with some sort of helpers to make up an authenticated user, instanceIdToken, and app?

Yeah this was copy/paste of the previous wrap implementation. I don't have strong opinions on this.

Are callable functions documented anywhere for V2?

I can see it either way. Convenient to have the same test harness work for all function types but that only holds if the test methods needed are meaningfully similar.

spec/v2.spec.ts

src/v2.ts

TheIronDev · 2022-04-26T16:40:37Z

@inlined Providing a default data might fix that then? I'll follow up with you on that.

This cl decouples the implementation logic from wrapping V1 functions and places it into a separate directory. I went back and re-exported the same exports from v1 back out to main.ts to reduce disruption from existing usage.

inlined

I suspect you'll have an easier time if you use a strategy pattern. For example, you can look up a Service (e.g. Pub/Sub or GCS) given an eventType and then call into that service's "generateMockEvent" method. This will help you, for example, only insert params on RTDB events or to insert appId as a field in FireAlerts events.

We're possibly going to have a lot of event types given that 3P events are coming. I think a little over-engineering to have a codebase where changes are decoupled from the rest of the codebase and have an obvious way to grow organically would be in our best interest.

src/main.ts

inlined · 2022-04-27T23:32:47Z

src/v1.ts

+
+  if (
+    has(cloudFunction, '__trigger.httpsTrigger') &&
+    get(cloudFunction, '__trigger.labels.deployment-callable') !== 'true'


Note: this is going to be __endpoint.callableTrigger in the endpoint world.

Oh neat, didn't realize v1 shared the same ManifestEndpoint.

I fear that changing the logic here will break existing usages (if people mocked out their own event with httpsTrigger), but maybe a major bump is the right time to be making a realignment to match spec...

inlined · 2022-04-27T23:34:29Z

src/v1.ts

+  );
+}
+
+function _checkOptionValidity(


I don't mind if, in the future, we try to solve these sorts of concerns with the type system rather than with helper functions (though having both is great too)

I agree, in the future type system checking would be better.

src/v2.ts

CloudEvent stuff is being decoupled shortly, stay tuned!™

TheIronDev · 2022-04-28T04:16:24Z

I suspect you'll have an easier time if you use a strategy pattern. For example, you can look up a Service (e.g. Pub/Sub or GCS) given an eventType and then call into that service's "generateMockEvent" method. This will help you, for example, only insert params on RTDB events or to insert appId as a field in FireAlerts events.

We're possibly going to have a lot of event types given that 3P events are coming. I think a little over-engineering to have a codebase where changes are decoupled from the rest of the codebase and have an obvious way to grow organically would be in our best interest.

I agree with the strategy pattern. We need a little more signal to differentiate some of the alerts, so I'm planning on also using eventTrigger?.eventFilters as well.

Will ping you when I get closer to cleaning everything up.

This commit introduces MockCloudEvent. Its very apparent the switch-case shenanigans are too brittle to handle the growing number of CloudEvents. So Instead, I'm introducing a wrapper that contains most of the implementation within `src/mock-cloud-event-partial-definitions.ts`. The wrapper uses a `match` function to see which implementation of MockCloudEventPartial to use, and then uses a `generatePartial` to get the expected Partial<CloudEvent>. This is not particularly DRY, but it seemed more useful to keep the building blocks basic rather than being too clever. I'm happy to be convinced otherwise though. I'll be breaking / updating other unit tests momentarily.

This commit refactors the V2 wrap function, removing the need for the end-user to generate a mock event to pass into the function. The spec tests are still mostly useless, and we are still missing data. I wanted to finish this refactor before implementing the data potion, otherwise it would be more work to clean up.

src/v2.ts

Because Partials' don't Partial down recursively.

TheIronDev · 2022-04-29T03:49:19Z

Alright, Storage data is done!
Still working on generating data for....

alerts
pubsub
eventarc

This is a quality-of-life improvement that splits up the giant `src/mock-cloud-event-partial-definitions.ts` by event type. `src/mock-cloud-event-partial-definitions.ts` is now just `partials.ts`, and it returns a list of partials that are all definined in different files. Its not the DRYest thing in thr world... but its much more readable.

src/v2.ts

src/main.ts

TheIronDev · 2022-04-28T04:04:55Z

src/v2.ts

+    throw new Error('This function can only wrap V2 CloudFunctions.');
+  }
+
+  return (cloudEvent: CloudEvent) => cloudFunction.run(cloudEvent);


I'm going to be splitting the mock-cloud-event into its own file, so the update will be coming shortly.

src/v2.ts

TheIronDev · 2022-05-02T17:16:21Z

src/v1.ts

+  );
+}
+
+function _checkOptionValidity(


I agree, in the future type system checking would be better.

inlined

I think this is fine for a checkpoint. I may dig in later to play with the type system, DRY things out, and/or improve the way one field can be generated from another.

src/cloudevent/generate.ts

inlined · 2022-05-02T20:15:26Z

spec/cloudevent/generate.ts

+        expect(cloudEvent.source).equal(
+          '//firebasealerts.googleapis.com/projects/__PROJECT_ID__');
+        expect(cloudEvent.subject).equal(undefined);
+      });


and eventually tests on data

Ack (will follow up!)

inlined · 2022-05-02T20:15:55Z

spec/cloudevent/generate.ts

+        expect(cloudEvent.source).equal(
+          `//storage.googleapis.com/projects/_/buckets/${bucketName}`);
+        expect(cloudEvent.subject).equal('objects/file_name');
+      });


and another test for when bucketName is not specified

inlined · 2022-05-02T20:16:58Z

spec/v2.spec.ts

+        it('should update CloudEvent appropriately', () => {
+          const cloudFn = alerts.onAlertPublished('alertType', handler);
+          const cloudFnWrap = wrapV2(cloudFn);
+          expect(cloudFnWrap().cloudEvent).to.include({});


Should I assume these tests come later?

Yes, these events will get updated later 👍

src/cloudevent/partials/alerts/billing-on-plan-automated-update-published.ts

src/cloudevent/partials/alerts/billing-on-plan-update-published.ts

src/cloudevent/partials/alerts/crashlytics-on-new-anr-issue-published.ts

src/cloudevent/partials/alerts/crashlytics-on-regression-alert-published.ts

src/cloudevent/partials/pubsub/pubsub-on-message-published.ts

spec/cloudevent/generate.ts

xiangkangjw · 2022-05-03T02:53:18Z

src/cloudevent/partials/alerts/alerts-on-alert-published.ts

+function getOnAlertPublishedData(): FirebaseAlertData<any> {
+  const now = new Date().toISOString();
+  return ({
+    // '@type': 'type.googleapis.com/google.events.firebase.firebasealerts.v1.AlertData',


+1. Why is this commented out?

src/cloudevent/partials/alerts/app-distribution-on-new-tester-ios-device-published.ts

src/cloudevent/partials/alerts/billing-on-plan-automated-update-published.ts

`<FunctionType, EventType>` -> `<EventType>`

Removing the hardcoded json from the partial. There needs to be a better pattern. Stay tuned ™

TheIronDev · 2022-05-03T05:39:45Z

I think this is fine for a checkpoint. I may dig in later to play with the type system, DRY things out, and/or improve the way one field can be generated from another.

Sounds good to me! I'm going to submit to unblock the other team members.

Implement firebase-functions-test wrapV2

Verified

This commit was signed with the committer’s verified signature.

arjo129 Arjo Chakravarty

GPG key ID: 12295BC3F3732EC8

Verified
Learn about vigilant mode

26bef11

inlined requested changes Apr 13, 2022

View reviewed changes

src/v2.ts Outdated Show resolved Hide resolved

src/v2.ts Outdated Show resolved Hide resolved

src/v2.ts Outdated Show resolved Hide resolved

src/v2.ts Outdated Show resolved Hide resolved

TheIronDev added 6 commits April 13, 2022 21:00

Minor updates (per pr comments.)

620efec

This commit makes some minor updates to resolve pr comments. I still need to go back and clean up the createCloudEvent usecase. Stay tuned!

Fix how V2 CloudEventSource gets populated

da6d7f6

Fix CloudEvent.subject and CloudEvent.source

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

GPG key ID: 4AEE18F83AFDEB23
Expired

Verified
Learn about vigilant mode

5ead878

Bumped semver to 1.0.0

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

GPG key ID: 4AEE18F83AFDEB23
Expired

Verified
Learn about vigilant mode

173244e

Added support for eventarc custom events

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

GPG key ID: 4AEE18F83AFDEB23
Expired

Verified
Learn about vigilant mode

7a6a36b

inlined requested changes Apr 26, 2022

View reviewed changes

Fix follow ups from PR feedback

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.

Learn about vigilant mode

7a43876

TheIronDev added 4 commits April 26, 2022 15:52

Handle V2 function behavior within the main.ts#wrap call

c6bc418

Decouple wrapV1 logic from main.ts#wrap

2614eb5

This cl decouples the implementation logic from wrapping V1 functions and places it into a separate directory. I went back and re-exported the same exports from v1 back out to main.ts to reduce disruption from existing usage.

Add in Partial Typing instead of CloudEventOverride

c535541

Fix issue with broken typing (CloudBaseEvent isn't exported)

4cb0038

inlined reviewed Apr 27, 2022

View reviewed changes

Follow up items from PR feedback

5c873ea

CloudEvent stuff is being decoupled shortly, stay tuned!™

TheIronDev added 2 commits April 28, 2022 10:23

inlined reviewed Apr 28, 2022

View reviewed changes

src/v2.ts Outdated Show resolved Hide resolved

TheIronDev added 2 commits April 28, 2022 16:19

Implemented StorageObjectData and improved typing

7c40120

Introduce DeepPartial

14805af

Because Partials' don't Partial down recursively.

TheIronDev added 5 commits April 29, 2022 12:34

Wire up deep merge

8d232d9

Implement data generation for Alerts

dfcdebe

Fix Types for AppDistributionEvent, BillingEvent, and CrashlyticsEvent

5726338

Implement mock MessagePublishedData for pubsub

afbeac8

TheIronDev added 2 commits May 2, 2022 10:30

Update EventArc sample CloudEvent

510a880

Updated Unit tests for PubSub data.message generation

c259fdf

TheIronDev commented May 2, 2022

View reviewed changes

TheIronDev requested a review from inlined May 2, 2022 20:44

Update returned type for Alert.OnAlertPublished partial

c6ad354

inlined approved these changes May 2, 2022

View reviewed changes

Remove param from default CloudEvent generation

614139b

xiangkangjw reviewed May 3, 2022

View reviewed changes

TheIronDev added 5 commits May 2, 2022 23:21

Fix all the imports (from /lib/v2/providers -> /v2)

b2eeddd

Fix redundant typing

3b92144

`<FunctionType, EventType>` -> `<EventType>`

DRY cleanup

87cd8cb

Patch Minor lint issues

c7e7845

Removed hardcoded json from pubsub partial generator

a530aac

Removing the hardcoded json from the partial. There needs to be a better pattern. Stay tuned ™

TheIronDev merged commit 0182a8a into firebase:master May 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement firebase-functions-test wrapV2 #131

Implement firebase-functions-test wrapV2 #131

TheIronDev commented Apr 12, 2022

TheIronDev commented Apr 13, 2022

inlined left a comment

inlined Apr 26, 2022

TheIronDev Apr 26, 2022

mbleigh May 2, 2022

TheIronDev commented Apr 26, 2022

inlined left a comment

inlined Apr 27, 2022

TheIronDev Apr 28, 2022

inlined Apr 27, 2022

TheIronDev May 2, 2022

TheIronDev commented Apr 28, 2022

TheIronDev commented Apr 29, 2022 •

edited

Loading

TheIronDev Apr 28, 2022

TheIronDev May 2, 2022

inlined left a comment

inlined May 2, 2022

TheIronDev May 3, 2022

inlined May 2, 2022

inlined May 2, 2022

TheIronDev May 3, 2022

xiangkangjw May 3, 2022

TheIronDev commented May 3, 2022

Implement firebase-functions-test wrapV2 #131

Implement firebase-functions-test wrapV2 #131

Conversation

TheIronDev commented Apr 12, 2022

Description

Code sample

TheIronDev commented Apr 13, 2022

inlined left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheIronDev commented Apr 26, 2022

inlined left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheIronDev commented Apr 28, 2022

TheIronDev commented Apr 29, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

inlined left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheIronDev commented May 3, 2022

TheIronDev commented Apr 29, 2022 •

edited

Loading