Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement bypass deferred binding #1462

Merged

Conversation

surgupta-msft
Copy link
Contributor

@surgupta-msft surgupta-msft commented Apr 11, 2023

Issue describing the changes in this PR

Resolves #1495

Design doc - Link

Implementation Summary

  1. Binding Attributes are updated as per design doc - (Refer section: class design)

    • Binding Attributes updated - BlobInputAttribute, BlobTriggerAttribute, CosmosDBInputAttribute
    • Converters updated - BlobStorageConverter, CosmosDBConverter
    • New Attributes added - AllowConverterFallback, SupportedConverterTypes
      // Binding Attributes to advertise all the converters they support and enable/disable fallback to registered converters
      [AllowConverterFallback]        // Attribute indicates that fallback to default converters is enabled by BindingAttribute
      [InputConverter(typeof(MyConverter1))] // Attribute to indicate the converter supported by BindingAttribute 
      [InputConverter(typeof(MyConverter2))]
      class MyTriggerAttribute // or MyInputAttribute
      {   
      
      }
    
      // Converters to advertise the support for deferred binding and types that are supported
      [SupportsDeferredBinding]        // Attribute indicates that converter supports deferred binding
      [SupportedConverterTypes(typeof(MyType1))]  // Attribute to indicate the type supported by converter
      [SupportedConverterTypes(typeof(MyType2))]
      [SupportedConverterTypes(typeof(IEnumerable<MyType2>))]
      class MyConverter
      {
      
      }
      ```
  2. Changes in Function Metadata Generator include -

    • Logic to correctly assign SupportsDeferredBinding by analyzing attributes advertised by BindingAttributes and Converters
    • Handling for cases such as POCOs, Collections - (Refer section: Metadata Generation flow)
  3. Changes in GrpcFunctionDefinition.cs -

    • This class will cache information about Binding Attributes and Converters in PropertyBags during Function indexing time to avoid type evaluation during invocation time.
    • Information cached here -
      • Whether BindingAttribute supports fallback to registered converters
      • A Dictionary with Key as converter and value as Properties for that converter such as all the types supported by it and support for Json Deserializable objects.
  4. Invocation flow - (Refer section: Invocation flow)

    • Changes in DefaultInputConversionFeature.cs
      • Get list of all the converters advertised by BindingAttribute and hence all the types supported by each converter.
      • Check if TargetType is supported by the converter before invoking it.
      • Check if Fallback to registered converters is enabled by Binding attribute. If yes. continue with default probing.

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • I have added all required tests (Unit tests, E2E tests)

Additional information

Additional PR information

Copy link
Member

@kshyju kshyju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looks good.

The PR title suggests that this change is to skip the new binding behavior (so things will work as hot it is today - populating binding data in host). But this PR has more changes than that (Ex: using converter types advertised by trigger, using supported types published by converter etc.) Consider updating title as needed.

sdk/Sdk/FunctionMetadataGenerator.cs Show resolved Hide resolved
@surgupta-msft surgupta-msft merged commit ad3a33c into feature/sdk-type-binding May 12, 2023
21 checks passed
@surgupta-msft surgupta-msft deleted the surgupta/bypass-deferredbinding branch May 12, 2023 22:19
surgupta-msft added a commit that referenced this pull request May 22, 2023
Implement bypass deferred binding
liliankasem pushed a commit that referenced this pull request May 22, 2023
Implement bypass deferred binding
aishwaryabh added a commit that referenced this pull request May 25, 2023
* Add E2E tests for blob SDK type bindings (#1360)

* Add analyzer for SupportsDeferredBindingAttribute (#1367)

* Update SupportsDeferredBinding diagnostic code & update docs (#1377)

* Revert "Remove types, tests & logic related to the SDK-binding feature (#1374)"

This reverts commit eac5b19.

* Added unit tests for BlobStorageConverter (#1370)

* Adding test project
* Added tests

* Initial changes

* Updating metadata

* Adding a test

* Code refactoring

* Cosmos DB converter for SDK-type support and samples (#1406)

* Execution flow added

* Update pipeline variables to include tags (#1427)

* Update extension variables to not use build number with tags (#1429)

* Execution flow

* cleanup

* Cleanup

* Cleanup

* Cleanup execution flow

* Added test

* Adding tests

* code cleanup

* Code cleanup

* code refactor

* Added logic to check type before sending to converter

* Updated tests

* Removing advertised converters from options

* Adding options back

* Changes to check type

* updating type check logic

* correction in metadata generation

* code cleanup

* Test cleanup

* Added changes to support poco

* Metadata generation changes

* Changes on Invocation side

* correcting type check and tests

* Test for poco invocation support

* Added tests for poco invocation flows

* Metadata generator minor update

* Grpc definition update

* Added poco and poco collection check

* cleanup metadata generation

* Adding support for type collection

* code cleanup

* Fixing metadata gen

* Test fix

* Grpc definition fix

* Removing BlobContainerClient advertisement

* Readding containerclient

* Updated metadata generator

* Changes as per latest design

* code cleanup

* updatin tests input

* code cleanup

* Cleanup Metadata generation

* correcting converters fallback logic

* Removing converter advertisement on cosmos trigger

* Fixing failing test

* test check in

* Undo testing changes in sample app

* spacing fix

* updated tests

* non secret changes

* Addressing PR feedback

* e2e test work

* adding comments

* Removed json deserialization attribute

* Rename to AllowConverterFallback

* Addressing feedback

* Metada generation cleanup

* Test cleanup

* Addressing feedback

* Updated grpcFunctionDefinition for multiple inputConverterAttribute

* Removing cosmos converter registration

* Address PR feedback

* Address PR feedback

* Test fix

* Minor

* addressing comments

* getting e2e working

* Implement bypass deferred binding (#1462)

Implement bypass deferred binding

* Build issue fix

* comments

* Update test/E2ETests/E2EApps/E2EApp/Table/TableInputBindingFunctions.cs

Co-authored-by: Lilian Kasem <likasem@microsoft.com>

* Adding check for PR in yml file (#1546)

* addressing some more comments

* dependency

* one more dependency

* trying to compile lol

* seeing if tables project compiles with dependency

* removing local instance

* see if this compiles

* trying to remove test

* nvm adding it back

* adding keys to nuget.config

* trying to get feed

* trying again

* more yml

* fixing spacing

* ci.yml

* Add ability to bind to SBReceivedMessage (#1313)

* another change

* trying again

* formatting changes

* changes

* updating worker.sdk package

* fixing unit test

* getting unit tests to pass

* fixing test

---------

Co-authored-by: Lilian Kasem <likasem@microsoft.com>
Co-authored-by: Surgupta <surgupta@microsoft.com>
Co-authored-by: JoshLove-msft <54595583+JoshLove-msft@users.noreply.github.com>
liliankasem pushed a commit that referenced this pull request Jun 21, 2023
Implement bypass deferred binding
liliankasem added a commit that referenced this pull request Jun 21, 2023
* Add E2E tests for blob SDK type bindings (#1360)

* Add analyzer for SupportsDeferredBindingAttribute (#1367)

* Update SupportsDeferredBinding diagnostic code & update docs (#1377)

* Revert "Remove types, tests & logic related to the SDK-binding feature (#1374)"

This reverts commit eac5b19.

* Added unit tests for BlobStorageConverter (#1370)

* Adding test project
* Added tests

* Initial changes

* Updating metadata

* Adding a test

* Code refactoring

* Cosmos DB converter for SDK-type support and samples (#1406)

* Execution flow added

* Update pipeline variables to include tags (#1427)

* Update extension variables to not use build number with tags (#1429)

* Execution flow

* cleanup

* Cleanup

* Cleanup

* Cleanup execution flow

* Added test

* Adding tests

* code cleanup

* Code cleanup

* code refactor

* Added logic to check type before sending to converter

* Updated tests

* Removing advertised converters from options

* Adding options back

* Changes to check type

* updating type check logic

* correction in metadata generation

* code cleanup

* Test cleanup

* Added changes to support poco

* Metadata generation changes

* Changes on Invocation side

* correcting type check and tests

* Test for poco invocation support

* Added tests for poco invocation flows

* Metadata generator minor update

* Grpc definition update

* Added poco and poco collection check

* cleanup metadata generation

* Adding support for type collection

* code cleanup

* Fixing metadata gen

* Test fix

* Grpc definition fix

* Removing BlobContainerClient advertisement

* Readding containerclient

* Updated metadata generator

* Changes as per latest design

* code cleanup

* updatin tests input

* code cleanup

* Cleanup Metadata generation

* correcting converters fallback logic

* Removing converter advertisement on cosmos trigger

* Fixing failing test

* test check in

* Undo testing changes in sample app

* spacing fix

* updated tests

* non secret changes

* Addressing PR feedback

* e2e test work

* adding comments

* Removed json deserialization attribute

* Rename to AllowConverterFallback

* Addressing feedback

* Metada generation cleanup

* Test cleanup

* Addressing feedback

* Updated grpcFunctionDefinition for multiple inputConverterAttribute

* Removing cosmos converter registration

* Address PR feedback

* Address PR feedback

* Test fix

* Minor

* addressing comments

* getting e2e working

* Implement bypass deferred binding (#1462)

Implement bypass deferred binding

* Build issue fix

* comments

* Update test/E2ETests/E2EApps/E2EApp/Table/TableInputBindingFunctions.cs

Co-authored-by: Lilian Kasem <likasem@microsoft.com>

* Adding check for PR in yml file (#1546)

* addressing some more comments

* dependency

* one more dependency

* trying to compile lol

* seeing if tables project compiles with dependency

* removing local instance

* see if this compiles

* trying to remove test

* nvm adding it back

* adding keys to nuget.config

* trying to get feed

* trying again

* more yml

* fixing spacing

* ci.yml

* Add ability to bind to SBReceivedMessage (#1313)

* another change

* trying again

* formatting changes

* changes

* updating worker.sdk package

* fixing unit test

* getting unit tests to pass

* fixing test

---------

Co-authored-by: Lilian Kasem <likasem@microsoft.com>
Co-authored-by: Surgupta <surgupta@microsoft.com>
Co-authored-by: JoshLove-msft <54595583+JoshLove-msft@users.noreply.github.com>
liliankasem pushed a commit that referenced this pull request Jun 22, 2023
Implement bypass deferred binding
liliankasem added a commit that referenced this pull request Jun 22, 2023
* Add E2E tests for blob SDK type bindings (#1360)

* Add analyzer for SupportsDeferredBindingAttribute (#1367)

* Update SupportsDeferredBinding diagnostic code & update docs (#1377)

* Revert "Remove types, tests & logic related to the SDK-binding feature (#1374)"

This reverts commit eac5b19.

* Added unit tests for BlobStorageConverter (#1370)

* Adding test project
* Added tests

* Initial changes

* Updating metadata

* Adding a test

* Code refactoring

* Cosmos DB converter for SDK-type support and samples (#1406)

* Execution flow added

* Update pipeline variables to include tags (#1427)

* Update extension variables to not use build number with tags (#1429)

* Execution flow

* cleanup

* Cleanup

* Cleanup

* Cleanup execution flow

* Added test

* Adding tests

* code cleanup

* Code cleanup

* code refactor

* Added logic to check type before sending to converter

* Updated tests

* Removing advertised converters from options

* Adding options back

* Changes to check type

* updating type check logic

* correction in metadata generation

* code cleanup

* Test cleanup

* Added changes to support poco

* Metadata generation changes

* Changes on Invocation side

* correcting type check and tests

* Test for poco invocation support

* Added tests for poco invocation flows

* Metadata generator minor update

* Grpc definition update

* Added poco and poco collection check

* cleanup metadata generation

* Adding support for type collection

* code cleanup

* Fixing metadata gen

* Test fix

* Grpc definition fix

* Removing BlobContainerClient advertisement

* Readding containerclient

* Updated metadata generator

* Changes as per latest design

* code cleanup

* updatin tests input

* code cleanup

* Cleanup Metadata generation

* correcting converters fallback logic

* Removing converter advertisement on cosmos trigger

* Fixing failing test

* test check in

* Undo testing changes in sample app

* spacing fix

* updated tests

* non secret changes

* Addressing PR feedback

* e2e test work

* adding comments

* Removed json deserialization attribute

* Rename to AllowConverterFallback

* Addressing feedback

* Metada generation cleanup

* Test cleanup

* Addressing feedback

* Updated grpcFunctionDefinition for multiple inputConverterAttribute

* Removing cosmos converter registration

* Address PR feedback

* Address PR feedback

* Test fix

* Minor

* addressing comments

* getting e2e working

* Implement bypass deferred binding (#1462)

Implement bypass deferred binding

* Build issue fix

* comments

* Update test/E2ETests/E2EApps/E2EApp/Table/TableInputBindingFunctions.cs

Co-authored-by: Lilian Kasem <likasem@microsoft.com>

* Adding check for PR in yml file (#1546)

* addressing some more comments

* dependency

* one more dependency

* trying to compile lol

* seeing if tables project compiles with dependency

* removing local instance

* see if this compiles

* trying to remove test

* nvm adding it back

* adding keys to nuget.config

* trying to get feed

* trying again

* more yml

* fixing spacing

* ci.yml

* Add ability to bind to SBReceivedMessage (#1313)

* another change

* trying again

* formatting changes

* changes

* updating worker.sdk package

* fixing unit test

* getting unit tests to pass

* fixing test

---------

Co-authored-by: Lilian Kasem <likasem@microsoft.com>
Co-authored-by: Surgupta <surgupta@microsoft.com>
Co-authored-by: JoshLove-msft <54595583+JoshLove-msft@users.noreply.github.com>
liliankasem pushed a commit that referenced this pull request Jul 11, 2023
liliankasem pushed a commit that referenced this pull request Jul 11, 2023
liliankasem pushed a commit that referenced this pull request Jul 11, 2023
JoshLove-msft pushed a commit to JoshLove-msft/azure-functions-dotnet-worker that referenced this pull request Oct 4, 2023
Implement bypass deferred binding
JoshLove-msft added a commit to JoshLove-msft/azure-functions-dotnet-worker that referenced this pull request Oct 4, 2023
* Add E2E tests for blob SDK type bindings (Azure#1360)

* Add analyzer for SupportsDeferredBindingAttribute (Azure#1367)

* Update SupportsDeferredBinding diagnostic code & update docs (Azure#1377)

* Revert "Remove types, tests & logic related to the SDK-binding feature (Azure#1374)"

This reverts commit eac5b19.

* Added unit tests for BlobStorageConverter (Azure#1370)

* Adding test project
* Added tests

* Initial changes

* Updating metadata

* Adding a test

* Code refactoring

* Cosmos DB converter for SDK-type support and samples (Azure#1406)

* Execution flow added

* Update pipeline variables to include tags (Azure#1427)

* Update extension variables to not use build number with tags (Azure#1429)

* Execution flow

* cleanup

* Cleanup

* Cleanup

* Cleanup execution flow

* Added test

* Adding tests

* code cleanup

* Code cleanup

* code refactor

* Added logic to check type before sending to converter

* Updated tests

* Removing advertised converters from options

* Adding options back

* Changes to check type

* updating type check logic

* correction in metadata generation

* code cleanup

* Test cleanup

* Added changes to support poco

* Metadata generation changes

* Changes on Invocation side

* correcting type check and tests

* Test for poco invocation support

* Added tests for poco invocation flows

* Metadata generator minor update

* Grpc definition update

* Added poco and poco collection check

* cleanup metadata generation

* Adding support for type collection

* code cleanup

* Fixing metadata gen

* Test fix

* Grpc definition fix

* Removing BlobContainerClient advertisement

* Readding containerclient

* Updated metadata generator

* Changes as per latest design

* code cleanup

* updatin tests input

* code cleanup

* Cleanup Metadata generation

* correcting converters fallback logic

* Removing converter advertisement on cosmos trigger

* Fixing failing test

* test check in

* Undo testing changes in sample app

* spacing fix

* updated tests

* non secret changes

* Addressing PR feedback

* e2e test work

* adding comments

* Removed json deserialization attribute

* Rename to AllowConverterFallback

* Addressing feedback

* Metada generation cleanup

* Test cleanup

* Addressing feedback

* Updated grpcFunctionDefinition for multiple inputConverterAttribute

* Removing cosmos converter registration

* Address PR feedback

* Address PR feedback

* Test fix

* Minor

* addressing comments

* getting e2e working

* Implement bypass deferred binding (Azure#1462)

Implement bypass deferred binding

* Build issue fix

* comments

* Update test/E2ETests/E2EApps/E2EApp/Table/TableInputBindingFunctions.cs

Co-authored-by: Lilian Kasem <likasem@microsoft.com>

* Adding check for PR in yml file (Azure#1546)

* addressing some more comments

* dependency

* one more dependency

* trying to compile lol

* seeing if tables project compiles with dependency

* removing local instance

* see if this compiles

* trying to remove test

* nvm adding it back

* adding keys to nuget.config

* trying to get feed

* trying again

* more yml

* fixing spacing

* ci.yml

* Add ability to bind to SBReceivedMessage (Azure#1313)

* another change

* trying again

* formatting changes

* changes

* updating worker.sdk package

* fixing unit test

* getting unit tests to pass

* fixing test

---------

Co-authored-by: Lilian Kasem <likasem@microsoft.com>
Co-authored-by: Surgupta <surgupta@microsoft.com>
Co-authored-by: JoshLove-msft <54595583+JoshLove-msft@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants