Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement bypass deferred binding #1462

Merged
Merged
Show file tree
Hide file tree
Changes from 67 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
f7853d7
Initial changes
surgupta-msft Mar 16, 2023
9d105a6
Updating metadata
surgupta-msft Mar 17, 2023
0a4342b
Adding a test
surgupta-msft Mar 20, 2023
d5206aa
Code refactoring
surgupta-msft Mar 21, 2023
a4da7dc
Execution flow added
surgupta-msft Mar 22, 2023
61479cb
Execution flow
surgupta-msft Mar 22, 2023
356b372
cleanup
surgupta-msft Mar 22, 2023
199e6cb
Cleanup
surgupta-msft Mar 23, 2023
379da90
Cleanup
surgupta-msft Mar 23, 2023
71e0be8
Cleanup execution flow
surgupta-msft Mar 23, 2023
ad21fb9
Added test
surgupta-msft Mar 23, 2023
6e14edf
Adding tests
surgupta-msft Apr 7, 2023
a398322
code cleanup
surgupta-msft Apr 7, 2023
df76646
Code cleanup
surgupta-msft Apr 7, 2023
bc9749f
code refactor
surgupta-msft Apr 7, 2023
0eb9b5f
Added logic to check type before sending to converter
surgupta-msft Apr 10, 2023
6630355
Updated tests
surgupta-msft Apr 10, 2023
15752e9
Removing advertised converters from options
surgupta-msft Apr 10, 2023
1ef5181
Adding options back
surgupta-msft Apr 11, 2023
63af451
Changes to check type
surgupta-msft Apr 13, 2023
29bcb97
updating type check logic
surgupta-msft Apr 14, 2023
dfac68e
correction in metadata generation
surgupta-msft Apr 14, 2023
359b38c
code cleanup
surgupta-msft Apr 17, 2023
7211f01
Test cleanup
surgupta-msft Apr 17, 2023
b565b45
Added changes to support poco
surgupta-msft Apr 18, 2023
478f7b9
Metadata generation changes
surgupta-msft Apr 18, 2023
0074cbd
Changes on Invocation side
surgupta-msft Apr 18, 2023
f06b7dd
correcting type check and tests
surgupta-msft Apr 18, 2023
e6b059f
Test for poco invocation support
surgupta-msft Apr 19, 2023
4f8431b
Added tests for poco invocation flows
surgupta-msft Apr 19, 2023
8f88495
Metadata generator minor update
surgupta-msft Apr 19, 2023
5af5a3d
Grpc definition update
surgupta-msft Apr 19, 2023
3aee9e6
Added poco and poco collection check
surgupta-msft Apr 19, 2023
321050d
cleanup metadata generation
surgupta-msft Apr 19, 2023
68b232a
Adding support for type collection
surgupta-msft Apr 21, 2023
ac548ee
Merge branch 'feature/sdk-type-binding' into surgupta/bypass-deferred…
surgupta-msft Apr 21, 2023
e8ebef0
code cleanup
surgupta-msft Apr 21, 2023
c6c922c
Fixing metadata gen
surgupta-msft Apr 21, 2023
b34e939
Test fix
surgupta-msft Apr 21, 2023
b4c9e17
Grpc definition fix
surgupta-msft Apr 22, 2023
54c1cee
Removing BlobContainerClient advertisement
surgupta-msft Apr 22, 2023
3323177
Readding containerclient
surgupta-msft Apr 22, 2023
12fd136
Updated metadata generator
surgupta-msft Apr 25, 2023
deed9ed
Changes as per latest design
surgupta-msft Apr 27, 2023
81f7712
code cleanup
surgupta-msft Apr 28, 2023
5ddcc76
updatin tests input
surgupta-msft Apr 28, 2023
78e4271
code cleanup
surgupta-msft May 1, 2023
02ec62f
Cleanup Metadata generation
surgupta-msft May 1, 2023
0a6468e
correcting converters fallback logic
surgupta-msft May 1, 2023
5073024
Removing converter advertisement on cosmos trigger
surgupta-msft May 1, 2023
0136fc7
Fixing failing test
surgupta-msft May 1, 2023
f1dece6
Undo testing changes in sample app
surgupta-msft May 1, 2023
daa23fc
spacing fix
surgupta-msft May 1, 2023
e918fcd
updated tests
surgupta-msft May 2, 2023
b8b89e0
Addressing PR feedback
surgupta-msft May 3, 2023
7ef0a40
Removed json deserialization attribute
surgupta-msft May 4, 2023
8391cf2
Rename to AllowConverterFallback
surgupta-msft May 4, 2023
bac43ab
Addressing feedback
surgupta-msft May 5, 2023
3971796
Metada generation cleanup
surgupta-msft May 5, 2023
b8507af
Test cleanup
surgupta-msft May 5, 2023
be00b04
Addressing feedback
surgupta-msft May 5, 2023
1780981
Updated grpcFunctionDefinition for multiple inputConverterAttribute
surgupta-msft May 5, 2023
f1bcde0
Removing cosmos converter registration
surgupta-msft May 5, 2023
6197db7
Address PR feedback
surgupta-msft May 5, 2023
9d803b7
Address PR feedback
surgupta-msft May 5, 2023
49e5d04
Test fix
surgupta-msft May 5, 2023
6d4978d
Minor
surgupta-msft May 5, 2023
744a9eb
Updating AllowConverterFallback value
surgupta-msft May 10, 2023
4380e9d
Adding release notes
surgupta-msft May 10, 2023
d27ba03
Updating bypass deferred binding packages
surgupta-msft May 11, 2023
a39601b
Revert "Updating bypass deferred binding packages"
surgupta-msft May 11, 2023
3af7780
Minor updates based on comments
surgupta-msft May 11, 2023
e36726b
Cleanup defaultInputConverterProvider
surgupta-msft May 11, 2023
958375d
Test refactor
surgupta-msft May 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
namespace Microsoft.Azure.Functions.Worker.Extensions.Abstractions
{
/// <summary>
/// Specifies if a binding attribute supports deferred binding when generating function metadata.
/// This is to be used on input, output or trigger attributes that support deferred binding.
/// Specifies if a converter supports deferred binding when generating function metadata.
/// This is to be used on converters that support deferred binding.
/// </summary>
[AttributeUsage(AttributeTargets.Class)]
public class SupportsDeferredBindingAttribute : Attribute
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,14 @@
using Microsoft.Azure.Functions.Worker.Extensions.CosmosDB;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.Logging;
using Microsoft.Azure.Functions.Worker.Extensions.Abstractions;

namespace Microsoft.Azure.Functions.Worker
{
/// <summary>
/// Converter to bind Cosmos DB type parameters.
/// </summary>
[SupportsDeferredBinding]
internal class CosmosDBConverter : IInputConverter
{
private readonly IOptionsSnapshot<CosmosDBBindingOptions> _cosmosOptions;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@
// Licensed under the MIT License. See License.txt in the project root for license information.

using System.Collections.Generic;
using Microsoft.Azure.Functions.Worker.Converters;
using Microsoft.Azure.Functions.Worker.Extensions.Abstractions;

namespace Microsoft.Azure.Functions.Worker
{
[SupportsDeferredBinding]
[AllowConverterFallback(true)]
surgupta-msft marked this conversation as resolved.
Show resolved Hide resolved
[InputConverter(typeof(CosmosDBConverter))]
public sealed class CosmosDBInputAttribute : InputBindingAttribute
{
/// <summary>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

using System;
Expand All @@ -24,11 +24,6 @@ public override void Configure(IFunctionsWorkerApplicationBuilder applicationBui
applicationBuilder.Services.AddAzureClientsCore(); // Adds AzureComponentFactory
applicationBuilder.Services.AddOptions<CosmosDBBindingOptions>();
applicationBuilder.Services.AddSingleton<IConfigureOptions<CosmosDBBindingOptions>, CosmosDBBindingOptionsSetup>();

applicationBuilder.Services.Configure<WorkerOptions>((workerOption) =>
{
workerOption.InputConverters.RegisterAt<CosmosDBConverter>(0);
});
}
}
}
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

using Microsoft.Azure.Functions.Worker.Converters;
using Microsoft.Azure.Functions.Worker.Extensions.Abstractions;

namespace Microsoft.Azure.Functions.Worker
{
[SupportsDeferredBinding]
[AllowConverterFallback(true)]
surgupta-msft marked this conversation as resolved.
Show resolved Hide resolved
[InputConverter(typeof(BlobStorageConverter))]
public sealed class BlobInputAttribute : InputBindingAttribute
{
private readonly string _blobPath;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@
using Microsoft.Azure.Functions.Worker.Extensions.Storage.Blobs;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.Logging;
using Microsoft.Azure.Functions.Worker.Extensions.Abstractions;

namespace Microsoft.Azure.Functions.Worker
{
/// <summary>
/// Converter to bind Blob Storage type parameters.
/// </summary>
[SupportsDeferredBinding]
internal class BlobStorageConverter : IInputConverter
{
private readonly IOptions<WorkerOptions> _workerOptions;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

using Microsoft.Azure.Functions.Worker.Converters;
using Microsoft.Azure.Functions.Worker.Extensions.Abstractions;

namespace Microsoft.Azure.Functions.Worker
{
[SupportsDeferredBinding]
[AllowConverterFallback(true)]
[InputConverter(typeof(BlobStorageConverter))]
public sealed class BlobTriggerAttribute : TriggerBindingAttribute
{
private readonly string _blobPath;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,6 @@ public override void Configure(IFunctionsWorkerApplicationBuilder applicationBui
applicationBuilder.Services.AddAzureClientsCore(); // Adds AzureComponentFactory
applicationBuilder.Services.AddOptions<BlobStorageBindingOptions>();
applicationBuilder.Services.AddSingleton<IConfigureOptions<BlobStorageBindingOptions>, BlobStorageBindingOptionsSetup>();

applicationBuilder.Services.Configure<WorkerOptions>((workerOption) =>
{
workerOption.InputConverters.RegisterAt<BlobStorageConverter>(0);
surgupta-msft marked this conversation as resolved.
Show resolved Hide resolved
});
}
}
}
6 changes: 6 additions & 0 deletions samples/WorkerBindingSamples/NuGet.Config
surgupta-msft marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
</packageSources>
</configuration>
55 changes: 28 additions & 27 deletions samples/WorkerBindingSamples/WorkerBindingSamples.csproj
Original file line number Diff line number Diff line change
@@ -1,29 +1,30 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net7.0</TargetFramework>
<AzureFunctionsVersion>v4</AzureFunctionsVersion>
<OutputType>Exe</OutputType>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.12.1-preview1" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.9.0-preview1" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.0.13" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Storage" Version="5.1.0-preview1" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
<PackageReference Include="System.Net.NameResolution" Version="4.3.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\extensions\Worker.Extensions.CosmosDB\src\Worker.Extensions.CosmosDB.csproj" />
</ItemGroup>
<ItemGroup>
<None Update="host.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
<None Update="local.settings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<CopyToPublishDirectory>Never</CopyToPublishDirectory>
</None>
</ItemGroup>
<PropertyGroup>
<TargetFramework>net7.0</TargetFramework>
<AzureFunctionsVersion>v4</AzureFunctionsVersion>
<OutputType>Exe</OutputType>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.9.0-preview1" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.0.13" />
<PackageReference Include="System.Net.NameResolution" Version="4.3.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\extensions\Worker.Extensions.Abstractions\src\Worker.Extensions.Abstractions.csproj" />
liliankasem marked this conversation as resolved.
Show resolved Hide resolved
<ProjectReference Include="..\..\extensions\Worker.Extensions.Storage\src\Worker.Extensions.Storage.csproj" />
<ProjectReference Include="..\..\src\DotNetWorker.ApplicationInsights\DotNetWorker.ApplicationInsights.csproj" />
<ProjectReference Include="..\..\src\DotNetWorker\DotNetWorker.csproj" />
<ProjectReference Include="..\..\extensions\Worker.Extensions.CosmosDB\src\Worker.Extensions.CosmosDB.csproj" />
surgupta-msft marked this conversation as resolved.
Show resolved Hide resolved
</ItemGroup>
<ItemGroup>
<None Update="host.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
<None Update="local.settings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<CopyToPublishDirectory>Never</CopyToPublishDirectory>
</None>
</ItemGroup>
</Project>
2 changes: 2 additions & 0 deletions sdk/Sdk/Constants.cs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ internal static class Constants
internal const string ExponentialBackoffRetryAttributeType = "Microsoft.Azure.Functions.Worker.ExponentialBackoffRetryAttribute";
internal const string BindingPropertyNameAttributeType = "Microsoft.Azure.Functions.Worker.Extensions.Abstractions.BindingPropertyNameAttribute";
internal const string SupportsDeferredBindingAttributeType = "Microsoft.Azure.Functions.Worker.Extensions.Abstractions.SupportsDeferredBindingAttribute";
internal const string InputConverterAttributeType = "Microsoft.Azure.Functions.Worker.Converters.InputConverterAttribute";
internal const string SupportedConverterTypeAttributeType = "Microsoft.Azure.Functions.Worker.Converters.SupportedConverterTypeAttribute";

// System types
internal const string IEnumerableType = "System.Collections.IEnumerable";
Expand Down
77 changes: 73 additions & 4 deletions sdk/Sdk/FunctionMetadataGenerator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
using System.IO;
using System.Linq;
using Mono.Cecil;
using Mono.Collections.Generic;

namespace Microsoft.Azure.Functions.Worker.Sdk
{
Expand Down Expand Up @@ -487,7 +488,7 @@ private static ExpandoObject BuildBindingMetadataFromAttribute(CustomAttribute a

// For extensions that support deferred binding, set the supportsDeferredBinding property so parameters are bound by the worker
// Only use deferred binding for input and trigger bindings, output is out not currently supported
if (SupportsDeferredBinding(attribute) && direction != Constants.OutputBindingDirection)
if (SupportsDeferredBinding(attribute, parameterType) && direction != Constants.OutputBindingDirection)
{
bindingProperties.Add(Constants.SupportsDeferredBindingProperty, Boolean.TrueString);
}
Expand Down Expand Up @@ -795,7 +796,8 @@ private static string GetBindingDirection(CustomAttribute attribute)
return Constants.InputBindingDirection;
}

private static bool SupportsDeferredBinding(CustomAttribute attribute)
// Input and Trigger Binding Attribute will be checked for supporting deferred binding
private static bool SupportsDeferredBinding(CustomAttribute attribute, TypeReference bindingType)
{
var typeDefinition = attribute?.AttributeType?.Resolve();

Expand All @@ -804,8 +806,75 @@ private static bool SupportsDeferredBinding(CustomAttribute attribute)
return false;
}

return typeDefinition.CustomAttributes
.Any(a => string.Equals(a.AttributeType.FullName, Constants.SupportsDeferredBindingAttributeType, StringComparison.Ordinal));
// checking attributes advertised by the binding attribute
foreach (CustomAttribute bindingAttribute in typeDefinition.CustomAttributes)
surgupta-msft marked this conversation as resolved.
Show resolved Hide resolved
{
if (string.Equals(bindingAttribute.AttributeType.FullName, Constants.InputConverterAttributeType, StringComparison.Ordinal))
{
// InputConverterAttribute will have supported converter type
foreach (var customAttribute in bindingAttribute.ConstructorArguments)
surgupta-msft marked this conversation as resolved.
Show resolved Hide resolved
{
if (DoesConverterSupportDeferredBinding(customAttribute, bindingType))
{
return true;
surgupta-msft marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
}

return false;
}

private static bool DoesConverterSupportDeferredBinding(CustomAttributeArgument customAttribute, TypeReference bindingType)
{
var typeReferenceValue = customAttribute.Value as TypeReference;
var typeReferenceCustomAttributes = typeReferenceValue?.Resolve().CustomAttributes;

// Attributes advertised by converter
if (typeReferenceCustomAttributes is not null)
{
bool converterAdvertisesDeferredBindingSupport = typeReferenceCustomAttributes.Any(a => string.Equals(a.AttributeType.FullName, Constants.SupportsDeferredBindingAttributeType, StringComparison.Ordinal));

if (converterAdvertisesDeferredBindingSupport)
{
bool converterAdvertisesTypes = typeReferenceCustomAttributes.Any(a => string.Equals(a.AttributeType.FullName, Constants.SupportedConverterTypeAttributeType, StringComparison.Ordinal));

if (!converterAdvertisesTypes)
{
// If a converter advertises deferred binding but does not explictly advertise any types then DeferredBinding will be supported for all the types
return true;
}

return DoesConverterSupportTargetType(typeReferenceCustomAttributes, bindingType);
}
}

return false;
}

private static bool DoesConverterSupportTargetType(Collection<CustomAttribute> customAttributes, TypeReference bindingType)
{
// Parse attributes advertised by converter
foreach (CustomAttribute attribute in customAttributes)
{
if (string.Equals(attribute.AttributeType.FullName, Constants.SupportedConverterTypeAttributeType, StringComparison.Ordinal))
{
foreach (CustomAttributeArgument element in attribute.ConstructorArguments)
{
if (string.Equals(element.Type.FullName, typeof(Type).FullName, StringComparison.Ordinal))
{
var supportedType = element.Value as TypeReference;

if (supportedType is not null && string.Equals(supportedType.FullName, bindingType.FullName, StringComparison.Ordinal))
{
return true;
}
}
}
}
}

return false;
}

private static bool IsOutputBindingType(CustomAttribute attribute)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure.Functions.Worker.Converters;
Expand Down Expand Up @@ -69,18 +70,29 @@ public async ValueTask<FunctionInputBindingResult> BindFunctionInputAsync(Functi
}
else
{
IReadOnlyDictionary<string, object> properties = ImmutableDictionary<string, object>.Empty;
var properties = new Dictionary<string, object>();

// Pass info about specific input converter type defined for this parameter, if present.
if (param.Properties.TryGetValue(PropertyBagKeys.ConverterType, out var converterTypeAssemblyFullName))
{
properties = new Dictionary<string, object>()
properties.Add(PropertyBagKeys.ConverterType, converterTypeAssemblyFullName);
}

// Pass info about the flag to allow fallback to default converters defined for this parameter, if present.
if (param.Properties.TryGetValue(PropertyBagKeys.AllowConverterFallback, out var flag))
{
properties.Add(PropertyBagKeys.AllowConverterFallback, flag);
}

// Pass info about input converter types defined for this parameter, if present.
if (param.Properties.TryGetValue(PropertyBagKeys.BindingAttributeSupportedConverters, out var converters))
{
{ PropertyBagKeys.ConverterType, converterTypeAssemblyFullName }
};
properties.Add(PropertyBagKeys.BindingAttributeSupportedConverters, converters);
}

var converterContext = _converterContextFactory.Create(param.Type, source, context, properties);
var converterContext = _converterContextFactory.Create(param.Type, source, context, properties.Count() != 0
? properties.ToImmutableDictionary()
: ImmutableDictionary<string, object>.Empty);

bindingResult = await inputConversionFeature.ConvertAsync(converterContext);
inputBindingCache[cacheKey] = bindingResult;
Expand Down