[webhookeventreceiver] add option to include headers as attributes

[tredman]

Description

Hi! I'm looking to add a feature to the webhookeventreceiver to allow HTTP request headers to be passed through as log attributes. Our initial use case for this is for receiving webhook events from Github/Github Enterprise - which stores important metadata in the webhook request headers that is absent from the payload.

This PR adds a new option, convert_headers_to_attributes, which will include any headers except the one specified in required_header in the payload as an attribute.

Link to tracking issue

Fixes

N/A - started with this PR

Testing

Added unit tests and I'm running this successfully in our environment using a local build.

Documentation

README

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: tredman / name: Travis Redman (ff9dc77, b421eae, fed620b, 7b6d09c, dbc45d3, 419154a)

[shalper2]

Hey this is really great! A couple of small things: First in factory.go the default config should be updated to include false for the new setting (since it is optional). Second, could you add two additional test cases for me, one which covers the normalizeHeader function and another where convert_headers_to_attributes is set but no non-required headers are passed in? other than that I think this looks good!

[tredman]

Thanks for the quick review and feedback! I think I've addressed it in my latest commit but let me know.

re: normalizeHeader - i renamed this to be a little more descriptive of what it was doing: generating an OTEL attribute key from a header.

[shalper2]

lgtm! Thanks!

[atoulme]

Any reason to transform to snake case? That can be done later in a processor

[tredman]

Was (maybe poorly) trying to conform to https://opentelemetry.io/docs/specs/semconv/general/naming/

[atoulme]

The issue is that this is input data, not a deterministic piece of data. You can have collision right now such as two headers with the names “content-length” and “Content Length”. It’s better to let processors perform data sanitization.

[tredman]

Fair enough. For our use case it would indeed be easy enough to do this in a processor as you say, without baking it in here. What do you think about maintaining the header. namespace? I think it would still be useful to differentiate the attributes this way.

[tredman]

Updated PR to address feedback on multi-values and switch to using regex instead. I also added a comment to headerAttributeKey to hopefully clarify the attribute key naming

Switching to regex was a little more complex, since I wanted to ensure the regex was validated on startup and also pass the compiled regex down to reqToLog. I made a couple of changes:

• reqToLog is now a receiver function, receiving an eventReciever - this simplifies the function header but does mean it's a little more complex to test. I think this makes sense because reqToLog requires several values from the calling eventReceiver anyway
• newLogsReceiver now calls Config.Validate() - this was previously only referenced in tests, and there was some duplicated validation code. I added a couple of test cases for some previously untested max values. Validate() now also checks the regex.

[mterhar]

I opened another PR against the same receiver. When this merges, I'll rebase. #38042

[tredman]

👋 Pushed a rebase after the last PR was merged. Is there any feedback here that still needs to be addressed?

[songy23]

Generated code is out of date, please run "make generate" and commit the changes in this PR.

also please fix lint http://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/13655296572/job/38267389141?pr=37815

[tredman]

ran make generate and fixed lint errors