Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syslog exporter does not format structured data with multiple fields properly #33300

Closed
peffis opened this issue May 29, 2024 · 7 comments · Fixed by #37927
Closed

syslog exporter does not format structured data with multiple fields properly #33300

peffis opened this issue May 29, 2024 · 7 comments · Fixed by #37927
Labels
bug Something isn't working exporter/syslog never stale Issues marked with this label will be never staled and automatically removed

Comments

@peffis
Copy link
Contributor

peffis commented May 29, 2024

Component(s)

exporter/syslog

What happened?

Description

In the case when you have several fields in the "attributes.structured_data" object, the resulting syslog entry will not be formatted properly according to RFC5424 (see examples here for instance: https://datatracker.ietf.org/doc/html/rfc5424#section-6.3.5)

Steps to Reproduce

Receive a log record with an attributes.structured_data with more than one field, such as for instance

{
    "body": "",
    "timeUnixNano": 1438811939693012000,
    "attributes": {
        "appname": "SecureAuth0",
        "hostname": "192.168.2.132",
        "message": "Found the user for retrieving user's profile",
        "msg_id": "ID52020",
        "priority": 86,
        "proc_id": "23108",
        "structured_data": {
            "a@193": {
                "a": "a"
            },
            "b@193": {
                "b": "b"
            }
        },
        "version": 1
    }
}

Expected Result

A syslog entry with one block per structured_data field, like so:

<86>1 2015-08-05T21:58:59.693012Z 192.168.2.132 SecureAuth0 23108 ID52020 [a@193 a="a"][b@193 b="b"] Found the user for retrieving user's profile

Actual Result

Instead the exporter builds only one array (here: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/syslogexporter/rfc5424_formatter.go#L72) which it does fmt.Sprint on, so end result looks like so instead:

<86>1 2015-08-05T21:58:59.693012Z 192.168.2.132 SecureAuth0 23108 ID52020 [a@193 a="a" b@193 b="b"] Found the user for retrieving user's profile

Collector version

816b1f9

Environment information

Environment

OS: Any OS really
Compiler(if manually compiled): go1.22.0

OpenTelemetry Collector configuration

receivers:
      otlp:
          protocols:
              grpc:
              http:
    processors:
    extensions:
      health_check: {}
    exporters:
      debug:
        verbosity: detailed
      syslog:
        tls:
            insecure: true
        network: tcp
        port: 5140
        endpoint: syslog
        protocol: rfc5424

    service:
      telemetry:
          logs:
            level: INFO
          metrics:
              address: ":8889"
      extensions: [health_check]
      pipelines:
        logs:
          receivers: [otlp]
          processors: []
          exporters: [debug,syslog]

Log output

No response

Additional context

No response

@peffis peffis added bug Something isn't working needs triage New item requiring triage labels May 29, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@andrzej-stencel
Copy link
Member

Thank you for reporting this issue @peffis. Looking at the examples you linked, this issue looks valid to me. @kkujawa-sumo @rnishtala-sumo do you agree?

Looking at the tests, it seems they don't cover the case with more than one structured data element. This should be fixed, the examples from the RFC could be used for the test cases.

@peffis Would you be open to submitting a fix for this? No worries if not, just checking.

@andrzej-stencel andrzej-stencel removed the needs triage New item requiring triage label Jul 8, 2024
@peffis
Copy link
Contributor Author

peffis commented Jul 8, 2024

@peffis Would you be open to submitting a fix for this? No worries if not, just checking.

Sorry, but I cannot formally be a contributor to the project right now other than reporting issues.

Copy link
Contributor

github-actions bot commented Sep 9, 2024

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Sep 9, 2024
@rnishtala-sumo
Copy link
Contributor

@andrzej-stencel yes this does look like a bug. Open to any contributions or will work on it myself.

@github-actions github-actions bot removed the Stale label Sep 10, 2024
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Nov 11, 2024
@andrzej-stencel andrzej-stencel added never stale Issues marked with this label will be never staled and automatically removed and removed Stale labels Nov 13, 2024
@peffis
Copy link
Contributor Author

peffis commented Feb 10, 2025

@andrzej-stencel yes this does look like a bug. Open to any contributions or will work on it myself.

Is anyone working on this issue? I would be able to submit my fix for it (my company has allowed contributions to this project now). But if someone has picked it up I can leave it be.

peffis added a commit to peffis/opentelemetry-collector-contrib that referenced this issue Feb 14, 2025

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
The current syslog exporter does not handle structured data well.
In particular, when there is more than one structured data field,
it will put them all in one structured data block rather than in
one block per field as the RFC5424 suggests.

This test is a test for this error.
peffis added a commit to peffis/opentelemetry-collector-contrib that referenced this issue Feb 14, 2025

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
…ata block

is built up as its own array (rather than as one single array as
it was done before). String builder (strings.Builder) was used
to buffer the final string that is built up.
@songy23 songy23 closed this as completed in 707bcb9 Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working exporter/syslog never stale Issues marked with this label will be never staled and automatically removed
Projects
None yet
3 participants