fix(protobuf): Correctly decode multi-messages streams #616
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sven Rebhan <srebhan@influxdata.com>
srebhan
force-pushed
the
fix_protobuf_decode
branch
from
d16ad4d
to
78585cf
Compare
beorn7
approved these changes
Apr 3, 2024
There was a problem hiding this comment.
Thanks for the catch. I guess this makes sense.
@stapelberg FYI (and just in case you want to raise any concerns about this fix)
beorn7
added a commit
to prometheus/pushgateway
that referenced
this pull request
Apr 3, 2024
Most notable, we need to pull in prometheus/common#616 . Signed-off-by: beorn7 <beorn@grafana.com>
|
Oh, I see the mistake! Sorry about that, and thanks for sending a fix which even increases test coverage :) |
beorn7
added a commit
to prometheus/pushgateway
that referenced
this pull request
Apr 3, 2024
Most notable, we need to pull in prometheus/common#616 . Signed-off-by: beorn7 <beorn@grafana.com>
mx-psi
pushed a commit
to open-telemetry/opentelemetry-collector-contrib
that referenced
this pull request
Apr 9, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/prometheus/common](https://togithub.com/prometheus/common) | `v0.51.1` -> `v0.52.2` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>prometheus/common (github.com/prometheus/common)</summary> ### [`v0.52.2`](https://togithub.com/prometheus/common/releases/tag/v0.52.2) [Compare Source](https://togithub.com/prometheus/common/compare/v0.51.1...v0.52.2) #### What's Changed - Drop support for Go older than 1.18 by [@​SuperQ](https://togithub.com/SuperQ) in [prometheus/common#612 - fix(protobuf): Correctly decode multi-messages streams by [@​srebhan](https://togithub.com/srebhan) in [prometheus/common#616 - Bump github.com/aws/aws-sdk-go from 1.50.31 to 1.51.11 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#615 #### New Contributors - [@​srebhan](https://togithub.com/srebhan) made their first contribution in [prometheus/common#616 **Full Changelog**: prometheus/common@v0.51.1...v0.52.2 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <107717825+opentelemetrybot@users.noreply.github.com>
codeboten
pushed a commit
to open-telemetry/opentelemetry-collector
that referenced
this pull request
Apr 11, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/prometheus/common](https://togithub.com/prometheus/common) | `v0.48.0` -> `v0.52.3` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>prometheus/common (github.com/prometheus/common)</summary> ### [`v0.52.3`](https://togithub.com/prometheus/common/compare/v0.52.2...v0.52.3) [Compare Source](https://togithub.com/prometheus/common/compare/v0.52.2...v0.52.3) ### [`v0.52.2`](https://togithub.com/prometheus/common/releases/tag/v0.52.2) [Compare Source](https://togithub.com/prometheus/common/compare/v0.51.1...v0.52.2) #### What's Changed - Drop support for Go older than 1.18 by [@​SuperQ](https://togithub.com/SuperQ) in [prometheus/common#612 - fix(protobuf): Correctly decode multi-messages streams by [@​srebhan](https://togithub.com/srebhan) in [prometheus/common#616 - Bump github.com/aws/aws-sdk-go from 1.50.31 to 1.51.11 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#615 #### New Contributors - [@​srebhan](https://togithub.com/srebhan) made their first contribution in [prometheus/common#616 **Full Changelog**: prometheus/common@v0.51.1...v0.52.2 ### [`v0.51.1`](https://togithub.com/prometheus/common/releases/tag/v0.51.1) [Compare Source](https://togithub.com/prometheus/common/compare/v0.51.0...v0.51.1) #### What's Changed - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [prometheus/common#606 - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [prometheus/common#609 - Retract v0.50.0 by [@​SuperQ](https://togithub.com/SuperQ) in [prometheus/common#610 **Full Changelog**: prometheus/common@v0.51.0...v0.51.1 ### [`v0.51.0`](https://togithub.com/prometheus/common/releases/tag/v0.51.0) [Compare Source](https://togithub.com/prometheus/common/compare/v0.50.0...v0.51.0) #### What's Changed - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [prometheus/common#604 - expfmt: Add a way to generate different OpenMetrics Formats by [@​ywwg](https://togithub.com/ywwg) in [prometheus/common#596 - Fix string slice definition for FormatFlagOptions. by [@​gizmoguy](https://togithub.com/gizmoguy) in [prometheus/common#607 - Correct logic in sample naming for counters, add new test by [@​vesari](https://togithub.com/vesari) in [prometheus/common#608 #### New Contributors - [@​gizmoguy](https://togithub.com/gizmoguy) made their first contribution in [prometheus/common#607 **Full Changelog**: prometheus/common@v0.50.0...v0.51.0 ### [`v0.50.0`](https://togithub.com/prometheus/common/releases/tag/v0.50.0) [Compare Source](https://togithub.com/prometheus/common/compare/v0.49.0...v0.50.0) #### What's Changed - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [prometheus/common#594 - Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#593 - Bump github.com/aws/aws-sdk-go from 1.50.27 to 1.50.29 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#592 - Bump github.com/aws/aws-sdk-go from 1.50.29 to 1.50.31 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#595 - Remove unused 'Host' member from HTTPClientConfig by [@​bboreham](https://togithub.com/bboreham) in [prometheus/common#597 - Add OpenMetrics unit support by [@​vesari](https://togithub.com/vesari) in [prometheus/common#544 - Remove deprecated version function by [@​SuperQ](https://togithub.com/SuperQ) in [prometheus/common#591 - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [prometheus/common#599 - Bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#600 - Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#601 **Full Changelog**: prometheus/common@v0.49.0...v0.50.0 ### [`v0.49.0`](https://togithub.com/prometheus/common/releases/tag/v0.49.0) [Compare Source](https://togithub.com/prometheus/common/compare/v0.48.0...v0.49.0) #### What's Changed - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [prometheus/common#574 - Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.8 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#571 - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [prometheus/common#581 - Update Go by [@​SuperQ](https://togithub.com/SuperQ) in [prometheus/common#588 - Deprecate version.NewCollector by [@​SuperQ](https://togithub.com/SuperQ) in [prometheus/common#579 - Bump github.com/aws/aws-sdk-go from 1.50.8 to 1.50.27 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#587 - Avoid off-spec openmetrics exposition when exemplars have empty labels by [@​orls](https://togithub.com/orls) in [prometheus/common#569 - Bump golang.org/x/oauth2 from 0.16.0 to 0.17.0 by [@​dependabot](https://togithub.com/dependabot) in [prometheus/common#585 - Write created lines when negotiating OpenMetrics by [@​ArthurSens](https://togithub.com/ArthurSens) in [prometheus/common#504 - Upgrade client_model to v.0.6.0 by [@​vesari](https://togithub.com/vesari) in [prometheus/common#589 - http_config: Add host by [@​jkroepke](https://togithub.com/jkroepke) in [prometheus/common#549 - LabelSet: Fix alphabetical sorting for prometheus LabelSet by [@​wasim-nihal](https://togithub.com/wasim-nihal) in [prometheus/common#575 - labelset: optimise String() function by [@​bboreham](https://togithub.com/bboreham) in [prometheus/common#590 #### New Contributors - [@​orls](https://togithub.com/orls) made their first contribution in [prometheus/common#569 - [@​vesari](https://togithub.com/vesari) made their first contribution in [prometheus/common#589 **Full Changelog**: prometheus/common@v0.48.0...v0.49.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yMjAuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <107717825+opentelemetrybot@users.noreply.github.com>
rimitchell
pushed a commit
to rimitchell/opentelemetry-collector-contrib
that referenced
this pull request
May 8, 2024
](https://renovatebot.com){kind=link}
…#32250) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/prometheus/common](https://togithub.com/prometheus/common) | `v0.51.1` -> `v0.52.2` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>prometheus/common (github.com/prometheus/common)</summary> ### [`v0.52.2`](https://togithub.com/prometheus/common/releases/tag/v0.52.2) [Compare Source](https://togithub.com/prometheus/common/compare/v0.51.1...v0.52.2) #### What's Changed - Drop support for Go older than 1.18 by [@&open-telemetry#8203;SuperQ](https://togithub.com/SuperQ) in [prometheus/common#612 - fix(protobuf): Correctly decode multi-messages streams by [@&open-telemetry#8203;srebhan](https://togithub.com/srebhan) in [prometheus/common#616 - Bump github.com/aws/aws-sdk-go from 1.50.31 to 1.51.11 in /sigv4 by [@&open-telemetry#8203;dependabot](https://togithub.com/dependabot) in [prometheus/common#615 #### New Contributors - [@&open-telemetry#8203;srebhan](https://togithub.com/srebhan) made their first contribution in [prometheus/common#616 **Full Changelog**: prometheus/common@v0.51.1...v0.52.2 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <107717825+opentelemetrybot@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When sending multiple metric-family samples in one stream the current implementation only decodes the first of those messages, all other ones are lost. This is the case since #567 which switches to directly using the
protodelimpackage. However, in the process or copying the code, the initialization of thereadermoved to theDecode()function and as such, the buffered reader continues somewhere in the actual data-stream (usually at the end) leading to anEOFon the nextDecode()call.The malfunction is present since release v0.46.0.
This PR moves the initialization out of
Decode()into the initialization of the decoder preventing the described issue. Furthermore, we add a unit-test including a multi-message data-sample verifying the issue is fixed.