Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/issue 3051 swagger preserve rpc order #3500

Conversation

CemGurhan
Copy link
Contributor

@CemGurhan CemGurhan commented Aug 15, 2023

References to other Issues or PRs

Fixes #3051

Have you read the Contributing Guidelines?

Yes

Brief description of what is fixed or changed

Currently, generated swagger files representing openapi definitions emit paths found in proto files in alphabetical order. This feature provides a new command line option called preserve_rpc_order. If set to true, once swagger files are generated, paths will maintain the order found in proto files. This should work for both json and yaml formats. Path ordering is also preserved in the following scenarios (as specified):

  1. When using additional bindings, paths should preserve this ordering within an RPC
  2. When using multiple services, paths should preserve the ordering between RPCs in the whole protobuf file
  3. When merging protobuf files, paths should preserve the ordering specified on the command line.

Alphabetical ordering of paths was due to paths being keys to the openapiPathsObject map, stored inside the openapiSwaggerObject. Hence, upon encoding to JSON or YAML, paths were automatically ordered alphabetically. In order to resolve this issue, an ordered data structure was used instead for the openapiPathsObject.

Other comments

This is my first open source contribution, so do let me know if there's anything I can improve on!

@google-cla
Copy link

google-cla bot commented Aug 15, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@CemGurhan CemGurhan force-pushed the feature/issue-3051-swagger-preserve-rpc-order branch from d8dabeb to b830ff3 Compare August 15, 2023 21:40
@CemGurhan CemGurhan marked this pull request as ready for review August 15, 2023 21:51
Copy link
Collaborator

@johanbrandhorst johanbrandhorst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Cem, thanks a lot for taking this on. I think there's a fundamental problem with the change in that it doesn't output the correct swagger structure anymore. Instead of

 "paths": {
    "/v1/example/a_bit_of_everything": {
      "get": {

It's now doing

  "paths": [
    {
      "Path": "/v1/example/a_bit_of_everything",
      "PathItemObject": {
        "get": {

I think you'll need to go back to the structure and make sure it produces the correct format. Let me know if you need me to dig deeper.

examples/internal/helloworld/helloworld.swagger.json Outdated Show resolved Hide resolved
Copy link
Collaborator

@johanbrandhorst johanbrandhorst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an excellent submission, sorry for the slow review!

@johanbrandhorst
Copy link
Collaborator

There are two warnings from golangci:

Error: func `isValidPathOrder` is unused (unused)
Error: S1001: should use copy(to, from) instead of a loop (gosimple)

Otherwise I think this is good to go!

@CemGurhan
Copy link
Contributor Author

Thanks for the review Johan, let me know if there is anything more I can do/I've missed!

Copy link
Collaborator

@johanbrandhorst johanbrandhorst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the fixes Cem. I just remembered that we should add a section to the docs about this new functionality so users can find it easily. Would you mind adding something to https://github.com/grpc-ecosystem/grpc-gateway/blob/main/docs/docs/mapping/customizing_openapi_output.md to explain what this does and how to use it?

@CemGurhan
Copy link
Contributor Author

Sure thing, will get on that now!

@CemGurhan CemGurhan force-pushed the feature/issue-3051-swagger-preserve-rpc-order branch from 46feefb to c80a170 Compare August 22, 2023 19:48
Copy link
Collaborator

@johanbrandhorst johanbrandhorst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you!

@johanbrandhorst johanbrandhorst enabled auto-merge (squash) August 22, 2023 21:18
@johanbrandhorst johanbrandhorst merged commit f3a72ed into grpc-ecosystem:main Aug 22, 2023
18 checks passed
@CemGurhan
Copy link
Contributor Author

No problem, thanks for the reviews! Will hopefully be working on more issues for gRPC gateway soon : )

michaelkedar pushed a commit to google/osv.dev that referenced this pull request Sep 11, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[github.com/grpc-ecosystem/grpc-gateway/v2](https://togithub.com/grpc-ecosystem/grpc-gateway)
| require | minor | `v2.16.0` -> `v2.18.0` |
|
[google.golang.org/protobuf](https://togithub.com/protocolbuffers/protobuf-go)
| require | minor | `v1.30.0` -> `v1.31.0` |
|  |  | lockFileMaintenance | All locks refreshed |
| [jekyll-feed](https://togithub.com/jekyll/jekyll-feed) | | minor |
`0.15.1` -> `0.17.0` |

---

### Release Notes

<details>
<summary>grpc-ecosystem/grpc-gateway
(github.com/grpc-ecosystem/grpc-gateway/v2)</summary>

###
[`v2.18.0`](https://togithub.com/grpc-ecosystem/grpc-gateway/releases/tag/v2.18.0)

[Compare
Source](https://togithub.com/grpc-ecosystem/grpc-gateway/compare/v2.17.1...v2.18.0)

#### What's Changed

- Enable a few more golangci-lint linters by
[@&#8203;pkwarren](https://togithub.com/pkwarren) in
[grpc-ecosystem/grpc-gateway#3546
- Add .golangci.yml config by
[@&#8203;pkwarren](https://togithub.com/pkwarren) in
[grpc-ecosystem/grpc-gateway#3548
- Fix: path item objects become block scalars
\[[#&#8203;3557](https://togithub.com/grpc-ecosystem/grpc-gateway/issues/3557)]
by [@&#8203;qawatake](https://togithub.com/qawatake) in
[grpc-ecosystem/grpc-gateway#3566
- Add remove_internal_comments option by
[@&#8203;same-id](https://togithub.com/same-id) in
[grpc-ecosystem/grpc-gateway#3560

#### New Contributors

- [@&#8203;benjx1990](https://togithub.com/benjx1990) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3552
- [@&#8203;qawatake](https://togithub.com/qawatake) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3566

**Full Changelog**:
grpc-ecosystem/grpc-gateway@v2.17.1...v2.18.0

###
[`v2.17.1`](https://togithub.com/grpc-ecosystem/grpc-gateway/releases/tag/v2.17.1)

[Compare
Source](https://togithub.com/grpc-ecosystem/grpc-gateway/compare/v2.17.0...v2.17.1)

#### What's Changed

- genopenapi: set source code info explicitly by
[@&#8203;johanbrandhorst](https://togithub.com/johanbrandhorst) in
[grpc-ecosystem/grpc-gateway#3544
- Record the filename of files that are missing SourceCodeInfo by
[@&#8203;ebilling](https://togithub.com/ebilling) in
[grpc-ecosystem/grpc-gateway#3545

#### New Contributors

- [@&#8203;ebilling](https://togithub.com/ebilling) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3545

**Full Changelog**:
grpc-ecosystem/grpc-gateway@v2.17.0...v2.17.1

###
[`v2.17.0`](https://togithub.com/grpc-ecosystem/grpc-gateway/releases/tag/v2.17.0)

[Compare
Source](https://togithub.com/grpc-ecosystem/grpc-gateway/compare/v2.16.2...v2.17.0)

#### What's Changed

- Fix renovate.yml overwriting .bazelrc by
[@&#8203;adambabik](https://togithub.com/adambabik) in
[grpc-ecosystem/grpc-gateway#3451
- fix: successful typo by
[@&#8203;testwill](https://togithub.com/testwill) in
[grpc-ecosystem/grpc-gateway#3510
- Update example buf.gen.yaml by
[@&#8203;pkwarren](https://togithub.com/pkwarren) in
[grpc-ecosystem/grpc-gateway#3522
- fix(deps): replace all uses of golang/protobuf/protobuf by
[@&#8203;aimuz](https://togithub.com/aimuz) in
[grpc-ecosystem/grpc-gateway#3516
- Feature/issue 3051 swagger preserve rpc order by
[@&#8203;CemGurhan](https://togithub.com/CemGurhan) in
[grpc-ecosystem/grpc-gateway#3500

#### New Contributors

- [@&#8203;testwill](https://togithub.com/testwill) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3510
- [@&#8203;pkwarren](https://togithub.com/pkwarren) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3522
- [@&#8203;aimuz](https://togithub.com/aimuz) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3516
- [@&#8203;CemGurhan](https://togithub.com/CemGurhan) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3500

**Full Changelog**:
grpc-ecosystem/grpc-gateway@v2.16.2...v2.17.0

###
[`v2.16.2`](https://togithub.com/grpc-ecosystem/grpc-gateway/releases/tag/v2.16.2)

[Compare
Source](https://togithub.com/grpc-ecosystem/grpc-gateway/compare/v2.16.1...v2.16.2)

#### What's Changed

- chore: replace the Goreleaser's deprecated option `--rm-dist` to
`--clean` by
[@&#8203;suzuki-shunsuke](https://togithub.com/suzuki-shunsuke) in
[grpc-ecosystem/grpc-gateway#3438
- chore: stop using the deprecated field `archives.replacements` by
[@&#8203;suzuki-shunsuke](https://togithub.com/suzuki-shunsuke) in
[grpc-ecosystem/grpc-gateway#3436

#### New Contributors

- [@&#8203;suzuki-shunsuke](https://togithub.com/suzuki-shunsuke) made
their first contribution in
[grpc-ecosystem/grpc-gateway#3438

**Full Changelog**:
grpc-ecosystem/grpc-gateway@v2.16.1...v2.16.2

###
[`v2.16.1`](https://togithub.com/grpc-ecosystem/grpc-gateway/releases/tag/v2.16.1)

[Compare
Source](https://togithub.com/grpc-ecosystem/grpc-gateway/compare/v2.16.0...v2.16.1)

#### What's Changed

- Rename LICENSE.txt to LICENSE by
[@&#8203;pgmitche](https://togithub.com/pgmitche) in
[grpc-ecosystem/grpc-gateway#3345
- feat: message supports enum field option for query params by
[@&#8203;zhb127](https://togithub.com/zhb127) in
[grpc-ecosystem/grpc-gateway#3352
- remove glog from a bunch of spots by
[@&#8203;kn100](https://togithub.com/kn100) in
[grpc-ecosystem/grpc-gateway#3361
- Improve AIP support in field_configuration.path_param_name containing
by [@&#8203;same-id](https://togithub.com/same-id) in
[grpc-ecosystem/grpc-gateway#3364
- Hide comments on omitted enum default values by
[@&#8203;same-id](https://togithub.com/same-id) in
[grpc-ecosystem/grpc-gateway#3366
- fix(docs): correcting mux routing error code example by
[@&#8203;tjasko](https://togithub.com/tjasko) in
[grpc-ecosystem/grpc-gateway#3409
- Fix the place where format property in placed for repeated fields by
[@&#8203;far4599](https://togithub.com/far4599) in
[grpc-ecosystem/grpc-gateway#3410
- Handle httpbody not having data to return by
[@&#8203;achew22](https://togithub.com/achew22) in
[grpc-ecosystem/grpc-gateway#3415
- bazel: bump Bazel repositories versions by
[@&#8203;adambabik](https://togithub.com/adambabik) in
[grpc-ecosystem/grpc-gateway#3413

#### New Contributors

- [@&#8203;pgmitche](https://togithub.com/pgmitche) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3345
- [@&#8203;zhb127](https://togithub.com/zhb127) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3352
- [@&#8203;kn100](https://togithub.com/kn100) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3361
- [@&#8203;tjasko](https://togithub.com/tjasko) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3409
- [@&#8203;far4599](https://togithub.com/far4599) made their first
contribution in
[grpc-ecosystem/grpc-gateway#3410

**Full Changelog**:
grpc-ecosystem/grpc-gateway@v2.16.0...v2.16.1

</details>

<details>
<summary>protocolbuffers/protobuf-go
(google.golang.org/protobuf)</summary>

###
[`v1.31.0`](https://togithub.com/protocolbuffers/protobuf-go/releases/tag/v1.31.0)

[Compare
Source](https://togithub.com/protocolbuffers/protobuf-go/compare/v1.30.0...v1.31.0)

#### Notable changes <a name="v1.31-notable-changes"></a>

**New Features**

-   [CL/489316](https://go.dev/cl/489316): types/dynamicpb: add NewTypes
- Add a function to construct a dynamic type registry from a
protoregistry.Files
- [CL/489615](https://go.dev/cl/489615): encoding: add MarshalAppend to
protojson and prototext

**Minor performance improvements**

- [CL/491596](https://go.dev/cl/491596): encoding/protodelim: If
UnmarshalFrom gets a bufio.Reader, try to reuse its buffer instead of
creating a new one
- [CL/500695](https://go.dev/cl/500695): proto: store the size of tag to
avoid multiple calculations

**Bug fixes**

- [CL/497935](https://go.dev/cl/497935): internal/order: fix sorting of
synthetic oneofs to be deterministic
- [CL/505555](https://go.dev/cl/505555): encoding/protodelim: fix
handling of io.EOF

</details>

<details>
<summary>jekyll/jekyll-feed (jekyll-feed)</summary>

###
[`v0.17.0`](https://togithub.com/jekyll/jekyll-feed/blob/HEAD/History.markdown#0170--2022-10-14)

[Compare
Source](https://togithub.com/jekyll/jekyll-feed/compare/v0.16.0...v0.17.0)

##### Documentation

- Update CI status badge
([#&#8203;363](https://togithub.com/jekyll/jekyll-feed/issues/363))

##### Development Fixes

- Add Ruby 3.1 to the CI matrix
([#&#8203;365](https://togithub.com/jekyll/jekyll-feed/issues/365))

##### Minor Enhancements

- Allow disabling of jekyll-feed while in development
([#&#8203;370](https://togithub.com/jekyll/jekyll-feed/issues/370))

###
[`v0.16.0`](https://togithub.com/jekyll/jekyll-feed/blob/HEAD/History.markdown#0160--2022-01-03)

[Compare
Source](https://togithub.com/jekyll/jekyll-feed/compare/v0.15.1...v0.16.0)

##### Minor Enhancements

- Add support for `page.description` in front matter to become entry
`<summary>`
([#&#8203;297](https://togithub.com/jekyll/jekyll-feed/issues/297))

##### Bug Fixes

- Fold private methods into the `:render` method as local variables
([#&#8203;327](https://togithub.com/jekyll/jekyll-feed/issues/327))
- Check `post.categories` instead of `post.category`
([#&#8203;357](https://togithub.com/jekyll/jekyll-feed/issues/357))
- Switched xml_escape for `<![CDATA[]]>` for post content
([#&#8203;332](https://togithub.com/jekyll/jekyll-feed/issues/332))

##### Development Fixes

- Add Ruby 3.0 to CI
([#&#8203;337](https://togithub.com/jekyll/jekyll-feed/issues/337))
- Lock RuboCop to v1.18.x
([#&#8203;348](https://togithub.com/jekyll/jekyll-feed/issues/348))
- Add workflow to release gem via GH Action
([#&#8203;355](https://togithub.com/jekyll/jekyll-feed/issues/355))

##### Documentation

- Use `.atom` extension in documented examples since we write an Atom
feed ([#&#8203;359](https://togithub.com/jekyll/jekyll-feed/issues/359))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 6am on wednesday" in timezone
Australia/Sydney, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config help](https://togithub.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/google/osv.dev).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi42OC4xIiwidXBkYXRlZEluVmVyIjoiMzYuODMuMCIsInRhcmdldEJyYW5jaCI6Im1hc3RlciJ9-->
@eccles
Copy link

eccles commented Sep 14, 2023

A big Thankyou to @CemGurhan and @johanbrandhorst for doing this. I have just implemented this in our system and it worked cleanly.
Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add option to preserve RPC paths order in swagger output
3 participants