-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add generate_repo.py
#2431
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What tool did you format the python files with? It'd be nice if that tool had a linter mode that fails on non-compliant files so we can add it as part of our linter step
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What tool did you format the python files with?
I use black
to format python code and it also has a validate function.
It'd be nice if that tool had a linter mode that fails on non-compliant files so we can add it as part of our linter step
Sure, I'll add a lint step in CI.
This tool is included when I install all requirements.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a python lint job in CI.
if library.rpc_documentation: | ||
repo_metadata["rpc_documentation"] = library.rpc_documentation | ||
|
||
# generate .repo-meta.json |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add more comments here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As discussed, we can either enable the IT for this split repo or leave it as a follow up, whichever is most convenient
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I'll remove the split repo integration test for now, since we mainly focus on generating google-cloud-java.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it should be fine with leaving the yaml file, although not touched by our scripts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I'll modify the integration test to reflect this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Restored the config file for java-bigtable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice if I can see how the real main entry point (library_generation/generate_repo.py?) is invoked in verify_library_generation.yaml. No need to replace the Python-based integraiton tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can leave a follow up after this PR to update the README to reflect that the main entrypoint is this one and not the now-lower-level generate_library.sh
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I'll follow up with this.
Quality Gate passed for 'gapic-generator-java-root'Kudos, no new issues were introduced! 0 New issues |
Quality Gate passed for 'java_showcase_integration_tests'Kudos, no new issues were introduced! 0 New issues |
🤖 I have created a release *beep* *boop* --- <details><summary>2.35.0</summary> ## [2.35.0](v2.34.0...v2.35.0) (2024-02-13) ### Features * add `generate_repo.py` ([#2431](#2431)) ([47b632a](47b632a)) * move synthtool templates to `library_generation/owlbot` ([#2443](#2443)) ([5c95472](5c95472)) ### Bug Fixes * Apiary Host returns user set host if set ([#2455](#2455)) ([5f17e62](5f17e62)) * Cancel the Timeout Task for HttpJson ([#2360](#2360)) ([b177d9e](b177d9e)) ### Dependencies * update dependency commons-codec:commons-codec to v1.16.1 ([#2473](#2473)) ([8c6e91d](8c6e91d)) * update google api dependencies ([#2469](#2469)) ([ad4d4e6](ad4d4e6)) * update google auth library dependencies to v1.23.0 ([#2466](#2466)) ([349a5d3](349a5d3)) * update google auth library dependencies to v1.23.0 ([#2476](#2476)) ([6c9127c](6c9127c)) * update google http client dependencies to v1.44.1 ([#2467](#2467)) ([87d1435](87d1435)) * update googleapis/java-cloud-bom digest to ac9893c ([#2472](#2472)) ([7fff34a](7fff34a)) * update grpc dependencies to v1.61.1 ([#2463](#2463)) ([9ec575b](9ec575b)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
In this PR: - Add `generate_repo.py` to generate libraries defined in a configuration file. - Regenerate `.repo-metadata.json` - Generate `.OwlBot.yaml` and `owlbot.py` for new clients - Invoke `generate_composed_library.py` to generate multiple versions of a GAPIC library. - Apply repo-level post processing using python - Regenerate root `pom.xml` - Regenerate `gapic-libraries-bom/pom.xml` - Format python script using `black`. - Add unit tests for utility functions. - Add an integration test using python, replacing shell integration tests. - Add a python lint check in CI. This is [step 2](https://docs.google.com/document/d/1JiCcG3X7lnxaJErKe0ES_JkyU7ECb40nf2Xez3gWvuo/edit?pli=1&tab=t.0#bookmark=id.h3n0hcp3ch2m) of milestone 2 of hermetic build project.
🤖 I have created a release *beep* *boop* --- <details><summary>2.35.0</summary> ## [2.35.0](v2.34.0...v2.35.0) (2024-02-13) ### Features * add `generate_repo.py` ([#2431](#2431)) ([47b632a](47b632a)) * move synthtool templates to `library_generation/owlbot` ([#2443](#2443)) ([5c95472](5c95472)) ### Bug Fixes * Apiary Host returns user set host if set ([#2455](#2455)) ([5f17e62](5f17e62)) * Cancel the Timeout Task for HttpJson ([#2360](#2360)) ([b177d9e](b177d9e)) ### Dependencies * update dependency commons-codec:commons-codec to v1.16.1 ([#2473](#2473)) ([8c6e91d](8c6e91d)) * update google api dependencies ([#2469](#2469)) ([ad4d4e6](ad4d4e6)) * update google auth library dependencies to v1.23.0 ([#2466](#2466)) ([349a5d3](349a5d3)) * update google auth library dependencies to v1.23.0 ([#2476](#2476)) ([6c9127c](6c9127c)) * update google http client dependencies to v1.44.1 ([#2467](#2467)) ([87d1435](87d1435)) * update googleapis/java-cloud-bom digest to ac9893c ([#2472](#2472)) ([7fff34a](7fff34a)) * update grpc dependencies to v1.61.1 ([#2463](#2463)) ([9ec575b](9ec575b)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
🤖 I have created a release *beep* *boop* --- <details><summary>2.35.0</summary> ## [2.35.0](v2.34.0...v2.35.0) (2024-02-13) ### Features * add `generate_repo.py` ([#2431](#2431)) ([47b632a](47b632a)) * move synthtool templates to `library_generation/owlbot` ([#2443](#2443)) ([5c95472](5c95472)) ### Bug Fixes * Apiary Host returns user set host if set ([#2455](#2455)) ([5f17e62](5f17e62)) * Cancel the Timeout Task for HttpJson ([#2360](#2360)) ([b177d9e](b177d9e)) ### Dependencies * update dependency commons-codec:commons-codec to v1.16.1 ([#2473](#2473)) ([8c6e91d](8c6e91d)) * update google api dependencies ([#2469](#2469)) ([ad4d4e6](ad4d4e6)) * update google auth library dependencies to v1.23.0 ([#2466](#2466)) ([349a5d3](349a5d3)) * update google auth library dependencies to v1.23.0 ([#2476](#2476)) ([6c9127c](6c9127c)) * update google http client dependencies to v1.44.1 ([#2467](#2467)) ([87d1435](87d1435)) * update googleapis/java-cloud-bom digest to ac9893c ([#2472](#2472)) ([7fff34a](7fff34a)) * update grpc dependencies to v1.61.1 ([#2463](#2463)) ([9ec575b](9ec575b)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
In this PR:
generate_repo.py
to generate libraries defined in a configuration file..repo-metadata.json
.OwlBot.yaml
andowlbot.py
for new clientsgenerate_composed_library.py
to generate multiple versions of a GAPIC library.pom.xml
gapic-libraries-bom/pom.xml
black
.This is step 2 of milestone 2 of hermetic build project.