Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KEP-1880 Multiple ServiceCIDR ( and IPAddress allocation) #116516

Merged
merged 31 commits into from Nov 1, 2023
Merged

KEP-1880 Multiple ServiceCIDR ( and IPAddress allocation) #116516

merged 31 commits into from Nov 1, 2023

Conversation

aojea
Copy link
Member

@aojea aojea commented Mar 12, 2023 •
edited

/kind feature
/kind api-change

After adding the new IPAddress object, this PR adds a new ServiceCIDR object that allows to dynamically configure the ranges used by Services in a cluster.

Add a new ServiceCIDR type that allows to dynamically configure the cluster range used to allocate Service ClusterIPs addresses

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1880-multiple-service-cidrs

Information for reviewers

The PR is split in independent commits

  • API definition of ServiceCIDR (it uses a Ready condition)
  • Integration API in the apiserver
  • apiserver controller to handle the default serviceCIDR
  • introduction of an IP radix tree structure that is the base of the functionality of all the controllers
  • extend the existing IPAllocator (allocator that uses the IPAddress object) to handle multiple and dynamic ServiceCIDRs
  • extend the existing repair loop for the IPAllocator to handle multiple ServiceCIDRs
  • new controller in the controller manager to handle the lifecycle of the ServiceCIDRs (do not leave orphan IPs)
  • integration tests to cover the whole feature, since it is API based only integration will give us great coverage
  • e2e tests
  • make update

Demo https://gist.github.com/aojea/c20eb117bf1c1214f8bba26c495be9c7

TODO

Dual stackness and IP families

kubectl apply -f https://gist.githubusercontent.com/aojea/90768935ab71cb31950b6a13078a7e92/raw/99ceac308f2b2658c7313198a39fbe24b155ae68/dual-stack.yaml
deployment.apps/httpd-deployment created
service/my-service-v4 created
service/my-service-prefer-dual created
Error from server (Invalid): error when creating "https://gist.githubusercontent.com/aojea/90768935ab71cb31950b6a13078a7e92/raw/99ceac308f2b2658c7313198a39fbe24b155ae68/dual-stack.yaml": Service "my-service-v6" is invalid: spec.ipFamilies[0]: Invalid value: "IPv6": not configured on this cluster
Error from server (Invalid): error when creating "https://gist.githubusercontent.com/aojea/90768935ab71cb31950b6a13078a7e92/raw/99ceac308f2b2658c7313198a39fbe24b155ae68/dual-stack.yaml": Service "my-service-require-dual" is invalid: spec.ipFamilyPolicy: Invalid value: "RequireDualStack": this cluster is not configured for dual-stack services

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/apiserver area/code-generation area/kubectl area/test sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Mar 12, 2023
@aojea aojea mentioned this pull request Mar 12, 2023
Merged
@aojea aojea force-pushed the servicecidr branch 7 times, most recently from 6cb9a45 to e32d378 Compare March 14, 2023 00:31
@aojea aojea changed the title [WIP] Multiple ServiceCIDR Multiple ServiceCIDR ( and IPAddress allocation) Mar 14, 2023
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 14, 2023
@aojea
describer: use new ServiceCIDR API
513fdb5 
Change-Id: Iff11c70f20bab3e55e4e569fb110ef25dd6dd97e
@aojea
default_servicecidr_controller: use new ServiceCIDR API
7e87806 
Change-Id: I2c53815136e0d985959420911339aacd62e01e78
@aojea
ipallocator/cidrallocator: use new ServiceCIDR API
881cf4d 
Change-Id: I5b300373e21cc65f6ef171790ef81ee391f1c752

cidrallocator

Change-Id: I13050a9d60360e555b75a13beaa9a923268ec4e8
@aojea
repairip controller: use new ServiceCIDR API
016c3c9
@aojea
service cidr controller manager: use new ServiceCIDR API
3edcce5
@aojea
integration etcd data: use new ServiceCIDR API
271900a 
Change-Id: I68a91815b3220a41e97667d8f8f4a3bdf5a91e8a
@aojea
test integration: use new ServiceCIDR API
5d90d07 
Change-Id: I7aaa978842fbe8d75383f8a3159ca4fce01ffdcb
@aojea
test e2e: use new ServiceCIDR API
9917c72 
Change-Id: I82075c4252d8734886f80dd68327fe73e2ed4ba5
@aojea
make update with the new API with arrays
fee07ad
@aojea
servicecidrs controller clarify condition false reevaluation
3b69bd6 
Change-Id: I0eb8d39abe9b7b0ce6472ff426e9a62e7155aae1
@aojea
hack/update-codegen
ce5a22d 
Change-Id: I929e8fe0313ec2e0aef2b26a5138cd88bdfe8914
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2023
@k8s-ci-robot
Copy link
Contributor

@aojea: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-verify-govet-levee ef5054723c1c081f42dfccc34e37ba20cc06e770 link true /test pull-kubernetes-verify-govet-levee
pull-kubernetes-e2e-kind-alpha-features 799f1c21ff4a1891574ab350e96ea5b06775d369 link false /test pull-kubernetes-e2e-kind-alpha-features
pull-kubernetes-linter-hints ce5a22d link false /test pull-kubernetes-linter-hints
pull-kubernetes-verify-lint ce5a22d link false /test pull-kubernetes-verify-lint
pull-kubernetes-e2e-ubuntu-gce-network-policies ce5a22d link false /test pull-kubernetes-e2e-ubuntu-gce-network-policies

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@dims
Copy link
Member

dims commented Oct 31, 2023

few more things to fix here @aojea

@thockin
Copy link
Member

thockin commented Oct 31, 2023

I'm still LGTM and the tests are still not

@aojea
Copy link
Member Author

aojea commented Oct 31, 2023

I'm still LGTM and the tests are still not

the jobs failing are unrelated to this PR, and not required

@thockin
Copy link
Member

thockin commented Oct 31, 2023

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: af57a22b8de921765055d485e79536dc5165487e

@aojea
Copy link
Member Author

aojea commented Nov 1, 2023

/milestone v1.29

Lgtm and approved before code freeze

@k8s-ci-robot k8s-ci-robot added this to the v1.29 milestone Nov 1, 2023
@k8s-ci-robot k8s-ci-robot merged commit 6abff74 into kubernetes:master Nov 1, 2023
15 of 18 checks passed
@pohly pohly mentioned this pull request Nov 1, 2023
Merged
liggitt
liggitt reviewed Nov 1, 2023
View reviewed changes
plugin/pkg/auth/authorizer/rbac/bootstrappolicy/controller_policy.go
@@ -369,6 +369,16 @@ func buildControllerRoles() ([]rbacv1.ClusterRole, []rbacv1.ClusterRoleBinding)
eventsRule(),
},
})
addControllerRole(&controllerRoles, &controllerRoleBindings, rbacv1.ClusterRole{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this added an unconditional role / binding to default cluster policy

this should be wrapped in a feature gate check so we don't create these permissions until the feature is enabled

fixed in #121682

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for fixing it

@pacoxu pacoxu mentioned this pull request Nov 14, 2023
Closed
@zhangzujian zhangzujian mentioned this pull request Dec 9, 2023
Open
@tnqn tnqn mentioned this pull request Dec 13, 2023
Closed
5 tasks
@sdmodi sdmodi mentioned this pull request Mar 15, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liggitt liggitt liggitt left review comments

@andrewsykim andrewsykim Awaiting requested review from andrewsykim

@ardaguclu ardaguclu Awaiting requested review from ardaguclu

@apelisse apelisse Awaiting requested review from apelisse

@caesarxuchao caesarxuchao Awaiting requested review from caesarxuchao

@danwinship danwinship Awaiting requested review from danwinship

@khenidak khenidak Awaiting requested review from khenidak

@smarterclayton smarterclayton Awaiting requested review from smarterclayton

@thockin thockin Awaiting requested review from thockin

Assignees

@danwinship danwinship

@smarterclayton smarterclayton

@thockin thockin

@khenidak khenidak

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/apiserver area/code-generation area/kubectl area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
SIG Apps
Archived in project
SIG Auth
Archived in project
SIG CLI
Archived in project
[sig-release] Bug Triage
Archived in project
Milestone
v1.29
Development

Successfully merging this pull request may close these issues.

None yet