Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multi-module releases in go-control-plane #714

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

alecholmez
Copy link
Contributor

@alecholmez alecholmez commented Jun 5, 2023

This PR introduces support for multi-module releases in go-control-plane following the plan of action outlined in issue #391.

A few things have happened:

  • Bumped modules to Go 1.20 (to inherit support for workspaces)
  • Introduced a go workspace that orchestrates all the subcomponents of this repo
  • Initialized new modules for ./envoy and ./contrib containing the generated go stubs
  • Renamed build/ to scripts/ so we have a bucket for more general repo automation

When this PR is merged, a tag will be cut at v0.11.2 but a new tag will be introduced with envoy/v1.26.2. closes #391

valerian-roche
valerian-roche previously approved these changes Jun 5, 2023
Copy link
Contributor

@valerian-roche valerian-roche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with go workspaces, but overall lgtm
Nice to see independent versioning through this though


go 1.20

replace github.com/envoyproxy/go-control-plane/envoy => ../envoy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding from go.work is that it should no longer be needed to add those replace commands
Is that actually not the case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually couldn't get go mod tidy to play nice without them. I can mess around with it a bit more but figured this was fine for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to drop the replace from ./envoy but that one still complained so I left it in.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is an indication that we don't have the right structure? Since the "envoy" and "contrib" protobufs are bound to the same Envoy release, does it make sense to allow the possibility for a program to import different versions of them?

How hard would it be to move the contrib protobufs to "envoy/contrib"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I suppose that a similar question arises for the "./ratelimit" directory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heres a branch of Contour using the above branch: https://github.com/sunjayBhatia/contour/tree/use-go-control-plane-multi-module

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the above go-control-plane branch also copies the ratelimit module to a package in the new api module)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this branch of Contour works with this PR's branch (plus this commit sunjayBhatia@4be6f62): sunjayBhatia/contour@dc8fa81

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think either approach will work just fine, though the upgrade/transition UX might be cleaner (no possibility for users complaining about ambiguous package paths if they are requiring mismatched versions) if the new modules have a new package path

Copy link
Contributor

@jpeach jpeach Jun 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the above go-control-plane branch also copies the ratelimit module to a package in the new api module)

I'll check later, but me recollection is that the ratelimit protos don't come from the envoy repository. So they probably need to be versioned differently and it might not make sense for them to be in the same module as the envoy api.

.gitignore Outdated Show resolved Hide resolved
contrib/go.mod Outdated
@@ -0,0 +1,23 @@
module github.com/envoyproxy/go-control-plane/contrib

go 1.20
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JFYI, we should be prepared for some pushback on this. It's possible that some consumer of this module will be using earlier Go versions (see #271 for example).

Copy link
Contributor Author

@alecholmez alecholmez Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try and roll back to 1.18 but that is a bare minimum requirement for workspaces. Wondering if we should see if it's that problematic and roll back if people complain.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 please don't use 1.20. In grpc-go, which uses this, we support 1.18+ and also haven't made any changes AFAIK that would prevent even 1.17 or maybe 1.16 from working (e.g. no use of any or generics). IMO this (and the use of new language features) should stay at the oldest version you can reasonably handle supporting.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alecholmez it sounds like this is the most contentious point within this change set.

@jpeach
Copy link
Contributor

jpeach commented Jun 6, 2023

  • Initialized new modules for ./envoy and ./contrib containing the generated go stubs

We probably want to do the same for ./ratelimit too?

@alecholmez alecholmez force-pushed the multi-module-release branch 2 times, most recently from 0ab1e02 to 1b350d3 Compare June 6, 2023 15:00
go.work Outdated
@@ -0,0 +1,10 @@
go 1.20
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the dumb question...what do go workspaces actually do for you? Someone wanted to add a go.work file to our repo, but I didn't fully understand how it would help. We have several modules and don't use a workspace, and things generally work fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think they are more useful in local development etc. when you have modules side-by-side rather than a module hierarchy like we have here, you should mainly be able to remove the replace directives but it doesn't seem (from my playing around with it as well) that we get that here because of the package/module structure

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. to get the other modules in to actually use the toher modules in this repo all the way, had to do this: sunjayBhatia@4be6f62

so I'm not sure how much the workspace is getting us

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the decision to ultimately remove the workspace? I'm fine with that

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot added the stale label Jul 10, 2023
@github-actions
Copy link

This pull request has been automatically closed because it has not had activity in the last 37 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot closed this Jul 17, 2023
@alecholmez alecholmez reopened this Jul 17, 2023
@alecholmez
Copy link
Contributor Author

Hey all, been away for quite a while. Starting to catch back up on this project. Did we ever come to a consensus on the way forward here? I have some cycles I can spend to push this across so we can get onto a more regular release cycle.

@mmorel-35
Copy link
Contributor

Hi @alecholmez ,
You might want to rebase your work.
The go version has changed.
Have you considered defining a module per extension in contrib instead of a contrib module ?

@valerian-roche
Copy link
Contributor

I merged head, but my understanding is that we need to create the tags for the sub-modules first in order to allow proper importing. Trying to import in a project with the current setup, I get

 % go mod tidy
go: downloading github.com/envoyproxy/go-control-plane v0.12.1-0.20240804223412-e9538224ae87
go: github.com/envoyproxy/[email protected] requires
	github.com/envoyproxy/go-control-plane/[email protected]: invalid version: unknown revision 000000000000

In this case this PR would need:

  • first create the changes within the submodules, and tag them
  • this would require to follow some specific order, to add the different steps (envoy, then contrib and ratelimit, then the rest)
  • update tooling to allow consistency. I'm still unclear how we can maintain consistency in tagging with a multi-module repository. We can auto-tag envoy and contrib when syncing from envoy, but we'd need to still manually update all the other modules

go.work Show resolved Hide resolved
@mmorel-35 mmorel-35 force-pushed the multi-module-release branch 2 times, most recently from 36e2e63 to a360f6f Compare August 30, 2024 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Develop release policy
8 participants