Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement buf alpha repo sync #2122

Merged

Conversation

saquibmian
Copy link
Contributor

Only base branch is supported today.

sync replicates pushed git state to the BSR, walking commits in order. New or moved tags are also pushed.

Only base branch is supported today.

`sync` replicates pushed git state to the BSR, walking commits in order.
New or moved tags are also pushed.
@saquibmian
Copy link
Contributor Author

Alright, this one is a rough draft, it's just to get it out there for initial assessment.


const (
errorFormatFlagName = "error-format"
moduleFlagName = "module"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not first arg, similar to all other commands?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be repeated, unless we're cool with repeated inputs?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could make the same arguments for most of our commands (buf lint could take multiple inputs), but there's no reason to, and deviating from that standard without a clear reason to do so makes the UX inconsistent. Is there a clear reason to do so here?

&f.Modules,
moduleFlagName,
nil,
"The module(s) to sync to the BSR; you can provide a module override in the format "+
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect sync to be congruent with other commands, namely taking a module or workspace as first arg

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't resolve comments that are not resolved.

) error {
// Assume that this command is run from the repository root. If not, `OpenRepository` will return
// a dir not found error.
repo, err := git.OpenRepository(git.DotGitDir, command.NewRunner())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

THIS feels like a flag that defaults to .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I explicitly don't want to let the user control the .git dir, I want them to run sync from the root of the repo they are syncing. The fewer options they can configure, the simpler the command is to use.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments should be resolved by the creator when there is a design disagreement

return err
}

func pushOrCreate(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks copied from push - worth making into a common command in bufcli? Or will this diverge?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's 90% a copy, yeah, namely the the message sent as part of PushManifestAndBlobs is different. I would definitely like to move this to a bufpush package.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do so

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has diverged, calling an entirely different RPC now.

Copy link
Member

@unmultimedio unmultimedio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, no blockers, only nits 😄

private/buf/bufsync/bufsync.go Outdated Show resolved Hide resolved
private/buf/bufsync/syncer.go Outdated Show resolved Hide resolved
private/buf/bufsync/syncer.go Outdated Show resolved Hide resolved
private/buf/bufsync/syncer.go Show resolved Hide resolved
private/buf/bufsync/syncer.go Outdated Show resolved Hide resolved
private/buf/bufsync/syncer.go Outdated Show resolved Hide resolved
@saquibmian saquibmian requested a review from bufdev May 30, 2023 21:00
Copy link
Member

@unmultimedio unmultimedio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

// We failed to build the module. We can warn on this
// and carry on. Note that because of resumption, we will typically only come
// across this commit once, we will not log this warning again.
s.logger.Warn(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something that is conceivably part of control flow - it is conceivable that the caller of bufsync will want to know that a module was not built, and take some type of action. By logging here, we've effectively swallowed the warning. Instead, such data should be returned up the stack in a manner that allows the caller to decide what to do (including potentially log it)

// We found a module but the module config is invalid. We can warn on this
// and carry on. Note that because of resumption, we will typically only come
// across this commit once, we will not log this warning again.
s.logger.Warn(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for all warnings here

@saquibmian saquibmian requested a review from bufdev June 1, 2023 15:22

// NewModule constructs a new module that can be synced with a Syncer.
func NewModule(dir string, identityOverride bufmoduleref.ModuleIdentity) (Module, error) {
path, err := normalpath.NormalizeAndValidate(dir)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should move this into newSyncableModule. Let me do it up.

@saquibmian saquibmian merged commit a3ed389 into main Jun 1, 2023
7 checks passed
@saquibmian saquibmian deleted the saquib/bsr-1894-implement-buf-alpha-repo-sync-command branch June 1, 2023 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants