Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

observe: warn on unknown field while JSON decoding #962

Merged
merged 1 commit into from
Apr 26, 2023

Conversation

kaworu
Copy link
Member

@kaworu kaworu commented Mar 27, 2023

Before this patch, when parsing JSON flows and encountering an unknown field, the CLI would log a warning and skip the flow.

Unknown fields is not uncommon however, e.g. when using the Hubble CLI against a more recent Hubble server version.

This patch

  • emit the flow if successfully parsed while ignoring unknown fields,
  • improve the warning message by hinting at upgrading the Hubble CLI,
  • make it so the warning message is logged only once.

@kaworu kaworu added kind/enhancement This would improve or streamline existing functionality. ⌨️ area/cli Impacts the command line interface of any command in the repository. labels Mar 27, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot added dont-merge/needs-release-note-label PR is blocked until the release note is set labels Mar 27, 2023
@kaworu kaworu added the release-note/misc This PR makes changes that have no direct user impact. label Mar 27, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label PR is blocked until the release note is set label Mar 27, 2023
@kaworu
Copy link
Member Author

kaworu commented Mar 27, 2023

Opening as draft as it will conflict with #958 and also unsure whether logging a warning message is the way to go.

@rolinh rolinh added the :basecamp: needs-rebase This PR needs to be rebased because it has merge conflicts. label Mar 28, 2023
Copy link
Collaborator

@michi-covalent michi-covalent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

time to ship

request *observer.GetFlowsRequest
allow filters.FilterFuncs
deny filters.FilterFuncs
discardUnknown bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is this going to be set?

Comment on lines 94 to 98
// the error might be that the JSON data contains an unknown field.
// This can happen we attempting to decode flows generated from a
// newer Hubble version than the CLI (having introduced a new
// field). Retry parsing discarding unknown fields and see whether
// the decoding is successful.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there nothing in the error that we can check to make sure it was unknown field, rather than attempting to parse twice?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracking the error from protobuf it's from

return d.newError(tok.Pos(), "unknown field %v", tok.RawString())
and
// newError returns an error object with position info.
func (d decoder) newError(pos int, f string, x ...interface{}) error {
line, column := d.Position(pos)
head := fmt.Sprintf("(line %d:%d): ", line, column)
return errors.New(head+f, x...)
}

So I see no way to errors.Is as it isn't wrapped. Matching on the error message doesn't feel right either.

Before this patch, when parsing JSON flows and encountering an unknown
field, the CLI would log a warning and skip the flow.

Unknown fields is not uncommon however, e.g. when using the Hubble CLI
against a more recent Hubble server version.

This patch

* emit the flow if successfully parsed while ignoring unknown fields,
* improve the warning message by hinting at upgrading the Hubble CLI,
* make it so the warning message is logged only once.

Signed-off-by: Alexandre Perrin <[email protected]>
@kaworu kaworu force-pushed the pr/kaworu/warn-unknown-field branch from bb32734 to c237c6c Compare April 3, 2023 10:29
@kaworu kaworu removed the :basecamp: needs-rebase This PR needs to be rebased because it has merge conflicts. label Apr 3, 2023
@kaworu kaworu marked this pull request as ready for review April 3, 2023 10:55
@kaworu kaworu requested a review from a team as a code owner April 3, 2023 10:55
@kaworu kaworu requested review from chancez and removed request for a team April 3, 2023 10:55
@chancez
Copy link
Contributor

chancez commented Apr 3, 2023

This only seems to update the io reader observer, what about the gRPC observer client?

@kaworu
Copy link
Member Author

kaworu commented Apr 3, 2023

This only seems to update the io reader observer, what about the gRPC observer client?

Yes good question, so functionally the gRPC client already ignore unknown fields. If we think the warning would be a good idea too for gRPC let's do it.

@chancez
Copy link
Contributor

chancez commented Apr 3, 2023

@kaworu Ahhh, I see. Perhaps you could update the PR title/commit message to reflect this is only when ingesting flows from a file/stdin.

Copy link
Contributor

@chancez chancez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes LGTM.

@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Apr 3, 2023
@michi-covalent michi-covalent merged commit 2149334 into master Apr 26, 2023
@michi-covalent michi-covalent deleted the pr/kaworu/warn-unknown-field branch April 26, 2023 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⌨️ area/cli Impacts the command line interface of any command in the repository. kind/enhancement This would improve or streamline existing functionality. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/misc This PR makes changes that have no direct user impact.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants