Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: adding explicit batching #107

Merged
merged 9 commits into from
Oct 2, 2024
Merged

feat: adding explicit batching #107

merged 9 commits into from
Oct 2, 2024

Conversation

vlastahajek
Copy link
Contributor

Closes #103

Proposed Changes

Added Batcher for simple explicit batching

Checklist

  • CHANGELOG.md updated
  • Rebased/mergeable
  • A test has been added if appropriate
  • Tests pass
  • Commit messages are conventional
  • Sign CLA (if not already signed)

Copy link

codecov bot commented Sep 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.18%. Comparing base (5671330) to head (bf9fb71).
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #107      +/-   ##
==========================================
+ Coverage   85.35%   86.18%   +0.82%     
==========================================
  Files          14       15       +1     
  Lines        1127     1194      +67     
==========================================
+ Hits          962     1029      +67     
  Misses        137      137              
  Partials       28       28              
Flag Coverage Δ
?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@bednar bednar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vlastahajek, thanks for your PR 👍. I have a small request regarding the formatting of our CHANGELOG.md:

CHANGELOG.md Outdated Show resolved Hide resolved
}

// Add a metric to the batcher and call the given callbacks if any
func (b *Batcher) Add(p *influxdb3.Point) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering if this could be a variadic function, so that []*influxdb3.Point could be used as an argument as well, or whether we might want o support adding []*influxdb3.Point in another function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Variadic parameters look good.

Copy link
Contributor

@karel-rehor karel-rehor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks Good. But I have two questions.

  1. Shouldn't there be a function in batcher to add a slice or array of *influxdb3.Point instead of just a single point.
  2. I noticed that using Option methods WithSize() and WithCapacity() it is possible to create a batcher with Size > Capacity. My understanding of Golang is that append will be lenient and reallocate as needed, but then the value of Batcher.capacity will no longer represent the true capacity of the internal buffer. Are we OK with this?

@vlastahajek
Copy link
Contributor Author

@karel-rehor,

  1. Answered in the inline comment
  2. The capacity is just an initial capacity. If the batch-size is bigger, reallocation will occur.

Copy link
Contributor

@karel-rehor karel-rehor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests pass locally. Examples work. Looks Good.

Copy link

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks @vlastahajek!

Copy link
Member

@bednar bednar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@bednar bednar merged commit 772c98f into main Oct 2, 2024
11 checks passed
@bednar bednar deleted the feat/simple-batching branch October 2, 2024 08:13
@bednar bednar added this to the 0.12.0 milestone Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add code to ease batching
4 participants