-
Notifications
You must be signed in to change notification settings - Fork 0
fix: benchmarks timeout #50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
fix: benchmarks timeout #50
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR addresses performance issues with CodSpeed's Go benchmark runner by implementing safeguards to prevent benchmarks from running excessively long. The changes add timeout limits for both b.Loop() and b.N benchmark execution patterns.
Changes:
- Added maximum execution time limits (2x the benchmark time) to prevent runaway benchmarks
- Refactored duplicate warmup logic into a shared
runBenchmarkWithWarmupfunction - Modified
b.Loop()to track overhead from timer start/stop operations
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| go-runner/overlay/codspeed.go | Adds BenchMaxTimeMult constant, stopOrScaleBLoopCodspeed() method for loop time limiting, and runBenchmarkWithWarmup() function containing shared benchmark execution logic |
| go-runner/overlay/benchmark1.25.0.go | Refactors inline warmup code to use shared runBenchmarkWithWarmup() function, initializes loopStartTime, and switches to stopOrScaleBLoopCodspeed() |
| go-runner/overlay/benchmark1.24.0.go | Identical changes to benchmark1.25.0.go for Go 1.24.0 compatibility |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
CodSpeed Performance ReportMerging this PR will not alter performanceComparing Summary
Footnotes
|
3eddbb4 to
f8b8d75
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
4f8bc49 to
6940d07
Compare
b.Loop()execution timeSets a maximum time for the benchmark execution, and exits early if we exceed it. Currently this is set to
3 * benchTime(so 9s in our case)