benchmark: add --csv option to compare.js with --analyze · nodejs/node@a079177 · GitHub
Skip to content

Commit a079177

Browse files
jasnelladuh95
authored andcommitted
benchmark: add --csv option to compare.js with --analyze
Add a `--csv {filename}` option to benchmark/compare.js to capture the CSV when the `--analyze` option is used Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode PR-URL: #65922 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
1 parent a578933 commit a079177

3 files changed

Lines changed: 123 additions & 42 deletions

File tree

‎benchmark/compare.js‎

Lines changed: 60 additions & 42 deletions

‎doc/contributing/writing-and-running-benchmarks.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -411,6 +411,8 @@ module, you can use the `--filter` option:_
411411
--set variable=value set benchmark variable (can be repeated)
412412
--no-progress don't show benchmark progress indicator
413413
--analyze perform statistical analysis inline (no R needed)
414+
--csv filename write csv output to filename (can be combined
415+
with --analyze)
414416
--scale 1000 rate multiplier for --analyze precision
415417
--max-regression N exit with code 1 if any significant regression
416418
exceeds N% (implies --analyze)
@@ -424,6 +426,14 @@ The simplest way to get statistical results is to pass `--analyze`:
424426
node benchmark/compare.js --old ./node-main --new ./node-pr-5134 --analyze string_decoder
425427
```
426428

429+
Use `--csv` to retain the raw benchmark results. If you pass both `--csv` and
430+
`--analyze`, both the raw results and the analysis are printed:
431+
432+
```bash
433+
node benchmark/compare.js --old ./node-main --new ./node-pr-5134 \
434+
--analyze --csv compare-pr-5134.csv string_decoder
435+
```
436+
427437
This runs the benchmarks and prints the analysis directly:
428438

429439
```console
@@ -433,6 +443,13 @@ string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=128 encoding='utf8'
433443
...
434444
```
435445

446+
Use `-csv -` to output the raw results to stdout with the analysis.
447+
448+
```bash
449+
node benchmark/compare.js --old ./node-main --new ./node-pr-5134 \
450+
--analyze --csv - string_decoder
451+
```
452+
436453
The `--analyze` mode uses the histogram API's `welchTest()` method to perform
437454
the same Welch's t-test that the R script uses. Benchmark rates are scaled to
438455
integers for the histogram (controlled by `--scale`, default 1000). With the
Lines changed: 46 additions & 0 deletions

0 commit comments

Comments
 (0)