benchmark: add --csv option to compare.js with --analyze · nodejs/node@c518831 · GitHub
Skip to content

Commit c518831

Browse files
jasnelladuh95
authored andcommitted
benchmark: add --csv option to compare.js with --analyze
Add a `--csv {filename}` option to benchmark/compare.js to capture the CSV when the `--analyze` option is used Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode PR-URL: #65922 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
1 parent 3666d6a commit c518831

3 files changed

Lines changed: 123 additions & 42 deletions

File tree

‎benchmark/compare.js‎

Lines changed: 60 additions & 42 deletions

‎doc/contributing/writing-and-running-benchmarks.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -416,6 +416,8 @@ module, you can use the `--filter` option:_
416416
--set variable=value set benchmark variable (can be repeated)
417417
--no-progress don't show benchmark progress indicator
418418
--analyze perform statistical analysis inline (no R needed)
419+
--csv filename write csv output to filename (can be combined
420+
with --analyze)
419421
--scale 1000 rate multiplier for --analyze precision
420422
--max-regression N exit with code 1 if any significant regression
421423
exceeds N% (implies --analyze)
@@ -429,6 +431,14 @@ The simplest way to get statistical results is to pass `--analyze`:
429431
node benchmark/compare.js --old ./node-main --new ./node-pr-5134 --analyze string_decoder
430432
```
431433

434+
Use `--csv` to retain the raw benchmark results. If you pass both `--csv` and
435+
`--analyze`, both the raw results and the analysis are printed:
436+
437+
```bash
438+
node benchmark/compare.js --old ./node-main --new ./node-pr-5134 \
439+
--analyze --csv compare-pr-5134.csv string_decoder
440+
```
441+
432442
This runs the benchmarks and prints the analysis directly:
433443

434444
```console
@@ -438,6 +448,13 @@ string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=128 encoding='utf8'
438448
...
439449
```
440450

451+
Use `-csv -` to output the raw results to stdout with the analysis.
452+
453+
```bash
454+
node benchmark/compare.js --old ./node-main --new ./node-pr-5134 \
455+
--analyze --csv - string_decoder
456+
```
457+
441458
The `--analyze` mode uses the histogram API's `welchTest()` method to perform
442459
the same Welch's t-test that the R script uses. Benchmark rates are scaled to
443460
integers for the histogram (controlled by `--scale`, default 1000). With the
Lines changed: 46 additions & 0 deletions

0 commit comments

Comments
 (0)