ENH: Added ``coverage``, ``gcov`` and ``generate-lcov-html`` flags in ``spin test`` by ganesh-k13 · Pull Request #24992 · numpy/numpy · GitHub
Skip to content

ENH: Added coverage, gcov and generate-lcov-html flags in spin test#24992

Closed
ganesh-k13 wants to merge 1 commit into
numpy:mainfrom
ganesh-k13:bld_24080_coverage
Closed

ENH: Added coverage, gcov and generate-lcov-html flags in spin test#24992
ganesh-k13 wants to merge 1 commit into
numpy:mainfrom
ganesh-k13:bld_24080_coverage

Conversation

@ganesh-k13

@ganesh-k13 ganesh-k13 commented Oct 24, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Changes

  • Added coverage flag in test

Example

$ spin test --coverage -- -v -k numpy/tests/test_numpy_config.py 
Removing `/workspaces/numpy/build/coverage`
Invoking `build` prior to running tests:

.
.
.

------------------------------------------------------------------------------
TOTAL                                                     109351  13626    88%
Coverage HTML written to dir /workspaces/numpy/build/coverage

In Spin, after merging of scientific-python/spin#146:

image

  1. spin test --coverage --gcov
  2. spin test --generate-lcov-html
~/os/numpy (bld_24080_coverage) » spin test --generate-lcov-html                                                                                                            ganesh@ganesh-MS-7B86
Deleting old HTML coverage report...
Generating HTML coverage report...
$ ninja coverage-html -C build
Coverage report generated successfully and written to /home/ganesh/os/numpy/build/meson-logs/coveragereport/index.html

Other error handling

  1. spin build
  2. spin test --generate-lcov-html
~/os/numpy (bld_24080_coverage) » spin test --generate-lcov-html                                                                                                            ganesh@ganesh-MS-7B86
Error: GCOV files missing... Cannot generate coverage reports. Coverage reports can be generated by `spin test --coverage --gcov`

Notes

related #24080, scientific-python/spin#100
depends on scientific-python/spin#146

@ganesh-k13 ganesh-k13 self-assigned this Oct 24, 2023
@ngoldbaum ngoldbaum changed the title ENH: Added coverage flag in test ENH: Added coverage flag in spin test Oct 24, 2023
@stefanv

stefanv commented Oct 30, 2023

Copy link
Copy Markdown
Contributor

@ganesh-k13

ganesh-k13 commented Oct 31, 2023

Copy link
Copy Markdown
Member Author

Ah yes, that's possible. But I was thinking from runtest.py perspective and migrate all it's functionalities.

@ngoldbaum

ngoldbaum commented Jan 3, 2024

Copy link
Copy Markdown
Member

Hey @ganesh-k13 do you have any followup for this? I'd like to bring it in and also add support for generating C coverage via gcov.

@ganesh-k13

ganesh-k13 commented Jan 4, 2024

Copy link
Copy Markdown
Member Author

Hey @ngoldbaum , I'd be happy to add gcov integration to spin if there is a need for it. I'll get working on it tonight IST. Thanks for following up on this PR.

@ngoldbaum

ngoldbaum commented Jan 4, 2024

Copy link
Copy Markdown
Member

That'd be really useful. It would be nice to be able to generate a coverage report locally for both C and python.

@ganesh-k13 ganesh-k13 mentioned this pull request Jan 4, 2024
Merged
@charris charris changed the title ENH: Added coverage flag in spin test ENH: Added coverage flag in spin test Jan 4, 2024
@ganesh-k13 ganesh-k13 marked this pull request as draft January 4, 2024 16:34
ganesh-k13
ganesh-k13 commented Jan 4, 2024
View reviewed changes

@ganesh-k13 ganesh-k13 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I'm doing something wrong here, the .gcda generated from old runtest.py is of the format:

./build/temp.linux-x86_64-3.10/build/src.linux-x86_64-3.10/numpy/core/src/umath/matmul.gcda

while the new one looks like:

./build/numpy/_core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so.p/meson-generated_matmul.c.gcda

This means lcov is not working as expected. I'll take a look at this tomorrow, something off with the flags I'm guessing.

@ganesh-k13

ganesh-k13 commented Jan 5, 2024

Copy link
Copy Markdown
Member Author

CI builds will fail till scientific-python/spin#146 is merged and released, I'll move it ready for review so it does not go stale.

@ganesh-k13 ganesh-k13 marked this pull request as ready for review January 5, 2024 14:56
@ganesh-k13

ganesh-k13 commented Jan 5, 2024 •
edited
Loading

Copy link
Copy Markdown
Member Author

Hi @ganesh-k13, any reason to not simply let the coverage flag get passed through to the built-in command?

Ohh I understood this statement now @stefanv . I have made the changes to reflect that via
c805f94.

@ganesh-k13 ganesh-k13 changed the title ENH: Added coverage flag in spin test ENH: Added coverage, gcov and generate-lcov-html flags in spin test Jan 5, 2024
@ngoldbaum ngoldbaum mentioned this pull request Jan 8, 2024
Merged
@rgommers rgommers mentioned this pull request Jul 17, 2024
Open
@ganesh-k13

ganesh-k13 commented Jul 18, 2024

Copy link
Copy Markdown
Member Author

This is currently blocked on #26977 as the spin changes needed for gcov are in Spin 0.9 and we are running 0.8

@ngoldbaum

ngoldbaum commented Jul 18, 2024

Copy link
Copy Markdown
Member

I think we can update spin now in CI, we were waiting on spin 0.11 to do that and it's out now.

@stefanv

stefanv commented Dec 4, 2024

Copy link
Copy Markdown
Contributor

@ganesh-k13 Now that we're on spin 0.13, do you want to take another look at this?

@ganesh-k13

ganesh-k13 commented Dec 4, 2024

Copy link
Copy Markdown
Member Author

Thanks @stefanv! I'll get the PR working this week.

@ganesh-k13 ganesh-k13 force-pushed the bld_24080_coverage branch 2 times, most recently from c76a1bf to e894f97 Compare December 5, 2024 06:54
@ganesh-k13
ENH: Added new flags in spin test
f555c8f 
- `coverage`
- `gcov`
- `generate-lcov-html`
@mattip

mattip commented Dec 5, 2024

Copy link
Copy Markdown
Member

Is anything here NumPy-specific, or in other words can this at some stage be migrated to spin itself so more people can use it?

@ganesh-k13

ganesh-k13 commented Dec 5, 2024

Copy link
Copy Markdown
Member Author

I thought the same thing and started testing this out in my local spin. I too feel we can move all these changes upstream.

@ganesh-k13 ganesh-k13 mentioned this pull request Dec 5, 2024
Closed
1 task
@ganesh-k13

ganesh-k13 commented Jan 9, 2025

Copy link
Copy Markdown
Member Author

@ganesh-k13 ganesh-k13 closed this Jan 9, 2025
@ganesh-k13 ganesh-k13 deleted the bld_24080_coverage branch January 9, 2025 16:42
@ganesh-k13 ganesh-k13 mentioned this pull request Mar 2, 2026
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@ganesh-k13 ganesh-k13

Labels

01 - Enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ganesh-k13 @stefanv @ngoldbaum @mattip