Coverage badge in Gitlab CI with Python coverage always unknown - Stack Overflow
How to create a Gitlab CI coverage badge for R - Stack Overflow
testing - Gitlab coverage badge always unknow - Stack Overflow
Gitlab Coverage badge not working - Stack Overflow
I finally got the coverage badge displaying a percentage instead of unknown today for my python project. Here's the relevant content from my .gitlab-ci.yml:
job:
script:
- 'python -m venv venv'
- '.\venv\Scripts\activate'
- 'python -m pip install -r requirements.txt'
- 'coverage run --source=python_project -m unittest discover ./tests'
- 'coverage report --omit=things_that_arent_mine/*'
- 'coverage xml'
artifacts:
reports:
cobertura: 'coverage.xml'
I'm also using the regex for gcovr listed in the repo CI/CD Settings > General Pipelines > Test coverage parsing which I found after reading this as well as the second to last comment on this:
^TOTAL.*\s+(\d+\%)$
In the repo General Settings > Badges, my badge link is:
http://gitlab-server/%{project_path}/-/jobs
and my badge image url is:
http://gitlab-server/%{project_path}/badges/%{default_branch}/coverage.svg
I don't quite know what the Cobertura report artifact is for (I think it specifically has to do with merge requests) but I have it in there because the tutorials took me down that road. I did confirm that removing the following from the .gitlab-ci.yml doesn't break the badge or coverage number on the jobs page:
- 'coverage xml'
artifacts:
reports:
cobertura: 'coverage.xml'
While removing or commenting:
- 'coverage report --omit=things_that_arent_mine/*'
does break the badge as well as the coverage number displayed on the CI/CD jobs page of the repo. I also tried some regex variations that I tested in rubular but the only one that didn't cause gitlab to barf was the gcovr one.
Hopefully this helps out, it was kind of difficult and arduous to piece together what was needed for this badge to work on a python project.
EDIT: Also just figured out how to add some sexy precision to the coverage percentage number. If you change the coverage report line of the .gitlab-ci.yml to:
- 'coverage report --omit=things_that_arent_mine/* --precision=2'
And the regex in CI/CD Settings > General Pipelines > Test coverage parsing to:
^TOTAL.+?(\d+.\d+\%)$
That should give you a very precise coverage number that effectively no one but you and I will care about. But by gosh we'll know for sure if coverage is 99.99% or 100%.
A solution that worked for me:
In the .gitlab-ci.yml file, in the job that runs the coverage report, I added the following lines:
script:
# some lines omitted for brevity
- coverage report --omit=venv/*
coverage: '/TOTAL.*\s+(\d+\%)/'
In particular, I couldn't get the badge to show anything but unknown until I added the coverage directive to my test job. Bear in mind, different tools may print different output and so you will likely have to change the regular expression, as I did.
I was struggling with this too for a python project and found that the test coverage results is now deprecated (https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-to-a-merge-request-deprecated)
The answer for me was to use the coverage keyword in my .gitlab-ci.yml with the required regex to extract the percentage from the job log
e.g.
test:
stage: test
extends: .python-reports
script:
- pipenv run pytest --cov --cov-report term-missing --cov-report xml:./coverage_out_report.xml --junitxml=./test_out_report.xml
coverage: '/TOTAL.*\s+(\d+%)$/'
And my badge defined as
Link https://gitlab.com/%{project_path}/-/commits/%{default_branch}
Badge image URL https://gitlab.com/%{project_path}/badges/%{default_branch}/coverage.svg
Now I see the coverage as 100% on main (or master) and a nice green badge
I had the same issue in a different stack (Java, Gradle, and Jacoco). I've already solved the issue, and I will share what I've done. I hope my solution gives you some idea.
First, In your project, go to Settings > CI/CD and expand the General pipelines section. Fill the Test coverage parsing textbox with an appropriate regular expression based on the tool you are using. You can find the sample list of these regular expressions on this page: Test coverage parsing samples
Gitlab expects that the test runner tool will print the test coverage percent based on the given regular expression somewhere on the CI job's log. If the test runner engine doesn't print it to the console, you can do it manually. For instance:
test:
stage: test
script:
- gradle test jacocoTestReport --info
# Print test coverage to console
- cat build/reports/jacoco/test/html/index.html | grep -o 'Total[^%]*%'
artifacts:
paths:
- build/reports/jacoco/test/*
After making sure that the test coverage exists in the log, the badge will work perfectly.
pipeline and coverage badge sample
I know this is an old question, but this may help others see this question in the future. Except setting the Test Coverage parsing regex in the Settings, CI/CD page, you need to make sure the test coverage percent is printed somewhere in the CI's console log. The printed line should match the given regex. So, Gitlab can parse and save it as the code coverage percent.
I've answered with more detail the same question: Gitlab coverage badge always unknow
Your regex works for me in Gitlab.
I've had similar problems though, because just rerunning a job does not pick up new settings, and so you need to do a fresh commit to get the coverage to show up in the job output.