Ziirish's Home :: Blog

Ziirish's Pub

Gitlab-ci coverage with python nose

Hi folks,

Starting from today, my blog posts will be written in English as often as I can.

The main reason is that I do most of my researches in English. This blog has always been some kind of a bookmark to me where I note a few tips I find useful.

Maybe these tips can be useful for someone else.

Anyway, today I wanted to give you a little tip about Gitlab-ci . Since Gitlab-ci 5.1, you can define a regex that will be processed against the output of your continuous integrations tests in order to extract the coverage ratio of your tests.

Most of the unit tests libraries can provide you this information.

For my project Burp-ui , I use the python nose library with the python coverage plugin.

The thing is I haven't been able to find the right regex to match the standard nose + coverage output which is something like that:

% nosetests --with-coverage --cover-package=burpui test/test_burpui.py
...........
Name                            Stmts   Miss  Cover   Missing
-------------------------------------------------------------
burpui                             41     11    73%   42, 48-56, 61-62
burpui.forms                        6      0   100%
burpui.misc                         0      0   100%
burpui.misc.auth                    0      0   100%
burpui.misc.auth.basic             45      4    91%   18-21
burpui.misc.auth.interface          8      3    63%   5, 8, 12
burpui.misc.backend                 0      0   100%
burpui.misc.backend.burp1         450    308    32%   108-109, 112-113, 115-116, 122-123, 126-127, 129-130, 136-137, 140-141, 143-144, 146, 148-149, 157, 193, 209-210, 213, 218-233, 239-257, 260-374, 381-465, 472-511, 518-528, 540-544, 553-571, 578-600, 607-646, 650, 653-706, 710, 715
burpui.misc.backend.interface      29     13    55%   5-7, 10, 13, 16, 19, 22, 25, 28, 31, 34, 37
burpui.misc.parser                  0      0   100%
burpui.misc.parser.burp1          110     22    80%   291, 297, 299-300, 327-328, 338, 358-371, 378-379
burpui.misc.parser.interface        8      2    75%   9, 12
burpui.misc.utils                  23     18    22%   24-54
burpui.routes                     367    196    47%   20, 58, 63-108, 142-149, 153-157, 164-165, 180, 184-193, 196-202, 213-220, 232-243, 252-270, 281-301, 310-317, 333, 343-345, 349-355, 359, 374-388, 399-404, 413-418, 427-429, 440-444, 455-463, 476, 494-495, 506
burpui.server                      85     29    66%   24, 27, 41-43, 46-48, 57-64, 82, 87-89, 94-107
-------------------------------------------------------------
TOTAL                            1172    606    48%
----------------------------------------------------------------------
Ran 11 tests in 1.782s

OK


I thought something like /TOTAL.*(\d+\%)/ would be enough, but it looks like I was wrong... So, in order to match the right coverage, I wrote a handler script like that:

#!/bin/bash

LOG=$(mktemp) nosetests --with-coverage --cover-package=burpui test/test_burpui.py 2>&1 >$LOG
cat $LOG grep TOTAL$LOG | awk '{ print "TOTAL: "$4; }' rm$LOG


That way the regex becomes: /TOTAL:\s+\d+\%/, and TADA, it works here