Code coverage integration


(Simon) #1

I had a look at codecov.io and it is very simple to set up. I think it could be useful to visualize our test coverage, especially for diffs on PRs. I am aware that coverage doesn’t mean, that the code actually does what it should, but no coverage means it’s certainly untested and that maybe warrants special attention then.

All that’s required is to run tests with coverage recording enabled and a little bit of scripting and calling a bash script to upload to codecov. That can either be a separate build step/job on the CI or adding the necessary options to one of the existing test stages. Both could be done as an additional command in build.go or a separate script. I tested it for unit tests, but it should even be doable on the integration tests as well.

What do you think?


(Audrius Butkevicius) #2

I think it needs proprietary binaries or some curl foo | sudo bash which sucks. I guess if I get this “for free” without having to do anything, sure.


(Simon) #3

What does it need proprietary binaries for? You just need to run a bash script, for which the “source” code is known, and apache licensed: https://codecov.io/bash

And sure, I don’t propose it and expect others to do it :wink:
I probably wont do it with high prio right now myself, just want to know where I stand if I find myself with some time and motivation for it.


(Simon) #4

This is really simple: Either we just enable a travis build for it (i.e. no new work for teamcity) or add a build step/job to teamcity (which requires to put a secret token somewhere on the build server). In either scenario it can just “run in the background”, providing coverage data, or a bot can be activated which posts on PRs with information about the coverage change and how much of the diff is covered (example: https://github.com/imsodin/syncthing/pull/4). So two things to decide:

  • Travis or TeamCity?
  • PR-bot or not?

PS: Our current coverage of lib/ is just shy over 50%.


(Audrius Butkevicius) #5

I vote team city + bot.