source: bfontaine.net
Coveralls is a service that keep track of your tests coverage for you. It can notifies you when your coverage decreases under a custom threshold, and their bot comments on pull requests to report their tests coverage. Like Travis-CI, it allows you to add a badge to your readme with an up-to-date tests coverage percentage.
If you already test your GitHub projects with a CI server like Travis, it’s very easy to add Coveralls to your workflow. Unfortunately, they have a library for Ruby, a couple user-provided libraries for other languages such as PHP, Java and Python, but nothing for Clojure. Fortunately, they provide an API for unsupported languages like Clojure. Here is how to use it.
Cloverage is a Clojure library and a Leiningen plugin to get form-level tests coverage of your projects. It’s dead easy to use and generates HTML reports for you as well as some other formats.
Add their plugin to your project.clj
:
Use lein cloverage
to run your tests and generate various reports. You can
specify which Cloverage version you want to use with CLOVERAGE_VERSION
. The
latest stable one is 1.0.3
, but we’ll need to use 1.0.4-SNAPSHOT
for this
article:
If you don’t already use it, signup on Coveralls using your GitHub account. Then, click on “Add Repo”, find your repo in the list and activate it. That’s all you have to do on their website for now.
You now need to custom your builds on Travis to use Coveralls. We’ll do it in a
Bash script, so add this to your .travis.yml
:
This tells Travis to run test/coveralls.sh
after each build. We use Bash
options -e
and -x
to respectively stop the script at the first failure and
print each command.
Now edit test/coveralls.sh
and add the following content:
It’ll run Cloverage and output reports in cov
(use another name if you
prefer), using a report for Coveralls. This generates a coveralls.json
file
which can then be sent to their API. That’s what we do on the next line, using
cURL.
That’s all! You can now push on GitHub, Travis will run your tests and send their tests coverage to Coveralls.
Coveralls doesn’t support partial-line tests coverage, so we’re cheating a little bit here using hits count. A line that it never covered has 0 hits, one that is partially covered has one hit, and a fully covered one has two hits. There’s an open issue regarding this.
Coveralls is a service that keep track of your tests coverage for you. It can notifies you when your coverage decreases under a custom threshold, and their bot comments on pull requests to report their tests coverage. Like Travis-CI, it allows you to add a badge to your readme with an up-to-date tests coverage percentage.
If you already test your GitHub projects with a CI server like Travis, it’s very easy to add Coveralls to your workflow. Unfortunately, they have a library for Ruby, a couple user-provided libraries for other languages such as PHP, Java and Python, but nothing for Clojure. Fortunately, they provide an API for unsupported languages like Clojure. Here is how to use it.
Cloverage is a Clojure library and a Leiningen plugin to get form-level tests coverage of your projects. It’s dead easy to use and generates HTML reports for you as well as some other formats.
Add their plugin to your project.clj
:
1 2 3 |
|
Use lein cloverage
to run your tests and generate various reports. You can
specify which Cloverage version you want to use with CLOVERAGE_VERSION
. The
latest stable one is 1.0.3
, but we’ll need to use 1.0.4-SNAPSHOT
for this
article:
1
|
|
If you don’t already use it, signup on Coveralls using your GitHub account. Then, click on “Add Repo”, find your repo in the list and activate it. That’s all you have to do on their website for now.
You now need to custom your builds on Travis to use Coveralls. We’ll do it in a
Bash script, so add this to your .travis.yml
:
1 2 |
|
This tells Travis to run test/coveralls.sh
after each build. We use Bash
options -e
and -x
to respectively stop the script at the first failure and
print each command.
Now edit test/coveralls.sh
and add the following content:
1 2 3 |
|
It’ll run Cloverage and output reports in cov
(use another name if you
prefer), using a report for Coveralls. This generates a coveralls.json
file
which can then be sent to their API. That’s what we do on the next line, using
cURL.
That’s all! You can now push on GitHub, Travis will run your tests and send their tests coverage to Coveralls.
Coveralls doesn’t support partial-line tests coverage, so we’re cheating a little bit here using hits count. A line that it never covered has 0 hits, one that is partially covered has one hit, and a fully covered one has two hits. There’s an open issue regarding this.
Black-box testing is a method of software testing that examines the
functionality of an application (e.g. what the software does) without peering
into its internal structures or workings
(Wikipedia). While it’s
usually done at a system level, I think the most obvious place it should be
used is at the function level. It’s even more efficient if you write tests for
someone else’s code.
Here is how I write unit tests for functions.
If you can’t tell what the function roughly does only by looking at its name, it’s usually because it’s badly named or it’s doing too many things, maybe you should break it in smaller functions. Look at its arguments names. Do they make sense to you? Do you understand what’s the function doing only by looking as its name and its arguments?
You sometimes need to read a function’s documentation if its name is unclear or if it’s a complicated code area. This is your latest chance to understand the function, because in function-level black-box testing you can’t read the code, it would not be black-box anymore. If you understand the function, you can now write tests. If not, you should improve its name, its arguments names, and its documentation, in that order because it’s the order one reads it.
Good unit tests are short, incremental, and test only a specific case. Start
with the simplest. Does the function takes a string? Give it an empty one. Does
it take a number? Give it 0
. Use null
for objects. Then move to more edge
cases: negative numbers, un-trimmed strings, uninitialized objects. Add more
simple edge cases, but only once at a time. Then, combine those edge cases.
Write more complex cases. Never combine two cases you never individually tested
before. After all these edge cases, test the legitimate ones. Again start with
the simplest, then add more complexity and combine cases.
Black-box testing is a method of software testing that examines the
functionality of an application (e.g. what the software does) without peering
into its internal structures or workings
(Wikipedia). While it’s
usually done at a system level, I think the most obvious place it should be
used is at the function level. It’s even more efficient if you write tests for
someone else’s code.
Here is how I write unit tests for functions.
If you can’t tell what the function roughly does only by looking at its name, it’s usually because it’s badly named or it’s doing too many things, maybe you should break it in smaller functions. Look at its arguments names. Do they make sense to you? Do you understand what’s the function doing only by looking as its name and its arguments?
You sometimes need to read a function’s documentation if its name is unclear or if it’s a complicated code area. This is your latest chance to understand the function, because in function-level black-box testing you can’t read the code, it would not be black-box anymore. If you understand the function, you can now write tests. If not, you should improve its name, its arguments names, and its documentation, in that order because it’s the order one reads it.
Good unit tests are short, incremental, and test only a specific case. Start
with the simplest. Does the function takes a string? Give it an empty one. Does
it take a number? Give it 0
. Use null
for objects. Then move to more edge
cases: negative numbers, un-trimmed strings, uninitialized objects. Add more
simple edge cases, but only once at a time. Then, combine those edge cases.
Write more complex cases. Never combine two cases you never individually tested
before. After all these edge cases, test the legitimate ones. Again start with
the simplest, then add more complexity and combine cases.
Black-box testing is a method of software testing that examines the
functionality of an application (e.g. what the software does) without peering
into its internal structures or workings
(Wikipedia). While it’s
usually done at a system level, I think the most obvious place it should be
used is at the function level. It’s even more efficient if you write tests for
someone else’s code.
Here is how I write unit tests for functions.
If you can’t tell what the function roughly does only by looking at its name, it’s usually because it’s badly named or it’s doing too many things, maybe you should break it in smaller functions. Look at its arguments names. Do they make sense to you? Do you understand what’s the function doing only by looking as its name and its arguments?
You sometimes need to read a function’s documentation if its name is unclear or if it’s a complicated code area. This is your latest chance to understand the function, because in function-level black-box testing you can’t read the code, it would not be black-box anymore. If you understand the function, you can now write tests. If not, you should improve its name, its arguments names, and its documentation, in that order because it’s the order one reads it.
Good unit tests are short, incremental, and test only a specific case. Start
with the simplest. Does the function takes a string? Give it an empty one. Does
it take a number? Give it 0
. Use null
for objects. Then move to more edge
cases: negative numbers, un-trimmed strings, uninitialized objects. Add more
simple edge cases, but only once at a time. Then, combine those edge cases.
Write more complex cases. Never combine two cases you never individually tested
before. After all these edge cases, test the legitimate ones. Again start with
the simplest, then add more complexity and combine cases.