
The following is provided as a reference for those interested in understanding
how the test suite works, and how to add and maintain tests.

Overview of how tests are run
=============================

Running tests starts in "GNUmakefile" which has a "make test" target.

This causes all the functional tests in "tests/" to be run against repositories
in the old, hashed, and darcs2 formats. Additionally, scripts in "bugs/" are
also run on all three times of repositories, with the expectation that they
currently fail. 

Running fewer tests
-----------------------------

Because "make test" can take a long time to run, it's useful to run fewer tests
at once.  To help with that, the following specific make targets are also
available through GNUmakefile:

  make test_shell
  make test_shell_format2
  make test_shell_hashed
  make test_perl
  make test_perl_format2
  make test_perl_hashed
  make bugs_shell
  make bugs_shell_format2
  make bugs_shell_hashed
  make bugs_perl
  make bugs_perl_format2
  make bugs_perl_hashed


Running a single test
---------------------

  ./bin/prove script.pl
  perl shell_harness script.sh


Overview of types of tests
==========================

Darcs has tests in three formats: Functional tests that test the darcs binary
are written in Shell and Perl. Unit tests are  written in Haskell and
directly test Haskell functions. These can be run via "make unit", but are
not otherwise documented here. 

Shell tests
---------------------------

Shell tests and are useful because they are easy to create from a copy/paste of
actual shell commands executed. They are considered successful if no bad exit
codes are returned from the commands in the script. 

Perl scripts
---------------------------

The Perl scripts are useful because of the featureful testing framework that
Perl provides, and the potential they offer to create more portable tests. 

Right now both kinds of tests are equally accepted by the project. 


Tips for writing tests
=======================

- Avoid including a repo format type to "darcs init"
  This insures that all three repo formats will be tested.
  However, if you know that the test only passes under some
  repo formats, *do* explicity include a format option to "darcs init". 


Tips for writing Shell tests
-------------------------------

- Consider adding this near the top of the script:

   set -ev 

 The "v" causes the contents of the script to be printed as part of the run,
 which is helpful for debugging.  The "e" causes the script to exit as soon as
 their is an error. 

- Set an email address in the environment:
   export EMAIL=tester

  This is easier than adding "-A" to every record command.  Eventually this
  service should provided in the testing infrastructure, like it is for the
  Perl scripts.


Tips for writing Perl tests
-------------------------------

- Be aware of the testing modules that are available. See the docs for each of
  these, in order of importance. They are in the tests/lib/perl directory.
  (Use perldoc file.pm to see the docs). 

   Test::More
   Test::Darcs
   Shell::Command
   File::Slurp

- Avoid direct system calls. It will be tempting to use backticks to drop in direct shell
  commands. By writing as much of the tests as possible in pure Perl, they will remain
  more portable. The modules above cover the common cases. 

- ./bin/prove has many options for running the Perl tests. 


The future of testing darcs
=================================

We should seek to for our shell and Perl scripts to generate consistent
output, which will provide some benefits. 

By switching the shell scripts to emit success and failure in the TAP* format,
like Perl does, we can remove the duplicate code we have in the Perl and Shell
harnesses, and generally simplify the infrastructure code. 

The "prove" script already supports running non-Perl test scripts through the
"--exec" option. Once the shell scripts emit TAP output, we can use "prove" to
run some of them in parallel, like we can for the Perl test scripts. 

The "git" project already has "t/test-lib.sh", which would provide the bulk of
the code needed for this, although the output is not quite TAP format. . Their
"t/README" provides a nice overview of their Shell testing framework. 

* TAP == Test Anything Protocol: http://testanything.org
