.\" Manpage for tesh, the TEsting SHell.
.\" 
.\" To read it locally (when not installed): 
.\"   man ./tesh.1
.\"
.TH tesh 1 "10 Oct 2012" "1.0" "tesh man page"
.SH NAME
tesh \- testing shell
.SH SYNOPSIS
.B tesh
[\fIOPTION\fR]... [\fIFILE\fR]...
.SH DESCRIPTION
This is the TESH tool. It constitutes a testing shell, ie a sort of shell specialized to run tests. The list of actions to take is parsed from files files called testsuite.
.SH OPTIONS
  --cd some/directory : ask tesh to switch the working directory before
                        launching the tests
  --setenv var=value  : set a specific environment variable
  --cfg arg           : add parameter --cfg=arg to each command line
  --enable-coverage   : ignore output lines starting with "profiling:"
.SH TESH FILE SYNTAX
Here is the syntax of these files:

The kind of each line is given by the first char (the second char should be
blank and is ignored):

 `$' command to run in foreground
 `&' command to run in background
 `<' input to pass to the command
 `>' output expected from the command
 `!' metacommand, which can be one of:
     `timeout' <integer>|no
     `expect signal' <signal name>
     `expect return' <integer>
     `output' <ignore|display>
     `setenv <key>=<val>'
 `p' a string to print
 `P' a string to print at the CRITICAL level (ease logging grepping)

If the expected output do not match what the command spits, TESH will produce
an error showing the diff (see OUTPUT below).
.SH IO ORDERS
The < and > lines add IO to the command defined in the current block (blocks
are separated by blank lines). It is possible to place these lines either after
the command or before. The difference between the two following chunks is
mainly cosmetic in your testsuites, TESH don't care. (cf IO-orders.tesh)

 $ cat
 < TOTO
 > TOTO

 > TOTO
 $ cat
 < TOTO

Nevertheless, it is possible to have several commands in the same block, but
none of them can have any output. It may seem a bit restrictive, as one could
say that a command gets all the IO until the next command, but I'm afraid of
errors such as the following:

 $ cd toto
 > TOTO
 $ mkfile file

TOTO will be passed to the cd command, where the user clearly want to pass it
to the mkfile built-in command (see below).
.SH STREAM REDIRECTION
Stream redirections (">", "<" and "|" constructs in sh) are not
implemented yet in tesh. This is a bit restrictive, but well, patch
welcome...

The situation in which it is mainly problematic is to create a
temporary file. The solution is to use the "mkfile" built-in command,
as in the following example:
$ mkfile myFile
> some content
> to the file

This will create a file called myFile (first argument of the mkfile
command). Its content will be all the input provided to the command.
.SH RETURN CODE
TESH spits an appropriate error message when the child do not return 0 as
return code (cf. catch-return.tesh), and returns code+40 itself.

It is also possible to specify that a given command must return another
value. For this, use the "expect return" metacommand, which takes an integer as
argument. The change only apply to the next command (cf. set-return.tesh).
.SH SIGNALS
TESH detects when the child is killed by a signal (like on segfaults), and
spits an appropriate error message (cf. catch-signal.tesh).

It is also possible to specify that a given command must raise a given
signal. For this, use the "expect signal" metacommand. It takes the signal name
as argument. The change only apply to the next command (cf. set-signal.tesh).
.SH TIMEOUTS
By default, all commands are given 5 seconds to execute
(cf. catch-timeout.tesh). You can change this with the "timeout", which
takes an integer as argument. The change only apply to the next command
(cf. set-timeout.tesh). If you pass "no" as argument, the command
cannot timeout.
.SH OUTPUT
By default, the commands output is matched against the one expected,
and an error is raised on discrepancy. Metacommands to change this:
 "output ignore"  -> output completely discarded
 "output display" -> output displayed (but not verified)
 "output sort"    -> sorts the display before verifying it (see below)
.SH SORTING OUTPUT

SimGrid is designed to produce perfectly reproducible results, so its
output can usualy be compared without any preprocessing. This is not
true anymore when the user activates parallel execution: User
processes are run in parallel at each timestamp, and the output is not
reproducible anymore. Until you sort the lines.

If you ask for 
.B ! output sort
then tesh will sort the whole lines. But it really complicates the
analysis of the error detected: the logical order of the output is
defeated by the lexicographical sort. 

The solution is to ask for
.B ! output sort 19
instead to sort only on the prefix of the line. Indeed, we run our simulation
tests with the flag: 
  --log=root.fmt:[%10.6r]%e(%i:%P@%h)%e%m%n

Then, the previous command sorts  lines on the first 19 chars, that is
exactly the length of the prefix indicating the timestamp and the
process. That's exactly what we need:
 - Every timestamps remain separated, as it should; 
 - In each timestamp, the output order of processes become
   reproducible: that's the lexicographical order of their name;
 - For each process, the order of its execution is preserved: its
   messages within a given timestamp are not reordered.

That way, tesh can do its job (no false positive, no false negative)
despite the unpredictable order of executions of processes within a
timestamp, and reported errors remain easy to analyze (execution of a
given process preserved).

This is of course very SimGrid oriented, but could even be usable by
others, who knows?

.SH ENVIRONMENT
You can add some content to the tested processes environment with the
setenv metacommand. It works as expected. For example:
  "setenv PATH=/bin"
.SH BUGS
No known bugs.