+++ /dev/null
-/*!
-@page inside_tests Testing SimGrid
-
-This page will teach you how to run the tests, selecting the ones you
-want, and how to add new tests to the archive.
-
-@tableofcontents
-
-SimGrid code coverage is usually between 70% and 80%, which is much
-more than most projects out there. This is because we consider SimGrid
-to be a rather complex project, and we want to modify it with less fear.
-
-We have two sets of tests in SimGrid: Each of the 10,000+ unit tests
-check one specific case for one specific function, while the 500+
-integration tests run a given simulation specifically intended to
-exercise a larger amount of functions together. Every example provided
-in examples/ is used as an integration test, while some other torture
-tests and corner cases integration tests are located in teshsuite/.
-For each integration test, we ensure that the output exactly matches
-the defined expectations. Since SimGrid displays the timestamp of
-every logged line, this ensures that every change of the models'
-prediction will be noticed. All these tests should ensure that SimGrid
-is safe to use and to depend on.
-
-@section inside_tests_runintegration Running the tests
-
-Running the tests is done using the ctest binary that comes with
-cmake. These tests are run for every commit and the result is publicly
-<a href="https://ci.inria.fr/simgrid/">available</a>.
-
-@verbatim
-ctest # Launch all tests
-ctest -R mc- # Launch only the tests which name match the string "mc-"
-ctest -j4 # Launch all tests in parallel, at most 4 at the same time
-ctest --verbose # Display all details on what's going on
-ctest --output-on-failure # Only get verbose for the tests that fail
-
-ctest -R mc- -j5 --output-on-failure # You changed the model checker and want to check that you didn't break anything, huh?
- # That's fine, I do so all the time myself.
-@endverbatim
-
-@section inside_tests_rununit Running the unit tests
-
-All unit tests are packed into the unit-tests binary, that lives at the
-source root. These tests are run when you launch ctest, don't worry.
-
-@verbatim
-make unit-tests # Rebuild the test runner on need
-./unit-tests # Launch all tests
-./unit-tests --help # revise how it goes if you forgot
-@endverbatim
-
-
-@section inside_tests_add_units Adding unit tests
-
-Our unit tests are written using the Catch2 library, that is included
-in the source tree. Please check for examples, listed at the end of
-tools/cmake/Tests.cmake.
-
-It is important to keep your tests fast. We run them very very often,
-and you should strive to make them as fast as possible, to not bother
-the other developers. Do not hesitate to stress test your code, but
-make sure that it runs reasonably fast, or nobody will run "ctest"
-before committing code.
-
-@section inside_tests_add_integration Adding integration tests
-
-TESH (the TEsting SHell) is the test runner that we wrote for our
-integration tests. It is distributed with the SimGrid source file, and
-even comes with a man page. TESH ensures that the output produced by a
-command perfectly matches the expected output. This is very precious
-to ensure that no change modifies the timings computed by the models
-without notice.
-
-To add a new integration test, you thus have 3 things to do:
-
- - <b>Write the code exercising the feature you target</b>. You should
- strive to make this code clear, well documented and informative for
- the users. If you manage to do so, put this somewhere under
- examples/ and modify the cmake files as explained on this page:
- @ref inside_cmake_examples. If you feel like you should write a
- torture test that is not interesting to the users (because nobody
- would sanely write something similar in user code), then put it under
- teshsuite/ somewhere.
-
- - <b>Write the tesh file</b>, containing the command to run, the
- provided input (if any, but almost no SimGrid test provide such an
- input) and the expected output. Check the tesh man page for more
- details.@n
- Tesh is sometimes annoying as you have to ensure that the expected
- output will always be exactly the same. In particular, your should
- not output machine dependent information such as absolute data
- path, nor memory addresses as they would change on each run. Several
- steps can be used here, such as the obfucation of the memory
- addresses unless the verbose logs are displayed (using the
- #XBT_LOG_ISENABLED() macro), or the modification of the log formats
- to hide the timings when they depend on the host machine.@n
- The script located in <project/directory>/tools/tesh/generate_tesh can
- help you a lot in particular if the output is large (though a smaller output is preferable).
- There are also example tesh files in the <project/directory>/tools/tesh/ directory, that can be useful to understand the tesh syntax.
-
- - <b>Add your test in the cmake infrastructure</b>. For that, modify
- the following file:
- @verbatim
- <project/directory>/teshsuite/<interface eg s4u>/CMakeLists.txt
- @endverbatim
- Make sure to pick a wise name for your test. It is often useful to
- check a category of tests together. The only way to do so in ctest
- is to use the -R argument that specifies a regular expression that
- the test names must match. For example, you can run all MC test
- with "ctest -R mc-". That explains the importance of the test
- names.
-
-Once the name is chosen, create a new test by adding a line similar to
-the following (assuming that you use tesh as expected).
-
-@verbatim
-# Usage: ADD_TEST(test-name ${CMAKE_BINARY_DIR}/bin/tesh <options> <tesh-file>)
-# option --setenv bindir set the directory containing the binary
-# --setenv srcdir set the directory containing the source file
-# --cd set the working directory
-ADD_TEST(my-test-name ${CMAKE_BINARY_DIR}/bin/tesh
- --setenv bindir=${CMAKE_BINARY_DIR}/examples/my-test/
- --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/my-test/
- --cd ${CMAKE_HOME_DIRECTORY}/examples/my-test/
- ${CMAKE_HOME_DIRECTORY}/examples/deprecated/msg/io/io.tesh
-)
-@endverbatim
-
-As usual, you must run "make distcheck" after modifying the cmake files,
-to ensure that you did not forget any files in the distributed archive.
-
-@section inside_tests_ci Continuous Integration
-
-We use several systems to automatically test SimGrid with a large set
-of parameters, across as many platforms as possible.
-We use <a href="https://ci.inria.fr/simgrid/">Jenkins on Inria
-servers</a> as a workhorse: it runs all of our tests for many
-configurations. It takes a long time to answer, and it often reports
-issues but when it's green, then you know that SimGrid is very fit!
-
-@subsection inside_tests_jenkins Jenkins on the Inria CI servers
-
-You should not have to change the configuration of the Jenkins tool
-yourself, although you could have to change the slaves' configuration
-using the <a href="https://ci.inria.fr">CI interface of INRIA</a> --
-refer to the <a href="https://wiki.inria.fr/ciportal/">CI documentation</a>.
-
-The result can be seen here: https://ci.inria.fr/simgrid/
-
-We have 2 interesting projects on Jenkins:
-@li <a href="https://ci.inria.fr/simgrid/job/SimGrid/">SimGrid</a>
- is the main project, running the tests that we spoke about.@n It is
- configured (on Jenkins) to run the script <tt>tools/jenkins/build.sh</tt>
-@li <a href="https://ci.inria.fr/simgrid/job/SimGrid-DynamicAnalysis/">SimGrid-DynamicAnalysis</a>
- should be called "nightly" because it does not only run dynamic
- tests, but a whole bunch of long lasting tests: valgrind (memory
- errors), gcovr (coverage), Sanitizers (bad pointer usage, threading
- errors, use of unspecified C constructs) and the clang static analyzer.@n It is configured
- (on Jenkins) to run the script <tt>tools/jenkins/DynamicAnalysis.sh</tt>
-
-In each case, SimGrid gets built in
-/builds/workspace/$PROJECT/build_mode/$CONFIG/label/$SERVER/build
-with $PROJECT being for instance "SimGrid", $CONFIG "DEBUG" or
-"ModelChecker" and $SERVER for instance "simgrid-fedora20-64-clang".
-
-If some configurations are known to fail on some systems (such as
-model-checking on non-linux systems), go to your Project and click on
-"Configuration". There, find the field "combination filter" (if your
-interface language is English) and tick the checkbox; then add a
-groovy-expression to disable a specific configuration. For example, in
-order to disable the "ModelChecker" build on host
-"small-netbsd-64-clang", use:
-
-@verbatim
-(label=="small-netbsd-64-clang").implies(build_mode!="ModelChecker")
-@endverbatim
-
-Just for the record, the slaves were created from the available
-template with the following commands:
-@verbatim
-#debian/ubuntu
-apt-get install gcc g++ gfortran automake cmake libboost-dev libxslt-dev libxml2-dev libevent-dev libunwind-dev libdw-dev htop git python3 xsltproc libboost-context-dev
-#for dynamicanalysis:
-apt-get install libns3-dev pcregrep gcovr sloccount
-
-#fedora
-dnf install libboost-devel libxslt-devel libxml2-devel xsltproc git python3 libdw-devel libevent-devel libunwind-devel htop
-
-#netbsd
-pkg_add cmake gcc7 boost boost-headers automake libxslt libxml2 libunwind git htop python36
-
-#opensuse
-zypper install cmake automake clang boost-devel libxslt-devel libxml2-devel xsltproc git python3 libdw-devel libevent-devel libunwind-devel htop binutils ggc7-fortran
-
-#freebsd
-pkg install boost-libs cmake automake libxslt libxml2 libunwind git htop python3 automake gcc6 flang elfutils libevent
-#+ clang-devel from ports
-
-#osx
-brew install cmake boost libunwind-headers libxslt git python3
-@endverbatim
-
-@subsection inside_tests_debian Debian builders
-
-Since SimGrid is packaged in Debian, we benefit from their huge
-testing infrastructure. That's an interesting torture test for our
-code base. The downside is that it's only for the released versions of
-SimGrid. That is why the Debian build does not stop when the tests
-fail: post-releases fixes do not fit well in our workflow and we fix
-only the most important breakages.
-
-The build results are here:
-https://buildd.debian.org/status/package.php?p=simgrid
-
-@subsection inside_tests_sonarqube SonarQube
-
-SonarQube is an open-source code quality analysis solution. Their nice
-code scanners are provided as plugin. The one for C++ is not free, but
-open-source project can use it at no cost. That is what we are doing.
-
-Don't miss the great looking dashboard here:
-https://sonarcloud.io/dashboard?id=simgrid_simgrid
-
-*/
