X-Git-Url: http://info.iut-bm.univ-fcomte.fr/pub/gitweb/simgrid.git/blobdiff_plain/c46aba70aa4ccf22cc19fac31d451c7158e922dd..3a35f27158e3b8ab786d435ae3d59a90dca364ed:/doc/doxygen/inside_tests.doc diff --git a/doc/doxygen/inside_tests.doc b/doc/doxygen/inside_tests.doc index 4502ac0828..7845eb2a75 100644 --- a/doc/doxygen/inside_tests.doc +++ b/doc/doxygen/inside_tests.doc @@ -49,44 +49,35 @@ make testall # Rebuild the test runner on need ./testall # Launch all tests ./testall --help # revise how it goes if you forgot ./testall --tests=-all # run no test at all (yeah, that's useless) -./testall --dump-only # Display all existing test suite -./testall --tests=-all,+dict # Only launch the tests from the dict testsuite +./testall --dump-only # Display all existing test suites +./testall --tests=-all,+dict # Only launch the tests from the dict test suite ./testall --tests=-all,+foo:bar # run only the bar test from the foo suite. \endverbatim \section inside_tests_add_units Adding unit tests -If you want to test a specific function or set of functions, you need -a unit test. Edit -/tools/cmake/UnitTesting.cmake to add your -source file to the TEST_CFILES list, and add the corresponding unit -file to the TEST_UNITS list. For example, if your file is toto.c, -your unit file will be toto_unit.c. The full path to your file must be -provided, but the unit file will always be in src/ directly. +\warning this section is outdated. New unit tests should be written +using the unit_test_framework component of Boost. There is no such +example so far in our codebase, but that's definitely the way to go +for the future. STOP USING XBT. -If you want to create unit tests in the file src/xbt/toto.c, your -changes should look similar to: +If you want to test a specific function or set of functions, you need +a unit test. Edit the file tools/cmake/UnitTesting.cmake to +add your source file to the FILES_CONTAINING_UNITTESTS list. For +example, if you want to create unit tests in the file src/xbt/plouf.c, +your changes should look like that: \verbatim --- a/tools/cmake/UnitTesting.cmake +++ b/tools/cmake/UnitTesting.cmake -@@ -11,6 +11,7 @@ set(TEST_CFILES - src/xbt/xbt_strbuff.c +@@ -11,6 +11,7 @@ set(FILES_CONTAINING_UNITTESTS src/xbt/xbt_sha.c src/xbt/config.c -+ src/xbt/toto.c - ) - set(TEST_UNITS - ${CMAKE_CURRENT_BINARY_DIR}/src/cunit_unit.c -@@ -22,6 +23,7 @@ set(TEST_UNITS - ${CMAKE_CURRENT_BINARY_DIR}/src/xbt_strbuff_unit.c - ${CMAKE_CURRENT_BINARY_DIR}/src/xbt_sha_unit.c - ${CMAKE_CURRENT_BINARY_DIR}/src/config_unit.c -+ ${CMAKE_CURRENT_BINARY_DIR}/src/toto_unit.c - - ${CMAKE_CURRENT_BINARY_DIR}/src/simgrid_units_main.c ++ src/xbt/plouf.c ) + + if(SIMGRID_HAVE_MC) \endverbatim Then, you want to actually add your tests in the source file. All the @@ -112,7 +103,8 @@ reporting any issue. Finally, #xbt_test_log can be used to report intermediate steps. The messages will be shown only if the corresponding test fails. -Here is a recaping example, inspired from the dynar implementation. +Here is a recaping example, inspired from src/xbt/dynar.h (see that +file for details). @code /* The rest of your module implementation */ @@ -201,23 +193,27 @@ To add a new integration test, you thus have 3 things to do: details.\n Tesh is sometimes annoying as you have to ensure that the expected output will always be exactly the same. In particular, your should - not output machine dependent informations such as absolute data path, nor memory adresses as - they would change on each run. Several steps can be used here, such - as the obfucation of the memory adresses unless the verbose logs - are displayed (using the #XBT_LOG_ISENABLED() macro), or the - modification of the log formats to hide the timings when they - depend on the host machine.\n - The script located in /tools/cmake/tesh/generate_tesh.sh can + not output machine dependent informations such as absolute data + path, nor memory adresses as they would change on each run. Several + steps can be used here, such as the obfucation of the memory + adresses unless the verbose logs are displayed (using the + #XBT_LOG_ISENABLED() macro), or the modification of the log formats + to hide the timings when they depend on the host machine.\n + The script located in /tools/tesh/generate_tesh can help you a lot in particular if the output is large (though a smaller output is preferable). - There are also example tesh files in the /tools/cmake/tesh/ directory, that can be useful to understand the tesh syntax. + There are also example tesh files in the /tools/tesh/ directory, that can be useful to understand the tesh syntax. - Add your test in the cmake infrastructure. For that, modify - the file /tools/cmake/Tests.cmake. Make sure to - pick a wise name for your test. It is often useful to check a - category of tests together. The only way to do so in ctest is to - use the -R argument that specifies a regular expression that the - test names must match. For example, you can run all MSG test with - "ctest -R msg". That explains the importance of the test names. + the following file: + @verbatim + /teshsuite//CMakeLists.txt + @endverbatim + Make sure to pick a wise name for your test. It is often useful to + check a category of tests together. The only way to do so in ctest + is to use the -R argument that specifies a regular expression that + the test names must match. For example, you can run all MSG test + with "ctest -R msg". That explains the importance of the test + names. Once the name is chosen, create a new test by adding a line similar to the following (assuming that you use tesh as expected). @@ -246,7 +242,7 @@ We use Jenkins on Inria servers as a workhorse: it runs all of our tests for many configurations. It takes a long time to answer, and it often reports issues but when it's green, then you know that SimGrid is very fit! -We use Travis to +We use Travis to quickly run some tests on Linux and Mac. It answers quickly but may miss issues. And we use AppVeyor to build and somehow test SimGrid on windows. @@ -260,16 +256,16 @@ refer to the CI documentation. The result can be seen here: https://ci.inria.fr/simgrid/ -We have 3 projects on Jenkins: +We have 2 interesting projects on Jenkins: \li SimGrid-Multi is the main project, running the tests that we spoke about.\n It is configured (on Jenkins) to run the script tools/jenkins/build.sh \li SimGrid-DynamicAnalysis - runs the tests both under valgrind to find the memory errors and - under gcovr to report the achieved test coverage.\n It is configured + should be called "nightly" because it does not only run dynamic + tests, but a whole bunch of long lasting tests: valgrind (memory + errors), gcovr (coverage), Sanitizers (bad pointer usage, threading + errors, use of unspecified C constructs) and the clang static analyzer.\n It is configured (on Jenkins) to run the script tools/jenkins/DynamicAnalysis.sh -\li SimGrid-Windows - is an ongoing attempt to get Windows tested on Jenkins too. In each case, SimGrid gets built in /builds/workspace/$PROJECT/build_mode/$CONFIG/label/$SERVER/build @@ -282,10 +278,35 @@ model-checking on non-linux systems), go to your Project and click on interface language is English) and tick the checkbox; then add a groovy-expression to disable a specific configuration. For example, in order to disable the "ModelChecker" build on host -"small-freebsd-64-clang", use: +"small-netbsd-64-clang", use: \verbatim -(label=="small-freebsd-64-clang").implies(build_mode!="ModelChecker") +(label=="small-netbsd-64-clang").implies(build_mode!="ModelChecker") +\endverbatim + +Just for the record, the slaves were created from the available +template with the following commands: +\verbatim +#debian/ubuntu +apt-get install gcc g++ gfortran automake cmake libboost-dev openjdk-8-jdk openjdk-8-jre libxslt-dev libxml2-dev libevent-dev libunwind-dev libdw-dev htop git python3 xsltproc libboost-context-dev +#for dynamicanalysis: +apt-get install jacoco libjacoco-java libns3-dev pcregrep gcovr ant lua5.3-dev sloccount + +#fedora +dnf install libboost-devel openjdk-8-jdk openjdk-8-jre libxslt-devel libxml2-devel xsltproc git python3 libdw-devel libevent-devel libunwind-devel htop lua5.3-devel + +#netbsd +pkg_add cmake gcc7 boost boost-headers automake openjdk8 libxslt libxml2 libunwind git htop python36 + +#opensuse +zypper install cmake automake clang boost-devel java-1_8_0-openjdk-devel libxslt-devel libxml2-devel xsltproc git python3 libdw-devel libevent-devel libunwind-devel htop binutils ggc7-fortran + +#freebsd +pkg install boost-libs cmake openjdk8 automake libxslt libxml2 libunwind git htop python3 automake gcc6 flang elfutils libevent +#+ clang-devel from ports + +#osx +brew install cmake boost libunwind-headers libxslt git python3 \endverbatim \subsection inside_tests_travis Travis @@ -294,7 +315,11 @@ Travis is a free (as in free beer) Continuous Integration system that open-sourced project can use freely. It is very well integrated in the GitHub ecosystem. There is a plenty of documentation out there. Our configuration is in the file .travis.yml as it should be, and the -result is here: https://travis-ci.org/mquinson/simgrid +result is here: https://travis-ci.org/simgrid/simgrid + +The .travis.yml configuration file can be useful if you fail to get +SimGrid to compile on modern mac systems. We use the \c brew package +manager there, and it works like a charm. \subsection inside_tests_appveyor AppVeyor @@ -303,15 +328,12 @@ mature than Travis, or maybe it is just that I'm less trained in Windows. Our configuration is in the file appveyor.yml as it should be, and the result is here: https://ci.appveyor.com/project/mquinson/simgrid -It should be noted that I miserably failed to use the environment -provided by AppVeyor, since SimGrid does not build with Microsoft -Visual Studio. Instead, we download a whole development environment -from the internet at each build. That's an archive of already compiled -binaries that are unpacked on the appveyor systems each time we start. -We re-use the ones from the -symengine -project. Thanks to them for compiling sane tools and constituting that -archive, it saved my mind! +We use \c Choco as a package manager on AppVeyor, and it is sufficient +for us. In the future, we will probably move to the ubuntu subsystem +of Windows 10: SimGrid performs very well under these settings, as +tested on Inria's CI servers. For the time being having a native +library is still useful for the Java users that don't want to install +anything beyond Java on their windows. \subsection inside_tests_debian Debian builders @@ -325,4 +347,16 @@ only the most important breakages. The build results are here: https://buildd.debian.org/status/package.php?p=simgrid +\subsection inside_tests_sonarqube SonarQube + +SonarQube is an open-source code quality analysis solution. Their nice +code scanners are provided as plugin. The one for C++ is not free, but +open-source project can use it at no cost. That is what we are doing. + +Don't miss the great looking dashboard here: +https://sonarcloud.io/dashboard?id=simgrid + +This tool is enriched by the script @c tools/internal/travis-sonarqube.sh +that is run from @c .travis.yml + */