From 3dea830fcf83870034f5542b0007d2799ad9e697 Mon Sep 17 00:00:00 2001 From: Christian Heinrich Date: Tue, 19 May 2015 16:06:07 +0200 Subject: [PATCH 1/1] Removed PIPOL documentation and added CI (Jenkins) - PIPOL seems to have been an old system used for that purpose; all references were removed or updated. - Added a file called "inside_ci.doc". There are sometimes tricky things that one wants to do with Jenkins and it's good for us to have a starting point here. --- buildtools/Cmake/DefinePackages.cmake | 2 +- doc/Doxyfile.in | 2 +- doc/doxygen/inside_autotests.doc | 287 -------------------------- doc/doxygen/inside_ci.doc | 50 +++++ doc/doxygen/inside_cmake.doc | 9 +- doc/doxygen/internals.doc | 2 +- 6 files changed, 57 insertions(+), 295 deletions(-) delete mode 100644 doc/doxygen/inside_autotests.doc create mode 100644 doc/doxygen/inside_ci.doc diff --git a/buildtools/Cmake/DefinePackages.cmake b/buildtools/Cmake/DefinePackages.cmake index abde10327c..dd757286ed 100644 --- a/buildtools/Cmake/DefinePackages.cmake +++ b/buildtools/Cmake/DefinePackages.cmake @@ -869,7 +869,7 @@ set(DOC_SOURCES doc/doxygen/header.html doc/doxygen/help.doc doc/doxygen/index.doc - doc/doxygen/inside_autotests.doc + doc/doxygen/inside_ci.doc doc/doxygen/inside_cmake.doc doc/doxygen/inside_doxygen.doc doc/doxygen/inside_extending.doc diff --git a/doc/Doxyfile.in b/doc/Doxyfile.in index c3d435b5ba..f193522e55 100644 --- a/doc/Doxyfile.in +++ b/doc/Doxyfile.in @@ -658,7 +658,7 @@ INPUT = doxygen/index.doc \ doxygen/inside_doxygen.doc \ doxygen/inside_extending.doc \ doxygen/inside_cmake.doc \ - doxygen/inside_autotests.doc \ + doxygen/inside_ci.doc \ doxygen/inside_release.doc \ doxygen/contributing.doc \ doxygen/tracing.doc \ diff --git a/doc/doxygen/inside_autotests.doc b/doc/doxygen/inside_autotests.doc deleted file mode 100644 index 0d5a7c1205..0000000000 --- a/doc/doxygen/inside_autotests.doc +++ /dev/null @@ -1,287 +0,0 @@ -/*! -\page inside_autotests Automatic Testing Infrastructure - -\section xps_dev_guide_pipol How to run test in pipol? - -In order to use PIPOL you have to create your account here: http://pipol.inria.fr/ - -\subsection inside_autotests_dev_guide_pipol_general What is PIPOL? - -The PIPOL platform is a porting, testing and compilation cluster. It hosts several hardware and software -architectures to perform its work. An user can reserve a resource (a specific hardware, an operating system...) -to be used interactively or automatically (with a "Nightly Builds" system) for software testing. - -\subsection inside_autotests_dev_guide_pipol_frontend From PIPOL frontend - -You first need to be connected into pipol ssh. -\verbatim -user@caraja:~$ ssh pipol.inria.fr -Linux pipol-server 2.6.26-2-amd64 #1 SMP Thu Nov 5 02:23:12 UTC 2009 x86_64 -*************************************************************************** -* You are just logged-on the PIPOL platform server * -* You can find some documentation about porting works on the platform * -* on the PIPOL's web site: http://pipol.inria.fr/docs/ * -* To report bugs you can write to: pipol-sysadm@lists.gforge.inria.fr * -*************************************************************************** - - -user@pipol:~$ -\endverbatim -Then you can see pipol images availables for deployment - -\verbatim -user@pipol:~$ pipol-sub si - amd64_2010-linux-centos-5.dd.gz - amd64_2010-linux-debian-squeeze.dd.gz - amd64_2010-linux-debian-testing.dd.gz - amd64_2010-linux-fedora-core13.dd.gz - amd64_2010-linux-fedora-core14.dd.gz - amd64_2010-linux-fedora-core16.dd.gz - amd64_2010-linux-ubuntu-lucid.dd.gz - amd64_2010-linux-ubuntu-maverick.dd.gz - amd64_2010-linux-ubuntu-natty.dd.gz - amd64_kvm-linux-debian-lenny - amd64_kvm-linux-debian-testing - amd64_kvm-windows-7 - amd64-linux-centos-5.dd.gz - amd64-linux-debian-etch.dd.gz - amd64-linux-debian-lenny.dd.gz -.... - i386-linux-ubuntu-lucid.dd.gz - i386-linux-ubuntu-maverick.dd.gz - i386-linux-ubuntu-natty.dd.gz - i386-linux-ubuntu-precise.dd.gz - i386_mac-mac-osx-server-leopard.dd.gz - i386-unix-freebsd-7.dd.gz - i386-unix-opensolaris-10.dd.gz - i386-unix-opensolaris-11.dd.gz - i386-unix-solaris-10.dd.gz - ia64-linux-debian-lenny.dd - ia64-linux-debian-squeeze.dd - ia64-linux-fedora-core9.dd - ia64-linux-redhatEL-5.0.dd - x86_64_mac-mac-osx-server-snow-leopard.dd.gz - x86_mac-mac-osx-server-snow-leopard.dd.gz -\endverbatim - -You can also see available architectures on host name: -\verbatim -navarro@pipol:~$ pipol-sub sa -================================================================= - Availables architectures: -================================================================= - -pipol18 -:i386_2010:amd64_2010: -pipol19 -:i386_2010:amd64_2010: -pipol20 -:i386_2010:amd64_2010: -pipol1 -:i386:amd64: -pipol2 -:i386:amd64: -pipol3 -:i386:amd64: -pipol4 -:i386:amd64: -pipol5 -:i386:amd64: -pipol6 -:i386:amd64: -pipol7 -:i386:amd64: -pipol8 -:i386:amd64: -pipol14 -:i386_kvm:amd64_kvm: -pipol15 -:i386_kvm:amd64_kvm: -pipol16 -:i386_kvm:amd64_kvm: -pipol17 -:i386_kvm:amd64_kvm: -pipol11 -:i386_mac:x86_mac: -pipol10 -:ia64: -pipol9 -:ia64: -pipol12 -:x86_64_mac: -\endverbatim - -When you have choose your image and host (not necessary) you deploy with command line: - -pipol-sub esn \ \ \ -\verbatim -user@pipol:~$pipol-sub esn amd64_2010-linux-ubuntu-maverick.dd.gz pipol20 02:00 -user@pipol:~$ssh pipol20 -\endverbatim - -You can now make all your tests. - -\subsection inside_autotests_dev_guide_pipol_home From a computer - -You have to renseign to simgrid configuration your pipol login. -\verbatim -$ cmake -Dpipol_user=user . -\endverbatim - -Then you have two kind of command: -\li make \ -\verbatim -$ make amd64_2010-linux-ubuntu-maverick -\endverbatim -This command copy your local simgrid directory to pipol and execute a configure, make and ctest. - -\li make \_experimental -\verbatim -$ make amd64_2010-linux-ubuntu-maverick_experimental -\endverbatim -Same as previous but report into cdash - -You can also see all available images from pipol -\verbatim -$ make pipol_test_list_images -\endverbatim - -\section inside_autotests_dev_guide_pipol_nightly How to use nightly builds? - -All the pipol scripts are located in \"/buildtools/pipol/\". -If you modify them you have to update the pipol home directory by using this command: make sync-pipol. - -It will copy those files: -\verbatim -simgrid@caraja:~/workspace/simgrid/build$ make sync-pipol -Update pipol script for user: simgrid -rc.debian 100% 1338 1.3KB/s 00:00 -rc.fedora 100% 1433 1.4KB/s 00:00 -rc.mac 100% 1129 1.1KB/s 00:00 -rc.ubuntu 100% 1875 1.8KB/s 00:00 -rc.windows 100% 494 0.5KB/s 00:00 -Nightly_simgrid.sh 100% 4004 3.9KB/s 00:00 -Experimental_bindings.sh 100% 1794 1.8KB/s 00:00 -\endverbatim - -\section inside_autotests_dev_guide_cdash How to report tests in cdash? - -Reporting experiment in cdash is very easy because it is done by ctest. - -The easier way is to execute command line "ctest -D Experiemntal" in build directory. More option is available by ctest: -\verbatim - ctest -D Continuous - ctest -D Continuous(Start|Update|Configure|Build) - ctest -D Continuous(Test|Coverage|MemCheck|Submit) - ctest -D Experimental - ctest -D Experimental(Start|Update|Configure|Build) - ctest -D Experimental(Test|Coverage|MemCheck|Submit) - ctest -D Nightly - ctest -D Nightly(Start|Update|Configure|Build) - ctest -D Nightly(Test|Coverage|MemCheck|Submit) - ctest -D NightlyMemoryCheck -\endverbatim - -If you want to have a code coverage, please add option on simgrid. -\verbatim -$ cmake -Denable_coverage=ON . -$ ctest -D ExperimentalStart -$ ctest -D ExperimentalConfigure -$ ctest -D ExperimentalBuild -$ ctest -D ExperimentalTest -$ ctest -D ExperimentalCoverage -$ ctest -D ExperimentalSubmit -\endverbatim - -\section inside_autotests_perf Infrastructure of the SimGrid Performance Regresion tests - -The goal of these tests is to detect the impact on simulation -performance of the commited changes. It is not quite effective right -now but at the end, it should be similar to -Phoromatic. - -\subsection inside_autotests_dev_guide_g5k_campaign How to execute g5k campaign? - -Quick steps deployment for rennes: - -\li 1/ Create a G5K account - -\li 2/ SSH to a frontend (must be rennes, nancy or toulouse for git protocol) - -\li 3/ Install g5k-campaign -\verbatim -$ gem install g5k-campaign --source http://g5k-campaign.gforge.inria.fr/pkg -p http://proxy:3128 --no-ri --no-rdoc --user-install -\endverbatim - -\li 4/ Configure the API -\verbatim -$ mkdir ~/.restfully -$ echo 'base_uri: https://api.grid5000.fr/stable/grid5000' > ~/.restfully/api.grid5000.fr.yml -$ chmod 0600 ~/.restfully/api.grid5000.fr.yml -\endverbatim - -\li 5/ Git clone the SimGrid Scalability project -\verbatim -$ git clone git@github.com:mquinson/simgrid-scalability-XPs.git -\endverbatim -An alternative URL is -git://scm.gforge.inria.fr/simgrid/simgrid-scalability-XPs.git, but the -repository on github is believed to be more uptodate. You may want to -double-check by pulling both: -\verbatim -$ git remote add g5k git://scm.gforge.inria.fr/simgrid/simgrid-scalability-XPs.git -$ git pull g5k master -\endverbatim - -\li 6/ Copy the run script into your home -\verbatim -$ cp simgrid-scalability-XPs/script-sh/run-g5k-scalab.sh ~/ -\endverbatim - -\li 7/ Create the result log directory (must be ~/log/) -\verbatim -$ mkdir ~/log -\endverbatim - -\li 8/ Execute the g5k campaign on a revision "rev" -\verbatim -$ sh run-g5k-scalab.sh "rev" -\endverbatim - -You can also have more parameters - -\li 1/ -> 5/ Same as before - -\li 6/ Open simgrid-scalability-XPs - -\li 7/ Execute SGXP.pl to see parameters -\verbatim -$ perl SGXP.pl --help -\endverbatim - -\li 8/ Execute SGXP.pl with your parameters like -\verbatim -$ ./SGXP.pl --site=nancy --cluster=graphene --test=chord,goal --rev="09bbc8de,3ca7b9a13" -\endverbatim - -\subsection inside_autotests_dev_guide_g5k_log How to analyze logs? - -To analyze log from g5k-campaign you must install R tool. - -\li 0/ You can copy logs from g5k to your computer (recommanded) - -\li 1/ Open ~/log/ - -\li 2/ Execute the perl analyzer for goal log -\verbatim -$ ~/simgrid-scalability-XPs/libperl/analyzer.pl goal.log.* > goal.csv -\endverbatim - -\li 3/ Execute the R analyer for goal log -\verbatim -$ ~/simgrid-scalability-XPs/script-R/goal.R goal.csv output.goal.pdf -\endverbatim - -Example of generated pdf here. - -*/ diff --git a/doc/doxygen/inside_ci.doc b/doc/doxygen/inside_ci.doc new file mode 100644 index 0000000000..7c5bba77eb --- /dev/null +++ b/doc/doxygen/inside_ci.doc @@ -0,0 +1,50 @@ +/*! +\page inside_ci Continous Integration (with Jenkins) + +\section ci_jenkins Jenkins Interface + +\subsection inside_jenkins_basics Where can I find Jenkins? + +The SimGrid team currently uses Jenkins to automate testing. Our Jenkins +interface can be found here: https://ci.inria.fr/simgrid/ + +If you need an account, talk to Martin Quinson. + +\subsection inside_jenkins_add_host How can I add a new host? + +You have to login to the CI interface of INRIA: https://ci.inria.fr +There you can manage the project and add new nodes. + +\subsection inside_jenkins_reboot_host How can I restart / reboot a host? + +Exactly the same as in \ref inside_jenkins_add_host. The only exception +is that you have to click on "restart" of the host you want to restart. + + +\subsection inside_jenkins_config_matrix Disable a certain build in the configuration matrix + +Jenkins uses a configuration matrix, i.e., a matrix consisting of configurations +on the one axis and hosts on the other. Normally, every host will try to build +every configuration but this may not be desirable. + +In order to disable a single configuration for a specific host (but leave +other configurations for the same host enabled), go to your Project and click +on "Configuration". There, find the field "combination filter" (if your interface +language is English) and tick the checkbox; then add a groovy-expression to +disable a specific configuration. For example, in order to disable the "ModelChecker" +build on host "small-freebsd-64-clang", use: + +\verbatim +(label=="small-freebsd-64-clang").implies(build_mode!="ModelChecker") +\endverbatim + +\section ci_servers CI Servers + +\subsection ci_servers_build_dir Where is SimGrid built? + +SimGrid gets built in /builds/workspace/$PROJECT/build_mode/$CONFIG/label/$SERVER/build + +Here, $PROJECT could be for instance "SimGrid-Multi", $CONFIG "DEBUG" or "ModelChecker" +and $SERVER for instance "simgrid-fedora20-64-clang". + +*/ diff --git a/doc/doxygen/inside_cmake.doc b/doc/doxygen/inside_cmake.doc index 4d95e0c836..aae6e291ae 100644 --- a/doc/doxygen/inside_cmake.doc +++ b/doc/doxygen/inside_cmake.doc @@ -246,11 +246,10 @@ ADD_TEST(NAME my-test-name As usual, you must run "make distcheck" after modifying the cmake files, to ensure that you did not forget any files in the distributed archive. -\subsection inside_cmake_addtest_perf Performance testing in SimGrid +\subsection inside_cmake_ci Continous Integration -We are currently in the process of adding an infrastructure for -performance regression testing, but this is not related to cmake -anyhow. They are thus documented elsewhere, in Section \ref -inside_autotests_perf +We are using Continous Integration to help us provide a stable build +across as many platforms as possible. %As this is not related to cmake, +you have to head over to Section \ref inside_ci. */ diff --git a/doc/doxygen/internals.doc b/doc/doxygen/internals.doc index 08a2ee1375..c5f74fec19 100644 --- a/doc/doxygen/internals.doc +++ b/doc/doxygen/internals.doc @@ -17,7 +17,7 @@ several pages, as follows: - @subpage inside_extending - @subpage inside_cmake - @subpage inside_release - - @subpage inside_autotests + - @subpage inside_ci \htmlonly -- 2.20.1