doc/overview-tree.html
doc/resources/
doc/serialized-form.html
-doc/stylesheet.css
doc/all_bib.html
doc/logcategories.sh
doc/realtoc.sh
sthread:
- Implement pthread_join in MC mode.
+Documentation:
+ - New section in the user guide on the provided performance models.
+ - New section presenting some technical good practices for (potential) contributors.
+
Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests)
(FG: issues on Framagit; GH: issues on GitHub)
- FG#18: Java bindings should be redone or removed
include README.md
include doc/doxygen/FAQ.doc
include doc/doxygen/inside.doc
-include doc/doxygen/inside_cmake.doc
include doc/doxygen/inside_extending.doc
include doc/doxygen/inside_release.doc
-include doc/doxygen/inside_tests.doc
-include doc/doxygen/module-index.doc
-include doc/doxygen/module-surf.doc
include doc/doxygen/outcomes_vizu.doc
-include doc/doxygen/platform.doc
include doc/doxygen/uhood.doc
include doc/doxygen/uhood_switch.doc
include docs/Build.sh
include docs/source/Configuring_SimGrid.rst
include docs/source/Deploying_your_application.rst
include docs/source/Design_goals.rst
+include docs/source/Developers_Documentation.rst
include docs/source/Doxyfile
include docs/source/Experimental_setup.rst
include docs/source/Installing_SimGrid.rst
several pages, as follows:
- @ref uhood_tech_inside
- - @subpage inside_tests
- @subpage inside_doxygen
- @subpage inside_extending
- - @subpage inside_cmake
- @subpage inside_release
@section uhood_tech_inside Insiders Considerations
-@subsection uhood_tech_inside_config Extra configuration
-
-The default build configuration of SimGrid fits the user needs, but
-they are not adapted to the ones actually working on SimGrid. See @ref
-install_src_config for more information. Note that this is very
-different from runtime configuration.
-
-In particular, the build is configured by default to produce highly
-optimized binaries, at the price of high compilation time. The
-rationale is that users will compile SimGrid only once, and use it many
-times. This is exactly the contrary for the insiders, so you want to
-turn off \b enable_compile_optimizations.
-
-Symmetrically, \b enable_compile_warnings is off for the users because
-we don't want to bother them with compiler warnings (that abort the
-build in SimGrid), but any insider must turn this option on, or your
-code will be refused from the main repository.
-
-@verbatim
- cmake -Denable_compile_optimizations=OFF \
- -Denable_compile_warnings=ON .
-@endverbatim
-
-@subsection uhood_tech_inside_commit Interacting with git
-
-During the Gran Refactoring to SimGrid4, things are evolving rather
-quickly, and some changes impact a large amount of files. You should
-thus not have long-standing branches, because they will rot very
-quickly and you will suffer to merge them back. Instead, you should
-work as much as possible with incremental changes that do not break
-things, and get them directly in master.
-
-Your commit message should follow the git habits, explained in this
-<a href="http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html">blog
-post</a>, or in the
-<a href="https://github.com/atom/atom/blob/master/CONTRIBUTING.md#git-commit-messages">
-git styleguide of Atom</a>.
-
-
-@subsection uhood_tech_inside_codstand Automatically Enforcing our Coding Standards
-
-If you plan to commit code to the SimGrid project, you definitely need
-to install the relevant tool to ensure that your changes follow our
-coding standards:
-
-@verbatim
-# install clang-format
-sudo apt-get install clang-format-3.9 # debian
-
-# tell git to call the script on each commit
-ln -s $(realpath tools/git-hooks/clang-format.pre-commit) .git/hooks/pre-commit
-@endverbatim
-
-This will add an extra verification before integrating any commit that
-you could prepare. If your code does not respects our formatting code,
-git will say so, and provide a ready to use patch that you can apply
-to improve your commit. Just carefully read the error message you get
-to find the exact command with git-apply to fix your formatting.
-
-If you find that for a specific commit, the formatter does a very bad
-job, then add --no-verify to your git commit command line.
@subsection uhood_tech_tricks Random Tricks
Over the years, we accumulated a few tricks that make it easier to
work with SimGrid. Here is a somewhat unsorted list of such tricks.
-### Easy testing
-
-Launching all tests can be very time consuming, so you want to build
-and run the tests in parallel. Also, you want to save the build output
-to disk, for further reference. This is exactly what the
-BuildSimGrid.sh script does. It is upper-cased so that the shell
-completion works and allows one to run it in 4 key press: `./B<tab>`
-
-Note that if you build out of tree (as you should, see below), the
-script builds the build/default directory. I usually copy the file in
-each build/ subdir to test each of them separately.
-
### Easy out of tree builds
It is easy to break one build configuration or another. That's
install_cmake_outsrc) in addition to your main build tree.
To not mess with git, you want to put your build tree under the build/
directory, which is ignored by git. For example, I have the following
-directories: build/clang build/full
-(but YMMV).
+directories: build/clang build/full build/mc (but YMMV).
Then, the problem is that when you traverse these directories, you
cannot edit the sources (that are in the srcdir, while you're in
Note that the links sometimes broken by git or others. Relaunching
`make hardlinks` may help if you're getting incoherent build results.
-### Unsorted hints
-
-* If you want to debug memory allocation problems, here are a few hints:
- - disable compiler optimizations, to have better backtraces;
- - disable the mallocators, or it will be hard to match malloc's with free's;
- - disable model checking, unless your problem lies in the model
- checker part of SimGrid (MC brings its own malloc implementation,
- which valgrind does not really love).
- All this is configured with:
-
- cmake -Denable_model-checking=OFF
- -Denable_mallocators=OFF
- -Denable_compile_optimizations=OFF .
-
-* If you break the logs, you want to define XBT_LOG_MAYDAY at the
- beginning of log.h. It deactivates the whole logging mechanism,
- switching to printfs instead. SimGrid becomes incredibly verbose
- when doing so, but it you let you fixing things.
-
-
*/
+++ /dev/null
-/*!
-@page inside_cmake Adding source files or examples
-
-@tableofcontents
-
-SimGrid uses CMake which is a family of tools designed to build, test, and package software.
-
-@section inside_cmake_addsrc How to add source files?
-
-If you want to rename, add, or delete source file(s) in the SimGrid distribution, you have to edit the
-tools/cmake/DefinePackages.cmake configuration file. Files are organized in sections, then find
-the section you are interested in and modify it.
-
-Once you're done, test your changes with ``make distcheck``.
-
-@section inside_cmake_examples How to add an example?
-
-The first rule is that the content of examples/ must be interesting to the users. It is expected that the users will
-take one of these examples and start editing it to make it fit their needs. So, it should be self-contained,
-informative, and should use only the public APIs.
-
-To ensure that all examples actually work as expected, every example is also used as an integration test (see
-@ref inside_tests), but you should still strive to keep the code under examples/ as informative as possible for the
-users. In particular, torture test cases should be placed in teshsuite/, not examples/, so that the users don't stumble
-upon them by error.
-
-The examples/ directory is organized as follows:
- - examples/cpp/ for examples using the S4U API
- - examples/smpi/ or examples using the SMPI API
- - examples/platforms/ only contains platforms descriptions in the XML format (see @ref platform for details)
-
-In each of these directories, there is a CMakeLists.txt file that has
-to be edited to include the new examples.
-
-Once you're done, test your changes with ``make distcheck``.
-
-*/
@tableofcontents
-@section simgrid_dev_guide_simcall How to add a new simcall?
-
-First of all you might want to avoid defining a new simcall if possible:
-@ref simgrid_dev_guide_generic_simcall.
-
-A simcall is used to go from user mode to kernel mode. There is some
-sort of popping dance involved, as we want to isolate the user
-contextes from their environment (so that they can run in parallel and
-so that we can model-check them).
-
-In short, just add a line to src/simix/simcalls.in and run the
-src/simix/simcalls.py script. It will guide you about how to implement
-your simcall. Please keep reading this section (only) if you want to
-understand how it goes.
-
-
-The workflow of a simcall is the following:
-
-- `<ret> simcall_<name>(<args>)`
- - `simcall_BODY_<name>(<args>)`
- - Initializes the simcall (store the arguments in position)
- - If maestro, executes the simcall directly (and return)
- - If not, call `ActorImpl::yield` to give back the control to maestro
- - ========== KERNEL MODE ==========
- - `ActorImpl::simcall_handle` large switch (on simcall) doing for each:
- - `simcall_HANDLER_<name>(simcall, <args>)` (the manual code handling the simcall)
- - If the simcall is not marked as "blocking" in its definition,
- call `ActorImpl::simcall_answer()` that adds back the issuer
- process to the list of processes to run in the next scheduling round.
- It is thus the responsibility of the blocking simcalls to call
- `ActorImpl::simcall_answer()` themselves in their handler.
-
-Note that empty HANDLERs can be omitted. These functions usually do
-some parameter checking, or retrieve some information about the
-simcall issuer, but when there no need for such things, the handler
-can be omitted. In that case, we directly call the function
-`simcall_<name>(<args>)`.
-
-To simplify the simcall creation, a python script generates most of
-the code and give helpers for the remaining stuff. That script reads
-the simcall definitions from src/simix/simcalls.in, checks that both
-`simcall_<name>()` and `simcall_HANDLER()` are defined somewhere, and
-generates the following files:
-
-- popping_accessors.hpp:
- Helper functions to get and set simcall arguments and results
-- popping_bodies.cpp:
- The BODY function of each simcall
-- popping_enum.hpp:
- Definition of type `enum class Simcall` (one value per existing simcall)
-- popping_generated.cpp:
- Definitions of `simcall_names[]` (debug name of each simcall), and
- ActorImpl::simcall_handle() that deals with the simcall from within the kernel
-
-The simcall.in file list all the simcalls in sections. A line starting by "##"
-define a new section which will be replace by a "ifdef" in the generated code.
-
-@section simgrid_dev_guide_generic_simcall How to avoid adding a new simcall?
+@section simgrid_dev_guide_generic_simcall The modern SimCall interface
We now have some generic simcalls which can be used to interface with the
Maestro without creating new simcalls. You might want to use them instead of
<b>Note:</b> `kernel_sync(f)` could be implemented as `kernel_async(f).get()`.
-@section simgrid_dev_guide_tag What is How to add a new tag for xml files?
-
-You should not do something like that. Please work instead to make XML
-avoidable, ie to make the C++ interface nice and usable.
-
*/
+++ /dev/null
-/*!
-@page inside_tests Testing SimGrid
-
-This page will teach you how to run the tests, selecting the ones you
-want, and how to add new tests to the archive.
-
-@tableofcontents
-
-SimGrid code coverage is usually between 70% and 80%, which is much
-more than most projects out there. This is because we consider SimGrid
-to be a rather complex project, and we want to modify it with less fear.
-
-We have two sets of tests in SimGrid: Each of the 10,000+ unit tests
-check one specific case for one specific function, while the 500+
-integration tests run a given simulation specifically intended to
-exercise a larger amount of functions together. Every example provided
-in examples/ is used as an integration test, while some other torture
-tests and corner cases integration tests are located in teshsuite/.
-For each integration test, we ensure that the output exactly matches
-the defined expectations. Since SimGrid displays the timestamp of
-every logged line, this ensures that every change of the models'
-prediction will be noticed. All these tests should ensure that SimGrid
-is safe to use and to depend on.
-
-@section inside_tests_runintegration Running the tests
-
-Running the tests is done using the ctest binary that comes with
-cmake. These tests are run for every commit and the result is publicly
-<a href="https://ci.inria.fr/simgrid/">available</a>.
-
-@verbatim
-ctest # Launch all tests
-ctest -R mc- # Launch only the tests which name match the string "mc-"
-ctest -j4 # Launch all tests in parallel, at most 4 at the same time
-ctest --verbose # Display all details on what's going on
-ctest --output-on-failure # Only get verbose for the tests that fail
-
-ctest -R mc- -j5 --output-on-failure # You changed the model checker and want to check that you didn't break anything, huh?
- # That's fine, I do so all the time myself.
-@endverbatim
-
-@section inside_tests_rununit Running the unit tests
-
-All unit tests are packed into the unit-tests binary, that lives at the
-source root. These tests are run when you launch ctest, don't worry.
-
-@verbatim
-make unit-tests # Rebuild the test runner on need
-./unit-tests # Launch all tests
-./unit-tests --help # revise how it goes if you forgot
-@endverbatim
-
-
-@section inside_tests_add_units Adding unit tests
-
-Our unit tests are written using the Catch2 library, that is included
-in the source tree. Please check for examples, listed at the end of
-tools/cmake/Tests.cmake.
-
-It is important to keep your tests fast. We run them very very often,
-and you should strive to make them as fast as possible, to not bother
-the other developers. Do not hesitate to stress test your code, but
-make sure that it runs reasonably fast, or nobody will run "ctest"
-before committing code.
-
-@section inside_tests_add_integration Adding integration tests
-
-TESH (the TEsting SHell) is the test runner that we wrote for our
-integration tests. It is distributed with the SimGrid source file, and
-even comes with a man page. TESH ensures that the output produced by a
-command perfectly matches the expected output. This is very precious
-to ensure that no change modifies the timings computed by the models
-without notice.
-
-To add a new integration test, you thus have 3 things to do:
-
- - <b>Write the code exercising the feature you target</b>. You should
- strive to make this code clear, well documented and informative for
- the users. If you manage to do so, put this somewhere under
- examples/ and modify the cmake files as explained on this page:
- @ref inside_cmake_examples. If you feel like you should write a
- torture test that is not interesting to the users (because nobody
- would sanely write something similar in user code), then put it under
- teshsuite/ somewhere.
-
- - <b>Write the tesh file</b>, containing the command to run, the
- provided input (if any, but almost no SimGrid test provide such an
- input) and the expected output. Check the tesh man page for more
- details.@n
- Tesh is sometimes annoying as you have to ensure that the expected
- output will always be exactly the same. In particular, your should
- not output machine dependent information such as absolute data
- path, nor memory addresses as they would change on each run. Several
- steps can be used here, such as the obfucation of the memory
- addresses unless the verbose logs are displayed (using the
- #XBT_LOG_ISENABLED() macro), or the modification of the log formats
- to hide the timings when they depend on the host machine.@n
- The script located in <project/directory>/tools/tesh/generate_tesh can
- help you a lot in particular if the output is large (though a smaller output is preferable).
- There are also example tesh files in the <project/directory>/tools/tesh/ directory, that can be useful to understand the tesh syntax.
-
- - <b>Add your test in the cmake infrastructure</b>. For that, modify
- the following file:
- @verbatim
- <project/directory>/teshsuite/<interface eg s4u>/CMakeLists.txt
- @endverbatim
- Make sure to pick a wise name for your test. It is often useful to
- check a category of tests together. The only way to do so in ctest
- is to use the -R argument that specifies a regular expression that
- the test names must match. For example, you can run all MC test
- with "ctest -R mc-". That explains the importance of the test
- names.
-
-Once the name is chosen, create a new test by adding a line similar to
-the following (assuming that you use tesh as expected).
-
-@verbatim
-# Usage: ADD_TEST(test-name ${CMAKE_BINARY_DIR}/bin/tesh <options> <tesh-file>)
-# option --setenv bindir set the directory containing the binary
-# --setenv srcdir set the directory containing the source file
-# --cd set the working directory
-ADD_TEST(my-test-name ${CMAKE_BINARY_DIR}/bin/tesh
- --setenv bindir=${CMAKE_BINARY_DIR}/examples/my-test/
- --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/my-test/
- --cd ${CMAKE_HOME_DIRECTORY}/examples/my-test/
- ${CMAKE_HOME_DIRECTORY}/examples/deprecated/msg/io/io.tesh
-)
-@endverbatim
-
-As usual, you must run "make distcheck" after modifying the cmake files,
-to ensure that you did not forget any files in the distributed archive.
-
-@section inside_tests_ci Continuous Integration
-
-We use several systems to automatically test SimGrid with a large set
-of parameters, across as many platforms as possible.
-We use <a href="https://ci.inria.fr/simgrid/">Jenkins on Inria
-servers</a> as a workhorse: it runs all of our tests for many
-configurations. It takes a long time to answer, and it often reports
-issues but when it's green, then you know that SimGrid is very fit!
-
-@subsection inside_tests_jenkins Jenkins on the Inria CI servers
-
-You should not have to change the configuration of the Jenkins tool
-yourself, although you could have to change the slaves' configuration
-using the <a href="https://ci.inria.fr">CI interface of INRIA</a> --
-refer to the <a href="https://wiki.inria.fr/ciportal/">CI documentation</a>.
-
-The result can be seen here: https://ci.inria.fr/simgrid/
-
-We have 2 interesting projects on Jenkins:
-@li <a href="https://ci.inria.fr/simgrid/job/SimGrid/">SimGrid</a>
- is the main project, running the tests that we spoke about.@n It is
- configured (on Jenkins) to run the script <tt>tools/jenkins/build.sh</tt>
-@li <a href="https://ci.inria.fr/simgrid/job/SimGrid-DynamicAnalysis/">SimGrid-DynamicAnalysis</a>
- should be called "nightly" because it does not only run dynamic
- tests, but a whole bunch of long lasting tests: valgrind (memory
- errors), gcovr (coverage), Sanitizers (bad pointer usage, threading
- errors, use of unspecified C constructs) and the clang static analyzer.@n It is configured
- (on Jenkins) to run the script <tt>tools/jenkins/DynamicAnalysis.sh</tt>
-
-In each case, SimGrid gets built in
-/builds/workspace/$PROJECT/build_mode/$CONFIG/label/$SERVER/build
-with $PROJECT being for instance "SimGrid", $CONFIG "DEBUG" or
-"ModelChecker" and $SERVER for instance "simgrid-fedora20-64-clang".
-
-If some configurations are known to fail on some systems (such as
-model-checking on non-linux systems), go to your Project and click on
-"Configuration". There, find the field "combination filter" (if your
-interface language is English) and tick the checkbox; then add a
-groovy-expression to disable a specific configuration. For example, in
-order to disable the "ModelChecker" build on host
-"small-netbsd-64-clang", use:
-
-@verbatim
-(label=="small-netbsd-64-clang").implies(build_mode!="ModelChecker")
-@endverbatim
-
-Just for the record, the slaves were created from the available
-template with the following commands:
-@verbatim
-#debian/ubuntu
-apt-get install gcc g++ gfortran automake cmake libboost-dev libxslt-dev libxml2-dev libevent-dev libunwind-dev libdw-dev htop git python3 xsltproc libboost-context-dev
-#for dynamicanalysis:
-apt-get install libns3-dev pcregrep gcovr sloccount
-
-#fedora
-dnf install libboost-devel libxslt-devel libxml2-devel xsltproc git python3 libdw-devel libevent-devel libunwind-devel htop
-
-#netbsd
-pkg_add cmake gcc7 boost boost-headers automake libxslt libxml2 libunwind git htop python36
-
-#opensuse
-zypper install cmake automake clang boost-devel libxslt-devel libxml2-devel xsltproc git python3 libdw-devel libevent-devel libunwind-devel htop binutils ggc7-fortran
-
-#freebsd
-pkg install boost-libs cmake automake libxslt libxml2 libunwind git htop python3 automake gcc6 flang elfutils libevent
-#+ clang-devel from ports
-
-#osx
-brew install cmake boost libunwind-headers libxslt git python3
-@endverbatim
-
-@subsection inside_tests_debian Debian builders
-
-Since SimGrid is packaged in Debian, we benefit from their huge
-testing infrastructure. That's an interesting torture test for our
-code base. The downside is that it's only for the released versions of
-SimGrid. That is why the Debian build does not stop when the tests
-fail: post-releases fixes do not fit well in our workflow and we fix
-only the most important breakages.
-
-The build results are here:
-https://buildd.debian.org/status/package.php?p=simgrid
-
-@subsection inside_tests_sonarqube SonarQube
-
-SonarQube is an open-source code quality analysis solution. Their nice
-code scanners are provided as plugin. The one for C++ is not free, but
-open-source project can use it at no cost. That is what we are doing.
-
-Don't miss the great looking dashboard here:
-https://sonarcloud.io/dashboard?id=simgrid_simgrid
-
-*/
+++ /dev/null
-/**
-@defgroup XBT_API XBT: SimGrid core toolbox
-@brief The core toolbox of SimGrid, containing useful datatypes and friends
-*/
-
-/**
-@defgroup TRACE_API TRACING
-@brief Gather data about your simulation for later analysis
-
-SimGrid can trace the resource (of hosts and links) utilization using
-any of its programming interfaces (S4U, SimDAG and SMPI). This means
-that the tracing will register how much power is used for each host
-and how much bandwidth is used for each link of the platform.
-
-The idea of the tracing facilities is to give SimGrid users to
-possibility to classify S4U and SimDAG tasks by category, tracing the
-platform utilization (hosts and links) for each of the categories.
-The API enables the declaration of categories and a function to
-associate them to the tasks (S4U and SD). The tasks that are not
-classified according to a category are not traced. If no categories
-are specified, simulations can still be traced using a special
-parameter in the command line (see @ref outcomes_vizu for details).
-*/
-
-/** @defgroup ROUTING_API Routing: Determining the communication paths
- @brief Organize the platform to determine the links used by each communication
-
-@section routing_basics Basic Concepts
-
-The purpose of the simgrid::kernel::routing module is to retrieve the
-routing path between two points in a time- and space-efficient manner.
-Indeed, the network model needs both the list of links that the convey
-the created communication, and the summed latency that these links
-represent. This operation is clearly on the critical path of most
-SimGrid simulations.
-
-When defining how the information is routed in the simulated network,
-it is certainly very tempting to use a formalism somehow similar to
-how it is defined on real network. One would have to define the
-routing tables of each routers interconnections sub-networks, just
-like in the real life. Given the daunting amount of configuration
-required, we could complete the information given by the user with
-classical protocols such as BGP and RIP. Many network simulator take
-such configuration as an input, for good reasons.
-
-This is not the way it goes in SimGrid: the network routing is defined
-in a global and compact way instead. This eases the modeling of very
-large systems, and allows highly optimized datastructures and
-algorithms in the simulator. The proposed description mechanism is
-thus much more convenient and efficient. In addition, it is more
-expressive than the classical solution based on forwarding tables on
-each host and router.
-
-The price to pay is that this representation of networks is very
-specific to SimGrid, so you will have to read further to understand
-it, even if you already know how real networks work.
-
-The central notion here are @b Networking @b Zones. NetZones represent
-network areas in which the routing is done in an homogeneous way.
-Conceptually, netzones generalize from the ideas of local networks
-(such as Ethernet switched networks) and Autonomous System. The
-network as a whole is represented as a single hierarchy of netzones,
-meaning that every netzone is part of another netzone (but the @c
-NetRoot, which is the top-level netzone).
-
-The main goal of the routing module is to provide a list of links
-traversed by a given communication and/or a latency to apply. These
-information are then used by the network model to compute the time
-that this communication takes. This information is retrieved by three
-combined algorithms: intra-zone routing, inter-zone routing, and the
-bypass mechanism.
-
-The <b>intra-zone level</b> is naturally handled by the netzones. Each
-netzone have to specify the routing algorithm it uses for that.
-@ref simgrid::kernel::routing::FullZone "FullZone" netzones have complete matrix where matrix(a,b)
-represents the full path (the list of links) between the hosts a and
-b. @ref simgrid::kernel::routing::FloydZone "FloydZone" apply the Floyd-Warshall algorithm to compute the
-paths. @ref simgrid::kernel::routing::ClusterZone "ClusterZone" model classical switched or hub networks,
-where each component is connected through a private link onto a common
-backbone. Many other routing algorithms are provided to model the
-classical needs, but you can naturally define your own routing if the
-provided ones do not fit your needs.
-
-The <b>inter-zone algorithm</b> is used when the communication
-traverses more than one zone. The overall path goes from the source up
-in the netzones' tree, until the first common ancestor zone, and moves
-down to the destination. It crawls the differing netzones on its path
-according to the user-defined inter-zone routes, moving from gateway
-to gateway.
-
-You can also use the <b>bypass mechanism</b> to specify manually some
-shortcuts that directly provide the list of links interconnecting two
-given processes.
-
-@section routing_declaring Declaring a platform
-
-For now, you can only declare a platform from an XML file, but we are
-working to make it possible from the C++ code (or even from bindings
-in other languages). Until then, please head to @ref platform.
-
-*/
+++ /dev/null
-/** @defgroup SURF_build_api Create a new API
- @ingroup SURF_API
- @brief How to build a new API on top of SURF
-
- SURF provides the functionalities to simulate the platform. There are two main data types in SURF:
- the actions and the resources. Several types of resources exist:
- - the host resource,
- - the network resource,
- - the CPU resource,
- - the timer resource.
-
- The implementation of these resources depends on the platform
- models you choose. You can select your model by calling
- #surf_host_model_init_current_default() (which will give you a
- CLM03 model), or similar (see @ref SURF_models).
-
- To initialize SURF, call
- #surf_host_model_init_current_default() or #surf_host_model_init_ptask_L07()
- to create the platform.
-
- Then you can access the hosts with the @ref simgrid::s4u::Engine::get_all_hosts.
- Some functions of the @ref SURF_host_interface and similar can give
- you some information about:
- - a host: get_speed(), get_available_speed();
- - a network link: get_link_name(), get_link_latency(), get_link_bandwidth();
- - a route: get_route(), get_route_size().
-
- During the simulation, call @a surf_host_model->execute() to schedule a
- computation task on a host, or @a surf_host_model->communicate()
- to schedule a communication task between two hosts. You can also create parallel task
- with @a surf_host_model->extension_public->execute_parallel_task(). These functions return
- a new action that represents the task you have just created.
-
- To execute the actions created with @a execute(), @a communicate() or @a execute_parallel_task(), call
- EngineImpl::solve(). This function is where the simulation takes place. It returns the
- time elapsed to execute the actions. You can know what actions have changed their state thanks
- to the states sets. For example, if your want to know what actions are finished,
- extract them from @a surf_host_model->common_public->states.done_action_set.
- Depending on these results, you can schedule other tasks and call solve() again.
-
- Have a look at the implementation ofd @ref SD_API "Simdag" to see how these module
- interact with SURF. But if you want to create a new API on top of SURF,
- we strongly recommend you to contact us before anyway.
-
-*/
-
+/**
+@defgroup TRACE_API TRACING
+@brief Gather data about your simulation for later analysis
+
+SimGrid can trace the resource (of hosts and links) utilization using
+any of its programming interfaces (S4U, SimDAG and SMPI). This means
+that the tracing will register how much power is used for each host
+and how much bandwidth is used for each link of the platform.
+
+The idea of the tracing facilities is to give SimGrid users to
+possibility to classify S4U and SimDAG tasks by category, tracing the
+platform utilization (hosts and links) for each of the categories.
+The API enables the declaration of categories and a function to
+associate them to the tasks (S4U and SD). The tasks that are not
+classified according to a category are not traced. If no categories
+are specified, simulations can still be traced using a special
+parameter in the command line (see @ref outcomes_vizu for details).
+*/
+
/*! @page outcomes_vizu Visualization and Statistical Analysis
SimGrid comes with an extensive support to trace and register what
+++ /dev/null
-/*! @page platform Describing the virtual platform
-
-
-@section pf_res Resource description
-
-@subsection pf_res_computing Computing Resources
-
-@subsubsection pf_cabinet <cabinet>
-
-@note
- This tag is only available when the routing mode of the network zone
- is set to ``Cluster``.
-
-The ``<cabinet />`` tag is, like the @ref pf_tag_cluster "<cluster>" tag,
-a meta-tag. This means that it is simply a shortcut for creating a set of (homogeneous) hosts and links quickly;
-unsurprisingly, this tag was introduced to setup cabinets in data centers quickly. Unlike
-<cluster>, however, the <cabinet> assumes that you create the backbone
-and routers yourself; see our examples below.
-
-#### Attributes ####
-
-Attribute name | Mandatory | Values | Description
---------------- | --------- | ------ | -----------
-id | yes | string | The identifier of the cabinet. Facilitates referring to this cluster.
-prefix | yes | string | Each node of the cabinet has to have a name. This name will be prefixed with this prefix.
-suffix | yes | string | Each node of the cabinet will be suffixed with this suffix
-radical | yes | string | Regexp used to generate cabinet nodes name. Syntax: "10-20" will give you 11 machines numbered from 10 to 20, "10-20;2" will give you 12 machines, one with the number 2, others numbered as before. The produced number is concatenated between prefix and suffix to form machine names.
-speed | yes | int | Same as the ``speed`` attribute of the @ref pf_tag_host "<host>" tag.
-bw | yes | int | Bandwidth for the links between nodes and backbone (if any). See the @ref pf_tag_link "link section" for syntax/details.
-lat | yes | int | Latency for the links between nodes and backbone (if any). See the @ref pf_tag_link "link section" for syntax/details.
-
-@note
- Please note that as of now, it is impossible to change attributes such as,
- amount of cores (always set to 1), the sharing policy of the links (always set to @ref pf_sharing_policy_splitduplex "SPLITDUPLEX").
-
-#### Example ####
-
-The following example was taken from ``examples/platforms/meta_cluster.xml`` and
-shows how to use the cabinet tag.
-
-@verbatim
- <zone id="my_cluster1" routing="Cluster">
- <cabinet id="cabinet1" prefix="host-" suffix=".cluster1"
- speed="1Gf" bw="125MBps" lat="100us" radical="1-10"/>
- <cabinet id="cabinet2" prefix="host-" suffix=".cluster1"
- speed="1Gf" bw="125MBps" lat="100us" radical="11-20"/>
- <cabinet id="cabinet3" prefix="host-" suffix=".cluster1"
- speed="1Gf" bw="125MBps" lat="100us" radical="21-30"/>
-
- <backbone id="backbone1" bandwidth="2.25GBps" latency="500us"/>
- </zone>
-@endverbatim
-
-@note
- Please note that you must specify the @ref pf_backbone "<backbone>"
- tag by yourself; this is not done automatically and there are no checks
- that ensure this backbone was defined.
-
-The hosts generated in the above example are named host-1.cluster, host-2.cluster1
-etc.
-
-
-@subsection pf_ne Network equipment
-
-There are two tags at all times available to represent network entities and
-several other tags that are available only in certain contexts.
-1. ``<link>``:
-
-
-3. ``<backbone/>``: This tag is only available when the containing network zone is
- used as a cluster (i.e., mode="Cluster")
-
-@remark
- If you want to represent an entity like a switch, you must use ``<link>`` (see section). Routers are used
- to run some routing algorithm and determine routes (see Section @ref pf_routing for details).
-
-@subsubsection pf_backbone <backbone/>
-
-@note
- This tag is <b>only available</b> when the containing network zone uses the "Cluster" routing mode!
-
-Using this tag, you can designate an already existing link to be a backbone.
-
-Attribute name | Mandatory | Values | Description
---------------- | --------- | ------ | -----------
-id | yes | string | Name of the link that is supposed to act as a backbone.
-
-
-
-@section pf_routing Routing
-
-
-@subsubsection pf_routing_model_simple Simple/fast models
-
-| Name | Description |
-| ---------------------------------------- | ------------------------------------------------------------------------------ |
-| @ref pf_routing_model_cluster "Cluster" | This is specific to the @ref pf_tag_cluster "<cluster/>" tag and should not be used by the user, as several assumptions are made. |
-| @ref pf_routing_model_none "None" | No routing at all. Unless you know what you're doing, avoid using this mode in combination with a non-constant network model. |
-| @ref pf_routing_model_vivaldi "Vivaldi" | Perfect when you want to use coordinates. Also see the corresponding @ref pf_P2P_tags "P2P section" below. |
-
-@anchor pf_routing_model_cluster
-### Cluster ###
-
-@note
- In this mode, the @ref pf_cabinet "<cabinet/>" tag is available.
-
-#### Example platform files ####
-
-This is an automatically generated list of example files that use the Cluster
-routing model (the path is given relative to SimGrid's source directory):
-
-@verbinclude example_filelist_routing_cluster
-
-@anchor pf_routing_model_none
-
-### None ###
-
-This model does exactly what it's name advertises: Nothing. There is no routing
-available within this model and if you try to communicate within the zone that
-uses this model, SimGrid will fail unless you have explicitly activated the
-@ref options_model_select_network_constant "Constant Network Model" (this model charges
-the same for every single communication). It should
-be noted, however, that you can still attach an @ref pf_tag_zoneroute "ZoneRoute",
-as is demonstrated in the example below:
-
-@verbinclude platforms/cluster_and_one_host.xml
-
-#### Example platform files ####
-
-This is an automatically generated list of example files that use the None
-routing model (the path is given relative to SimGrid's source directory):
-
-@verbinclude example_filelist_routing_none
-
-
-
-
-@anchor pf_routing_model_vivaldi
-### Vivaldi ###
-
-For more information on how to use the [Vivaldi Coordinates](https://en.wikipedia.org/wiki/Vivaldi_coordinates),
-see also Section @ref pf_P2P_tags "P2P tags".
-
-Note that it is possible to combine the Vivaldi routing model with other routing models;
-an example can be found in the file @c examples/platforms/cloud.xml. This
-examples models a NetZone using Vivaldi that contains other NetZones that use different
-routing models.
-
-#### Example platform files ####
-
-This is an automatically generated list of example files that use the None
-routing model (the path is given relative to SimGrid's source directory):
-
-@verbinclude example_filelist_routing_vivaldi
-
-
-@subsection ps_dec Defining routes
-
-There are currently four different ways to define routes:
-
-| Name | Description |
-| ------------------------------------------------- | ----------------------------------------------------------------------------------- |
-| @ref pf_tag_route "route" | Used to define route between host/router |
-| @ref pf_tag_zoneroute "zoneRoute" | Used to define route between different zones |
-| @ref pf_tag_bypassroute "bypassRoute" | Used to supersede normal routes as calculated by the network model between host/router; e.g., can be used to use a route that is not the shortest path for any of the shortest-path routing models. |
-| @ref pf_tag_bypassasroute "bypassZoneRoute" | Used in the same way as bypassRoute, but for zones |
-
-Basically all those tags will contain an (ordered) list of references
-to link that compose the route you want to define.
-
-Consider the example below:
-
-@subsubsection pf_tag_bypassasroute bypasszoneroute
-
-As said before, once you choose
-a model, it (most likely; the constant network model, for example, doesn't) calculates routes for you. But maybe you want to
-define some of your routes, which will be specific. You may also want
-to bypass some routes defined in lower level zone at an upper stage:
-<b>bypasszoneroute</b> is the tag you're looking for. It allows one to
-bypass routes defined between already defined between zone (if you want
-to bypass route for a specific host, you should just use byPassRoute).
-The principle is the same as zoneroute: <b>bypasszoneroute</b> contains
-list of links that are in the path between src and dst.
-
-#### Attributes ####
-
-| Attribute name | Mandatory | Values | Description |
-| --------------- | --------- | ---------------------- | ----------- |
-| src | yes | String | The value given to the source zone's "id" attribute |
-| dst | yes | String | The value given to the destination zone's "id" attribute. |
-| gw_src | yes | String | The value given to the source gateway's "id" attribute; this can be any host or router within the src zone |
-| gw_dst | yes | String | The value given to the destination gateway's "id" attribute; this can be any host or router within the dst zone|
-| symmetrical | no | YES@| NO (Default: YES) | If this route is symmetric, the opposite route (from dst to src) will also be declared implicitly. |
-
-#### Example ####
-
-@verbatim
-<bypasszoneRoute src="my_cluster_1" dst="my_cluster_2"
- gw_src="my_cluster_1_router"
- gw_dst="my_cluster_2_router">
- <link_ctn id="link_tmp"/>
-</bypasszoneroute>
-@endverbatim
-
-This example shows that link @c link_tmp (definition not displayed here) directly
-connects the router @c my_cluster_1_router in the source cluster to the router
-@c my_cluster_2_router in the destination router. Additionally, as the @c symmetrical
-attribute was not given, this route is presumed to be symmetrical.
-
-@subsubsection pf_tag_bypassroute bypassRoute
-
-As said before, once you choose
-a model, it (most likely; the constant network model, for example, doesn't) calculates routes for you. But maybe you want to
-define some of your routes, which will be specific. You may also want
-to bypass some routes defined in lower level zone at an upper stage:
-<b>bypassRoute</b> is the tag you're looking for. It allows one to bypass
-routes defined between <b>host/router</b>. The principle is the same
-as route: <b>bypassRoute</b> contains list of links references of
-links that are in the path between src and dst.
-
-#### Attributes ####
-
-| Attribute name | Mandatory | Values | Description |
-| --------------- | --------- | ---------------------- | ----------- |
-| src | yes | String | The value given to the source zone's "id" attribute |
-| dst | yes | String | The value given to the destination zone's "id" attribute. |
-| symmetrical | no | YES @| NO (Default: YES) | If this route is symmetric, the opposite route (from dst to src) will also be declared implicitly. |
-
-#### Examples ####
-
-@verbatim
-<bypassRoute src="host_1" dst="host_2">
- <link_ctn id="link_tmp"/>
-</bypassRoute>
-@endverbatim
-
-This example shows that link @c link_tmp (definition not displayed here) directly
-connects host @c host_1 to host @c host_2. Additionally, as the @c symmetrical
-attribute was not given, this route is presumed to be symmetrical.
-
-@section pf_other Other tags
-
-The following tags can be used inside a @<platform@> tag even if they are not
-directly describing the platform:
-
- - @ref pf_tag_config passes configuration options, e.g. to change the network model;
- - @ref pf_tag_prop gives user-defined properties to various elements
-
-@subsection pf_trace trace and trace_connect
-
-Both tags are an alternate way to pass files containing information on
-availability, state etc. to an entity. (See also @ref howto_churn).
-Instead of referring to the file directly in the host, link, or
-cluster tag, you proceed by defining a trace with an id corresponding
-to a file, later a host/link/cluster, and finally using trace_connect
-you say that the file trace must be used by the entity.
-
-
-#### Example ####
-
-@verbatim
-<zone id="zone0" routing="Full">
- <host id="bob" speed="1000000000"/>
-</zone>
-<trace id="myTrace" file="bob.trace" periodicity="1.0"/>
-<trace_connect trace="myTrace" element="bob" kind="POWER"/>
-@endverbatim
-
-@note
- The order here is important. @c trace_connect must come
- after the elements @c trace and @c host, as both the host
- and the trace definition must be known when @c trace_connect
- is parsed; the order of @c trace and @c host is arbitrary.
-
-
-#### @c trace attributes ####
-
-
-| Attribute name | Mandatory | Values | Description |
-| --------------- | --------- | ---------------------- | ----------- |
-| id | yes | String | Identifier of this trace; this is the name you pass on to @c trace_connect. |
-| file | no | String | Filename of the file that contains the information - the path must follow the style of your OS. You can omit this, but then you must specify the values inside of <trace> and </trace> - see the example below. |
-| trace_periodicity | yes | String | This is the same as for @ref pf_tag_host "hosts" (see there for details) |
-
-Here is an example of trace when no file name is provided:
-
-@verbatim
- <trace id="myTrace" periodicity="1.0">
- 0.0 1.0
- 11.0 0.5
- 20.0 0.8
- </trace>
-@endverbatim
-
-#### @c trace_connect attributes ####
-
-| Attribute name | Mandatory | Values | Description |
-| --------------- | --------- | ---------------------- | ----------- |
-| kind | no | HOST_AVAIL@|POWER@|<br/>LINK_AVAIL@|BANDWIDTH@|LATENCY (Default: HOST_AVAIL) | Describes the kind of trace. |
-| trace | yes | String | Identifier of the referenced trace (specified of the trace's @c id attribute) |
-| element | yes | String | The identifier of the referenced entity as given by its @c id attribute |
-
-@section pf_hints Hints, tips and frequently requested features
-
-Now you should know at least the syntax and be able to create a
-platform by your own. However, after having ourselves wrote some platforms, there
-are some best practices you should pay attention to in order to
-produce good platform and some choices you can make in order to have
-faster simulations. Here's some hints and tips, then.
-
-@subsection pf_hints_search Finding the platform example that you need
-
-Most platform files that we ship are in the @c examples/platforms
-folder. The good old @c grep tool can find the examples you need when
-wondering on a specific XML tag. Here is an example session searching
-for @ref pf_trace "trace_connect":
-
-@verbatim
-% cd examples/platforms
-% grep -R -i -n --include="*.xml" "trace_connect" .
-./two_hosts_platform_with_availability_included.xml:26:<trace_connect kind="SPEED" trace="A" element="Cpu A"/>
-./two_hosts_platform_with_availability_included.xml:27:<trace_connect kind="HOST_AVAIL" trace="A_failure" element="Cpu A"/>
-./two_hosts_platform_with_availability_included.xml:28:<trace_connect kind="SPEED" trace="B" element="Cpu B"/>
-./two_hosts.xml:17: <trace_connect trace="Tremblay_power" element="Tremblay" kind="SPEED"/>
-@endverbatim
-
-@subsection pf_hint_generating How to generate different platform files?
-
-This is actually a good idea to search for a better platform file,
-that better fit the need of your study. To be honest, the provided
-examples are not representative of anything. They exemplify our XML
-syntax, but that's all. small_platform.xml for example was generated
-without much thought beyond that.
-
-The best thing to do when possible is to write your own platform file,
-that model the platform on which you run your code. For that, you
-could use <a href="https://gitlab.inria.fr/simgrid/platform-calibration">our
-calibration scripts</a>. This leads to very good fits between the
-platform, the model and the needs. The g5k.xml example resulted of
-such an effort, which also lead to <a href="https://github.com/lpouillo/topo5k/">an
-ongoing attempt</a> to automatically extract the SimGrid platform from
-the <a href="http://grid5000.fr/">Grid'5000</a> experimental platform.
-But it's hard to come up with generic models. Don't take these files
-too seriously. Actually, you should always challenge our models and
-your instantiation if the accuracy really matters to you (see <a
-href="https://hal.inria.fr/hal-00907887">this discussion</a>).
-
-But such advices only hold if you have a real platform and a real
-application at hand. It's moot for more abstract studies working on
-ideas and algorithms instead of technical artefacts. Well, in this
-case, there unfortunately is nothing better than this old and rusty
-<a href="http://pda.gforge.inria.fr/tools/download.html">simulacrum</a>.
-This project is dormant since over 10 years (and you will have to
-update the generated platforms with <tt>bin/simgrid_update_xml</tt> to
-use them), but that's the best we have for this right now....
-
-@subsection pf_zone_h Zone Hierarchy
-The network zone design allows SimGrid to go fast, because computing route is
-done only for the set of resources defined in the current zone. If you're using
-only a big zone containing all resource with no zone into it and you're
-using Full model, then ... you'll loose all interest into it. On the
-other hand, designing a binary tree of zone with, at the lower level,
-only one host, then you'll also loose all the good zone hierarchy can
-give you. Remind you should always be "reasonable" in your platform
-definition when choosing the hierarchy. A good choice if you try to
-describe a real life platform is to follow the zone described in
-reality, since this kind of trade-off works well for real life
-platforms.
-
-@subsection pf_exit_zone Exit Zone: why and how
-Users that have looked at some of our platforms may have notice a
-non-intuitive schema ... Something like that:
-
-
-@verbatim
-<zone id="zone_4" routing="Full">
-<zone id="exitzone_4" routing="Full">
- <router id="router_4"/>
-</zone>
-<cluster id="cl_4_1" prefix="c_4_1-" suffix="" radical="1-20" speed="1000000000" bw="125000000" lat="5E-5" bb_bw="2250000000" bb_lat="5E-4"/>
-<cluster id="cl_4_2" prefix="c_4_2-" suffix="" radical="1-20" speed="1000000000" bw="125000000" lat="5E-5" bb_bw="2250000000" bb_lat="5E-4"/>
-<link id="4_1" bandwidth="2250000000" latency="5E-5"/>
-<link id="4_2" bandwidth="2250000000" latency="5E-5"/>
-<link id="bb_4" bandwidth="2250000000" latency="5E-4"/>
-<zoneroute src="cl_4_1"
- dst="cl_4_2"
- gw_src="c_4_1-cl_4_1_router"
- gw_dst="c_4_2-cl_4_2_router">
- <link_ctn id="4_1"/>
- <link_ctn id="bb_4"/>
- <link_ctn id="4_2"/>
-</zoneroute>
-<zoneroute src="cl_4_1"
- dst="exitzone_4"
- gw_src="c_4_1-cl_4_1_router"
- gw_dst="router_4">
- <link_ctn id="4_1"/>
- <link_ctn id="bb_4"/>
-</zoneroute>
-<zoneroute src="cl_4_2"
- dst="exitzone_4"
- gw_src="c_4_2-cl_4_2_router"
- gw_dst="router_4">
- <link_ctn id="4_2"/>
- <link_ctn id="bb_4"/>
-</zoneroute>
-</zone>
-@endverbatim
-
-In the zone_4, you have an exitzone_4 defined, containing only one router,
-and routes defined to that zone from all other zone (as cluster is only a
-shortcut for a zone, see cluster description for details). If there was
-an upper zone, it would define routes to and from zone_4 with the gateway
-router_4. It's just because, as we did not allowed (for performances
-issues) to have routes from a zone to a single host/router, you have to
-enclose your gateway, when you have zone included in your zone, within a
-zone to define routes to it.
-
-@subsection pf_routing_howto_choose_wisely Choosing wisely the routing model to use
-
-
-Choosing wisely the routing model to use can significantly fasten your
-simulation/save your time when writing the platform/save tremendous
-disk space. Here is the list of available model and their
-characteristics (lookup: time to resolve a route):
-
-@li <b>Full</b>: Full routing data (fast, large memory requirements,
- fully expressive)
-@li <b>Floyd</b>: Floyd routing data (slow initialization, fast
- lookup, lesser memory requirements, shortest path routing only).
- Calculates all routes at once at the beginning.
-@li <b>Dijkstra</b>: Dijkstra routing data (fast initialization, slow
- lookup, small memory requirements, shortest path routing only).
- Calculates a route when necessary.
-@li <b>DijkstraCache</b>: Dijkstra routing data (fast initialization,
- fast lookup, small memory requirements, shortest path routing
- only). Same as Dijkstra, except it handles a cache for latest used
- routes.
-@li <b>None</b>: No routing (usable with Constant network only).
- Defines that there is no routes, so if you try to determine a
- route without constant network within this zone, SimGrid will raise
- an exception.
-@li <b>Vivaldi</b>: Vivaldi routing, so when you want to use coordinates
-@li <b>Cluster</b>: Cluster routing, specific to cluster tag, should
- not be used.
-
-@subsection pf_loopback I want to specify the characteristics of the loopback link!
-
-Each routing model automatically adds a loopback link for each declared host, i.e.,
-a network route from the host to itself, if no such route is declared in the XML
-file. This default link has a bandwidth of 498 Mb/s, a latency of 15 microseconds,
-and is <b>not</b> shared among network flows.
-
-If you want to specify the characteristics of the loopback link for a given host, you
-just have to specify a route from this host to itself with the desired characteristics
-in the XML file. This will prevent the routing model to add and use the default
-loopback link.
-
-@subsection pf_switch I want to describe a switch but there is no switch tag!
-
-Actually we did not include switch tag. But when you're trying to
-simulate a switch, assuming
-fluid bandwidth models are used (which SimGrid uses by default unless
-ns-3 or constant network models are activated), the limiting factor is
-switch backplane bandwidth. So, essentially, at least from
-the simulation perspective, a switch is similar to a
-link: some device that is traversed by flows and with some latency and
-so,e maximum bandwidth. Thus, you can simply simulate a switch as a
-link. Many links
-can be connected to this "switch", which is then included in routes just
-as a normal link.
-
-
-@subsection pf_multicabinets I want to describe multi-cabinets clusters!
-
-You have several possibilities, as usual when modeling things. If your
-cabinets are homogeneous and the intercabinet network negligible for
-your study, you should just create a larger cluster with all hosts at
-the same layer.
-
-In the rare case where your hosts are not homogeneous between the
-cabinets, you can create your cluster completely manually. For that,
-create an As using the Cluster routing, and then use one
-<cabinet> for each cabinet. This cabinet tag can only be used an
-As using the Cluster routing schema, and creating
-
-Be warned that creating a cluster manually from the XML with
-<cabinet>, <backbone> and friends is rather tedious. The
-easiest way to retrieve some control of your model without diving into
-the <cluster> internals is certainly to create one separate
-<cluster> per cabinet and interconnect them together. This is
-what we did in the G5K example platform for the Graphen cluster.
-
-@subsection pf_platform_multipath I want to express multipath routing in platform files!
-
-It is unfortunately impossible to express the fact that there is more
-than one routing path between two given hosts. Let's consider the
-following platform file:
-
-@verbatim
-<route src="A" dst="B">
- <link_ctn id="1"/>
-</route>
-<route src="B" dst="C">
- <link_ctn id="2"/>
-</route>
-<route src="A" dst="C">
- <link_ctn id="3"/>
-</route>
-@endverbatim
-
-Although it is perfectly valid, it does not mean that data traveling
-from A to C can either go directly (using link 3) or through B (using
-links 1 and 2). It simply means that the routing on the graph is not
-trivial, and that data do not following the shortest path in number of
-hops on this graph. Another way to say it is that there is no implicit
-in these routing descriptions. The system will only use the routes you
-declare (such as <route src="A" dst="C"><link_ctn
-id="3"/></route>), without trying to build new routes by aggregating
-the provided ones.
-
-You are also free to declare platform where the routing is not
-symmetrical. For example, add the following to the previous file:
-
-@verbatim
-<route src="C" dst="A">
- <link_ctn id="2"/>
- <link_ctn id="1"/>
-</route>
-@endverbatim
-
-This makes sure that data from C to A go through B where data from A
-to C go directly. Don't worry about realism of such settings since
-we've seen ways more weird situation in real settings (in fact, that's
-the realism of very regular platforms which is questionable, but
-that's another story).
-
-*/
}
~~~
-@section simgrid_uhood_async Asynchronous operations
-
-@subsection simgrid_uhood_futures Futures
-
-The `simgrid::kernel::Future` class has been added to SimGrid as an abstraction
-to represent asynchronous operations in the SimGrid maestro. Its API is based
-on `std::experimental::future` from the [C++ Extensions for Concurrency Technical
-Specification](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0159r0.html):
-
- - `simgrid::kernel::Future<T>` represents the result an asynchronous operations
- in the simulation inside the SimGrid maestro/kernel;
-
- - `simgrid::kernel::Promise<T>` can be used to set the value of an assocaiated
- `simgrid::kernel::Future<T>`.
-
-The expected way to work with `simgrid::kernel::Future<T>` is to add a
-completion handler/continuation:
-
-~~~
-// This code is executed in the maestro context, we cannot block for the result
-// to be ready:
-simgrid::kernel::Future<std::vector<char>> result = simgrid::kernel::readFile(file);
-
-// Add a completion handler:
-result.then([file](simgrid::kernel::Future<std::vector<char>> result) {
- // At this point, the operation is complete and we can safely call .get():
- xbt_assert(result.is_ready());
- try {
- std::vector<char> data = result.get();
- XBT_DEBUG("Finished reading file %s: length %zu", file.c_str(), data.size());
- }
- // If the operation failed, .get() throws an exception:
- catch (std::runtime_error& e) {
- XBT_ERROR("Could not read file %s", file.c_str());
- }
-});
-~~~
-
-The SimGrid kernel cannot block so calling `.get()` or `.wait()` on a
-`simgrid::kernel::Future<T>` which is not ready will deadlock. In practice, the
-simulator detects this and aborts after reporting an error.
-
-In order to generate your own future, you might need to use a
-`simgrid::kernel::Promise<T>`. The promise is a one-way channel which can be
-used to set the result of an associated `simgrid::kernel::Future<T>`
-(with either `.set_value()` or `.set_exception()`):
-
-~~~
-simgrid::kernel::Future<void> kernel_wait_until(double date)
-{
- auto promise = std::make_shared<simgrid::kernel::Promise<void>>();
- auto future = promise->get_future();
- simgrid::simix::Timer::set(date, [promise] { promise->set_value(); });
- return future;
-}
-~~~
-
-Like the experimental futures, we support chaining `.then()` methods with
-automatic future unwrapping.
-You might want to look at some [tutorial on C++ futures](https://www.youtube.com/watch?v=mPxIegd9J3w&list=PLHTh1InhhwT75gykhs7pqcR_uSiG601oh&index=43)
-for more details and examples. Some operations of the proposed experimental
-futures are currently not implemented in our futures however such as
-`.wait_for()`, `.wait_until()`,
-[`shared_future`](http://en.cppreference.com/w/cpp/thread/shared_future),
-[`when_any()`](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0159r0.html#futures.when_any).
-
-@subsection simgrid_uhood_timer Timers
-
@section simgrid_uhood_mc Model Checker
The current implementation of the model-checker uses two distinct processes:
typically defines volumes of computation, communication, and time to pass to methods such as :cpp:func:`execute()
<simgrid::s4u::this_actor::execute>`, :cpp:func:`put() <simgrid::s4u::Mailbox::put>`, or :cpp:func:`sleep_for()
<simgrid::s4u::this_actor::sleep_for>`). Regardless of the potential accuracy of the simulation models, if they are
-instantiated with unrealistic parameter values, then the simulation will be inaccurate.
+instantiated with unrealistic parameter values, then the simulation will be inaccurate.
Given the above, an integral and crucial part of simulation-driven research is **simulation calibration**: the process by
which one picks simulation parameter values based on observed real-world executions so that simulated executions have high
--- /dev/null
+.. _dev_doc:
+
+Developer's Documentation
+=========================
+
+This page describes the software infrastructure for SimGrid potential contributors. This page is particularly interesting if you plan to contribute to the
+project.
+
+Enforcing the coding standards
+------------------------------
+
+If you plan to contribute code to the SimGrid project, you must ensure that your changes follow our coding standards. For that, simply install the relevant
+tools and scripts.
+
+.. code-block:: console
+
+ # install clang-format
+ sudo apt-get install clang-format # debian/ubuntu
+
+ # tell git to call the script on each commit
+ ln -s $(realpath tools/git-hooks/clang-format.pre-commit) .git/hooks/pre-commit
+
+This will add an extra verification before integrating any commit that you could prepare. If your code does not respects our formatting code, git will say so,
+and provide a ready to use patch that you can apply to improve your commit. Just read the error message you get to find how to apply the fix after review.
+
+If you prefer not to apply the fix on a specific commit (because the formatter fails on this case), then add ``--no-verify`` to your git commit command line.
+
+Setting up the development environment
+--------------------------------------
+
+The :ref:`default build configuration <install_src_config>` is intended for users, but contributors should change these settings for their comfort, and to
+produce better code. In particular, the default is to build highly optimized binaries, at the price of high compilation time and somewhat harder debug sessions.
+Instead, contributors want to disable the ``enable_compile_optimizations`` compile option. Likewise, ``enable_compile_warnings`` is off by default to not bother
+the users, but the contributors should really enable it, as your code will not be integrated if it raises warnings from the compiler.
+
+.. code-block:: console
+
+ cmake -Denable_compile_optimizations=OFF \
+ -Denable_compile_warnings=ON \
+ -GNinja .
+
+As you can see, we advise to use Ninja instead of the good old Makefiles. In our experience, Ninja is a bit faster because it does not fork as many external
+commands as make for basic operations.
+
+Interacting with the tests
+--------------------------
+
+We use unit tests based on `Catch2 <https://github.com/catchorg/Catch2/>`_, but most of our testing is done through integration tests that launch a specific
+simulation example and enforce that the output remains unchanged. Since SimGrid displays the timestamp of every logged event, such integration tests ensure that
+every change of the models' prediction will be noticed. We designed a specific tool for these integration tests, called TESH (the TEsting SHell). ``ctest`` is
+used to run both unit and integration tests. You can :ref:`run only some of the tests <install_src_test>` if you prefer. To launch the unit tests out of ctest,
+build and execute the ``./unit-tests`` binary (it accepts a ``--help`` parameter). To rebuild and run all tests, use the ``./BuildSimGrid.sh`` script.
+
+The SimGrid code base contains a lot of tests for a coverage well over 80%. We want the contributors to feel confident, and improve the code without fearing of
+breaking something without noticing. Tests must be fast (because we run them very often), the should stress many aspects of the code (to actually catch problems) and
+clear to read (understanding and debugging failures should be easy).
+
+In fact, our integration tests fall in two categories. The ones located in ``examples/`` are part of the documentation, so readability is utterly important
+here. The goal is that novice users could grab some of the provided examples and assemble them to constitute a first version of their simulator. We tend to
+avoid advanced C++ constructs that would hinder the readability of these files. On the other hand, the integration tests located in ``teshsuite/`` are usually
+torture tests trying to exhaustively cover all cases. They may be less readable and some even rely on a small DSL to describe all test cases and the expected
+result (check for example ``teshsuite/models/cloud-sharing``), but they should still be rather easy to debug on need. For this reason, we tend to avoid random
+testing that is difficult to reproduce. For example, the tests under ``teshsuite/models/`` try to be very verbose to make the model intend clear, and the code
+easier to debug on need. The WiFi tests seem to be a good example of that trend.
+
+We often say that a feature that is not tested is a feature that could soon disappear. So you want to write tests for the features you add. To add new unit
+tests, please refer to the end of ``tools/cmake/Tests.cmake``) for some examples. Catch2 comes with a good documentation and many examples online. If you add a
+new feature, you should add an integration test in ``examples/``. Your code should be instructive, documented, reusable, and its output must be perfectly
+reproducible. Some debugging information can be hidden when run from TESH with the :c:macro:`XBT_LOG_ISENABLED` macro. You then need to write a tesh file,
+following the syntax described in this man page: ``man -l manpages/tesh.1`` Finally, you should add your source code and tesh file to the cmake infrastructure
+by modifying for example the ``examples/cpp/CMakeLists.txt`` file. The test name shall allow the filtering of tests with ``ctest -R``. Do not forget to run
+``make distcheck`` once you're done to check that you did not forget to add your new files to the distribution.
+
+We have many online build bots that launch the tests on various configurations and operating systems. The results are centralized on two Jenkins jobs: `the main
+one <https://ci.inria.fr/simgrid/job/SimGrid/>`_ runs all tests on a variety of systems for each commit, while `Nightly
+<https://ci.inria.fr/simgrid/job/SimGrid-Nightly/>`_ runs some additional code analysis every night. Several sister projects built on top of SimGrid regularly
+test our git too. The FramaGit project gathers some additional `build badges <https://framagit.org/simgrid/simgrid>`_, and results are posted on the `bot-office
+channel <https://framateam.org/simgrid/channels/bot-office>`_ on Mattermost. Our code quality is tested every night using `SonarQube
+<https://sonarcloud.io/dashboard?id=simgrid_simgrid>`_ , and the `Debian build daemon <https://buildd.debian.org/status/package.php?p=simgrid>`_ test each
+release on several CPU architectures. We maintain some scripts to interact with these tools under ``tools/cmake``.
+
+Breaking tests once in a while is completely OK in SimGrid. We accept temporarily breakages because we want things to evolve, but if you break something, it is
+your duty to fix it within 24 hours to not hinder the work of others.
+
+Interacting with git
+--------------------
+
+The SimGrid community is relatively small and active. We are not used to maintain long-standing branches but instead, we strive to do our changes in an
+incremental way, commiting all intermediary steps to the main branch. The only thing is to ensure that you don't break things on the path, i.e. that all tests
+pass for all commits. If you prefer to do a branch to accumulate some commits in a branch, you should strive to make so for a short period of time to reduce the
+burden of the branch maintenance. Large changes happen from time to time to the SimGrid source code, and your branch may become hard to rebase.
+
+It is nice if your commit message could follow the git habits, explained in this `blog post
+<http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html>`_, or in the `style guide of Atom
+<https://github.com/atom/atom/blob/master/CONTRIBUTING.md#git-commit-messages>`_.
+
+Unsorted hints
+--------------
+
+* If you break the logs, you want to define XBT_LOG_MAYDAY at the beginning of log.h. It deactivates the whole logging mechanism, switching to printfs instead.
+ SimGrid then becomes incredibly verbose, but it you let you fixing things.
$ tar xf simgrid-3-XX.tar.gz
$ cd simgrid-*
- $ cmake -DCMAKE_INSTALL_PREFIX=/opt/simgrid -GNinja.
+ $ cmake -DCMAKE_INSTALL_PREFIX=/opt/simgrid -GNinja .
$ make
$ make install
:caption: SimGrid's Internals:
Design goals <Design_goals.rst>
+ Developer's documentation <Developers_Documentation.rst>
.. Cheat Sheet on the sublevels
..
set(DOC_SOURCES
doc/doxygen/FAQ.doc
doc/doxygen/inside.doc
- doc/doxygen/inside_tests.doc
- doc/doxygen/inside_cmake.doc
doc/doxygen/inside_extending.doc
doc/doxygen/inside_release.doc
- doc/doxygen/module-surf.doc
- doc/doxygen/module-index.doc
doc/doxygen/outcomes_vizu.doc
- doc/doxygen/platform.doc
doc/doxygen/uhood.doc
doc/doxygen/uhood_switch.doc
docs/source/community.rst
docs/source/Configuring_SimGrid.rst
docs/source/Deploying_your_application.rst
+ docs/source/Developers_Documentation.rst
docs/source/Experimental_setup.rst
docs/source/index.rst
docs/source/intl.rst
