-/*! \page options Simgrid options and configurations
+/*! \page options Configure SimGrid
A number of options can be given at runtime to change the default
SimGrid behavior. For a complete list of all configuration options
the --help configuration flag to your program. If some of the options
are not documented on this page, this is a bug that you should please
report so that we can fix it. Note that some of the options presented
-here may not be available in your simulators, depending on the
+here may not be available in your simulators, depending on the
@ref install_src_config "compile-time options" that you used.
+\tableofcontents
+
\section options_using Passing configuration options to the simulators
There is several way to pass configuration options to the simulators.
within that tag, you can pass one or several \c \<prop\> tags to specify
the configuration to use. For example, setting \c Item to \c Value
can be done by adding the following to the beginning of your platform
-file:
+file:
\verbatim
<config>
<prop id="Item" value="Value"/>
directly as follows. Check the \ref XBT_config "relevant page" for
details on all the functions you can use in this context, \c
_sg_cfg_set being the only configuration set currently used in
-SimGrid.
+SimGrid.
@code
#include <xbt/config.h>
\section options_model Configuring the platform models
+\anchor options_storage_model
+\anchor options_vm_model
\subsection options_model_select Selecting the platform models
-SimGrid comes with several network and CPU models built in, and you
+SimGrid comes with several network, CPU and storage models built in, and you
can change the used model at runtime by changing the passed
configuration. The three main configuration items are given below.
For each of these items, passing the special \c help value gives
should provide information about all models for all existing resources.
- \b network/model: specify the used network model
- \b cpu/model: specify the used CPU model
- - \b workstation/model: specify the used workstation model
+ - \b host/model: specify the used host model
+ - \b storage/model: specify the used storage model (there is currently only one such model - this option is hence only useful for future releases)
+ - \b vm/model: specify the model for virtual machines (there is currently only one such model - this option is hence only useful for future releases)
-As of writting, the accepted network models are the following. Over
+%As of writing, the following network models are accepted. Over
the time new models can be added, and some experimental models can be
removed; check the values on your simulators for an uptodate
information. Note that the CM02 model is described in the research report
- \b LV08 (default one): Realistic network analytic model
(slow-start modeled by multiplying latency by 10.4, bandwidth by
.92; bottleneck sharing uses a payload of S=8775 for evaluating RTT)
- - \b Constant: Simplistic network model where all communication
+ - \anchor options_model_select_network_constant \b Constant: Simplistic network model where all communication
take a constant time (one second). This model provides the lowest
realism, but is (marginally) faster.
- \b SMPI: Realistic network model specifically tailored for HPC
settings (accurate modeling of slow start with correction factors on
three intervals: < 1KiB, < 64 KiB, >= 64 KiB). See also \ref
options_model_network_coefs "this section" for more info.
+ - \b IB: Realistic network model specifically tailored for HPC
+ settings with InfiniBand networks (accurate modeling contention
+ behavior, based on the model explained in
+ http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf).
+ See also \ref options_model_network_coefs "this section" for more info.
- \b CM02: Legacy network analytic model (Very similar to LV08, but
without corrective factors. The timings of small messages are thus
poorly modeled)
simulators as network models (see \ref pls). In that case, you have
two extra models, described below, and some \ref options_pls "specific
additional configuration flags".
- - \b GTNets: Network pseudo-model using the GTNets simulator instead
- of an analytic model
- \b NS3: Network pseudo-model using the NS3 tcp model instead of an
analytic model
Concerning the CPU, we have only one model for now:
- \b Cas01: Simplistic CPU model (time=size/power)
-The workstation concept is the aggregation of a CPU with a network
+The host concept is the aggregation of a CPU with a network
card. Three models exists, but actually, only 2 of them are
interesting. The "compound" one is simply due to the way our internal
code is organized, and can easily be ignored. So at the end, you have
-two workstation models: The default one allows to aggregate an
+two host models: The default one allows to aggregate an
existing CPU model with an existing network model, but does not allow
parallel tasks because these beasts need some collaboration between
the network and CPU model. That is why, ptask_07 is used by default
when using SimDag.
- - \b default: Default workstation model. Currently, CPU:Cas01 and
+ - \b default: Default host model. Currently, CPU:Cas01 and
network:LV08 (with cross traffic enabled)
- - \b compound: Workstation model that is automatically chosen if
+ - \b compound: Host model that is automatically chosen if
you change the network and CPU models
- - \b ptask_L07: Workstation model somehow similar to Cas01+CM02 but
+ - \b ptask_L07: Host model somehow similar to Cas01+CM02 but
allowing parallel tasks
+\subsection options_generic_plugin Plugins
+
+SimGrid supports the use of plugins; currently, no known plugins
+can be activated but there are use-cases where you may want to write
+your own plugin (for instance, for logging).
+
+Plugins can for instance define own classes that inherit from
+existing classes (for instance, a class "CpuEnergy" inherits from
+"Cpu" to assess energy consumption).
+
+The plugin connects to the code by registering callbacks using
+``signal.connect(callback)`` (see file ``src/surf/plugins/energy.cpp`` for
+details).
+
+\verbatim
+ --cfg=plugin:Energy
+\endverbatim
+
+\note
+ This option is case-sensitive: Energy and energy are not the same!
+
\subsection options_model_optim Optimization level of the platform models
The network and CPU models that are based on lmm_solve (that
now).
- \b Full: Full update of remaining and variables. Slow but may be
useful when debugging.
- - items \b network/maxmin_selective_update and
- \b cpu/maxmin_selective_update: configure whether the underlying
+ - items \b network/maxmin-selective-update and
+ \b cpu/maxmin-selective-update: configure whether the underlying
should be lazily updated or not. It should have no impact on the
computed timings, but should speed up the computation.
-It is still possible to disable the \c maxmin_selective_update feature
+It is still possible to disable the \c maxmin-selective-update feature
because it can reveal counter-productive in very specific scenarios
where the interaction level is high. In particular, if all your
communication share a given backbone link, you should disable it:
-without \c maxmin_selective_update, every communications are updated
+without \c maxmin-selective-update, every communications are updated
at each step through a simple loop over them. With that feature
enabled, every communications will still get updated in this case
(because of the dependency induced by the backbone), but through a
The analytical models need to know the maximal TCP window size to take
the TCP congestion mechanism into account. This is set to 20000 by
-default, but can be changed using the \b network/TCP_gamma item.
+default, but can be changed using the \b network/TCP-gamma item.
On linux, this value can be retrieved using the following
commands. Both give a set of values, and you should use the last one,
cat /proc/sys/net/ipv4/tcp_wmem # gives the receiver window
\endverbatim
-\subsubsection options_model_network_coefs Corrective simulation factors
+\subsubsection options_model_network_coefs Correcting important network parameters
+
+SimGrid can take network irregularities such as a slow startup or
+changing behavior depending on the message size into account.
+You should not change these values unless you really know what you're doing.
-These factors allow to betterly take the slow start into account.
The corresponding values were computed through data fitting one the
-timings of packet-level simulators. You should not change these values
-unless you are really certain of what you are doing. See
+timings of packet-level simulators.
+
+See
<a href="http://mescal.imag.fr/membres/arnaud.legrand/articles/simutools09.pdf">Accuracy Study and Improvement of Network Simulation in the SimGrid Framework</a>
-for more informations about these coeficients.
+for more information about these parameters.
-If you are using the SMPI model, these correction coeficients are
+If you are using the SMPI model, these correction coefficients are
themselves corrected by constant values depending on the size of the
exchange. Again, only hardcore experts should bother about this fact.
+InfiniBand network behavior can be modeled through 3 parameters, as explained in
+<a href="http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf">this PhD thesis</a>.
+These factors can be changed through the following option:
+
+\verbatim
+smpi/IB-penalty-factors:"βe;βs;γs"
+\endverbatim
+
+By default SMPI uses factors computed on the Stampede Supercomputer at TACC, with optimal
+deployment of processes on nodes.
+
\subsubsection options_model_network_crosstraffic Simulating cross-traffic
-As of SimGrid v3.7, cross-traffic effects can be taken into account in
+%As of SimGrid v3.7, cross-traffic effects can be taken into account in
analytical simulations. It means that ongoing and incoming
communication flows are treated independently. In addition, the LV08
model adds 0.05 of usage on the opposite direction for each new
For that to work, your platform must have two links for each
pair of interconnected hosts. An example of usable platform is
-available in <tt>examples/msg/gtnets/crosstraffic-p.xml</tt>.
+available in <tt>examples/platforms/crosstraffic.xml</tt>.
This is activated through the \b network/crosstraffic item, that
can be set to 0 (disable this feature) or 1 (enable it).
-Note that with the default workstation model this option is activated by default.
+Note that with the default host model this option is activated by default.
\subsubsection options_model_network_coord Coordinated-based network models
When you want to use network coordinates, as it happens when you use
-an \<AS\> in your platform file with \c Vivaldi as a routing, you must
+an \<AS\> in your platform file with \c Vivaldi as a routing (see also
+Section \ref pf_routing_model_vivaldi "Vivaldi Routing Model"), you must
set the \b network/coordinates to \c yes so that all mandatory
initialization are done in the simulator.
(this configuration item is experimental and may change or disapear)
It is possible to specify a timing gap between consecutive emission on
-the same network card through the \b network/sender_gap item. This
+the same network card through the \b network/sender-gap item. This
is still under investigation as of writting, and the default value is
to wait 10 microseconds (1e-5 seconds) between emissions.
(this configuration item is experimental and may change or disapear)
-It is possible to specify that messages below a certain size will be sent
-as soon as the call to MPI_Send is issued, without waiting for the
-correspondant receive. This threshold can be configured through the
-\b smpi/async_small_thres item. The default value is 0. This behavior can also be
-manually set for MSG mailboxes, by setting the receiving mode of the mailbox
-with a call to \ref MSG_mailbox_set_async . For MSG, all messages sent to this
-mailbox will have this behavior, so consider using two mailboxes if needed.
+It is possible to specify that messages below a certain size will be sent
+as soon as the call to MPI_Send is issued, without waiting for the
+correspondant receive. This threshold can be configured through the
+\b smpi/async-small-thresh item. The default value is 0. This behavior can also be
+manually set for MSG mailboxes, by setting the receiving mode of the mailbox
+with a call to \ref MSG_mailbox_set_async . For MSG, all messages sent to this
+mailbox will have this behavior, so consider using two mailboxes if needed.
-This value needs to be smaller than or equals to the threshold set at
-\ref options_model_smpi_detached , because asynchronous messages are
+This value needs to be smaller than or equals to the threshold set at
+\ref options_model_smpi_detached , because asynchronous messages are
meant to be detached as well.
\subsubsection options_pls Configuring packet-level pseudo-models
in NS3. The only valid values (enforced on the SimGrid side) are
'NewReno' or 'Reno' or 'Tahoe'.
-When using GTNeTS, two items exist:
- - \b gtnets/jitter, that is a double value to oscillate
- the link latency, uniformly in random interval
- [-latency*gtnets_jitter,latency*gtnets_jitter). It defaults to 0.
- - \b gtnets/jitter_seed, the positive seed used to reproduce jitted
- results. Its value must be in [1,1e8] and defaults to 10.
-
\section options_modelchecking Configuring the Model-Checking
-To enable the experimental SimGrid model-checking support the program should
-be executed with the command line argument
+To enable the SimGrid model-checking support the program should
+be executed using the simgrid-mc wrapper:
\verbatim
---cfg=model-check:1
+simgrid-mc ./my_program
\endverbatim
+
Safety properties are expressed as assertions using the function
\verbatim
void MC_assert(int prop);
If you want to specify liveness properties (beware, that's
experimental), you have to pass them on the command line, specifying
-the name of the file containing the property, as formated by the
+the name of the file containing the property, as formatted by the
ltl2ba program.
\verbatim
--cfg=model-check/property:<filename>
\endverbatim
-Of course, specifying a liveness property enables the model-checking
-so that you don't have to give <tt>--cfg=model-check:1</tt> in
-addition.
-
\subsection options_modelchecking_steps Going for stateful verification
By default, the system is backtracked to its initial state to explore
--cfg=model-check/checkpoint:1
\endverbatim
-Of course, specifying this option enables the model-checking so that
-you don't have to give <tt>--cfg=model-check:1</tt> in addition.
-
\subsection options_modelchecking_reduction Specifying the kind of reduction
The main issue when using the model-checking is the state space
* dpor: Apply Dynamic Partial Ordering Reduction. Only valid if you
verify local safety properties.
-Of course, specifying a reduction technique enables the model-checking
-so that you don't have to give <tt>--cfg=model-check:1</tt> in
-addition.
+\subsection options_modelchecking_visited model-check/visited, Cycle detection
+
+In order to detect cycles, the model-checker needs to check if a new explored
+state is in fact the same state than a previous one. In order to do this,
+the model-checker can take a snapshot of each visited state: this snapshot is
+then used to compare it with subsequent states in the exploration graph.
+
+The \b model-check/visited is the maximum number of states which are stored in
+memory. If the maximum number of snapshotted state is reached some states will
+be removed from the memory and some cycles might be missed.
+
+By default, no state is snapshotted and cycles cannot be detected.
+
+\subsection options_modelchecking_termination model-check/termination, Non termination detection
+
+The \b model-check/termination configuration item can be used to report if a
+non-termination execution path has been found. This is a path with a cycle
+which means that the program might never terminate.
+
+This only works in safety mode.
+
+This options is disabled by default.
+
+\subsection options_modelchecking_dot_output model-check/dot-output, Dot output
+
+If set, the \b model-check/dot-output configuration item is the name of a file
+in which to write a dot file of the path leading the found property (safety or
+liveness violation) as well as the cycle for liveness properties. This dot file
+can then fed to the graphviz dot tool to generate an corresponding graphical
+representation.
+
+\subsection options_modelchecking_max_depth model-check/max_depth, Depth limit
+
+The \b model-checker/max-depth can set the maximum depth of the exploration
+graph of the model-checker. If this limit is reached, a logging message is
+sent and the results might not be exact.
+
+By default, there is not depth limit.
+
+\subsection options_modelchecking_timeout Handling of timeout
+
+By default, the model-checker does not handle timeout conditions: the `wait`
+operations never time out. With the \b model-check/timeout configuration item
+set to \b yes, the model-checker will explore timeouts of `wait` operations.
+
+\subsection options_modelchecking_comm_determinism Communication determinism
+
+The \b model-check/communications-determinism and
+\b model-check/send-determinism items can be used to select the communication
+determinism mode of the model-checker which checks determinism properties of
+the communications of an application.
+
+\subsection options_modelchecking_sparse_checkpoint Per page checkpoints
+
+When the model-checker is configured to take a snapshot of each explored state
+(with the \b model-checker/visited item), the memory consumption can rapidly
+reach GiB ou Tib of memory. However, for many workloads, the memory does not
+change much between different snapshots and taking a complete copy of each
+snapshot is a waste of memory.
+
+The \b model-check/sparse-checkpoint option item can be set to \b yes in order
+to avoid making a complete copy of each snapshot: instead, each snapshot will be
+decomposed in blocks which will be stored separately.
+If multiple snapshots share the same block (or if the same block
+is used in the same snapshot), the same copy of the block will be shared leading
+to a reduction of the memory footprint.
+
+For many applications, this option considerably reduces the memory consumption.
+In somes cases, the model-checker might be slightly slower because of the time
+taken to manage the metadata about the blocks. In other cases however, this
+snapshotting strategy will be much faster by reducing the cache consumption.
+When the memory consumption is important, by avoiding to hit the swap or
+reducing the swap usage, this option might be much faster than the basic
+snapshotting strategy.
+
+This option is currently disabled by default.
\subsection options_mc_perf Performance considerations for the model checker
The size of the stacks can have a huge impact on the memory
-consumption when using model-checking. Currently each snapshot, will
-save a copy of the whole stack and not only of the part which is
+consumption when using model-checking. By default, each snapshot will
+save a copy of the whole stacks and not only of the part which is
really meaningful: you should expect the contribution of the memory
consumption of the snapshots to be \f$ \mbox{number of processes}
\times \mbox{stack size} \times \mbox{number of states} \f$.
-However, when compiled against the model checker, the stacks are not
+The \b model-check/sparse-checkpoint can be used to reduce the memory
+consumption by trying to share memory between the different snapshots.
+
+When compiled against the model checker, the stacks are not
protected with guards: if the stack size is too small for your
application, the stack will silently overflow on other parts of the
memory.
+\subsection options_modelchecking_hash Hashing of the state (experimental)
+
+Usually most of the time of the model-checker is spent comparing states. This
+process is complicated and consumes a lot of bandwidth and cache.
+In order to speedup the state comparison, the experimental \b model-checker/hash
+configuration item enables the computation of a hash summarizing as much
+information of the state as possible into a single value. This hash can be used
+to avoid most of the comparisons: the costly comparison is then only used when
+the hashes are identical.
+
+Currently most of the state is not included in the hash because the
+implementation was found to be buggy and this options is not as useful as
+it could be. For this reason, it is currently disabled by default.
+
+\subsection options_modelchecking_recordreplay Record/replay (experimental)
+
+As the model-checker keeps jumping at different places in the execution graph,
+it is difficult to understand what happens when trying to debug an application
+under the model-checker. Event the output of the program is difficult to
+interpret. Moreover, the model-checker does not behave nicely with advanced
+debugging tools such as valgrind. For those reason, to identify a trajectory
+in the execution graph with the model-checker and replay this trajcetory and
+without the model-checker black-magic but with more standard tools
+(such as a debugger, valgrind, etc.). For this reason, Simgrid implements an
+experimental record/replay functionnality in order to record a trajectory with
+the model-checker and replay it without the model-checker.
+
+When the model-checker finds an interesting path in the application execution
+graph (where a safety or liveness property is violated), it can generate an
+identifier for this path. In order to enable this behavious the
+\b model-check/record must be set to \b yes. By default, this behaviour is not
+enabled.
+
+This is an example of output:
+
+<pre>
+[ 0.000000] (0:@) Check a safety property
+[ 0.000000] (0:@) **************************
+[ 0.000000] (0:@) *** PROPERTY NOT VALID ***
+[ 0.000000] (0:@) **************************
+[ 0.000000] (0:@) Counter-example execution trace:
+[ 0.000000] (0:@) Path = 1/3;1/4
+[ 0.000000] (0:@) [(1)Tremblay (app)] MC_RANDOM(3)
+[ 0.000000] (0:@) [(1)Tremblay (app)] MC_RANDOM(4)
+[ 0.000000] (0:@) Expanded states = 27
+[ 0.000000] (0:@) Visited states = 68
+[ 0.000000] (0:@) Executed transitions = 46
+</pre>
+
+This path can then be replayed outside of the model-checker (and even in
+non-MC build of simgrid) by setting the \b model-check/replay item to the given
+path. The other options should be the same (but the model-checker should
+be disabled).
+
+The format and meaning of the path may change between different releases so
+the same release of Simgrid should be used for the record phase and the replay
+phase.
+
\section options_virt Configuring the User Process Virtualization
\subsection options_virt_factory Selecting the virtualization factory
-In SimGrid, the user code is virtualized in a specific mecanism
-allowing the simulation kernel to control its execution: when a user
+In SimGrid, the user code is virtualized in a specific mechanism
+that allows the simulation kernel to control its execution: when a user
process requires a blocking action (such as sending a message), it is
interrupted, and only gets released when the simulated clock reaches
the point where the blocking operation is done.
pthreads or windows native threads)
- \b ucontext: fast factory using System V contexts (or a portability
layer of our own on top of Windows fibers)
- - \b raw: amazingly fast factory using a context switching mecanism
+ - \b raw: amazingly fast factory using a context switching mechanism
of our own, directly implemented in assembly (only available for x86
and amd64 platforms for now)
+ - \b boost: This uses the [context implementation](http://www.boost.org/doc/libs/1_59_0/libs/context/doc/html/index.html)
+ of the boost library; you must have this library installed before
+ you compile SimGrid. (On Debian GNU/Linux based systems, this is
+ provided by the libboost-contexts-dev package.)
The only reason to change this setting is when the debugging tools get
fooled by the optimized context factories. Threads are the most
-debugging-friendly contextes.
+debugging-friendly contextes, as they allow to set breakpoints anywhere with gdb
+ and visualize backtraces for all processes, in order to debug concurrency issues.
+Valgrind is also more comfortable with threads, but it should be usable with all factories.
\subsection options_virt_stacksize Adapting the used stack size
stacks), leading to segfaults with corrupted stack traces.
If you want to push the scalability limits of your code, you might
-want to reduce the \b contexts/stack_size item. Its default value
+want to reduce the \b contexts/stack-size item. Its default value
is 8192 (in KiB), while our Chord simulation works with stacks as small
-as 16 KiB, for example. For the thread factory, the default value
-is the one of the system, if it is too large/small, it has to be set
+as 16 KiB, for example. For the thread factory, the default value
+is the one of the system, if it is too large/small, it has to be set
with this parameter.
The operating system should only allocate memory for the pages of the
In some cases, no stack guard page is used and the stack will silently
overflow on other parts of the memory if the stack size is too small
-for your application. This happens :
+for your application. This happens :
- on Windows systems;
- when the model checker is enabled;
request to execute the user code in parallel. Several threads are
launched, each of them handling as much user contexts at each run. To
actiave this, set the \b contexts/nthreads item to the amount of
-cores that you have in your computer (or lower than 1 to have
+cores that you have in your computer (or lower than 1 to have
the amount of cores auto-detected).
Even if you asked several worker threads using the previous option,
you can request to start the parallel execution (and pay the
associated synchronization costs) only if the potential parallelism is
-large enough. For that, set the \b contexts/parallel_threshold
+large enough. For that, set the \b contexts/parallel-threshold
item to the minimal amount of user contexts needed to start the
parallel execution. In any given simulation round, if that amount is
not reached, the contexts will be run sequentially directly by the
- Add the contents of a textual file on top of the trace file as comment:
\verbatim
---cfg=tracing/comment_file:my_file_with_additional_information.txt
+--cfg=tracing/comment-file:my_file_with_additional_information.txt
\endverbatim
Please, use these two parameters (for comments) to make reproducible
simulations. For additional details about this and all tracing
options, check See the \ref tracing_tracing_options.
+\section options_msg Configuring MSG
+
+\subsection options_msg_debug_multiple_use Debugging MSG
+
+Sometimes your application may try to send a task that is still being
+executed somewhere else, making it impossible to send this task. However,
+for debugging purposes, one may want to know what the other host is/was
+doing. This option shows a backtrace of the other process.
+
+Enable this option by adding
+
+\verbatim
+--cfg=msg/debug-multiple-use:on
+\endverbatim
+
\section options_smpi Configuring SMPI
The SMPI interface provides several specific configuration items.
These are uneasy to see since the code is usually launched through the
\c smiprun script directly.
-\subsection options_smpi_bench Automatic benchmarking of SMPI code
+\subsection options_smpi_bench smpi/bench: Automatic benchmarking of SMPI code
In SMPI, the sequential code is automatically benchmarked, and these
computations are automatically reported to the simulator. That is to
code, and create an execution task within the simulator to take this
into account. For that, the actual duration is measured on the host
machine and then scaled to the power of the corresponding simulated
-machine. The variable \b smpi/running_power allows to specify the
+machine. The variable \b smpi/running-power allows to specify the
computational power of the host machine (in flop/s) to use when
scaling the execution times. It defaults to 20000, but you really want
to update it to get accurate simulation results.
When the code is constituted of numerous consecutive MPI calls, the
previous mechanism feeds the simulation kernel with numerous tiny
-computations. The \b smpi/cpu_threshold item becomes handy when this
-impacts badly the simulation performance. It specify a threshold (in
-second) under which the execution chunks are not reported to the
-simulation kernel (default value: 1e-6). Please note that in some
-circonstances, this optimization can hinder the simulation accuracy.
+computations. The \b smpi/cpu-threshold item becomes handy when this
+impacts badly the simulation performance. It specifies a threshold (in
+seconds) below which the execution chunks are not reported to the
+simulation kernel (default value: 1e-6).
+
+
+\note
+ The option smpi/cpu-threshold ignores any computation time spent
+ below this threshold. SMPI does not consider the \a amount of these
+ computations; there is no offset for this. Hence, by using a
+ value that is too low, you may end up with unreliable simulation
+ results.
In some cases, however, one may wish to disable simulation of
application computation. This is the case when SMPI is used not to
be simulated using SMPI by calling internal smpi_execute*() functions.
To disable the benchmarking/simulation of computation in the simulated
-application, the variable \b
-smpi/simulation_computation should be set to no
+application, the variable \b smpi/simulate-computation should be set to no.
+Equivalently, setting \b smpi/cpu-threshold to -1 also ignores all
+computation.
-\subsection options_smpi_timing Reporting simulation time
+\note
+ This option just ignores the timings in your simulation; it still executes
+ the computations itself. If you want to stop SMPI from doing that,
+ you should check the SMPI_SAMPLE macros, documented in the chapter
+ \ref SMPI_adapting_speed.
-Most of the time, you run MPI code through SMPI to compute the time it
-would take to run it on a platform that you don't have. But since the
+\subsection options_model_smpi_bw_factor smpi/bw-factor: Bandwidth factors
+
+The possible throughput of network links is often dependent on the
+message sizes, as protocols may adapt to different message sizes. With
+this option, a series of message sizes and factors are given, helping
+the simulation to be more realistic. For instance, the current
+default value is
+
+\verbatim
+65472:0.940694;15424:0.697866;9376:0.58729;5776:1.08739;3484:0.77493;1426:0.608902;732:0.341987;257:0.338112;0:0.812084
+\endverbatim
+
+So, messages with size 65472 and more will get a total of MAX_BANDWIDTH*0.940694,
+messages of size 15424 to 65471 will get MAX_BANDWIDTH*0.697866 and so on.
+Here, MAX_BANDWIDTH denotes the bandwidth of the link.
+
+\note
+ The SimGrid-Team has developed a script to help you determine these
+ values. You can find more information and the download here:
+ 1. http://simgrid.gforge.inria.fr/contrib/smpi-calibration-doc.html
+ 2. http://simgrid.gforge.inria.fr/contrib/smpi-saturation-doc.html
+
+\subsection options_smpi_timing smpi/display-timing: Reporting simulation time
+
+\b Default: 0 (false)
+
+Most of the time, you run MPI code with SMPI to compute the time it
+would take to run it on a platform. But since the
code is run through the \c smpirun script, you don't have any control
-on the launcher code, making difficult to report the simulated time
-when the simulation ends. If you set the \b smpi/display_timing item
+on the launcher code, making it difficult to report the simulated time
+when the simulation ends. If you set the \b smpi/display-timing item
to 1, \c smpirun will display this information when the simulation ends. \verbatim
Simulation time: 1e3 seconds.
\endverbatim
-\subsection options_smpi_global Automatic privatization of global variables
+\subsection options_model_smpi_lat_factor smpi/lat-factor: Latency factors
+
+The motivation and syntax for this option is identical to the motivation/syntax
+of smpi/bw-factor, see \ref options_model_smpi_bw_factor for details.
+
+There is an important difference, though: While smpi/bw-factor \a reduces the
+actual bandwidth (i.e., values between 0 and 1 are valid), latency factors
+increase the latency, i.e., values larger than or equal to 1 are valid here.
+
+This is the default value:
+
+\verbatim
+65472:11.6436;15424:3.48845;9376:2.59299;5776:2.18796;3484:1.88101;1426:1.61075;732:1.9503;257:1.95341;0:2.01467
+\endverbatim
+
+\note
+ The SimGrid-Team has developed a script to help you determine these
+ values. You can find more information and the download here:
+ 1. http://simgrid.gforge.inria.fr/contrib/smpi-calibration-doc.html
+ 2. http://simgrid.gforge.inria.fr/contrib/smpi-saturation-doc.html
+
+\subsection options_smpi_global smpi/privatize-global-variables: Automatic privatization of global variables
-MPI executables are meant to be executed in separated processes, but SMPI is
-executed in only one process. Global variables from executables will be placed
+MPI executables are meant to be executed in separated processes, but SMPI is
+executed in only one process. Global variables from executables will be placed
in the same memory zone and shared between processes, causing hard to find bugs.
To avoid this, several options are possible :
- Manual edition of the code, for example to add __thread keyword before data
- declaration, which allows the resulting code to work with SMPI, but only
- if the thread factory (see \ref options_virt_factory) is used, as global
- variables are then placed in the TLS (thread local storage) segment.
- - Source-to-source transformation, to add a level of indirection
- to the global variables. SMPI does this for F77 codes compiled with smpiff,
+ declaration, which allows the resulting code to work with SMPI, but only
+ if the thread factory (see \ref options_virt_factory) is used, as global
+ variables are then placed in the TLS (thread local storage) segment.
+ - Source-to-source transformation, to add a level of indirection
+ to the global variables. SMPI does this for F77 codes compiled with smpiff,
and used to provide coccinelle scripts for C codes, which are not functional anymore.
- - Compilation pass, to have the compiler automatically put the data in
- an adapted zone.
- - Runtime automatic switching of the data segments. SMPI stores a copy of
- each global data segment for each process, and at each context switch replaces
+ - Compilation pass, to have the compiler automatically put the data in
+ an adapted zone.
+ - Runtime automatic switching of the data segments. SMPI stores a copy of
+ each global data segment for each process, and at each context switch replaces
the actual data with its copy from the right process. This mechanism uses mmap,
- and is for now limited to systems supporting this functionnality (all Linux
+ and is for now limited to systems supporting this functionnality (all Linux
and some BSD should be compatible).
Another limitation is that SMPI only accounts for global variables defined in
- the executable. If the processes use external global variables from dynamic
- libraries, they won't be switched correctly. To avoid this, using static
- linking is advised (but not with the simgrid library, to avoid replicating
- its own global variables).
+ the executable. If the processes use external global variables from dynamic
+ libraries, they won't be switched correctly. To avoid this, using static
+ linking is advised (but not with the simgrid library, to avoid replicating
+ its own global variables).
- To use this runtime automatic switching, the variable \b smpi/privatize_global_variables
+ To use this runtime automatic switching, the variable \b smpi/privatize-global-variables
should be set to yes
\subsection options_model_smpi_detached Simulating MPI detached send
-This threshold specifies the size in bytes under which the send will return
+This threshold specifies the size in bytes under which the send will return
immediately. This is different from the threshold detailed in \ref options_model_network_asyncsend
because the message is not effectively sent when the send is posted. SMPI still waits for the
-correspondant receive to be posted to perform the communication operation. This threshold can be set
-by changing the \b smpi/send_is_detached item. The default value is 65536.
+correspondant receive to be posted to perform the communication operation. This threshold can be set
+by changing the \b smpi/send-is-detached item. The default value is 65536.
\subsection options_model_smpi_collectives Simulating MPI collective algorithms
-SMPI implements more than 100 different algorithms for MPI collective communication, to accurately
-simulate the behavior of most of the existing MPI libraries. The \b smpi/coll_selector item can be used
+SMPI implements more than 100 different algorithms for MPI collective communication, to accurately
+simulate the behavior of most of the existing MPI libraries. The \b smpi/coll-selector item can be used
to use the decision logic of either OpenMPI or MPICH libraries (values: ompi or mpich, by default SMPI
-uses naive version of collective operations). Each collective operation can be manually selected with a
+uses naive version of collective operations). Each collective operation can be manually selected with a
\b smpi/collective_name:algo_name. Available algorithms are listed in \ref SMPI_collective_algorithms .
+\subsection options_model_smpi_iprobe smpi/iprobe: Inject constant times for calls to MPI_Iprobe
+
+\b Default value: 0.0001
+
+The behavior and motivation for this configuration option is identical with \a smpi/test, see
+Section \ref options_model_smpi_test for details.
+
+\subsection options_model_smpi_ois smpi/ois: Inject constant times for asynchronous send operations
+
+This configuration option works exactly as \a smpi/os, see Section \ref options_model_smpi_os.
+Of course, \a smpi/ois is used to account for MPI_Isend instead of MPI_Send.
+
+\subsection options_model_smpi_os smpi/os: Inject constant times for send operations
+
+In several network models such as LogP, send (MPI_Send, MPI_Isend) and receive (MPI_Recv)
+operations incur costs (i.e., they consume CPU time). SMPI can factor these costs in as well, but the
+user has to configure SMPI accordingly as these values may vary by machine.
+This can be done by using smpi/os for MPI_Send operations; for MPI_Isend and
+MPI_Recv, use \a smpi/ois and \a smpi/or, respectively. These work exactly as
+\a smpi/ois.
+
+\a smpi/os can consist of multiple sections; each section takes three values, for example:
+
+\verbatim
+ 1:3:2;10:5:1
+\endverbatim
+
+Here, the sections are divided by ";" (that is, this example contains two sections).
+Furthermore, each section consists of three values.
+
+1. The first value denotes the minimum size for this section to take effect;
+ read it as "if message size is greater than this value (and other section has a larger
+ first value that is also smaller than the message size), use this".
+ In the first section above, this value is "1".
+
+2. The second value is the startup time; this is a constant value that will always
+ be charged, no matter what the size of the message. In the first section above,
+ this value is "3".
+
+3. The third value is the \a per-byte cost. That is, it is charged for every
+ byte of the message (incurring cost messageSize*cost_per_byte)
+ and hence accounts also for larger messages. In the first
+ section of the example above, this value is "2".
+
+Now, SMPI always checks which section it should take for a given message; that is,
+if a message of size 11 is sent with the configuration of the example above, only
+the second section will be used, not the first, as the first value of the second
+section is closer to the message size. Hence, a message of size 11 incurs the
+following cost inside MPI_Send:
+
+\verbatim
+ 5+11*1
+\endverbatim
+
+%As 5 is the startup cost and 1 is the cost per byte.
+
+\note
+ The order of sections can be arbitrary; they will be ordered internally.
+
+\subsection options_model_smpi_or smpi/or: Inject constant times for receive operations
+
+This configuration option works exactly as \a smpi/os, see Section \ref options_model_smpi_os.
+Of course, \a smpi/or is used to account for MPI_Recv instead of MPI_Send.
+
+\subsection options_model_smpi_test smpi/test: Inject constant times for calls to MPI_Test
+
+\b Default value: 0.0001
+
+By setting this option, you can control the amount of time a process sleeps
+when MPI_Test() is called; this is important, because SimGrid normally only
+advances the time while communication is happening and thus,
+MPI_Test will not add to the time, resulting in a deadlock if used as a
+break-condition.
+
+Here is an example:
+
+\code{.unparsed}
+ while(!flag) {
+ MPI_Test(request, flag, status);
+ ...
+ }
+\endcode
+
+\note
+ Internally, in order to speed up execution, we use a counter to keep track
+ on how often we already checked if the handle is now valid or not. Hence, we
+ actually use counter*SLEEP_TIME, that is, the time MPI_Test() causes the process
+ to sleep increases linearly with the number of previously failed testk.
+
+
+\subsection options_model_smpi_use_shared_malloc smpi/use-shared-malloc: Factorize malloc()s
+
+\b Default: 1
+
+SMPI can use shared memory by calling shm_* functions; this might speed up the simulation.
+This opens or creates a new POSIX shared memory object, kept in RAM, in /dev/shm.
+
+If you want to disable this behavior, set the value to 0.
+
+\subsection options_model_smpi_wtime smpi/wtime: Inject constant times for calls to MPI_Wtime
+
+\b Default value: 0
+
+By setting this option, you can control the amount of time a process sleeps
+when MPI_Wtime() is called; this is important, because SimGrid normally only
+advances the time while communication is happening and thus,
+MPI_Wtime will not add to the time, resulting in a deadlock if used as a
+break-condition.
+
+Here is an example:
+
+\code{.unparsed}
+ while(MPI_Wtime() < some_time_bound) {
+ ...
+ }
+\endcode
+
+If the time is never advanced, this loop will clearly never end as MPI_Wtime()
+always returns the same value. Hence, pass a (small) value to the smpi/wtime
+option to force a call to MPI_Wtime to advance the time as well.
+
\section options_generic Configuring other aspects of SimGrid
+\subsection options_generic_clean_atexit Cleanup before termination
+
+The C / C++ standard contains a function called \b [atexit](http://www.cplusplus.com/reference/cstdlib/atexit/).
+atexit registers callbacks, which are called just before the program terminates.
+
+By setting the configuration option clean-atexit to 1 (true), a callback
+is registered and will clean up some variables and terminate/cleanup the tracing.
+
+TODO: Add when this should be used.
+
\subsection options_generic_path XML file inclusion path
It is possible to specify a list of directories to search into for the
\subsection options_generic_exit Behavior on Ctrl-C
By default, when Ctrl-C is pressed, the status of all existing
-simulated processes is displayed. This is very useful to debug your
+simulated processes is displayed before exiting the simulation. This is very useful to debug your
code, but it can reveal troublesome in some cases (such as when the
amount of processes becomes really big). This behavior is disabled
when \b verbose-exit is set to 0 (it is to 1 by default).
+\subsection options_exception_cutpath Truncate local path from exception backtrace
+
+<b>This configuration option is an internal option and should normally not be used
+by the user.</b> It is used to remove the path from the backtrace
+shown when an exception is thrown; if we didn't remove this part, the tests
+testing the exception parts of simgrid would fail on most machines, as we are
+currently comparing output. Clearly, the path used on different machines are almost
+guaranteed to be different and hence, the output would
+mismatch, causing the test to fail.
\section options_log Logging Configuration
A stack guard page is usually used which prevents the stack from
overflowing on other parts of the memory. However this might have a
performance impact if a huge number of processes is created. The
-option \b contexts:guard_size is the number of stack guard pages
+option \b contexts:guard-size is the number of stack guard pages
used. By setting it to 0, no guard pages will be used: in this case,
-you should avoid using small stacks (\b stack_size) as the stack will
+you should avoid using small stacks (\b stack-size) as the stack will
silently overflow on other parts of the memory.
-\section options_index Index of all existing configuration items
+\section options_index Index of all existing configuration options
+
+\note
+ Almost all options are defined in <i>src/simgrid/sg_config.c</i>. You may
+ want to check this file, too, but this index should be somewhat complete
+ for the moment (May 2015).
+
+\note
+ \b Please \b note: You can also pass the command-line option "--help" and
+ "--help-cfg" to an executable that uses simgrid.
+
+- \c clean-atexit: \ref options_generic_clean_atexit
- \c contexts/factory: \ref options_virt_factory
+- \c contexts/guard-size: \ref options_virt_parallel
- \c contexts/nthreads: \ref options_virt_parallel
- \c contexts/parallel_threshold: \ref options_virt_parallel
-- \c contexts/stack_size: \ref options_virt_stacksize
+- \c contexts/stack-size: \ref options_virt_stacksize
- \c contexts/synchro: \ref options_virt_parallel
-- \c contexts/guard_size: \ref options_virt_parallel
-- \c cpu/maxmin_selective_update: \ref options_model_optim
+- \c cpu/maxmin-selective-update: \ref options_model_optim
- \c cpu/model: \ref options_model_select
- \c cpu/optim: \ref options_model_optim
-- \c gtnets/jitter: \ref options_pls
-- \c gtnets/jitter_seed: \ref options_pls
+- \c exception/cutpath: \ref options_exception_cutpath
+
+- \c host/model: \ref options_model_select
- \c maxmin/precision: \ref options_model_precision
+- \c msg/debug-multiple-use: \ref options_msg_debug_multiple_use
+
- \c model-check: \ref options_modelchecking
-- \c model-check/property: \ref options_modelchecking_liveness
- \c model-check/checkpoint: \ref options_modelchecking_steps
-- \c model-check/reduce: \ref options_modelchecking_reduction
-
-- \c network/bandwidth_factor: \ref options_model_network_coefs
+- \c model-check/communications-determinism: \ref options_modelchecking_comm_determinism
+- \c model-check/dot-output: \ref options_modelchecking_dot_output
+- \c model-check/hash: \ref options_modelchecking_hash
+- \c model-check/property: \ref options_modelchecking_liveness
+- \c model-check/max-depth: \ref options_modelchecking_max_depth
+- \c model-check/record: \ref options_modelchecking_recordreplay
+- \c model-check/reduction: \ref options_modelchecking_reduction
+- \c model-check/replay: \ref options_modelchecking_recordreplay
+- \c model-check/send-determinism: \ref options_modelchecking_comm_determinism
+- \c model-check/sparse-checkpoint: \ref options_modelchecking_sparse_checkpoint
+- \c model-check/termination: \ref options_modelchecking_termination
+- \c model-check/timeout: \ref options_modelchecking_timeout
+- \c model-check/visited: \ref options_modelchecking_visited
+
+- \c network/bandwidth-factor: \ref options_model_network_coefs
- \c network/coordinates: \ref options_model_network_coord
- \c network/crosstraffic: \ref options_model_network_crosstraffic
-- \c network/latency_factor: \ref options_model_network_coefs
-- \c network/maxmin_selective_update: \ref options_model_optim
+- \c network/latency-factor: \ref options_model_network_coefs
+- \c network/maxmin-selective-update: \ref options_model_optim
- \c network/model: \ref options_model_select
- \c network/optim: \ref options_model_optim
- \c network/sender_gap: \ref options_model_network_sendergap
-- \c network/TCP_gamma: \ref options_model_network_gamma
-- \c network/weight_S: \ref options_model_network_coefs
+- \c network/TCP-gamma: \ref options_model_network_gamma
+- \c network/weight-S: \ref options_model_network_coefs
- \c ns3/TcpModel: \ref options_pls
+- \c path: \ref options_generic_path
+- \c plugin: \ref options_generic_plugin
- \c surf/nthreads: \ref options_model_nthreads
+- \c surf/precision: \ref options_model_precision
+
+- \c <b>For collective operations of SMPI, please refer to Section \ref options_index_smpi_coll</b>
+- \c smpi/async-small-thresh: \ref options_model_network_asyncsend
+- \c smpi/bw-factor: \ref options_model_smpi_bw_factor
+- \c smpi/coll-selector: \ref options_model_smpi_collectives
+- \c smpi/cpu-threshold: \ref options_smpi_bench
+- \c smpi/display-timing: \ref options_smpi_timing
+- \c smpi/lat-factor: \ref options_model_smpi_lat_factor
+- \c smpi/IB-penalty-factors: \ref options_model_network_coefs
+- \c smpi/iprobe: \ref options_model_smpi_iprobe
+- \c smpi/ois: \ref options_model_smpi_ois
+- \c smpi/or: \ref options_model_smpi_or
+- \c smpi/os: \ref options_model_smpi_os
+- \c smpi/privatize-global-variables: \ref options_smpi_global
+- \c smpi/running-power: \ref options_smpi_bench
+- \c smpi/send-is-detached-thresh: \ref options_model_smpi_detached
+- \c smpi/simulate-computation: \ref options_smpi_bench
+- \c smpi/test: \ref options_model_smpi_test
+- \c smpi/use-shared-malloc: \ref options_model_smpi_use_shared_malloc
+- \c smpi/wtime: \ref options_model_smpi_wtime
+
+- \c <b>Tracing configuration options can be found in Section \ref tracing_tracing_options</b>.
+
+- \c storage/model: \ref options_storage_model
+- \c verbose-exit: \ref options_generic_exit
-- \c smpi/simulation_computation: \ref options_smpi_bench
-- \c smpi/running_power: \ref options_smpi_bench
-- \c smpi/display_timing: \ref options_smpi_timing
-- \c smpi/cpu_threshold: \ref options_smpi_bench
-- \c smpi/async_small_thres: \ref options_model_network_asyncsend
-- \c smpi/send_is_detached: \ref options_model_smpi_detached
-- \c smpi/coll_selector: \ref options_model_smpi_collectives
-- \c smpi/privatize_global_variables: \ref options_smpi_global
+- \c vm/model: \ref options_vm_model
-- \c path: \ref options_generic_path
-- \c verbose-exit: \ref options_generic_exit
+\subsection options_index_smpi_coll Index of SMPI collective algorithms options
-- \c workstation/model: \ref options_model_select
+TODO: All available collective algorithms will be made available via the ``smpirun --help-coll`` command.
*/
