the --help configuration flag to your program. If some of the options
are not documented on this page, this is a bug that you should please
report so that we can fix it. Note that some of the options presented
-here may not be available in your simulators, depending on the
+here may not be available in your simulators, depending on the
@ref install_src_config "compile-time options" that you used.
\section options_using Passing configuration options to the simulators
within that tag, you can pass one or several \c \<prop\> tags to specify
the configuration to use. For example, setting \c Item to \c Value
can be done by adding the following to the beginning of your platform
-file:
+file:
\verbatim
<config>
<prop id="Item" value="Value"/>
directly as follows. Check the \ref XBT_config "relevant page" for
details on all the functions you can use in this context, \c
_sg_cfg_set being the only configuration set currently used in
-SimGrid.
+SimGrid.
@code
#include <xbt/config.h>
- \b cpu/model: specify the used CPU model
- \b workstation/model: specify the used workstation model
-As of writting, the accepted network models are the following. Over
+%As of writing, the following network models are accepted. Over
the time new models can be added, and some experimental models can be
removed; check the values on your simulators for an uptodate
information. Note that the CM02 model is described in the research report
settings (accurate modeling of slow start with correction factors on
three intervals: < 1KiB, < 64 KiB, >= 64 KiB). See also \ref
options_model_network_coefs "this section" for more info.
+ - \b IB: Realistic network model specifically tailored for HPC
+ settings with InfiniBand networks (accurate modeling contention
+ behavior, based on the model explained in
+ http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf).
+ See also \ref options_model_network_coefs "this section" for more info.
- \b CM02: Legacy network analytic model (Very similar to LV08, but
without corrective factors. The timings of small messages are thus
poorly modeled)
cat /proc/sys/net/ipv4/tcp_wmem # gives the receiver window
\endverbatim
-\subsubsection options_model_network_coefs Corrective simulation factors
+\subsubsection options_model_network_coefs Correcting important network parameters
+
+SimGrid can take network irregularities such as a slow startup or
+changing behavior depending on the message size into account.
+You should not change these values unless you really know what you're doing.
-These factors allow to betterly take the slow start into account.
The corresponding values were computed through data fitting one the
-timings of packet-level simulators. You should not change these values
-unless you are really certain of what you are doing. See
+timings of packet-level simulators.
+
+See
<a href="http://mescal.imag.fr/membres/arnaud.legrand/articles/simutools09.pdf">Accuracy Study and Improvement of Network Simulation in the SimGrid Framework</a>
-for more informations about these coeficients.
+for more information about these parameters.
-If you are using the SMPI model, these correction coeficients are
+If you are using the SMPI model, these correction coefficients are
themselves corrected by constant values depending on the size of the
exchange. Again, only hardcore experts should bother about this fact.
+InfiniBand network behavior can be modeled through 3 parameters, as explained in
+<a href="http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf">this PhD thesis</a>.
+These factors can be changed through the following option:
+
+\verbatim
+smpi/IB_penalty_factors:"βe;βs;γs"
+\endverbatim
+
+By default SMPI uses factors computed on the Stampede Supercomputer at TACC, with optimal
+deployment of processes on nodes.
+
\subsubsection options_model_network_crosstraffic Simulating cross-traffic
-As of SimGrid v3.7, cross-traffic effects can be taken into account in
+%As of SimGrid v3.7, cross-traffic effects can be taken into account in
analytical simulations. It means that ongoing and incoming
communication flows are treated independently. In addition, the LV08
model adds 0.05 of usage on the opposite direction for each new
(this configuration item is experimental and may change or disapear)
-It is possible to specify that messages below a certain size will be sent
-as soon as the call to MPI_Send is issued, without waiting for the
-correspondant receive. This threshold can be configured through the
-\b smpi/async_small_thres item. The default value is 0. This behavior can also be
-manually set for MSG mailboxes, by setting the receiving mode of the mailbox
-with a call to \ref MSG_mailbox_set_async . For MSG, all messages sent to this
-mailbox will have this behavior, so consider using two mailboxes if needed.
+It is possible to specify that messages below a certain size will be sent
+as soon as the call to MPI_Send is issued, without waiting for the
+correspondant receive. This threshold can be configured through the
+\b smpi/async_small_thres item. The default value is 0. This behavior can also be
+manually set for MSG mailboxes, by setting the receiving mode of the mailbox
+with a call to \ref MSG_mailbox_set_async . For MSG, all messages sent to this
+mailbox will have this behavior, so consider using two mailboxes if needed.
-This value needs to be smaller than or equals to the threshold set at
-\ref options_model_smpi_detached , because asynchronous messages are
+This value needs to be smaller than or equals to the threshold set at
+\ref options_model_smpi_detached , because asynchronous messages are
meant to be detached as well.
\subsubsection options_pls Configuring packet-level pseudo-models
If you want to specify liveness properties (beware, that's
experimental), you have to pass them on the command line, specifying
-the name of the file containing the property, as formated by the
+the name of the file containing the property, as formatted by the
ltl2ba program.
\verbatim
The only reason to change this setting is when the debugging tools get
fooled by the optimized context factories. Threads are the most
-debugging-friendly contextes.
+debugging-friendly contextes, as they allow to set breakpoints anywhere with gdb
+ and visualize backtraces for all processes, in order to debug concurrency issues.
+Valgrind is also more comfortable with threads, but it should be usable with all factories.
\subsection options_virt_stacksize Adapting the used stack size
If you want to push the scalability limits of your code, you might
want to reduce the \b contexts/stack_size item. Its default value
is 8192 (in KiB), while our Chord simulation works with stacks as small
-as 16 KiB, for example. For the thread factory, the default value
-is the one of the system, if it is too large/small, it has to be set
+as 16 KiB, for example. For the thread factory, the default value
+is the one of the system, if it is too large/small, it has to be set
with this parameter.
The operating system should only allocate memory for the pages of the
In some cases, no stack guard page is used and the stack will silently
overflow on other parts of the memory if the stack size is too small
-for your application. This happens :
+for your application. This happens :
- on Windows systems;
- when the model checker is enabled;
request to execute the user code in parallel. Several threads are
launched, each of them handling as much user contexts at each run. To
actiave this, set the \b contexts/nthreads item to the amount of
-cores that you have in your computer (or lower than 1 to have
+cores that you have in your computer (or lower than 1 to have
the amount of cores auto-detected).
Even if you asked several worker threads using the previous option,
These are uneasy to see since the code is usually launched through the
\c smiprun script directly.
-\subsection options_smpi_bench Automatic benchmarking of SMPI code
+\subsection options_smpi_bench smpi/bench: Automatic benchmarking of SMPI code
In SMPI, the sequential code is automatically benchmarked, and these
computations are automatically reported to the simulator. That is to
application, the variable \b
smpi/simulation_computation should be set to no
-\subsection options_smpi_timing Reporting simulation time
+\subsection options_model_smpi_bw_factor smpi/bw_factor: Bandwidth factors
+
+The possible throughput of network links is often dependent on the
+message sizes, as protocols may adapt to different message sizes. With
+this option, a series of message sizes and factors are given, helping
+the simulation to be more realistic. For instance, the current
+default value is
+
+\verbatim
+65472:0.940694;15424:0.697866;9376:0.58729;5776:1.08739;3484:0.77493;1426:0.608902;732:0.341987;257:0.338112;0:0.812084
+\endverbatim
+
+So, messages with size 65472 and more will get a total of MAX_BANDWIDTH*0.940694,
+messages of size 15424 to 65471 will get MAX_BANDWIDTH*0.697866 and so on.
+Here, MAX_BANDWIDTH denotes the bandwidth of the link.
+
+\subsection options_smpi_timing smpi/display_timing: Reporting simulation time
Most of the time, you run MPI code through SMPI to compute the time it
would take to run it on a platform that you don't have. But since the
Simulation time: 1e3 seconds.
\endverbatim
-\subsection options_smpi_global Automatic privatization of global variables
+\subsection options_model_smpi_lat_factor smpi/lat_factor: Latency factors
+
+The motivation and syntax for this option is identical to the motivation/syntax
+of smpi/bw_factor, see \ref options_model_smpi_bw_factor for details.
+
+There is an important difference, though: While smpi/bw_factor \a reduces the
+actual bandwidth (i.e., values between 0 and 1 are valid), latency factors
+increase the latency, i.e., values larger than or equal to 1 are valid here.
+
+This is the default value:
+
+\verbatim
+65472:11.6436;15424:3.48845;9376:2.59299;5776:2.18796;3484:1.88101;1426:1.61075;732:1.9503;257:1.95341;0:2.01467
+\endverbatim
+
+\subsection options_smpi_global smpi/privatize_global_variables: Automatic privatization of global variables
-MPI executables are meant to be executed in separated processes, but SMPI is
-executed in only one process. Global variables from executables will be placed
+MPI executables are meant to be executed in separated processes, but SMPI is
+executed in only one process. Global variables from executables will be placed
in the same memory zone and shared between processes, causing hard to find bugs.
To avoid this, several options are possible :
- Manual edition of the code, for example to add __thread keyword before data
- declaration, which allows the resulting code to work with SMPI, but only
- if the thread factory (see \ref options_virt_factory) is used, as global
- variables are then placed in the TLS (thread local storage) segment.
- - Source-to-source transformation, to add a level of indirection
- to the global variables. SMPI does this for F77 codes compiled with smpiff,
+ declaration, which allows the resulting code to work with SMPI, but only
+ if the thread factory (see \ref options_virt_factory) is used, as global
+ variables are then placed in the TLS (thread local storage) segment.
+ - Source-to-source transformation, to add a level of indirection
+ to the global variables. SMPI does this for F77 codes compiled with smpiff,
and used to provide coccinelle scripts for C codes, which are not functional anymore.
- - Compilation pass, to have the compiler automatically put the data in
- an adapted zone.
- - Runtime automatic switching of the data segments. SMPI stores a copy of
- each global data segment for each process, and at each context switch replaces
+ - Compilation pass, to have the compiler automatically put the data in
+ an adapted zone.
+ - Runtime automatic switching of the data segments. SMPI stores a copy of
+ each global data segment for each process, and at each context switch replaces
the actual data with its copy from the right process. This mechanism uses mmap,
- and is for now limited to systems supporting this functionnality (all Linux
+ and is for now limited to systems supporting this functionnality (all Linux
and some BSD should be compatible).
Another limitation is that SMPI only accounts for global variables defined in
- the executable. If the processes use external global variables from dynamic
- libraries, they won't be switched correctly. To avoid this, using static
- linking is advised (but not with the simgrid library, to avoid replicating
- its own global variables).
+ the executable. If the processes use external global variables from dynamic
+ libraries, they won't be switched correctly. To avoid this, using static
+ linking is advised (but not with the simgrid library, to avoid replicating
+ its own global variables).
To use this runtime automatic switching, the variable \b smpi/privatize_global_variables
should be set to yes
\subsection options_model_smpi_detached Simulating MPI detached send
-This threshold specifies the size in bytes under which the send will return
+This threshold specifies the size in bytes under which the send will return
immediately. This is different from the threshold detailed in \ref options_model_network_asyncsend
because the message is not effectively sent when the send is posted. SMPI still waits for the
-correspondant receive to be posted to perform the communication operation. This threshold can be set
+correspondant receive to be posted to perform the communication operation. This threshold can be set
by changing the \b smpi/send_is_detached item. The default value is 65536.
\subsection options_model_smpi_collectives Simulating MPI collective algorithms
-SMPI implements more than 100 different algorithms for MPI collective communication, to accurately
+SMPI implements more than 100 different algorithms for MPI collective communication, to accurately
simulate the behavior of most of the existing MPI libraries. The \b smpi/coll_selector item can be used
to use the decision logic of either OpenMPI or MPICH libraries (values: ompi or mpich, by default SMPI
-uses naive version of collective operations). Each collective operation can be manually selected with a
+uses naive version of collective operations). Each collective operation can be manually selected with a
\b smpi/collective_name:algo_name. Available algorithms are listed in \ref SMPI_collective_algorithms .
+\subsection options_model_smpi_iprobe smpi/iprobe: Inject constant times for calls to MPI_Iprobe
+
+\b Default value: 0.0001
+
+The behavior and motivation for this configuration option is identical with \a smpi/test, see
+Section \ref options_model_smpi_test for details.
+
+\subsection options_model_smpi_test smpi/test: Inject constant times for calls to MPI_Test
+
+\b Default value: 0.0001
+
+By setting this option, you can control the amount of time a process sleeps
+when MPI_Test() is called; this is important, because SimGrid normally only
+advances the time while communication is happening and thus,
+MPI_Test will not add to the time, resulting in a deadlock if used as a
+break-condition.
+
+Here is an example:
+
+\code{.unparsed}
+ while(!flag) {
+ MPI_Test(request, flag, status);
+ ...
+ }
+\endcode
+
+\note
+ Internally, in order to speed up execution, we use a counter to keep track
+ on how often we already checked if the handle is now valid or not. Hence, we
+ actually use counter*SLEEP_TIME, that is, the time MPI_Test() causes the process
+ to sleep increases linearly with the number of previously failed testk.
+
+
+\subsection options_model_smpi_wtime smpi/wtime: Inject constant times for calls to MPI_Wtime
+
+\b Default value: 0
+
+By setting this option, you can control the amount of time a process sleeps
+when MPI_Wtime() is called; this is important, because SimGrid normally only
+advances the time while communication is happening and thus,
+MPI_Wtime will not add to the time, resulting in a deadlock if used as a
+break-condition.
+
+Here is an example:
+
+\code{.unparsed}
+ while(MPI_Wtime() < some_time_bound) {
+ ...
+ }
+\endcode
+
+If the time is never advanced, this loop will clearly never end as MPI_Wtime()
+always returns the same value. Hence, pass a (small) value to the smpi/wtime
+option to force a call to MPI_Wtime to advance the time as well.
+
\section options_generic Configuring other aspects of SimGrid
+\subsection options_generic_clean_atexit Cleanup before termination
+
+The C / C++ standard contains a function called \b [atexit](http://www.cplusplus.com/reference/cstdlib/atexit/).
+atexit registers callbacks, which are called just before the program terminates.
+
+By setting the configuration option clean_atexit to 1 (true), a callback
+is registered and will clean up some variables and terminate/cleanup the tracing.
+
+TODO: Add when this should be used.
+
\subsection options_generic_path XML file inclusion path
It is possible to specify a list of directories to search into for the
amount of processes becomes really big). This behavior is disabled
when \b verbose-exit is set to 0 (it is to 1 by default).
+\subsection options_exception_cutpath Truncate local path from exception backtrace
+
+<b>This configuration option is an internal option and should normally not be used
+by the user.</b> It is used to remove the path from the backtrace
+shown when an exception is thrown; if we didn't remove this part, the tests
+testing the exception parts of simgrid would fail on most machines, as we are
+currently comparing output. Clearly, the path used on different machines are almost
+guaranteed to be different and hence, the output would
+mismatch, causing the test to fail.
\section options_log Logging Configuration
you should avoid using small stacks (\b stack_size) as the stack will
silently overflow on other parts of the memory.
-\section options_index Index of all existing configuration items
+\section options_index Index of all existing configuration options
+
+\note
+ Almost all options are defined in <i>src/simgrid/sg_config.c</i>. You may
+ want to check this file, too, but this index should be somewhat complete
+ for the moment (May 2015).
+
+\note
+ \b Please \b note: You can also pass the command-line option "\b --help" and
+ "--help-cfg" to an executable that uses simgrid.
+
+- \c clean_atexit: \ref options_generic_clean_atexit
- \c contexts/factory: \ref options_virt_factory
+- \c contexts/guard_size: \ref options_virt_parallel
- \c contexts/nthreads: \ref options_virt_parallel
- \c contexts/parallel_threshold: \ref options_virt_parallel
- \c contexts/stack_size: \ref options_virt_stacksize
- \c contexts/synchro: \ref options_virt_parallel
-- \c contexts/guard_size: \ref options_virt_parallel
- \c cpu/maxmin_selective_update: \ref options_model_optim
- \c cpu/model: \ref options_model_select
- \c cpu/optim: \ref options_model_optim
+- \c exception/cutpath: \ref options_exception_cutpath
+
- \c gtnets/jitter: \ref options_pls
- \c gtnets/jitter_seed: \ref options_pls
- \c maxmin/precision: \ref options_model_precision
+- \c msg/debug_multiple_use: \ref options_msg_debug_multiple_use
+
- \c model-check: \ref options_modelchecking
-- \c model-check/property: \ref options_modelchecking_liveness
- \c model-check/checkpoint: \ref options_modelchecking_steps
-- \c model-check/reduce: \ref options_modelchecking_reduction
+- \c model-check/communications_determinism: \ref options_modelchecking_comm_determinism
+- \c model-check/dot_output: \ref options_modelchecking_dot_output
+- \c model-check/hash: \ref options_modelchecking_hash
+- \c model-check/property: \ref options_modelchecking_liveness
+- \c model-check/max_depth: \ref options_modelchecking_max_depth
+- \c model-check/record: \ref options_modelchecking_record
+- \c model-check/reduction: \ref options_modelchecking_reduction
+- \c model-check/replay: \ref options_modelchecking_replay
+- \c model-check/send_determinism: \ref options_modelchecking_sparse_checkpoint
+- \c model-check/snapshot_fds: \ref options_modelchecking_snapshot_fds
+- \c model-check/sparse-checkpoint: \ref options_modelchecking_sparse_checkpoint
+- \c model-check/termination: \ref options_modelchecking_termination
+- \c model-check/timeout: \ref options_modelchecking_timeout
+- \c model-check/visited: \ref options_modelchecking_visited
- \c network/bandwidth_factor: \ref options_model_network_coefs
- \c network/coordinates: \ref options_model_network_coord
- \c ns3/TcpModel: \ref options_pls
- \c surf/nthreads: \ref options_model_nthreads
+- \c surf/precision: \ref options_model_precision
-- \c smpi/simulation_computation: \ref options_smpi_bench
-- \c smpi/running_power: \ref options_smpi_bench
-- \c smpi/display_timing: \ref options_smpi_timing
-- \c smpi/cpu_threshold: \ref options_smpi_bench
+- \c <b>For collective operations of SMPI, please refer to Section \ref options_index_smpi_coll</b>
- \c smpi/async_small_thres: \ref options_model_network_asyncsend
-- \c smpi/send_is_detached: \ref options_model_smpi_detached
+- \c smpi/bw_factor: \ref options_model_smpi_bw_factor
- \c smpi/coll_selector: \ref options_model_smpi_collectives
+- \c smpi/cpu_threshold: \ref options_smpi_bench
+- \c smpi/display_timing: \ref options_smpi_timing
+- \c smpi/lat_factor: \ref options_model_smpi_lat_factor
+- \c smpi/IB_penalty_factors: \ref options_model_network_coefs
+- \c smpi/iprobe: \ref options_model_smpi_iprobe
+- \c smpi/ois: \ref options_model_smpi_ois
+- \c smpi/or: \ref options_model_smpi_or
+- \c smpi/os: \ref options_model_smpi_os
- \c smpi/privatize_global_variables: \ref options_smpi_global
+- \c smpi/running_power: \ref options_smpi_bench
+- \c smpi/send_is_detached_thresh: \ref options_model_smpi_detached
+- \c smpi/simulation_computation: \ref options_smpi_bench
+- \c smpi/test: \ref options_model_smpi_test
+- \c smpi/use_shared_malloc: \ref options_model_smpi_use_shared_malloc
+- \c smpi/wtime: \ref options_model_smpi_wtime
+
+- \c <b>Tracing configuration options can be found in Section \ref tracing_tracing_options</b>.
+- \c storage/model: \ref options_storage_model
- \c path: \ref options_generic_path
+- \c plugin: \ref options_generic_plugin
- \c verbose-exit: \ref options_generic_exit
+- \c vm_workstation/model: \ref options_vm_workstation_model
- \c workstation/model: \ref options_model_select
+\subsection options_index_smpi_coll Index of SMPI collective algorithms options
+- \c smpi/allgather: \ref options_model_smpi_coll_allgather
+- \c smpi/allgatherv: \ref options_model_smpi_coll_allgatherv
+- \c smpi/allreduce: \ref options_model_smpi_coll_allreduce
+- \c smpi/alltoall: \ref options_model_smpi_coll_alltoall
+- \c smpi/alltoallv: \ref options_model_smpi_coll_alltoallv
+- \c smpi/barrier: \ref options_model_smpi_coll_barrier
+- \c smpi/bcast: \ref options_model_smpi_coll_bcast
+- \c smpi/gather: \ref options_model_smpi_coll_gather
+- \c smpi/reduce: \ref options_model_smpi_coll_reduce
+- \c smpi/reduce_scatter: \ref options_model_smpi_coll_reduce_scatter
+- \c smpi/scatter: \ref options_model_smpi_coll_scatter
+
*/
