settings (accurate modeling of slow start with correction factors on
three intervals: < 1KiB, < 64 KiB, >= 64 KiB). See also \ref
options_model_network_coefs "this section" for more info.
+ - \b IB: Realistic network model specifically tailored for HPC
+ settings with InfiniBand networks (accurate modeling contention
+ behavior, based on the model explained in
+ http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf).
+ See also \ref options_model_network_coefs "this section" for more info.
- \b CM02: Legacy network analytic model (Very similar to LV08, but
without corrective factors. The timings of small messages are thus
poorly modeled)
themselves corrected by constant values depending on the size of the
exchange. Again, only hardcore experts should bother about this fact.
+InfiniBand network behavior can be modeled through 3 parameters, as explained in
+http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf . These
+factors can be changed through option smpi/IB_penalty_factors:"βe;βs;γs". By
+default SMPI uses factors computed one Stampede cluster from TACC, with optimal
+deployment of processes on nodes.
+
\subsubsection options_model_network_crosstraffic Simulating cross-traffic
As of SimGrid v3.7, cross-traffic effects can be taken into account in
so that you don't have to give <tt>--cfg=model-check:1</tt> in
addition.
+\subsection options_mc_perf Performance considerations for the model checker
+
+The size of the stacks can have a huge impact on the memory
+consumption when using model-checking. Currently each snapshot, will
+save a copy of the whole stack and not only of the part which is
+really meaningful: you should expect the contribution of the memory
+consumption of the snapshots to be \f$ \mbox{number of processes}
+\times \mbox{stack size} \times \mbox{number of states} \f$.
+
+However, when compiled against the model checker, the stacks are not
+protected with guards: if the stack size is too small for your
+application, the stack will silently overflow on other parts of the
+memory.
+
\section options_virt Configuring the User Process Virtualization
\subsection options_virt_factory Selecting the virtualization factory
The only reason to change this setting is when the debugging tools get
fooled by the optimized context factories. Threads are the most
-debugging-friendly contextes.
+debugging-friendly contextes, as they allow to set breakpoints anywhere with gdb
+ and visualize backtraces for all processes, in order to debug concurrency issues.
+Valgrind is also more comfortable with threads, but it should be usable with all factories.
\subsection options_virt_stacksize Adapting the used stack size
rather disturbing: this leads to stack overflow (overwriting other
stacks), leading to segfaults with corrupted stack traces.
-If you want to push the scalability limits of your code, you really
+If you want to push the scalability limits of your code, you might
want to reduce the \b contexts/stack_size item. Its default value
-is 8 (in MiB), while our Chord simulation works with stacks as small
+is 8192 (in KiB), while our Chord simulation works with stacks as small
as 16 KiB, for example. For the thread factory, the default value
is the one of the system, if it is too large/small, it has to be set
with this parameter.
+The operating system should only allocate memory for the pages of the
+stack which are actually used and you might not need to use this in
+most cases. However, this setting is very important when using the
+model checker (see \ref options_mc_perf).
+
+In some cases, no stack guard page is used and the stack will silently
+overflow on other parts of the memory if the stack size is too small
+for your application. This happens :
+
+- on Windows systems;
+- when the model checker is enabled;
+- when stack guard pages are explicitely disabled (see \ref options_perf_guard_size).
+
\subsection options_virt_parallel Running user code in parallel
Parallel execution of the user code is only considered stable in
simulation kernel (default value: 1e-6). Please note that in some
circonstances, this optimization can hinder the simulation accuracy.
+ In some cases, however, one may wish to disable simulation of
+application computation. This is the case when SMPI is used not to
+simulate an MPI applications, but instead an MPI code that performs
+"live replay" of another MPI app (e.g., ScalaTrace's replay tool,
+various on-line simulators that run an app at scale). In this case the
+computation of the replay/simulation logic should not be simulated by
+SMPI. Instead, the replay tool or on-line simulator will issue
+"computation events", which correspond to the actual MPI simulation
+being replayed/simulated. At the moment, these computation events can
+be simulated using SMPI by calling internal smpi_execute*() functions.
+
+To disable the benchmarking/simulation of computation in the simulated
+application, the variable \b
+smpi/simulation_computation should be set to no
+
\subsection options_smpi_timing Reporting simulation time
Most of the time, you run MPI code through SMPI to compute the time it
Simulation time: 1e3 seconds.
\endverbatim
-\subsection options_model_smpi_detached Simulating MPI detached send
+\subsection options_smpi_global Automatic privatization of global variables
+
+MPI executables are meant to be executed in separated processes, but SMPI is
+executed in only one process. Global variables from executables will be placed
+in the same memory zone and shared between processes, causing hard to find bugs.
+To avoid this, several options are possible :
+ - Manual edition of the code, for example to add __thread keyword before data
+ declaration, which allows the resulting code to work with SMPI, but only
+ if the thread factory (see \ref options_virt_factory) is used, as global
+ variables are then placed in the TLS (thread local storage) segment.
+ - Source-to-source transformation, to add a level of indirection
+ to the global variables. SMPI does this for F77 codes compiled with smpiff,
+ and used to provide coccinelle scripts for C codes, which are not functional anymore.
+ - Compilation pass, to have the compiler automatically put the data in
+ an adapted zone.
+ - Runtime automatic switching of the data segments. SMPI stores a copy of
+ each global data segment for each process, and at each context switch replaces
+ the actual data with its copy from the right process. This mechanism uses mmap,
+ and is for now limited to systems supporting this functionnality (all Linux
+ and some BSD should be compatible).
+ Another limitation is that SMPI only accounts for global variables defined in
+ the executable. If the processes use external global variables from dynamic
+ libraries, they won't be switched correctly. To avoid this, using static
+ linking is advised (but not with the simgrid library, to avoid replicating
+ its own global variables).
+
+ To use this runtime automatic switching, the variable \b smpi/privatize_global_variables
+ should be set to yes
+
-(this configuration item is experimental and may change or disapear)
+
+\subsection options_model_smpi_detached Simulating MPI detached send
This threshold specifies the size in bytes under which the send will return
immediately. This is different from the threshold detailed in \ref options_model_network_asyncsend
uses naive version of collective operations). Each collective operation can be manually selected with a
\b smpi/collective_name:algo_name. Available algorithms are listed in \ref SMPI_collective_algorithms .
+
\section options_generic Configuring other aspects of SimGrid
\subsection options_generic_path XML file inclusion path
\subsection options_generic_exit Behavior on Ctrl-C
By default, when Ctrl-C is pressed, the status of all existing
-simulated processes is displayed. This is very useful to debug your
+simulated processes is displayed before exiting the simulation. This is very useful to debug your
code, but it can reveal troublesome in some cases (such as when the
amount of processes becomes really big). This behavior is disabled
when \b verbose-exit is set to 0 (it is to 1 by default).
It can be done by using XBT. Go to \ref XBT_log for more details.
+\section options_perf Performance optimizations
+
+\subsection options_perf_context Context factory
+
+In order to achieve higher performance, you might want to use the raw
+context factory which avoids any system call when switching between
+tasks. If it is not possible you might use ucontext instead.
+
+\subsection options_perf_guard_size Disabling stack guard pages
+
+A stack guard page is usually used which prevents the stack from
+overflowing on other parts of the memory. However this might have a
+performance impact if a huge number of processes is created. The
+option \b contexts:guard_size is the number of stack guard pages
+used. By setting it to 0, no guard pages will be used: in this case,
+you should avoid using small stacks (\b stack_size) as the stack will
+silently overflow on other parts of the memory.
\section options_index Index of all existing configuration items
- \c contexts/parallel_threshold: \ref options_virt_parallel
- \c contexts/stack_size: \ref options_virt_stacksize
- \c contexts/synchro: \ref options_virt_parallel
+- \c contexts/guard_size: \ref options_virt_parallel
- \c cpu/maxmin_selective_update: \ref options_model_optim
- \c cpu/model: \ref options_model_select
- \c surf/nthreads: \ref options_model_nthreads
+- \c smpi/simulation_computation: \ref options_smpi_bench
- \c smpi/running_power: \ref options_smpi_bench
- \c smpi/display_timing: \ref options_smpi_timing
- \c smpi/cpu_threshold: \ref options_smpi_bench
- \c smpi/async_small_thres: \ref options_model_network_asyncsend
- \c smpi/send_is_detached: \ref options_model_smpi_detached
- \c smpi/coll_selector: \ref options_model_smpi_collectives
+- \c smpi/privatize_global_variables: \ref options_smpi_global
- \c path: \ref options_generic_path
- \c verbose-exit: \ref options_generic_exit
