- **surf/precision:** :ref:`cfg=surf/precision`
- **For collective operations of SMPI,** please refer to Section :ref:`cfg=smpi/coll-selector`
+- **smpi/auto-shared-malloc-thresh:** :ref:`cfg=smpi/auto-shared-malloc-thresh`
- **smpi/async-small-thresh:** :ref:`cfg=smpi/async-small-thresh`
- **smpi/buffering:** :ref:`cfg=smpi/buffering`
- **smpi/bw-factor:** :ref:`cfg=smpi/bw-factor`
- **smpi/coll-selector:** :ref:`cfg=smpi/coll-selector`
- **smpi/comp-adjustment-file:** :ref:`cfg=smpi/comp-adjustment-file`
- **smpi/cpu-threshold:** :ref:`cfg=smpi/cpu-threshold`
+- **smpi/display-allocs:** :ref:`cfg=smpi/display-allocs`
- **smpi/display-timing:** :ref:`cfg=smpi/display-timing`
+- **smpi/finalization-barrier:** :ref:`cfg=smpi/finalization-barrier`
- **smpi/grow-injected-times:** :ref:`cfg=smpi/grow-injected-times`
- **smpi/host-speed:** :ref:`cfg=smpi/host-speed`
- **smpi/IB-penalty-factors:** :ref:`cfg=smpi/IB-penalty-factors`
- **smpi/simulate-computation:** :ref:`cfg=smpi/simulate-computation`
- **smpi/test:** :ref:`cfg=smpi/test`
- **smpi/wtime:** :ref:`cfg=smpi/wtime`
+- **smpi/list-leaks** :ref:`cfg=smpi/list-leaks`
- **Tracing configuration options** can be found in Section :ref:`tracing_tracing_options`
cat /proc/sys/net/ipv4/tcp_rmem # gives the sender window
cat /proc/sys/net/ipv4/tcp_wmem # gives the receiver window
-.. _cfg=smpi/IB-penalty-factors:
.. _cfg=network/bandwidth-factor:
.. _cfg=network/latency-factor:
.. _cfg=network/weight-S:
Supercomputer at TACC, with optimal deployment of processes on
nodes. Again, only hardcore experts should bother about this fact.
-InfiniBand network behavior can be modeled through 3 parameters
-``smpi/IB-penalty-factors:"βe;βs;γs"``, as explained in `this PhD
-thesis
-<http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf>`_.
.. todo:: This section should be rewritten, and actually explain the
options network/bandwidth-factor, network/latency-factor,
network/weight-S.
+.. _cfg=smpi/IB-penalty-factors:
+
+Infiniband model
+^^^^^^^^^^^^^^^^
+
+InfiniBand network behavior can be modeled through 3 parameters
+``smpi/IB-penalty-factors:"βe;βs;γs"``, as explained in `this PhD
+thesis
+<http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf>`_ (in French)
+or more concisely in `this paper <https://hal.inria.fr/hal-00953618/document>`_,
+even if that paper does only describe models for myrinet and ethernet.
+You can see in Fig 2 some results for Infiniband, for example. This model
+may be outdated by now for modern infiniband, anyway, so a new
+validation would be good.
+
+The three paramaters are defined as follows:
+
+- βs: penalty factor for outgoing messages, computed by running a simple send to
+ two nodes and checking slowdown compared to a single send to one node,
+ dividing by 2
+- βe: penalty factor for ingoing messages, same computation method but with one
+ node receiving several messages
+- γr: slowdown factor when communication buffer memory is saturated. It needs a
+ more complicated pattern to run in order to be computed (5.3 in the thesis,
+ page 107), and formula in the end is γr = time(c)/(3×βe×time(ref)), where
+ time(ref) is the time of a single comm with no contention).
+
+Once these values are computed, a penalty is assessed for each message (this is
+the part implemented in the simulator) as shown page 106 of the thesis. Here is
+a simple translation of this text. First, some notations:
+
+- ∆e(e) which corresponds to the incoming degree of node e, that is to say the number of communications having as destination node e.
+- ∆s (s) which corresponds to the degree outgoing from node s, that is to say the number of communications sent by node s.
+- Φ (e) which corresponds to the number of communications destined for the node e but coming from a different node.
+- Ω (s, e) which corresponds to the number of messages coming from node s to node e. If node e only receives communications from different nodes then Φ (e) = ∆e (e). On the other hand if, for example, there are three messages coming from node s and going from node e then Φ (e) 6 = ∆e (e) and Ω (s, e) = 3
+
+To determine the penalty for a communication, two values need to be calculated. First, the penalty caused by the conflict in transmission, noted ps.
+
+
+- if ∆s (i) = 1 then ps = 1.
+- if ∆s (i) ≥ 2 and ∆e (i) ≥ 3 then ps = ∆s (i) × βs × γr
+- else, ps = ∆s (i) × βs
+
+
+Then, the penalty caused by the conflict in reception (noted pe) should be computed as follows:
+
+- if ∆e (i) = 1 then pe = 1
+- else, pe = Φ (e) × βe × Ω (s, e)
+
+Finally, the penalty associated with the communication is:
+p = max (ps ∈ s, pe)
+
.. _cfg=network/crosstraffic:
Simulating Cross-Traffic
Exploration Depth Limit
.......................
-The ``model-checker/max-depth`` can set the maximum depth of the
+The ``model-check/max-depth`` can set the maximum depth of the
exploration graph of the model checker. If this limit is reached, a
logging message is sent and the results might not be exact.
-By default, there is no depth limit.
+By default, the exploration is limited to the depth of 1000.
.. _cfg=model-check/timeout:
and b are the return values of their simcalls. In the previous
example, ``1/3;1/4``, you can see from the full output that the actor
1 is doing MC_RANDOM simcalls, so the 3 and 4 simply denote the values
-that these simcall return.
+that these simcall return on the execution branch leading to the
+violation.
Configuring the User Code Virtualization
----------------------------------------
-----------------------
The :ref:`tracing subsystem <outcomes_vizu>` can be configured in
-several different ways depending on the nature of the simulator (MSG,
-SimDag, SMPI) and the kind of traces that need to be obtained. See the
+several different ways depending on the used interface (S4U, SMPI, SimDag)
+and the kind of traces that needs to be obtained. See the
:ref:`Tracing Configuration Options subsection
-<tracing_tracing_options>` to get a detailed description of each
+<tracing_tracing_options>` for a full description of each
configuration option.
We detail here a simple way to get the traces working for you, even if
simulation ends. If you enable the ``smpi/display-timing`` item,
``smpirun`` will display this information when the simulation
ends.
+SMPI will also display information about the amout of real time spent
+in application code and in SMPI internals, to provide hints about the
+need to use sampling to reduce simulation time.
+
+.. _cfg=smpi/display-allocs:
+
+Reporting memory allocations
+............................
+
+**Option** ``smpi/display-allocs`` **Default:** 0 (false)
+
+SMPI intercepts malloc and calloc calls performed inside the running
+application, if it wasn't compiled with SMPI_NO_OVERRIDE_MALLOC.
+With this option, SMPI will show at the end of execution the amount of
+memory allocated through these calls, and locate the most expensive one.
+This helps finding the targets for manual memory sharing, or the threshold
+to use for smpi/auto-shared-malloc-thresh option (see :ref:`cfg=smpi/auto-shared-malloc-thresh`).
.. _cfg=smpi/keep-temps:
.. TODO:: All available collective algorithms will be made available
via the ``smpirun --help-coll`` command.
+Add a barrier in MPI_Finalize
+.............................
+
+.. _cfg=smpi/finalization-barrier:
+
+**Option** ``smpi/finalization-barrier`` **default:** off
+
+By default, SMPI processes are destroyed as soon as soon as their code ends,
+so after a successful MPI_Finalize call returns. In some rare cases, some data
+might have been attached to MPI objects still active in the remaining processes,
+and can be destroyed eagerly by the finished process.
+If your code shows issues at finalization, such as segmentation fault, triggering
+this option will add an explicit MPI_Barrier(MPI_COMM_WORLD) call inside the
+MPI_Finalize, so that all processes will terminate at almost the same point.
+It might affect the total timing by the cost of a barrier.
+
.. _cfg=smpi/iprobe:
Inject constant times for MPI_Iprobe
``--cfg=smpi/shared-malloc-hugepage:/home/huge`` to smpirun to
actually activate the huge page support in shared mallocs.
+.. _cfg=smpi/auto-shared-malloc-thresh:
+
+Automatically share allocations
+...............................
+
+**Option** ``smpi/auto-shared-malloc-thresh:`` **Default:** 0 (false)
+ This value in bytes represents the size above which all allocations
+ will be "shared" by default (as if they were performed through
+ SMPI_SHARED_MALLOC macros). Default = 0 = disabled feature.
+ The value must be carefully chosen to only select data buffers which
+ will not modify execution path or cause crash if their content is false.
+ Option :ref:`cfg=smpi/display-allocs` can be used to locate the largest
+ allocation detected in a run, and provide a good starting threshold.
+ Note : malloc, calloc and free are overridden by smpicc/cxx by default.
+ This can cause some troubles if codes are already overriding these. If this
+ is the case, defining SMPI_NO_OVERRIDE_MALLOC in the compilation flags can
+ help, but will make this feature unusable.
+
.. _cfg=smpi/wtime:
Inject constant times for MPI_Wtime, gettimeofday and clock_gettime
If the simulation speed really matters to you, you can avoid this
extra delay by setting smpi/wtime to 0.
+.. _cfg=smpi/list-leaks:
+
+Report leaked MPI objects
+.........................
+
+**Option** ``smpi/list-leaks`` **default:** 0
+
+This option controls whether to report leaked MPI objects.
+The parameter is the number of leaks to report.
+
Other Configurations
--------------------
the paths of source files depend of the build settings. That would
break most of the tests since their output is continually compared.
-Logging Configuration
+.. _logging_config:
+
+Logging configuration
---------------------
As introduced in :ref:`outcome_logs`, the SimGrid logging mechanism allows to configure at runtime the messages that should be displayed and those that should be omitted. Each
that threshold are displayed), a layout (deciding how the messages in this category are formatted), and an appender (deciding what to do with the message: either print on stderr or
to a file).
-This section explains how to configure this logging features. You can also refer to the documentation of the :ref:`programmer's interface <xbt_log_prog>`, that allows to produce
+This section explains how to configure this logging features. You can also refer to the documentation of the :ref:`programmer's interface <logging_prog>`, that allows to produce
messages from your code.
-Most of the time, the logging mechanism is configured at runtime using the ``--log`` command-line argument, even if you can also use :ref:`xbt_log_control_set()` to control it from
+Most of the time, the logging mechanism is configured at runtime using the ``--log`` command-line argument, even if you can also use :c:func:`xbt_log_control_set()` to control it from
your program. To pass configure more than one setting, you can either pass several ``--log`` arguments, or separate your settings with spaces, that must be quoted accordingly. In
practice, the following is equivalent to the above settings: ``--log=root.thresh:error --log=s4u_host.thresh:debug``.
- %p: Priority name (LOG4J compatible)
- %h: Hostname (SimGrid extension)
- - %P: Actor name (SimGrid extension -- note that with SMPI this is the integer value of the process rank)
+ - %a: Actor name (SimGrid extension -- note that with SMPI this is the integer value of the process rank)
- %i: Actor PID (SimGrid extension -- this is a 'i' as in 'i'dea)
- %t: Thread "name" (LOG4J compatible -- actually the address of the thread in memory)
- %r: application age (time elapsed since the beginning of the application)
-``--log=root.fmt:'[%h:%P:(%i) %r] %l: %m%n'`` gives you the default layout used for info messages while ``--log=root.fmt:'[%h:%P:(%i) %r] %l: [%c/%p] %m%n'`` gives you the default
+``--log=root.fmt:'[%h:%a:(%i) %r] %l: %m%n'`` gives you the default layout used for info messages while ``--log=root.fmt:'[%h:%a:(%i) %r] %l: [%c/%p] %m%n'`` gives you the default
layout for the other priorities (it adds the source code location). Also, the actor identification is omitted by the default layout for the messages coming directly from the
SimGrid kernel, so info messages are formatted with ``[%r] [%c/%p] %m%n`` in this case. When specifying the layout manually, such distinctions are currently impossible, and the
provided layout is used for every messages.
The ``rollfile`` appender uses one file only, but the file is emptied and recreated when its size reaches the specified maximum. For example, ``--log=root.app:rollfile:500:mylog``
ensures that the log file ``mylog`` will never overpass 500 bytes in size.
-Any appender setup this way have its own layout format (simple one by default),
-so you may have to change it too afterward. Moreover, the additivity of the log category
-is also set to false to prevent log event displayed by this appender to "leak" to any other
-appender higher in the hierarchy. If it is not what you wanted, you can naturally change it
-manually.
+Any appender setup this way have its own layout format, that you may change afterward. When specifying a new appender, its additivity is set to false to prevent log event displayed
+by this appender to "leak" to any other appender higher in the hierarchy. You can naturally change that if you want your messages to be displayed twice.
Category additivity
...................
``iface.log`` file; the logging of the xbt toolbox will be sent to both the ``xbt.log`` file and the ``all.log`` file (because xbt additivity was enabled); and every other loggings
will only be sent to ``all.log``.
+Other options
+.............
+
+``--help-logs`` displays a complete help message about logging in SimGrid.
+
+``--help-log-categories`` displays the actual hierarchy of log categories for this binary.
+
+``--log=no_loc`` hides the source locations (file names and line numbers) from the messages. This is useful to make tests reproducible.
+
+
.. |br| raw:: html
<br />
