3 <object id="TOC" data="graphical-toc.svg" type="image/svg+xml"></object>
5 window.onload=function() { // Wait for the SVG to be loaded before changing it
6 var elem=document.querySelector("#TOC").contentDocument.getElementById("ModelBox")
7 elem.style="opacity:0.93999999;fill:#ff0000;fill-opacity:0.1;stroke:#000000;stroke-width:0.35277778;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1";
18 As for any simulator, the models are very important components in SimGrid. This page first introduces the several kind of models
19 used in SimGrid before focusing on the **performance models** that compute the duration of :ref:`every activities
20 <S4U_main_concepts>` in the simulator depending on the platform characteristics and on the other activities that are sharing the
23 The **routing models** constitute advanced elements of the platform description. This description naturally entails
24 :ref:`components<platform>` that are very related to the performance models, because determining for example the execution time
25 of a task obviously depends on the characteristics of the machine executing it. Furthermore, networking zones can be
26 interconnected to form larger platforms `in a scalable way <http://hal.inria.fr/hal-00650233/>`_. Each of these zone can be given
27 a specific :ref:`routing model<platform_routing>` that efficiently computes the list of links entailing a network path between
30 The model checker uses an abstraction of the performance simulations. Mc SimGrid explores every causally possible executions of
31 the application, completely abstracting the performance away. The simulated time not even computed in this mode! The abstraction
32 involved in this process also models the mutual impacts between actions, to not re-explore histories that only differ by the
33 order of independent and unrelated actions. As with the rest of the model checker, these models are unfortunately still to be
36 Finally, the `SimGrid-FMI external plugin <https://framagit.org/simgrid/simgrid-FMI>`_ can be used to integrate any FMI-based
37 models into SimGrid. This was used to accurately study a *Smart grid* through co-simulation: `PandaPower
38 <http://www.pandapower.org/>`_ was used to simulate the power grid, `ns-3 <https://nsnam.org/>`_ co-simulate was used the
39 communication network while SimGrid was simulating the IT infrastructure. Please refer to the `relevant publication
40 <https://hal.archives-ouvertes.fr/hal-01762540/>`_ for more details.
45 The main objective of SimGrid is to provide timing information for three kind of resources: network, CPU and disk.
47 The **network models** are improved and assessed since almost 20 years. It should be possible to get accurate predictions once
48 you properly :ref:`calibrate the models for your settings<models_calibration>`. As detailed in the next section, SimGrid
49 provides several network models. Two plugins can be used to compute the network energy consumption: One for the :ref:`wired
50 networks<plugin_link_energy>`, and another one for the :ref:`Wi-Fi networks<plugin_link_energy>`. Some users find :ref:`TCP
51 simulated performance counter-intuitive<understanding_lv08>` at first in SimGrid, sometimes because of a misunderstanding of the
52 TCP behavior in real networks.
54 The **computing models** are less developed in SimGrid. With the S4U interface, the user specifies the amount of flops that each
55 computation "consumes", and the model simply divides this amount by the host's flops rate to compute the duration of this
56 execution. In SMPI, the user code is automatically timed, and the :ref:`computing speed<cfg=smpi/host-speed>` of the host
57 machine is used to evaluate the corresponding amount of flops. This model should be sufficient for most users, even if assuming
58 a constant flops rate for each machine is a simplification. In reality, the flops rate varies because of I/O, memory and cache
59 effects. It is somehow possible to :ref:`overcome this simplification<cfg=smpi/comp-adjustment-file>`, but the required
60 calibration process is rather intricate and not documented yet (feel free to :ref:`contact the community<community>` on need).
61 In the future, more advanced models may be added but the existing model proved good enough for all experiments done on
62 distributed applications during the last two decades. The CPU energy consumption can be computed with the :ref:`relevant
63 plugin<plugin_host_energy>`.
65 The **disk models** of SimGrid are more recent than for the network and computing resources, but they should still be correct
66 for most users. `Studies have shown <https://hal.inria.fr/hal-01197128>`_ that they are sensible under some conditions, and a
67 :ref:`calibration process<howto_disk>` is provided. As usual, you probably want to double-check their predictions through an
68 appropriate validation campaign.
73 SimGrid aims at the sweet spot between accuracy and simulation speed. Concerning the accuracy, our goal is to report correct
74 performance trends when comparing competing designs with minimal burden on the user, while allowing power users to fine tune the
75 simulation models for predictions that are within 5% or below of the results on real machines. For example, we determined the
76 `speedup achieved by the Tibidabo ARM-based cluster <http://hal.inria.fr/hal-00919507>`_ before its construction. On the other
77 side, the tool must be fast and scalable enough to study modern IT systems at scale. SimGrid was for example used to `simulate a
78 Chord ring involving millions of actors <https://hal.inria.fr/inria-00602216>`_ (even if that not really more instructive for
79 this protocol than smaller simulations), or `a qualification run at full-scale of the Stampede supercomputer
80 <https://hal.inria.fr/hal-02096571>`_.
82 Most of our models are based on a linear max-min solver (LMM), as depicted below. The actors' activities are represented by
83 actions in the simulation kernel, accounting the initial amount of work of the corresponding activity (in flops for computing
84 activities or bytes for networking and disk activities), and the remaining amount of work. At each simulation step, the
85 instantaneous computing and communicating speed of each action is computed according to the model. A set of constraints is used
86 to express for example that the instantaneous speed of actions on a given resource must remain smaller than the instantaneous
87 speed of that resource. In the example below, it is stated that the speed :math:`\varrho_1` of activity 1 plus the speed :math:`\varrho_n`
88 of activity :math:`n` must remain smaller than the capacity :math:`C_A` of the corresponding host A.
90 .. image:: img/lmm-overview.svg
92 There is obviously many valuation of :math:`\varrho_1 \ldots{} \varrho_n` that respect such as set of constraints. SimGrid usually computes
93 the instantaneous speeds according to a Max-Mix objective function, that maximizing the minimum over all :math:`\varrho_i`. The
94 coefficients associated to each variable in the inequalities are used to model some performance effects, such as the fact that
95 TCP tend to favor communications with small RTTs. These coefficients computed from both hardcoded values and from the
96 :ref:`latency and bandwidth factors<cfg=network/latency-factor>`.
98 Once the instantaneous speeds are computed, the simulation kernel computes the earliest terminating action from their speeds and
99 remaining work. The simulated time is then updated along with the values in the LMM. The corresponding activities terminate,
100 unblocking the corresponding actors that can further execute.
102 Most of the SimGrid models build upon the LMM solver, that they adapt and configure for a given usage. **CM02** is the simplest
103 LMM model as it does not introduce any correction factors. This model should be used if you prefer understandable results over
104 realistic ones. **LV08** (the default model) uses constant factors that are intended to capture common effects such as
105 slow-start, or the fact that TCP headers reduce the *effective* bandwidth. **SMPI** use more advanced factors that also capture
106 the MPI-specific effects such as the eager vs. rendez-vous communication mode. You can :ref:`pick another
107 model<options_model_select>` on the command line, and these models can be :ref:`further configured<options_model>`.
109 **L07** is rather distinct because it uses another objective function called *bottleneck*. This is because this model is
110 intended to be used for parallel tasks that are actions mixing flops and bytes while the Max-Min objective function requires
111 that all variables are expressed using the same unit. This is also why in reality, we have one LMM system per resource kind in
112 the simulation, but the idea remains similar.
114 .. _understanding_lv08:
116 The default TCP model
117 =====================
119 When simulating a data transfer between two hosts, you may be surprised by the obtained simulation time. Lets consider the
124 <host id="A" speed="1Gf" />
125 <host id="B" speed="1Gf" />
127 <link id="link1" latency="10ms" bandwidth="1Mbps" />
129 <route src="A" dst="B">
130 <link_ctn id="link1" />
133 If host `A` sends `100kB` (a hundred kilobytes) to host `B`, one could expect that this communication would take `0.81` seconds
134 to complete according to a simple latency-plus-size-divided-by-bandwidth model (0.01 + 8e5/1e6 = 0.81). However, the default TCP
135 model of SimGrid is a bit more complex than that. It accounts for three phenomena that directly impact the simulation time even
136 on such a simple example:
138 - The size of a message at the application level (i.e., 100kB in this example) is not the size that will actually be
139 transferred over the network. To mimic the fact that TCP and IP headers are added to each packet of the original payload,
140 the TCP model of SimGrid empirically considers that `only 97% of the nominal bandwidth` are available. In other words, the
141 size of your message is increased by a few percents, whatever this size be.
143 - In the real world, the TCP protocol is not able to fully exploit the bandwidth of a link from the emission of the first
144 packet. To reflect this `slow start` phenomenon, the latency declared in the platform file is multiplied by `a factor of
145 13.01`. Here again, this is an empirically determined value that may not correspond to every TCP implementations on every
146 networks. It can be tuned when more realistic simulated times for short messages are needed though.
148 - When data is transferred from A to B, some TCP ACK messages travel in the opposite direction. To reflect the impact of this
149 `cross-traffic`, SimGrid simulates a flow from B to A that represents an additional bandwidth consumption of `0.05`. The
150 route from B to A is implicitly declared in the platform file and uses the same link `link1` as if the two hosts were
151 connected through a communication bus. The bandwidth share allocated to the flow from A to B is then the available bandwidth
152 of `link1` (i.e., 97% of the nominal bandwidth of 1Mb/s) divided by 1.05 (i.e., the total consumption). This feature,
153 activated by default, can be disabled by adding the `--cfg=network/crosstraffic:0` flag to command line.
155 As a consequence, the time to transfer 100kB from A to B as simulated by the default TCP model of SimGrid is not 0.81 seconds
158 .. code-block:: python
160 0.01 * 13.01 + 800000 / ((0.97 * 1e6) / 1.05) = 0.996079 seconds.
166 In SimGrid, WiFi networks are modeled with WiFi zones, where a zone contains the access point of the WiFi network and the hosts
167 connected to it (called station in the WiFi world). Links inside WiFi zones are modeled as regular links with a specific
168 attribute, and these links are then added to routes between hosts. The main difference of WiFi networks is that their
169 performance is not given by the link bandwidth and latency but by both the access point WiFi characteristics and the distance
170 between the access point and the hosts.
172 Such WiFi zones can be used in both the LMM-based model or with ns-3, and are supposed to behave similarly in both cases.
174 Declaring a WiFi zone
175 ---------------------
177 To declare a new WiFi network, simply declare a network zone with the ``WIFI`` routing.
181 <zone id="SSID_1" routing="WIFI">
183 Inside this zone you must declare which host or router will be the access point of the WiFi network.
187 <prop id="access_point" value="alice"/>
189 Afterward simply declare the hosts and routers inside the WiFi network. Remember that one must have the same name as declared in
190 the property "access point".
194 <router id="alice" speed="1Gf"/>
195 <host id="STA0-0" speed="1Gf"/>
196 <host id="STA0-1" speed="1Gf"/>
198 Finally, close the WiFi zone.
204 The WiFi zone may be connected to another zone using a traditional link and a zoneRoute. Note that the connection between two
205 zones is always wired.
209 <link id="wireline" bandwidth="100Mbps" latency="2ms" sharing_policy="SHARED"/>
211 <zoneRoute src="SSID_1" dst="SSID_2" gw_src="alice" gw_dst="bob">
212 <link_ctn id="wireline"/>
215 WiFi network performance
216 ------------------------
218 The performance of a wifi network is controlled by 3 property that can be added to hosts connected to the wifi zone:
220 * ``mcs`` (`Modulation and Coding Scheme <https://en.wikipedia.org/wiki/Link_adaptation>`_)
221 Roughly speaking, it defines the speed at which the access point is
222 exchanging data with all stations. It depends on its model and configuration,
223 and the possible values are listed for example on Wikipedia.
224 |br| By default, ``mcs=3``.
225 It is a property of the WiFi zone.
226 * ``nss`` (Number of Spatial Streams, or `number of antennas <https://en.wikipedia.org/wiki/IEEE_802.11n-2009#Number_of_antennas>`_)
227 defines the amount of simultaneous data streams that the AP can sustain.
228 Not all value of MCS and NSS are valid nor compatible (cf. `802.11n standard <https://en.wikipedia.org/wiki/IEEE_802.11n-2009#Data_rates>`_).
229 |br| By default, ``nss=1``.
230 It is a property of the WiFi zone.
231 * ``wifi_distance`` is the distance from the station to the access point. Each
232 station can have a specific value.
233 |br| By default, ``wifi_distance=10``.
234 It is a property of stations of the WiFi network.
236 Here is an example of a zone changing ``mcs`` and ``nss`` values.
240 <zone id="SSID_1" routing="WIFI">
241 <prop id="access_point" value="alice"/>
242 <prop id="mcs" value="2"/>
243 <prop id="nss" value="2"/>
247 Here is an example of a host changing ``wifi_distance`` value.
251 <host id="STA0-0" speed="1Gf">
252 <prop id="wifi_distance" value="37"/>
258 SimGrid provides two other models in addition to the LMM-based ones.
260 First, the **constant-time model** is a simplistic network model where all communication take a constant time (one second). It
261 provides the lowest realism, but is marginally faster and much simpler to understand. This model may reveal interesting if you
262 plan to study abstract distributed algorithms such as leader election or causal broadcast.
264 On the contrary, the **ns-3 based model** is the most accurate network model that you can get in SimGrid. It relies on the
265 well-known `ns-3 packet-level network simulator <http://www.nsnam.org>`_ to compute every timing information of your simulation.
266 For example, this may be used to investigate the validity of a simulation. Note that this model is much slower than the
267 LMM-based models, because ns-3 simulates every network packet involved in as communication while SimGrid only recompute the
268 instantaneous speeds when one of the communications starts or stops. Both simulators are linear in the size of their input, but
269 ns-3 has a much larger input in case of large steady communications.
271 ns-3 as a SimGrid model
272 =======================
274 You need to install ns-3 and recompile SimGrid accordingly to use this model.
276 The SimGrid/ns-3 binding only contains features that are common to both systems.
277 Not all ns-3 models are available from SimGrid (only the TCP and WiFi ones are),
278 while not all SimGrid platform files can be used in conjunction ns-3 (routes
279 must be of length 1). Also, the platform built in ns-3 from the SimGrid
280 description is very basic. Finally, communicating from a host to
281 itself is forbidden in ns-3, so every such communication completes
282 immediately upon startup.
285 Compiling the ns-3/SimGrid binding
286 ----------------------------------
291 SimGrid requires ns-3 version 3.26 or higher, and you probably want the most
292 recent version of both SimGrid and ns-3. While the Debian package of SimGrid
293 don't have the ns-3 bindings activated, you can still use the packaged version
294 of ns-3 by grabbing the ``libns3-dev ns3`` packages. Alternatively, you can
295 install ns-3 from scratch (see the `ns-3 documentation <http://www.nsnam.org>`_).
297 Enabling ns-3 in SimGrid
298 ^^^^^^^^^^^^^^^^^^^^^^^^
300 SimGrid must be recompiled with the ``enable_ns3`` option activated in cmake.
301 Optionally, use ``NS3_HINT`` to tell cmake where ns3 is installed on
304 .. code-block:: console
306 $ cmake . -Denable_ns3=ON -DNS3_HINT=/opt/ns3 # or change the path if needed
308 By the end of the configuration, cmake reports whether ns-3 was found,
309 and this information is also available in ``include/simgrid/config.h``
310 If your local copy defines the variable ``SIMGRID_HAVE_NS3`` to 1, then ns-3
311 was correctly detected. Otherwise, explore ``CMakeFiles/CMakeOutput.log`` and
312 ``CMakeFiles/CMakeError.log`` to diagnose the problem.
314 Test that ns-3 was successfully integrated with the following (from your SimGrid
315 build directory). It will run all SimGrid tests that are related to the ns-3
316 integration. If no test is run at all, you probably forgot to enable ns-3 in cmake.
318 .. code-block:: console
325 If you use a version of ns-3 that is not known to SimGrid yet, edit
326 ``tools/cmake/Modules/FindNS3.cmake`` in your SimGrid tree, according to the
327 comments on top of this file. Conversely, if something goes wrong with an old
328 version of either SimGrid or ns-3, try upgrading everything.
330 Note that there is a known bug with version 3.31 of ns3, when it's built with
331 MPI support, like it is with the package libns3-dev in Debian 11 « Bullseye ».
332 A simple workaround is to edit the file
333 ``/usr/include/ns3.31/ns3/point-to-point-helper.h`` to remove the ``#ifdef NS3_MPI``
334 include guard. This can be achieved with the following command (as root):
336 .. code-block:: console
338 # sed -i '/^#ifdef NS3_MPI/,+2s,^#,//&,' /usr/include/ns3.31/ns3/point-to-point-helper.h
342 Using ns-3 from SimGrid
343 -----------------------
345 Platform files compatibility
346 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
348 Any route longer than one will be ignored when using ns-3. They are
349 harmless, but you still need to connect your hosts using one-hop routes.
350 The best solution is to add routers to split your route. Here is an
351 example of an invalid platform:
355 <?xml version='1.0'?>
356 <!DOCTYPE platform SYSTEM "https://simgrid.org/simgrid.dtd">
357 <platform version="4.1">
358 <zone id="zone0" routing="Floyd">
359 <host id="alice" speed="1Gf" />
360 <host id="bob" speed="1Gf" />
362 <link id="l1" bandwidth="1Mbps" latency="5ms" />
363 <link id="l2" bandwidth="1Mbps" latency="5ms" />
365 <route src="alice" dst="bob">
366 <link_ctn id="l1"/> <!-- !!!! IGNORED WHEN USED WITH ns-3 !!!! -->
367 <link_ctn id="l2"/> <!-- !!!! ROUTES MUST CONTAIN ONE LINK ONLY !!!! -->
372 This can be reformulated as follows to make it usable with the ns-3 binding.
373 There is no direct connection from alice to bob, but that's OK because ns-3
374 automatically routes from point to point (using
375 ``ns3::Ipv4GlobalRoutingHelper::PopulateRoutingTables``).
379 <?xml version='1.0'?>
380 <!DOCTYPE platform SYSTEM "https://simgrid.org/simgrid.dtd">
381 <platform version="4.1">
382 <zone id="zone0" routing="Full">
383 <host id="alice" speed="1Gf" />
384 <host id="bob" speed="1Gf" />
386 <router id="r1" /> <!-- routers are compute-less hosts -->
388 <link id="l1" bandwidth="1Mbps" latency="5ms"/>
389 <link id="l2" bandwidth="1Mbps" latency="5ms"/>
391 <route src="alice" dst="r1">
395 <route src="r1" dst="bob">
401 Once your platform is OK, just change the :ref:`network/model
402 <options_model_select>` configuration option to `ns-3` as follows. The other
403 options can be used as usual.
405 .. code-block:: console
407 $ ./network-ns3 --cfg=network/model:ns-3 (other parameters)
409 Many other files from the ``examples/platform`` directory are usable with the
410 ns-3 model, such as `examples/platforms/dogbone.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/dogbone.xml>`_.
411 Check the file `examples/cpp/network-ns3/network-ns3.tesh <https://framagit.org/simgrid/simgrid/tree/master/examples/cpp/network-ns3/network-ns3.tesh>`_
412 to see which ones are used in our regression tests.
414 Alternatively, you can manually modify the ns-3 settings by retrieving
415 the ns-3 node from any given host with the
416 :cpp:func:`simgrid::get_ns3node_from_sghost` function (defined in
417 ``simgrid/plugins/ns3.hpp``).
419 .. doxygenfunction:: simgrid::get_ns3node_from_sghost
423 It is possible to define a fixed or random seed to the ns3 random number generator using the config tag.
427 <?xml version='1.0'?><!DOCTYPE platform SYSTEM "https://simgrid.org/simgrid.dtd">
428 <platform version="4.1">
430 <prop id = "network/model" value = "ns-3" />
431 <prop id = "ns3/seed" value = "time" />
436 The first property defines that this platform will be used with the ns3 model.
437 The second property defines the seed that will be used. Defined to ``time``
438 it will use a random seed, defined to a number it will use this number as
444 A ns-3 platform is automatically created from the provided SimGrid
445 platform. However, there are some known caveats:
447 * The default values (e.g., TCP parameters) are the ns-3 default values.
448 * ns-3 networks are routed using the shortest path algorithm, using ``ns3::Ipv4GlobalRoutingHelper::PopulateRoutingTables``.
449 * End hosts cannot have more than one interface card. So, your SimGrid hosts
450 should be connected to the platform through only one link. Otherwise, your
451 SimGrid host will be considered as a router (FIXME: is it still true?).
453 Our goal is to keep the ns-3 plugin of SimGrid as easy (and hopefully readable)
454 as possible. If the current state does not fit your needs, you should modify
455 this plugin, and/or create your own plugin from the existing one. If you come up
456 with interesting improvements, please contribute them back.
461 If your simulation hangs in a communication, this is probably because one host
462 is sending data that is not routable in your platform. Make sure that you only
463 use routes of length 1, and that any host is connected to the platform.
464 Arguably, SimGrid could detect this situation and report it, but unfortunately,
465 this is still to be done.