6 SimGrid was conceived as a tool to study distributed algorithms. Its
7 modern S4U interface makes it easy to assess Cloud, P2P, HPC, IoT and
10 A typical SimGrid simulation is composed of several **Actors**
11 |api_s4u_Actor|_ , that execute user-provided functions. The actors
12 have to explicitly use the S4U interface to express their computation,
13 communication, disk usage and other **Activities** |api_s4u_Activity|_
14 , so that they get reflected within the simulator. These activities
15 take place on **Resources** (CPUs, links, disks). SimGrid predicts the
16 time taken by each activity and orchestrates accordingly the actors
17 waiting for the completion of these activities.
19 .. |api_s4u_Actor| image:: /images/extlink.png
22 .. _api_s4u_Actor: api/classsimgrid_1_1s4u_1_1Actor.html#class-documentation
24 .. |api_s4u_Activity| image:: /images/extlink.png
27 .. _api_s4u_Activity: api/classsimgrid_1_1s4u_1_1Activity.html#class-documentation
30 Each actor executes a user-provided function on a simulated **Host**
31 |api_s4u_Host|_ with which it can interact. Communications are not
32 directly sent to actors, but posted onto **Mailboxes**
33 |api_s4u_Mailbox|_ that serve as rendez-vous points between
34 communicating processes.
36 .. |api_s4u_Host| image:: /images/extlink.png
39 .. _api_s4u_Host: api/classsimgrid_1_1s4u_1_1Host.html#class-documentation
41 .. |api_s4u_Mailbox| image:: /images/extlink.png
44 .. _api_s4u_Mailbox: api/classsimgrid_1_1s4u_1_1Mailbox.html#class-documentation
47 Discover the Master/Workers
48 ---------------------------
50 This section introduces a first example of SimGrid simulation. This
51 simple application is composed of two kind of actors: the **master**
52 is in charge of distributing some computational tasks to a set of
53 **workers** that execute them.
55 .. image:: /images/tuto-masterworkers-intro.svg
58 We first present a round-robin version of this application, where the
59 master dispatches the tasks to the workers, one after the other, until
60 all tasks are dispatched. Later in this tutorial, you will be given
61 the opportunity to improve this scheme.
66 Let's start with the code of the worker. It is represented by the
67 *master* function below. This simple function takes at least 3
68 parameters (the amount of tasks to dispatch, their computational size
69 in flops to compute and their communication size in bytes to
70 exchange). Every parameter after the third one must be the name of an
71 host on which a worker is waiting for something to compute.
73 Then, the tasks are sent one after the other, each on a mailbox named
74 after the worker's hosts. On the other side, a given worker (which
75 code is given below) wait for incoming tasks on its own
80 At the end, once all tasks are dispatched, the master dispatches
81 another task per worker, but this time with a negative amount of flops
82 to compute. Indeed, this application decided by convention, that the
83 workers should stop when encountering such a negative compute_size.
85 At the end of the day, the only SimGrid specific functions used in
86 this example are :cpp:func:`simgrid::s4u::Mailbox::by_name` and
87 :cpp:func:`simgrid::s4u::Mailbox::put`. Also, :c:macro:`XBT_INFO` is used
88 as a replacement to printf() or to cout to ensure that the messages
89 are nicely logged along with the simulated time and actor name.
92 .. literalinclude:: ../../examples/s4u/app-masterworkers/s4u-app-masterworkers-fun.cpp
94 :start-after: master-begin
95 :end-before: master-end
97 Here comes the code of the worker actors. This function expects no
98 parameter from its vector of strings. Its code is very simple: it
99 expects messages on the mailbox that is named after its own host. As long as it gets valid
100 computation requests (whose compute_amount is positive), it compute
101 this task and waits for the next one.
103 The worker retrieves its own host with
104 :cpp:func:`simgrid::s4u::this_actor::get_host`. The
105 :ref:`simgrid::s4u::this_actor <namespace_simgrid__s4u__this_actor>`
106 namespace contains many such helping functions.
108 .. literalinclude:: ../../examples/s4u/app-masterworkers/s4u-app-masterworkers-fun.cpp
110 :start-after: worker-begin
111 :end-before: worker-end
113 Starting the Simulation
114 .......................
116 And this is it. In only a few lines, we defined the algorithm of our
117 master/workers examples.
119 That being said, an algorithm alone is not enough to define a
120 simulation: SimGrid is a library, not a program. So you need to define
121 your own ``main()`` function as follows. This function is in charge of
122 creating a SimGrid simulation engine (on line 3), register the actor
123 functions to the engine (on lines 7 and 8), load the virtual platform
124 from its description file (on line 11), map actors onto that platform
125 (on line 12) and run the simulation until its completion on line 15.
127 .. literalinclude:: ../../examples/s4u/app-masterworkers/s4u-app-masterworkers-fun.cpp
129 :start-after: main-begin
130 :end-before: main-end
133 As you can see, this also requires a platform file and a deployment
139 Platform files define the virtual platform on which the provided
140 application will take place. In contains one or several **Network
141 Zone** |api_s4u_NetZone|_ that contain both **Host-** |api_s4u_Host|_
142 and **Link-** |api_s4u_Link|_ Resources, as well as routing
145 Such files can get rather long and boring, so the example below is
146 only an excerpts of the full ``examples/platforms/small_platform.xml``
147 file. For example, most routing information are missing, and only the
148 route between the hosts Tremblay and Fafard is given. This path
149 traverses 6 links (named 4, 3, 2, 0, 1 and 8). There are several
150 examples of platforms in the archive under ``examples/platforms``.
152 .. |api_s4u_NetZone| image:: /images/extlink.png
155 .. _api_s4u_NetZone: api/classsimgrid_1_1s4u_1_1NetZone.html#class-documentation
157 .. |api_s4u_Link| image:: /images/extlink.png
160 .. _api_s4u_Link: api/classsimgrid_1_1s4u_1_1Link.html#class-documentation
162 .. literalinclude:: ../../examples/platforms/small_platform.xml
164 :lines: 1-10,12-20,56-62,192-
165 :caption: (excerpts of the small_platform.xml file)
170 Deployment files specify the execution scenario: it lists the actors
171 that should be started, along with their parameter. In the following
172 example, we start 6 actors: one master and 5 workers.
174 .. literalinclude:: ../../examples/s4u/app-masterworkers/s4u-app-masterworkers_d.xml
180 This time, we have all parts: once the program is compiled, we can
181 execute it as follows. Note how the XBT_INFO() requests turned into
182 informative messages.
184 .. literalinclude:: ../../examples/s4u/app-masterworkers/s4u-app-masterworkers.tesh
186 :start-after: s4u-app-masterworkers-fun
187 :prepend: $$$ ./masterworkers platform.xml deploy.xml
195 In this section, you will modify the example presented earlier to
196 explore the quality of the proposed algorithm. For now, it works and
197 the simulation prints things, but the truth is that we have no idea of
198 whether this is a good algorithm to dispatch tasks to the workers.
199 This very simple setting raises many interesting questions:
201 .. image:: /images/tuto-masterworkers-question.svg
204 - Which algorithm should the master use? Or should the worker decide
207 Round Robin is not an efficient algorithm when all tasks are not
208 processed at the same speed. It would probably be more efficient
209 if the workers were asking for tasks when ready.
211 - Should tasks be grouped in batches or sent separately?
213 The workers will starve if they don't get the tasks fast
214 enough. One possibility to reduce latency would be to send tasks
215 in pools instead of one by one. But if the pools are too big, the
216 load balancing will likely get uneven, in particular when
217 distributing the last tasks.
219 - How does the quality of such algorithm dependent on the platform
220 characteristics and on the task characteristics?
222 Whenever the input communication time is very small compared to
223 processing time and workers are homogeneous, it is likely that the
224 round-robin algorithm performs very well. Would it still hold true
225 when transfer time is not negligible? What if some tasks are
226 performed faster on some specific nodes?
228 - The network topology interconnecting the master and the workers
229 may be quite complicated. How does such a topology impact the
232 When data transfers are the bottleneck, it is likely that a good
233 modeling of the platform becomes essential. The SimGrid platform
234 models are particularly handy to account for complex platform
237 - What is the best applicative topology?
239 Is a flat master worker deployment sufficient? Should we go for a
240 hierarchical algorithm, with some forwarders taking large pools of
241 tasks from the master, each of them distributing their tasks to a
242 sub-pool of workers? Or should we introduce super-peers,
243 dupplicating the master's role in a peer-to-peer manner? Do the
244 algorithms require a perfect knowledge of the network?
246 - How is such an algorithm sensitive to external workload variation?
248 What if bandwidth, latency and computing speed can vary with no
249 warning? Shouldn't you study whether your algorithm is sensitive
250 to such load variations?
252 - Although an algorithm may be more efficient than another, how does
253 it interfere with unrelated applications executing on the same
256 **SimGrid was invented to answer such questions.** Do not believe the
257 fools saying that all you need to study such settings is a simple
258 discrete event simulator. Do you really want to reinvent the wheel,
259 debug and optimize your own tool, and validate its models against real
260 settings for ages, or do you prefer to sit on the shoulders of a
261 giant? With SimGrid, you can focus on your algorithm. The whole
262 simulation mechanism is already working.
264 Here is the visualization of a SimGrid simulation of two master worker
265 applications (one in light gray and the other in dark gray) running in
266 concurrence and showing resource usage over a long period of time. It
267 was obtained with the Triva software.
269 .. image:: /images/tuto-masterworkers-result.png
275 Before your proceed, you need to :ref:`install SimGrid <install>`, a
276 C++ compiler and also ``pajeng`` to visualize the traces. The provided
277 code template requires cmake to compile. On Debian and Ubuntu for
278 example, you can get them as follows:
280 .. code-block:: shell
282 sudo apt install simgrid pajeng cmake g++
284 An initial version of the source code is provided on framagit. This
285 template compiles with cmake. If SimGrid is correctly installed, you
286 should be able to clone the `repository
287 <https://framagit.org/simgrid/simgrid-template-s4u>`_ and recompile
288 everything as follows:
290 .. code-block:: shell
292 git clone git@framagit.org:simgrid/simgrid-template-s4u.git
293 cd simgrid-template-s4u/
297 If you struggle with the compilation, then you should double check
298 your :ref:`SimGrid installation <install>`. On need, please refer to
299 the :ref:`Troubleshooting your Project Setup
300 <install_yours_troubleshooting>` section.
302 Discovering the Provided Code
303 .............................
305 Please compile and execute the provided simulator as follows:
308 .. code-block:: shell
311 ./master-workers small_platform.xml master-workers_d.xml
313 For a more "fancy" output, you can use simgrid-colorizer.
315 .. code-block:: shell
317 ./master-workers small_platform.xml master-workers_d.xml 2>&1 | simgrid-colorizer
319 If you installed SimGrid to a non-standard path, you may have to
320 specify the full path to simgrid-colorizer on the above line, such as
321 ``/opt/simgrid/bin/simgrid-colorizer``. If you did not install it at all,
322 you can find it in <simgrid_root_directory>/bin/colorize.
326 Explain how to generate a Gantt-Chart with S4U and pajeng.
328 Exercise 1: Simplifying the deployment file
329 ...........................................
331 In the provided example, adding more workers quickly becomes a pain:
332 You need to start them (at the bottom of the file), and to inform the
333 master of its availability with an extra parameter. This is mandatory
334 if you want to inform the master of where the workers are running. But
335 actually, the master does not need to have this information.
337 We could leverage the mailbox mechanism flexibility, and use a sort of
338 yellow page system: Instead of sending data to the worker running on
339 Fafard, the master could send data to the third worker. Ie, instead of
340 using the worker location (which should be filled in two locations),
341 we could use their ID (which should be filled in one location
344 This could be done with the following deployment file. It's clearly
345 not shorter than the previous one, but it's still simpler because the
346 information is only written once. It thus follows the `DRY
347 <https://en.wikipedia.org/wiki/Don't_repeat_yourself>`_ `SPOT
348 <http://wiki.c2.com/?SinglePointOfTruth>`_ design principle.
350 .. literalinclude:: tuto_s4u/deployment1.xml
354 Copy your ``master-workers.cpp`` into ``master-workers-exo1.cpp`` and
355 add a new executable into ``CMakeLists.txt``. Then modify your worker
356 function so that it gets its mailbox name not from the name of its
357 host, but from the string passed as ``args[1]``. The master will send
358 messages to all workers based on their number, for example as follows:
362 for (int i = 0; i < tasks_count; i++) {
363 std::string worker_rank = std::to_string(i % workers_count);
364 std::string mailbox_name = std::string("worker-") + worker_rank;
365 simgrid::s4u::MailboxPtr mailbox = simgrid::s4u::Mailbox::by_name(mailbox_name);
376 The mailboxes are a very powerful mechanism in SimGrid, allowing many
377 interesting application settings. They may feel surprising if you are
378 used to BSD sockets or other classical systems, but you will soon
379 appreciate their power. They are only used to match the
380 communications, but have no impact on the communication
381 timing. ``put()`` and ``get()`` are matched regardless of their
382 initiators' location and then the real communication occures between
383 the involved parties.
385 Please refer to the full `API of Mailboxes
386 <api/classsimgrid_1_1s4u_1_1Mailbox.html#class-documentation>`_
387 |api_s4u_Mailbox|_ for more details.
390 Exercise 2: Using the whole platform
391 ....................................
393 It is now easier to add a new worker, but you still has to do it
394 manually. It would be much easier if the master could start the
395 workers on its own, one per available host in the platform. The new
396 deployment file should be as simple as:
398 .. literalinclude:: tuto_s4u/deployment2.xml
402 Creating the workers from the master
403 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
405 For that, the master needs to retrieve the list of hosts declared in
406 the platform with :cpp:func:`simgrid::s4u::Engine::get_all_host()`.
407 Then, the master should start the worker processes with
408 :cpp:func:`simgrid::s4u::Actor::create`.
410 ``Actor::create(name, host, func, params...)`` is a very flexible
411 function. Its third parameter is the function that the actor should
412 execute. This function can take any kind of parameter, provided that
413 you pass similar parameters to ``Actor::create()``. For example, you
414 could have something like this:
418 void my_actor(int param1, double param2, std::string param3) {
421 int main(int argc, char argv**) {
423 simgrid::s4u::ActorPtr actor;
424 actor = simgrid::s4u::Actor::create("name", simgrid::s4u::Host::by_name("the_host"),
425 &my_actor, 42, 3.14, "thevalue");
430 Master-Workers Communication
431 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
433 Previously, the workers got from their parameter the name of the
434 mailbox they should use. We can still do so: the master should build
435 such a parameter before using it in the ``Actor::create()`` call. The
436 master could even pass directly the mailbox as a parameter to the
439 Since we want later to study concurrent applications, it is advised to
440 use a mailbox name that is unique over the simulation even if there is
441 more than one master.
443 One possibility for that is to use the actor ID (aid) of each worker
444 as a mailbox name. The master can retrieve the aid of the newly
445 created actor with ``actor->get_pid()`` while the actor itself can
446 retrieve its own aid with ``simgrid::s4u::this_actor::get_pid()``.
447 The retrieved value is an ``aid_t``, which is an alias for ``long``.
449 Instead of having one mailbox per worker, you could also reorganize
450 completely your application to have only one mailbox per master. All
451 the workers of a given master would pull their work from the same
452 mailbox, which should be passed as parameter to the workers. This
453 reduces the amount of mailboxes, but prevents the master from taking
454 any scheduling decision. It really depends on how you want to organize
455 your application and what you want to study with your simulator.
460 In this exercise, we reduced the amount of configuration that our
461 simulator requests. This is both a good idea, and a dangerous
462 trend. This simplification is another application of the good old DRY/SPOT
463 programming principle (`Don't Repeat Yourself / Single Point Of Truth
464 <https://en.wikipedia.org/wiki/Don%27t_repeat_yourself>`_), and you
465 really want your programming artefacts to follow these software
466 engineering principles.
468 But at the same time, you should be careful in separating your
469 scientific contribution (the master/workers algorithm) and the
470 artefacts used to test it (platform, deployment and workload). This is
471 why SimGrid forces you to express your platform and deployment files
472 in XML instead of using a programming interface: it forces a clear
473 separation of concerns between things of very different nature.
476 .. LocalWords: SimGrid