:width: 12
.. _api_s4u_Mailbox: api/classsimgrid_1_1s4u_1_1Mailbox.html#class-documentation
+**In the remainder of this tutorial**, you will discover a simple yet
+fully functioning example of SimGrid simulation: the Master/Workers
+application. We will detail each part of the code and necessary
+configuration to make it working. After this tour, several exercises
+are proposed to let you discover some of the SimGrid features, hands
+on the keyboard. This practical session will be given in C++, that you
+are supposed to know beforehand.
+
Discover the Master/Workers
---------------------------
is in charge of distributing some computational tasks to a set of
**workers** that execute them.
-.. image:: /images/tuto-masterworkers-intro.svg
+.. image:: /tuto_s4u/img/intro.svg
:align: center
We first present a round-robin version of this application, where the
whether this is a good algorithm to dispatch tasks to the workers.
This very simple setting raises many interesting questions:
-.. image:: /images/tuto-masterworkers-question.svg
+.. image:: /tuto_s4u/img/question.svg
:align: center
- Which algorithm should the master use? Or should the worker decide
concurrence and showing resource usage over a long period of time. It
was obtained with the Triva software.
-.. image:: /images/tuto-masterworkers-result.png
+.. image:: /tuto_s4u/img/result.png
:align: center
Prerequisite
............
Before your proceed, you need to :ref:`install SimGrid <install>`, a
-C++ compiler and also ``pajeng`` to visualize the traces. The provided
-code template requires cmake to compile. On Debian and Ubuntu for
-example, you can get them as follows:
+C++ compiler and also ``pajeng`` to visualize the traces. You may want
+to install `Vite <http://vite.gforge.inria.fr/>`_ to get a first
+glance at the traces. The provided code template requires cmake to
+compile. On Debian and Ubuntu for example, you can get them as
+follows:
.. code-block:: shell
- sudo apt install simgrid pajeng cmake g++
+ sudo apt install simgrid pajeng cmake g++ vite
An initial version of the source code is provided on framagit. This
template compiles with cmake. If SimGrid is correctly installed, you
``/opt/simgrid/bin/simgrid-colorizer``. If you did not install it at all,
you can find it in <simgrid_root_directory>/bin/colorize.
-.. todo::
+For a classical Gantt-Chart vizualisation, you can use `Vite
+<http://vite.gforge.inria.fr/>`_ if you have it installed, as
+follows. But do not spend too much time installing Vite, because there
+is a better way to visualize SimGrid traces (see below).
+
+.. code-block:: shell
+
+ ./master-workers small_platform.xml master-workers_d.xml --cfg=tracing:yes --cfg=tracing/msg/process:yes
+ vite simgrid.trace
+
+.. image:: /tuto_s4u/img/vite-screenshot.png
+ :align: center
+
+If you want the full power to visualize SimGrid traces, you need
+to use R. As a start, you can download this `starter script
+<https://framagit.org/simgrid/simgrid/raw/master/docs/source/tuto_s4u/draw_gantt.R>`_
+and use it as follows:
+
+.. code-block:: shell
+
+ ./master-workers small_platform.xml master-workers_d.xml --cfg=tracing:yes --cfg=tracing/msg/process:yes
+ pj_dump --ignore-incomplete-links simgrid.trace | grep STATE > gantt.csv
+ Rscript draw_gantt.R gantt.csv
+
+It produces a ``Rplots.pdf`` with the following content:
+
+.. image:: /tuto_s4u/img/Rscript-screenshot.png
+ :align: center
- Explain how to generate a Gantt-Chart with S4U and pajeng.
-Exercise 1: Simplifying the deployment file
-...........................................
+Lab 1: Simpler Deployments
+--------------------------
In the provided example, adding more workers quickly becomes a pain:
You need to start them (at the bottom of the file), and to inform the
:language: xml
-Copy your ``master-workers.cpp`` into ``master-workers-exo1.cpp`` and
+Copy your ``master-workers.cpp`` into ``master-workers-lab1.cpp`` and
add a new executable into ``CMakeLists.txt``. Then modify your worker
function so that it gets its mailbox name not from the name of its
host, but from the string passed as ``args[1]``. The master will send
Wrap up
-^^^^^^^
+.......
The mailboxes are a very powerful mechanism in SimGrid, allowing many
interesting application settings. They may feel surprising if you are
|api_s4u_Mailbox|_ for more details.
-Exercise 2: Using the whole platform
-....................................
+Lab 2: Using the Whole Platform
+-------------------------------
It is now easier to add a new worker, but you still has to do it
manually. It would be much easier if the master could start the
Creating the workers from the master
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+....................................
For that, the master needs to retrieve the list of hosts declared in
-the platform with :cpp:func:`simgrid::s4u::Engine::get_all_host()`.
+the platform with :cpp:func:`simgrid::s4u::Engine::get_all_hosts`.
Then, the master should start the worker processes with
:cpp:func:`simgrid::s4u::Actor::create`.
Master-Workers Communication
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+............................
Previously, the workers got from their parameter the name of the
mailbox they should use. We can still do so: the master should build
mailbox, which should be passed as parameter to the workers. This
reduces the amount of mailboxes, but prevents the master from taking
any scheduling decision. It really depends on how you want to organize
-your application and what you want to study with your simulator.
+your application and what you want to study with your simulator. In
+this tutorial, that's probably not a good idea.
Wrap up
-^^^^^^^
+.......
In this exercise, we reduced the amount of configuration that our
simulator requests. This is both a good idea, and a dangerous
in XML instead of using a programming interface: it forces a clear
separation of concerns between things of very different nature.
+Lab 3: Fixed Experiment Duration
+--------------------------------
+
+In the current version, the number of tasks is defined through the
+worker arguments. Hence, tasks are created at the very beginning of
+the simulation. Instead, have the master dispatching tasks for a
+predetermined amount of time. The tasks must now be created on demand
+instead of beforehand.
+
+Of course, usual time functions like ``gettimeofday`` will give you the
+time on your real machine, which is prety useless in the
+simulation. Instead, retrieve the time in the simulated world with
+:cpp:func:`simgrid::s4u::Engine::get_clock`.
+
+You can still stop your workers with a specific task as previously,
+or you may kill them forcefully with
+:cpp:func:`simgrid::s4u::Actor::kill` (if you already have a reference
+to the actor you want to kill) or
+:cpp:func:`void simgrid::s4u::Actor::kill(aid_t)` (if you only have its ID).
+
+
+Anyway, the new deployment `deployment3.xml` file should thus look
+like this:
+
+.. literalinclude:: tuto_s4u/deployment3.xml
+ :language: xml
+
+Controlling the message verbosity
+.................................
+
+Not all messages are equally informative, so you probably want to
+change some of the ``XBT_INFO`` into ``XBT_DEBUG`` so that they are
+hidden by default. For example, you may want to use ``XBT_INFO`` once
+every 100 tasks and ``XBT_DEBUG`` when sending all the other tasks. Or
+you could show only the total number of tasks processed by
+default. You can still see the debug messages as follows:
+
+.. code-block:: shell
+
+ ./master-workers-lab3 small_platform.xml deployment3.xml --log=msg_test.thres:debug
+
+
+Lab 4: Competing Applications
+-----------------------------
+
+It is now time to start several applications at once, with the following ``deployment4.xml`` file.
+
+.. literalinclude:: tuto_s4u/deployment4.xml
+ :language: xml
+
+Things happen when you do so, but it remains utterly difficult to
+understand what's happening exactely. Even Gantt visualizations
+contain too much information to be useful: it is impossible to
+understand which task belong to which application. To fix this, we
+will categorize the tasks.
+
+Instead of starting the execution in one function call only with
+``this_actor::execute(cost)``, you need to
+create the execution activity, set its tracing category, and then start
+it and wait for its completion, as follows:
+
+.. code-block:: cpp
+
+ simgrid::s4u::ExecPtr exec = simgrid::s4u::this_actor::exec_init(compute_cost);
+ exec->set_tracing_category(category);
+ // exec->start() is optional here as wait() starts the activity on need
+ exec->wait();
+
+You can make the same code shorter as follows:
+
+.. code-block:: cpp
+
+ simgrid::s4u::this_actor::exec_init(compute_cost)->set_tracing_category(category)->wait();
+
+Visualizing the result
+.......................
+
+vite is not enough to understand the situation, because it does not
+deal with categorization. This time, you absolutely must switch to R,
+as explained on `this page
+<http://simgrid.gforge.inria.fr/contrib/R_visualization.php>`_.
+
+.. todo::
+
+ Include here the minimal setting to view something in R.
+
+
+Lab 5: Better Scheduling
+------------------------
+
+You don't need a very advanced visualization solution to notice that
+round-robin is completely suboptimal: most of the workers keep waiting
+for more work. We will move to a First-Come First-Served mechanism
+instead.
+
+For that, your workers should explicitely request for work with a
+message sent to a channel that is specific to their master. The name
+of that private channel can be the one used to categorize the
+executions, as it is already specific to each master.
+
+The master should serve in a round-robin manner the requests it
+receives, until the time is up. Changing the communication schema can
+be a bit hairy, but once it works, you will see that such as simple
+FCFS schema allows to double the amount of tasks handled over time
+here. Things may be different with another platform file.
+
+Further Improvements
+....................
+
+From this, many things can easily be added. For example, you could:
+
+- Allow workers to have several pending requests so as to overlap
+ communication and computations as much as possible. Non-blocking
+ communication will probably become handy here.
+- Add a performance measurement mechanism, enabling the master to make smart scheduling choices.
+- Test your code on other platforms, from the ``examples/platforms``
+ directory in your archive.
+
+ What is the largest number of tasks requiring 50e6 flops and 1e5
+ bytes that you manage to distribute and process in one hour on
+ ``g5k.xml`` ?
+- Optimize not only for the amount of tasks handled, but also for the
+ total energy dissipated.
+- And so on. If you come up with a really nice extension, please share
+ it with us so that we can extend this tutorial.
+
+After this Tutorial
+-------------------
+
+This tutorial is now terminated. You could keep reading the [online documentation][fn:4] or
+[tutorials][fn:7], or you could head up to the example section to read some code.
+
+.. todo::
+
+ TODO: Points to improve for the next time
+
+ - Propose equivalent exercises and skeleton in java.
+ - Propose a virtualbox image with everything (simgrid, pajeng, ...) already set up.
+ - Ease the installation on mac OS X (binary installer) and windows.
.. LocalWords: SimGrid
