=====================
SimGrid was conceived as a tool to study distributed algorithms. Its
-modern S4U interface makes it easy to assess Cloud, P2P, HPC, IoT and
-similar settings.
-
-A typical SimGrid simulation is composed of several **Actors**
-|api_s4u_Actor|_ , that execute user-provided functions. The actors
-have to explicitly use the S4U interface to express their computation,
-communication, disk usage and other **Activities** |api_s4u_Activity|_
-, so that they get reflected within the simulator. These activities
-take place on **Resources** (CPUs, links, disks). SimGrid predicts the
-time taken by each activity and orchestrates accordingly the actors
-waiting for the completion of these activities.
-
-.. |api_s4u_Actor| image:: /images/extlink.png
- :align: middle
- :width: 12
-.. _api_s4u_Actor: api/classsimgrid_1_1s4u_1_1Actor.html#class-documentation
+modern :ref:`S4U interface <S4U_doc>` makes it easy to assess Cloud,
+P2P, HPC, IoT and similar settings.
-.. |api_s4u_Activity| image:: /images/extlink.png
- :align: middle
- :width: 12
-.. _api_s4u_Activity: api/classsimgrid_1_1s4u_1_1Activity.html#class-documentation
+A typical SimGrid simulation is composed of several |Actors|_, that
+execute user-provided functions. The actors have to explicitly use the
+S4U interface to express their computation, communication, disk usage
+and other |Activities|_, so that they get reflected within the
+simulator. These activities take place on **Resources** (|Hosts|_,
+|Links|_, |Storages|_). SimGrid predicts the time taken by each
+activity and orchestrates accordingly the actors waiting for the
+completion of these activities.
+Each actor executes a user-provided function on a simulated |Host|_
+with which it can interact. Communications are not directly sent to
+actors, but posted onto a |Mailbox|_ that serve as rendez-vous point
+between communicating actors.
-Each actor executes a user-provided function on a simulated **Host**
-|api_s4u_Host|_ with which it can interact. Communications are not
-directly sent to actors, but posted onto **Mailboxes**
-|api_s4u_Mailbox|_ that serve as rendez-vous points between
-communicating processes.
+.. |Actors| replace:: **Actors**
+.. _Actors: api/classsimgrid_1_1s4u_1_1Actor.html
-.. |api_s4u_Host| image:: /images/extlink.png
- :align: middle
- :width: 12
-.. _api_s4u_Host: api/classsimgrid_1_1s4u_1_1Host.html#class-documentation
+.. |Activities| replace:: **Activities**
+.. _Activities: api/classsimgrid_1_1s4u_1_1Activity.html
-.. |api_s4u_Mailbox| image:: /images/extlink.png
- :align: middle
- :width: 12
-.. _api_s4u_Mailbox: api/classsimgrid_1_1s4u_1_1Mailbox.html#class-documentation
+.. |Hosts| replace:: **Hosts**
+.. _Hosts: api/classsimgrid_1_1s4u_1_1Host.html
+
+.. |Links| replace:: **Links**
+.. _Links: api/classsimgrid_1_1s4u_1_1Link.html
+
+.. |Storages| replace:: **Storages**
+.. _Storages: api/classsimgrid_1_1s4u_1_1Storage.html
+
+.. |VirtualMachines| replace:: **VirtualMachines**
+.. _VirtualMachines: api/classsimgrid_1_1s4u_1_1VirtualMachine.html
+
+.. |Host| replace:: **Host**
+.. _Host: api/classsimgrid_1_1s4u_1_1Host.html
+
+.. |Link| replace:: **Link**
+.. _Link: api/classsimgrid_1_1s4u_1_1Link.html
+
+.. |Mailbox| replace:: **Mailbox**
+.. _Mailbox: api/classsimgrid_1_1s4u_1_1Mailbox.html
+
+.. |Barrier| replace:: **Barrier**
+.. _Barrier: api/classsimgrid_1_1s4u_1_1Barrier.html
+
+.. |ConditionVariable| replace:: **ConditionVariable**
+.. _ConditionVariable: api/classsimgrid_1_1s4u_1_1ConditionVariable.html
+
+.. |Mutex| replace:: **Mutex**
+.. _Mutex: api/classsimgrid_1_1s4u_1_1Mutex.html
+
+
+**In the remainder of this tutorial**, you will discover a simple yet
+fully functioning example of SimGrid simulation: the Master/Workers
+application. We will detail each part of the code and necessary
+configuration to make it working. After this tour, several exercises
+are proposed to let you discover some of the SimGrid features, hands
+on the keyboard. This practical session will be given in C++, that you
+are supposed to know beforehand.
Discover the Master/Workers
This section introduces a first example of SimGrid simulation. This
simple application is composed of two kind of actors: the **master**
is in charge of distributing some computational tasks to a set of
-**workers** that execute them.
+**workers** that execute them.
.. image:: /tuto_s4u/img/intro.svg
:align: center
:cpp:func:`simgrid::s4u::Mailbox::put`. Also, :c:macro:`XBT_INFO` is used
as a replacement to printf() or to cout to ensure that the messages
are nicely logged along with the simulated time and actor name.
-
-
+
+
.. literalinclude:: ../../examples/s4u/app-masterworkers/s4u-app-masterworkers-fun.cpp
:language: c++
:start-after: master-begin
Starting the Simulation
.......................
-
+
And this is it. In only a few lines, we defined the algorithm of our
master/workers examples.
Platform files define the virtual platform on which the provided
application will take place. In contains one or several **Network
-Zone** |api_s4u_NetZone|_ that contain both **Host-** |api_s4u_Host|_
-and **Link-** |api_s4u_Link|_ Resources, as well as routing
-information.
+Zone** |api_s4u_NetZone|_ that contain both |Host|_ and |Link|_
+Resources, as well as routing information.
Such files can get rather long and boring, so the example below is
only an excerpts of the full ``examples/platforms/small_platform.xml``
route between the hosts Tremblay and Fafard is given. This path
traverses 6 links (named 4, 3, 2, 0, 1 and 8). There are several
examples of platforms in the archive under ``examples/platforms``.
-
-.. |api_s4u_NetZone| image:: /images/extlink.png
+
+.. |api_s4u_NetZone| image:: /img/extlink.png
:align: middle
:width: 12
.. _api_s4u_NetZone: api/classsimgrid_1_1s4u_1_1NetZone.html#class-documentation
-.. |api_s4u_Link| image:: /images/extlink.png
+.. |api_s4u_Link| image:: /img/extlink.png
:align: middle
:width: 12
.. _api_s4u_Link: api/classsimgrid_1_1s4u_1_1Link.html#class-documentation
This time, we have all parts: once the program is compiled, we can
execute it as follows. Note how the XBT_INFO() requests turned into
informative messages.
-
+
.. literalinclude:: ../../examples/s4u/app-masterworkers/s4u-app-masterworkers.tesh
:language: shell
:start-after: s4u-app-masterworkers-fun
:prepend: $$$ ./masterworkers platform.xml deploy.xml
:append: $$$
:dedent: 2
-
+
Improve it Yourself
-------------------
round-robin algorithm performs very well. Would it still hold true
when transfer time is not negligible? What if some tasks are
performed faster on some specific nodes?
-
+
- The network topology interconnecting the master and the workers
may be quite complicated. How does such a topology impact the
previous result?
.. image:: /tuto_s4u/img/result.png
:align: center
-Prerequisite
+Using Docker
............
-Before your proceed, you need to :ref:`install SimGrid <install>`, a
-C++ compiler and also ``pajeng`` to visualize the traces. You may want
-to install `Vite <http://vite.gforge.inria.fr/>`_ to get a first
-glance at the traces. The provided code template requires cmake to
-compile. On Debian and Ubuntu for example, you can get them as
-follows:
+The easiest way to take the tutorial is to use the dedicated Docker
+image. Once you `installed Docker itself
+<https://docs.docker.com/install/>`_, simply do the following:
+
+.. code-block:: shell
+
+ docker pull simgrid/tuto-s4u
+ docker run -it --rm --name simgrid --volume ~/simgrid-tutorial:/source/tutorial simgrid/tuto-s4u bash
+
+This will start a new container with all you need to take this
+tutorial, and create a ``simgrid-tutorial`` directory in your home on
+your host machine that will be visible as ``/source/tutorial`` within the
+container. You can then edit the files you want with your favorite
+editor in ``~/simgrid-tutorial``, and compile them within the
+container to enjoy the provided dependencies.
+
+.. warning::
+
+ Any change to the container out of ``/source/tutorial`` will be lost
+ when you log out of the container, so don't edit the other files!
+
+All needed dependencies are already installed in this container
+(SimGrid, a C++ compiler, cmake, pajeng and R). Vite being only
+optional in this tutorial, it is not installed to reduce the image
+size.
+
+The code template is available under ``/source/simgrid-template-s4u`` in
+the image. You should copy it to your working directory when you first
+log in:
+
+.. code-block:: shell
+
+ cp -r /source/simgrid-template-s4u/* /source/tutorial
+ cd /source/tutorial
+
+Using your Computer Natively
+............................
+
+To take the tutorial on your machine, you first need to :ref:`install
+SimGrid <install>`, a C++ compiler and also ``pajeng`` to visualize
+the traces. You may want to install `Vite
+<http://vite.gforge.inria.fr/>`_ to get a first glance at the
+traces. The provided code template requires cmake to compile. On
+Debian and Ubuntu for example, you can get them as follows:
.. code-block:: shell
Please compile and execute the provided simulator as follows:
-
.. code-block:: shell
make master-workers
./master-workers small_platform.xml master-workers_d.xml
-For a more "fancy" output, you can use simgrid-colorizer.
+For a more "fancy" output, you can use simgrid-colorizer.
.. code-block:: shell
you can find it in <simgrid_root_directory>/bin/colorize.
For a classical Gantt-Chart vizualisation, you can use `Vite
-<http://vite.gforge.inria.fr/>`_ as follows:
+<http://vite.gforge.inria.fr/>`_ if you have it installed, as
+follows. But do not spend too much time installing Vite, because there
+is a better way to visualize SimGrid traces (see below).
.. code-block:: shell
.. image:: /tuto_s4u/img/vite-screenshot.png
:align: center
-
-But if you want the full power to visualize SimGrid traces, you need
+
+If you want the full power to visualize SimGrid traces, you need
to use R. As a start, you can download this `starter script
<https://framagit.org/simgrid/simgrid/raw/master/docs/source/tuto_s4u/draw_gantt.R>`_
and use it as follows:
...
}
-
+
Wrap up
.......
the involved parties.
Please refer to the full `API of Mailboxes
-<api/classsimgrid_1_1s4u_1_1Mailbox.html#class-documentation>`_
-|api_s4u_Mailbox|_ for more details.
+<api/classsimgrid_1_1s4u_1_1Mailbox.html#class-documentation>`_ for
+more details.
Lab 2: Using the Whole Platform
For that, the master needs to retrieve the list of hosts declared in
the platform with :cpp:func:`simgrid::s4u::Engine::get_all_hosts`.
-Then, the master should start the worker processes with
+Then, the master should start the worker actors with
:cpp:func:`simgrid::s4u::Actor::create`.
``Actor::create(name, host, func, params...)`` is a very flexible
mailbox they should use. We can still do so: the master should build
such a parameter before using it in the ``Actor::create()`` call. The
master could even pass directly the mailbox as a parameter to the
-workers.
+workers.
Since we want later to study concurrent applications, it is advised to
use a mailbox name that is unique over the simulation even if there is
-more than one master.
+more than one master.
One possibility for that is to use the actor ID (aid) of each worker
as a mailbox name. The master can retrieve the aid of the newly
or you may kill them forcefully with
:cpp:func:`simgrid::s4u::Actor::kill` (if you already have a reference
to the actor you want to kill) or
-:cpp:func:`simgrid::s4u::Actor::kill(aid_t)` (if you only have its ID).
+:cpp:func:`void simgrid::s4u::Actor::kill(aid_t)` (if you only have its ID).
Anyway, the new deployment `deployment3.xml` file should thus look
will categorize the tasks.
Instead of starting the execution in one function call only with
-``simgrid::s4u::this_actor::execute(compute_cost)``, you need to
-create the execution activity, set its tracing category and then start
+``this_actor::execute(cost)``, you need to
+create the execution activity, set its tracing category, and then start
it and wait for its completion, as follows:
.. code-block:: cpp
simgrid::s4u::this_actor::exec_init(compute_cost)->set_tracing_category(category)->wait();
-The outcome can then be visualized as a Gantt-chart as follows:
-
-.. code-block:: shell
+Visualizing the result
+.......................
- ./master-workers-lab4 small_platform.xml deployment4.xml --cfg=tracing:yes --cfg=tracing/msg/process:yes
- vite simgrid.trace
+vite is not enough to understand the situation, because it does not
+deal with categorization. This time, you absolutely must switch to R,
+as explained on `this page
+<http://simgrid.gforge.inria.fr/contrib/R_visualization.php>`_.
.. todo::
- Make it work
+ Include here the minimal setting to view something in R.
-Going further
-.............
-vite is not enough to understand the situation, because it does not
-deal with categorization. That is why you should switch to R to
-visualize your outcomes, as explained on `this page
-<http://simgrid.gforge.inria.fr/contrib/R_visualization.php>`_.
+Lab 5: Better Scheduling
+------------------------
+
+You don't need a very advanced visualization solution to notice that
+round-robin is completely suboptimal: most of the workers keep waiting
+for more work. We will move to a First-Come First-Served mechanism
+instead.
+
+For that, your workers should explicitely request for work with a
+message sent to a channel that is specific to their master. The name
+of that private channel can be the one used to categorize the
+executions, as it is already specific to each master.
+
+The master should serve in a round-robin manner the requests it
+receives, until the time is up. Changing the communication schema can
+be a bit hairy, but once it works, you will see that such as simple
+FCFS schema allows to double the amount of tasks handled over time
+here. Things may be different with another platform file.
+
+Further Improvements
+....................
+
+From this, many things can easily be added. For example, you could:
+
+- Allow workers to have several pending requests so as to overlap
+ communication and computations as much as possible. Non-blocking
+ communication will probably become handy here.
+- Add a performance measurement mechanism, enabling the master to make smart scheduling choices.
+- Test your code on other platforms, from the ``examples/platforms``
+ directory in your archive.
+
+ What is the largest number of tasks requiring 50e6 flops and 1e5
+ bytes that you manage to distribute and process in one hour on
+ ``g5k.xml`` ?
+- Optimize not only for the amount of tasks handled, but also for the
+ total energy dissipated.
+- And so on. If you come up with a really nice extension, please share
+ it with us so that we can extend this tutorial.
+
+After this Tutorial
+-------------------
+
+This tutorial is now terminated. You could keep reading the [online documentation][fn:4] or
+[tutorials][fn:7], or you could head up to the example section to read some code.
+
+.. todo::
+ TODO: Points to improve for the next time
+ - Propose equivalent exercises and skeleton in java.
+ - Propose a virtualbox image with everything (simgrid, pajeng, ...) already set up.
+ - Ease the installation on mac OS X (binary installer) and windows.
.. LocalWords: SimGrid
