-/*! @page introduction Introduction to SimGrid
+/*! @page introduction Introduction to SimGrid
+
[SimGrid](http://simgrid.gforge.inria.fr/) is a toolkit
that provides core functionalities for the simulation of distributed
platforms ranging from simple network of workstations to Computational
Grids.
-# Scenario
+\tableofcontents
+
+\section Scenario
The goal of this practical session is to illustrate various usage of
the MSG interface. To this end we will use the following simple setting:
The most obvious algorithm would be to send tasks to workers in a
round-robin fashion. This is the initial code we provide you.
- A less obvious one but probably more efficient would be to set up
- a request mechanism where client first ask for tasks, which allows
+ A less obvious but probably more efficient approach would be to set up
+ a request mechanism where a client first ask for tasks, which allows
the server to decide which request to answer and possibly to send
the tasks to the fastest machines. Maybe you can think of a
smarter mechanism...
-- How much tasks should the client ask for?
-
- Indeed, if we set up a request mechanism and that workers only
+- How many tasks should the client ask for?
+
+ Indeed, if we set up a request mechanism so that workers only
send request whenever they have no more task to process, they are
likely to be poorly exploited since they will have to wait for the
master to consider their request and for the input data to be
transferred. A client should thus probably request a pool of tasks
- but if it requests too much task, it is likely to lead to a poor
+ but if it requests too many tasks, it is likely to lead to a poor
load-balancing...
-
+
- How is the quality of such algorithm dependent on the platform
- characteristics? on the task characteristics?
-
+ characteristics and on the task characteristics?
+
Whenever the input communication time is very small compared to
processing time and workers are homogeneous, it is likely that the
round-robin algorithm performs very well. Would it still hold true
a volunteer computing system ?
- The network topology interconnecting the master and the workers
- may be quite complicated. How does such topology impact the
+ may be quite complicated. How does such a topology impact the
previous result?
When data transfers are the bottleneck, it is likely that a good
- modeling of the platform becomes essential, in which case, you may
+ modeling of the platform becomes essential. In this case, you may
want to be able to account for complex platform topologies.
- Do the algorithms depend on a perfect knowledge of this
topology?
Should we still use a flat master worker deployment or should we
- use a
+ use a
-- How is such algorithm sensitive to external workload variation?
+- How is such an algorithm sensitive to external workload variation?
What if bandwidth, latency and power can vary with no warning?
Shouldn't you study whether your algorithm is sensitive to such
- Although an algorithm may be more efficient than another, how
does it interfere with other applications?
- As you can see, this very simple setting may need to evolve way
- beyond what you initially imagined.
+ %As you can see, this very simple setting may need to evolve way
+ beyond what you initially imagined.
<blockquote> Premature optimization is the root of all evil. -- D.E.Knuth</blockquote>
- Furthermore, writing your own simulator is much harder that what you
- may imagine. This is why should rely on an established and flexible
+ Furthermore, writing your own simulator is much harder than you
+ may imagine. This is why you should rely on an established and flexible
one.
The following figure is a screenshot of [triva][fn:1] visualizing a [SimGrid
-# Prerequisites
+\section Prerequisites
+
+Of course, you need to install SimGrid before taking this tutorial.
+Please refer to the relevant Section: \ref install.
## Tutorials
A lot of information on how to install and use Simgrid are
-available on the [online documentation][fn:4] and in the tutorials:
+provided by the [online documentation][fn:4] and by several tutorials:
- http://simgrid.gforge.inria.fr/tutorials/simgrid-use-101.pdf
- http://simgrid.gforge.inria.fr/tutorials/simgrid-tracing-101.pdf
- http://simgrid.gforge.inria.fr/tutorials/simgrid-platf-101.pdf
-## Installing SimGrid
-
- sudo apt-get install simgrid
-
-This tutorial requires simgrid 3.8 at last so you may need to get
-the [debian package](http://packages.debian.org/unstable/main/simgrid). Here is a shortcut:
+\section intro_recommendation Recommended Steps
-- AMD64: http://ftp.de.debian.org/debian/pool/main/s/simgrid/simgrid_3.8.1-2_amd64.deb
-- i386: http://ftp.de.debian.org/debian/pool/main/s/simgrid/simgrid_3.8.1-2_i386.deb
+## Installing Viva
-Then
-
-~~~~{.sh}
-sudo dpkg -i simgrid_3.8*.deb
-~~~~
-
-# Recommended Steps
-
-## Installing Viva
-
This [software][fn:1] will be useful to make fancy graph or treemap
visualizations and get a better understanding of simulations. You
will first need to install pajeng:
~~~~{.sh}
sudo apt-get install git cmake build-essential libqt4-dev libboost-dev freeglut3-dev ;
git clone https://github.com/schnorr/pajeng.git
-cd pajeng && mkdir -p build && cd build && cmake ../ -DCMAKE_INSTALL_PREFIX=$HOME && make -j install
+cd pajeng && mkdir -p build && cd build && cmake ../ -DCMAKE_INSTALL_PREFIX=$HOME && make -j install
cd ../../
~~~~
~~~~{.sh}
sudo apt-get install libboost-dev libconfig++-dev libconfig8-dev libgtk2.0-dev freeglut3-dev
git clone https://github.com/schnorr/viva.git
-cd viva && mkdir -p build_graph && cd build_graph && cmake ../ -DTUPI_LIBRARY=ON -DVIVA=ON -DCMAKE_INSTALL_PREFIX=$HOME && make -j install
+cd viva && mkdir -p build_graph && cd build_graph && cmake ../ -DTUPI_LIBRARY=ON -DVIVA=ON -DCMAKE_INSTALL_PREFIX=$HOME && make -j install
cd ../../
~~~~
-## Installing Paje
+## Installing Paje
This [software][fn:5] provides a Gantt-chart visualization.
sudo apt-get install paje.app
~~~~
-## Installing Vite
+## Installing Vite
This software provides a [Gantt-chart visualization][fn:6].
sudo apt-get install vite
~~~~
-# Let's get Started
-## Setting up and Compiling.
-
-The corresponding archive with all source files and platform files
-can be obtained [here](http://simgrid.gforge.inria.fr/tutorials/msg-tuto/msg-tuto.tgz).
+\section intro_start Let's get started
+
+\anchor intro_setup
+## Setting up and Compiling
+
+The corresponding archive with all source files can be obtained
+[here](http://simgrid.gforge.inria.fr/tutorials/msg-tuto/msg-tuto.tgz),
+while the simgrid archive contains
+[several platform files](https://github.com/mquinson/simgrid/tree/master/examples/platforms)
+(click on the "Raw" button of files you want to download from GitHub).
~~~~{.sh}
tar zxf msg-tuto.tgz
make
~~~~
-As you can see, there is already a nice Makefile that compiles
+%As you can see, there is already a nice Makefile that compiles
everything for you. Now the tiny example has been compiled and it
can be easily run as follows:
For a really fancy output, you should use [viva/triva][fn:1]:
~~~~{.sh}
-./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:1 \
- --cfg=tracing/uncategorized:1 --cfg=viva/uncategorized:uncat.plist
+./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:yes \
+ --cfg=tracing/uncategorized:yes --cfg=viva/uncategorized:uncat.plist
LANG=C ; viva simgrid.trace uncat.plist
~~~~
-
+
For a more classical Gantt-Chart visualization, you can produce a
[Paje][fn:5] trace:
~~~~{.sh}
-./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:1 \
- --cfg=tracing/msg/process:1
+./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:yes \
+ --cfg=tracing/msg/process:yes
LANG=C ; Paje simgrid.trace
~~~~
Alternatively, you can use [vite][fn:6].
~~~~{.sh}
-./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:1 \
- --cfg=tracing/msg/process:1 --cfg=tracing/basic:1
+./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:yes \
+ --cfg=tracing/msg/process:yes --cfg=tracing/basic:yes
vite simgrid.trace
~~~~
</platform>
~~~~
-To this end you may need the following MSG functions, whose
-behavior is described in the [online documentation](http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html) (hint: use the
-search field to access directly the function you are looking for):
+To this end you may need the following MSG functions (click on the links
+to see their descriptions):
~~~~{.c}
-int MSG_get_host_number (void)
+int MSG_get_host_number(void);
xbt_dynar_t MSG_hosts_as_dynar(void);
void * xbt_dynar_to_array (xbt_dynar_t dynar);
msg_process_t MSG_process_create(const char *name, xbt_main_func_t code,
void *data, msg_host_t host);
~~~~
-Note that it may avoid bugs later to avoid launching a worker on
-the master host so you probably want to remove it from the host
-list.
+\note
+ It may avoid bugs later to avoid launching a worker on
+ the master host so you probably want to remove it from the host
+ list.
-The `data` field of the `MSG_process_create` can be used to pass
+The `data` field of the @ref MSG_process_create can be used to pass
a channel name that will be private between master
and workers (e.g., `master_name:worker_name`). Adding the
`master_name` in the channel name will allow to easily have several
msg_process_t MSG_process_self(void);
void * MSG_process_get_data(msg_process_t process);
~~~~
-
-Again, you should check the [online documentation](http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html)
-for more information. If you are not too much familiar with string
+
+If you are not too familiar with string
manipulation in C, you may want to use the following functions
+(see the C reference for details):
~~~~{.c}
char *strcpy(char *dest, const char *src);
## Setting up a Time Limit Mechanism
-In the current version, the number of tasks is defined in the
+In the current version, the number of tasks is defined through the
worker arguments. Hence, tasks are created at the very beginning of
the simulation. Instead, create tasks as needed and provide a time
limit indicating when it stops sending tasks. To this end, you will
-obviously need to know what time it is[fn:7]:
+obviously need to know what time it is:
~~~~{.c}
double MSG_get_clock(void);
~~~~
Otherwise, a quite effective way of terminating the simulation
-would be to use some of the following function[fn:7]:
+would be to use some of the following functions:
~~~~{.c}
void MSG_process_kill(msg_process_t process);
## Using the Tracing Mechanism
SimGrid can trace all resource consumption and the outcome can be
-displayed with viva as illustrated [[*Setting%20up%20and%20Compiling.][here]]. However, when several
-masters are deployed, it is hard to understand what happens.
+displayed with viva as illustrated in the section \ref intro_setup. However, when several
+masters are deployed, it is hard to understand what happens.
~~~~{.xml}
<?xml version='1.0'?>
</platform>
~~~~
-So let's use categories to track more precisely who does what and
-when[fn:7].
+So let's use categories to track more precisely who does what and when:
~~~~{.c}
void TRACE_category(const char *category);
void MSG_task_set_category (msg_task_t task, const char *category);
~~~~
-
+
The outcome can then be visualized as follows:
~~~~{.sh}
-./masterworker3 platforms/platform.xml deployment3.xml --cfg=tracing:1\
- --cfg=tracing/categorized:1 --cfg=viva/categorized:viva_cat.plist
+./masterworker3 platforms/platform.xml deployment3.xml --cfg=tracing:yes\
+ --cfg=tracing/categorized:yes --cfg=viva/categorized:viva_cat.plist
LANG=C; viva simgrid.trace viva_cat.plist
~~~~
Gantt-chart visualization may help:
~~~~{.sh}
-./masterworker3 platforms/platform.xml deployment3.xml --cfg=tracing:1 \
- --cfg=tracing/msg/process:1
+./masterworker3 platforms/platform.xml deployment3.xml --cfg=tracing:yes \
+ --cfg=tracing/msg/process:yes
LANG=C; Paje simgrid.trace
~~~~
and willing to work.
To know whether it has pending requests, the master can use the
-following function[fn:7]:
+following [function][fn:7]:
~~~~{.c}
int MSG_task_listen(const char *alias);
If so, it should get the request and push the corresponding host
into a dynar so that they can later be retrieved when sending a
-real task[fn:7].
+real [task][fn:7].
~~~~{.c}
xbt_dynar_t xbt_dynar_new(const unsigned long elm_size,
unsigned long xbt_dynar_length(const xbt_dynar_t dynar);
~~~~
-As you will soon realize, with such simple mechanisms, simple
+%As you will soon realize, with such simple mechanisms, simple
deadlocks will soon appear. They can easily be removed with a
simple polling mechanism, hence the need for the following
-function[fn:7]:
+[function][fn:7]:
~~~~{.c}
msg_error_t MSG_process_sleep(double nb_sec);
~~~~
-As you should quickly realize, on the simple previous example, it
+%As you should quickly realize, on the simple previous example, it
will double the throughput of the platform but will be quite
ineffective when input size of the tasks is not negligible anymore.
## Using More Elaborate Platforms
SimGrid offers a rather powerful platform modeling mechanism. The
-`src/platform/` repository comprises a variety of platform ranging
-from simple ones to quite elaborated ones. Associated to a good
+`src/examples/platforms/` repository comprises a variety of platforms ranging
+from simple to elaborate. Associated to a good
visualization tool to ensure your simulation is meaningful, they
can allow you to study to which extent your algorithm scales...
bytes that you manage to distribute and process in one hour on
`g5k.xml` (you should use `deployment_general.xml`)?
-# Points to improve for the next time
+\section intro_todo TODO: Points to improve for the next time
- Propose equivalent exercises and skeleton in java.
- Propose a virtualbox image with everything (simgrid, paje, viva,
[fn:4]: http://simgrid.gforge.inria.fr/documentation.html
[fn:5]: http://paje.sourceforge.net/
[fn:6]: http://vite.gforge.inria.fr/
-[fn:7]: http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html
