Logo AND Algorithmique Numérique Distribuée

Public GIT Repository
Merge remote-tracking branch 'github/master'
[simgrid.git] / examples / s4u / README.rst
1 .. S4U (Simgrid for you) is the next interface of SimGrid, expected to be released with SimGrid 4.0.
2 ..
3 .. Even if it is not completely rock stable yet, it may well already fit
4 .. your needs. You are welcome to try it and report any interface
5 .. glitches that you see. Be however warned that the interface may change
6 .. until the final release.  You will have to adapt your code on the way.
7 .. 
8 .. This file follows the ReStructured syntax to be included in the
9 .. documentation, but it should remain readable directly.
10
11
12 S4U Examples
13 ************
14
15 SimGrid comes with an extensive set of examples, documented on this
16 page. Most of them only demonstrate one single feature, with some
17 larger examplars listed below. 
18
19 The C++ examples can be found under examples/s4u while python examples
20 are in examples/python. Each such directory contains the source code (also listed
21 from this page), and the so-called tesh file containing how to call
22 the binary obtained by compiling this example and also the expected
23 output. Tesh files are used to turn each of our examples into an
24 integration test. Some examples also contain other files, on need.
25
26 A good way to bootstrap your own project is to copy and combine some
27 of the provided examples to constitute the skeleton of what you plan
28 to simulate.
29
30 ===========================
31 Actors: the Active Entities
32 ===========================
33
34
35 Starting and Stoping Actors
36 ---------------------------
37
38   - **Creating actors:**
39     Most actors are started from the deployment XML file, but there is other methods.
40     This example show them all.
41     `examples/python/actor-create/actor-create_d.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/python/actor-create/actor-create_d.xml>`_
42     
43     - |cpp| `examples/s4u/actor-create/s4u-actor-create.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-create/s4u-actor-create.cpp>`_
44     - |py|  `examples/python/actor-create/actor-create.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/actor-create/actor-create.py>`_
45
46   - **React to the end of actors:**
47     You can attach a callback to the end of actors. There is two ways
48     of doing so, depending of whether you want your callback to be
49     executed when a specific actor ends (with ```this_actor::on_exit()```)
50     or whether it should be executed when any actor ends (with
51     ```Actor::on_destruction()```)
52
53     - |cpp| `examples/s4u/actor-exiting/s4u-actor-exiting.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-exiting/s4u-actor-exiting.cpp>`_
54
55   - **Kill actors:**
56     Actors can forcefully stop other actors.
57     
58     - |cpp| `examples/s4u/actor-kill/s4u-actor-kill.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-kill/s4u-actor-kill.cpp>`_
59       :cpp:func:`void simgrid::s4u::Actor::kill(void)`,
60       :cpp:func:`void simgrid::s4u::Actor::kill_all()`,
61       :cpp:func:`simgrid::s4u::this_actor::exit`.
62     - |py| `examples/python/actor-kill/actor-kill.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/actor-kill/actor-kill.py>`_
63       :py:func:`simgrid.Actor.kill`,
64       :py:func:`simgrid.Actor.kill_all`, 
65       :py:func:`simgrid.this_actor.exit`.
66
67   - **Controling the actor life cycle from the XML:**
68     You can specify a start time and a kill time in the deployment
69     file.
70     |br| `examples/s4u/actor-lifetime/s4u-actor-lifetime.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-lifetime/s4u-actor-lifetime.cpp>`_
71     |br| `examples/s4u/actor-lifetime/s4u-actor-lifetime_d.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-lifetime/s4u-actor-lifetime_d.xml>`_
72
73   - **Daemonize actors:**
74     Some actors may be intended to simulate daemons that run in background. This example show how to transform a regular
75     actor into a daemon that will be automatically killed once the simulation is over.
76     
77     - |cpp| `examples/s4u/actor-daemon/s4u-actor-daemon.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-daemon/s4u-actor-daemon.cpp>`_
78     - |py|  `examples/python/actor-daemon/actor-daemon.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/actor-daemon/actor-daemon.py>`_
79     
80 Inter-Actors Interactions
81 -------------------------
82
83   - **Suspend and Resume actors:**    
84     Actors can be suspended and resumed during their executions.
85     
86     - |cpp| `examples/s4u/actor-suspend/s4u-actor-suspend.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-suspend/s4u-actor-suspend.cpp>`_
87       :cpp:func:`simgrid::s4u::this_actor::suspend()`,
88       :cpp:func:`simgrid::s4u::Actor::suspend()`, :cpp:func:`simgrid::s4u::Actor::resume()`, :cpp:func:`simgrid::s4u::Actor::is_suspended()`.
89     - |py|  `examples/python/actor-suspend/actor-suspend.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/actor-suspend/actor-suspend.py>`_
90       :py:func:`simgrid.this_actor.suspend()`,
91       :py:func:`simgrid.Actor.suspend()`, :py:func:`simgrid.Actor.resume()`, :py:func:`simgrid.Actor.is_suspended()`.
92
93   - **Migrating Actors:**
94     Actors can move or be moved from a host to another very easily.
95     
96     - |cpp| `examples/s4u/actor-migrate/s4u-actor-migrate.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-migrate/s4u-actor-migrate.cpp>`_
97       :cpp:func:`simgrid::s4u::this_actor::migrate()`
98     - |py|  `examples/python/actor-migrate/actor-migrate.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/actor-migrate/actor-migrate.py>`_
99       :py:func:`simgrid.this_actor.migrate()`
100
101   - **Waiting for the termination of an actor:** (joining on it)
102     You can block the current actor until the end of another actor.
103     
104     - |cpp| `examples/s4u/actor-join/s4u-actor-join.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-join/s4u-actor-join.cpp>`_
105       :cpp:func:`simgrid::s4u::Actor::join()`
106     - |py|  `examples/python/actor-join/actor-join.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/actor-join/actor-join.py>`_
107       :py:func:`simgrid.Actor.join()`
108
109   - **Yielding to other actors**.
110     The ```yield()``` function interrupts the execution of the current
111     actor, leaving a chance to the other actors that are ready to run
112     at this timestamp.
113     
114     - |cpp| `examples/s4u/actor-yield/s4u-actor-yield.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/actor-yield/s4u-actor-yield.cpp>`_
115       :cpp:func:`simgrid::s4u::this_actor::yield()`
116     - |py|  `examples/python/actor-yield/actor-yield.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/actor-yield/actor-yield.py>`_
117       :py:func:`simgrid.this_actor.yield_()`
118
119 Traces Replay as a Workload
120 ---------------------------
121
122 This section details how to run trace-driven simulations. It is very
123 handy when you want to test an algorithm or protocol that only react
124 to external events. For example, many P2P protocols react to user
125 requests, but do nothing if there is no such event.
126
127 In such situations, you should write your protocol in C++, and separate
128 the workload that you want to play onto your protocol in a separate
129 text file. Declare a function handling each type of the events in your
130 trace, register them using :cpp:func:`xbt_replay_action_register()` in
131 your main, and then run the simulation.
132
133 Then, you can either have one trace file containing all your events,
134 or a file per simulated process: the former may be easier to work
135 with, but the second is more efficient on very large traces. Check
136 also the tesh files in the example directories for details.
137
138   - **Communication replay:**
139     Presents a set of event handlers reproducing classical communication
140     primitives (asynchronous send/receive at the moment).
141     |br| `examples/s4u/replay-comm/s4u-replay-comm.cpp  <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/replay-comm/s4u-replay-comm.cpp>`_
142
143   - **I/O replay:**
144     Presents a set of event handlers reproducing classical I/O
145     primitives (open, read, close).
146     |br| `examples/s4u/replay-storage/s4u-replay-storage.cpp  <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/replay-storage/s4u-replay-storage.cpp>`_
147
148 ==========================
149 Activities: what Actors do
150 ==========================
151
152 Communications on the Network
153 -----------------------------
154
155  - **Basic asynchronous communications:**
156    Illustrates how to have non-blocking communications, that are
157    communications running in the background leaving the process free
158    to do something else during their completion. 
159    
160    - |cpp| `examples/s4u/async-wait/s4u-async-wait.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/async-wait/s4u-async-wait.cpp>`_
161      :cpp:func:`simgrid::s4u::Mailbox::put_async()` and :cpp:func:`simgrid::s4u::Comm::wait()`
162    - |py|  `examples/python/async-wait/async-wait.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/async-wait/async-wait.py>`_
163      :py:func:`simgrid.Mailbox.put_async()` :py:func:`simgrid.Comm.wait()`
164
165  - **Waiting for all communications in a set:**
166    The :cpp:func:`simgrid::s4u::Comm::wait_all()` function is useful
167    when you want to block until all activities in a given set have
168    completed. 
169    |br| `examples/s4u/async-waitall/s4u-async-waitall.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/async-waitall/s4u-async-waitall.cpp>`_
170
171  - **Waiting for the first completed communication in a set:**
172    The :cpp:func:`simgrid::s4u::Comm::wait_any()` function is useful
173    when you want to block until one activity of the set completes, no
174    matter which terminates first.    
175    |br| `examples/s4u/async-waitany/s4u-async-waitany.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/async-waitany/s4u-async-waitany.cpp>`_
176
177 .. todo:: review the `ready` and `waituntil` examples and add them here.
178    
179 .. _s4u_ex_execution:
180
181 Executions on the CPU
182 ---------------------
183
184   - **Basic execution:**
185     The computations done in your program are not reported to the
186     simulated world, unless you explicitely request the simulator to pause
187     the actor until a given amount of flops gets computed on its simulated
188     host. Some executions can be given an higher priority so that they
189     get more resources.
190     
191     - |cpp| `examples/s4u/exec-basic/s4u-exec-basic.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/exec-basic/s4u-exec-basic.cpp>`_
192     - |py|  `examples/python/exec-basic/exec-basic.py <https://framagit.org/simgrid/simgrid/tree/master/examples/python/exec-basic/exec-basic.py>`_
193
194   - **Asynchronous execution:**
195     You can start asynchronous executions, just like you would fire
196     background threads.
197     |br| `examples/s4u/exec-async/s4u-exec-async.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/exec-async/s4u-exec-async.cpp>`_
198     
199   - **Monitoring asynchronous executions:**
200     This example shows how to start an asynchronous execution, and
201     monitor its status.
202     |br| `examples/s4u/exec-monitor/s4u-exec-monitor.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/exec-monitor/s4u-exec-monitor.cpp>`_
203     
204   - **Remote execution:**
205     Before its start, you can change the host on which a given execution will occur.
206     |br| `examples/s4u/exec-remote/s4u-exec-remote.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/exec-remote/s4u-exec-remote.cpp>`_
207
208   - **Using Pstates on a host:**
209     Shows how define a set of pstatesfor a host in the XML, and how the current
210     pstate can be accessed/changed with :cpp:func:`simgrid::s4u::Host::get_pstate_speed` and :cpp:func:`simgrid::s4u::Host::set_pstate`.
211     |br| `examples/s4u/exec-dvfs/s4u-exec-dvfs.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/exec-dvfs/s4u-exec-dvfs.cpp>`_
212     |br| `examples/platforms/energy_platform.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/energy_platform.xml>`_
213
214   - **Parallel executions:**
215     These objects are convenient abstractions of parallel
216     computational kernels that span over several machines, such as a
217     PDGEM and the other ScaLAPACK routines.
218     |br| `examples/s4u/exec-ptask/s4u-exec-ptask.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/exec-ptask/s4u-exec-ptask.cpp>`_
219
220 I/O on Disks and Files
221 ----------------------
222
223 SimGrid provides two levels of abstraction to interact with the
224 simulated storages. At the simplest level, you simply create read and
225 write actions on the storage resources.
226
227   - **Access to raw storage devices:**
228     This example illustrates how to simply read and write data on a
229     simulated storage resource.
230     |br| `examples/s4u/io-storage-raw/s4u-io-storage-raw.cpp  <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/io-storage-raw/s4u-io-storage-raw.cpp>`_
231
232 The FileSystem plugin provides a more detailed view, with the
233 classical operations over files: open, move, unlink, and of course
234 read and write. The file and disk sizes are also dealt with and can
235 result in short reads and short write, as in reality.
236
237   - **File Management:**
238     This example illustrates the use of operations on files
239     (read, write, seek, tell, unlink, etc).
240     |br| `examples/s4u/io-file-system/s4u-io-file-system.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/io-file-system/s4u-io-file-system.cpp>`_
241
242   - **Remote I/O:**
243     I/O operations on files can also be done in a remote fashion, 
244     i.e. when the accessed disk is not mounted on the caller's host.
245     |br| `examples/s4u/io-file-remote/s4u-io-file-remote.cpp  <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/io-file-remote/s4u-io-file-remote.cpp>`_
246
247 Classical synchronization objects
248 ---------------------------------
249
250  - **Mutex:**
251    Shows how to use simgrid::s4u::Mutex synchronization objects.
252    |br| `examples/s4u/synchro-mutex/s4u-synchro-mutex.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/synchro-mutex/s4u-synchro-mutex.cpp>`_
253
254  - **Barrier:**
255    Shows how to use simgrid::s4u::Barrier synchronization objects.
256    |br| `examples/s4u/synchro-barrier/s4u-synchro-barrier.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/synchro-barrier/s4u-synchro-barrier.cpp>`_
257
258  - **Semaphore:**
259    Shows how to use simgrid::s4u::Semaphore synchronization objects.
260    |br| `examples/s4u/synchro-semaphore/s4u-synchro-semaphore.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/synchro-semaphore/s4u-synchro-semaphore.cpp>`_
261
262 =============================
263 Interacting with the Platform
264 =============================
265
266  - **Retrieving the list of hosts matching a given criteria:**
267    Shows how to filter the actors that match a given criteria.
268    |br| `examples/s4u/engine-filtering/s4u-engine-filtering.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/engine-filtering/s4u-engine-filtering.cpp>`_
269
270  - **User-defined properties:**
271    You can attach arbitrary information to most platform elements from
272    the XML file, and then interact with these values from your
273    program. Note that the changes are not written permanently on disk,
274    in the XML file nor anywhere else. They only last until the end of
275    your simulation.
276    
277    - :cpp:func:`simgrid::s4u::Actor::get_property()` and :cpp:func:`simgrid::s4u::Actor::set_property()`
278    - :cpp:func:`simgrid::s4u::Host::get_property()` and :cpp:func:`simgrid::s4u::Host::set_property()`
279    - :cpp:func:`simgrid::s4u::Link::get_property()` and :cpp:func:`simgrid::s4u::Link::set_property()`
280    - :cpp:func:`simgrid::s4u::NetZone::get_property()` and :cpp:func:`simgrid::s4u::NetZone::set_property()`
281      
282    |br| `examples/s4u/platform-properties/s4u-platform-properties.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/platform-properties/s4u-platform-properties.cpp>`_
283    |br| `examples/s4u/platform-properties/s4u-platform-properties_d.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/platform-properties/s4u-platform-properties_d.xml>`_
284    |br| `examples/platforms/prop.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/prop.xml>`_
285
286  - **Specifying state profiles:** shows how to specify when the
287    resources must be turned off and on again, and how to react to such
288    failures in your code.
289    
290    |br| `examples/platforms/small_platform_with_failure.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/small_platform_with_failure.xml>`_
291    |br| The state profiles in `examples/platforms/profiles <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/profiles>`_
292
293  - **Specifying speed profiles:** shows how to specify an external
294    load to resources, variating their peak speed over time.
295    
296    |br| `examples/platforms/small_platform_profile.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/small_platform_profile.xml>`_
297    |br| The speed, bandwidth and latency profiles in `examples/platforms/profiles  <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/profiles>`_
298
299 =================
300 Energy Simulation
301 =================
302
303   - **Describing the energy profiles in the platform:**
304     This platform file contains the energy profile of each links and
305     hosts, which is necessary to get energy consumption predictions.
306     As usual, you should not trust our example, and you should strive
307     to double-check that your instanciation matches your target platform.
308     |br| `examples/platforms/energy_platform.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/energy_platform.xml>`_
309
310   - **Consumption due to the CPU:** 
311     This example shows how to retrieve the amount of energy consumed
312     by the CPU during computations, and the impact of the pstate.
313     |br| `examples/s4u/energy-exec/s4u-energy-exec.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/energy-exec/s4u-energy-exec.cpp>`_
314
315   - **Consumption due to the network:**
316     This example shows how to retrieve and display the energy consumed
317     by the network during communications.
318     |br| `examples/s4u/energy-link/s4u-energy-link.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/energy-link/s4u-energy-link.cpp>`_
319
320   - **Modeling the shutdown and boot of hosts:**
321     Simple example of model of model for the energy consumption during
322     the host boot and shutdown periods.
323     |br| `examples/s4u/energy-boot/platform_boot.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/energy-boot/platform_boot.xml>`_
324     |br| `examples/s4u/energy-boot/s4u-energy-boot.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/energy-boot/s4u-energy-boot.cpp>`_
325
326 =======================
327 Tracing and Visualizing
328 =======================
329
330 Tracing can be activated by various configuration options which
331 are illustrated in these example. See also the 
332 :ref:`full list of options related to tracing <tracing_tracing_options>`.
333
334 It is interesting to run the process-create example with the following
335 options to see the task executions:
336
337   - **Platform Tracing:**
338     This program is a toy example just loading the platform, so that
339     you can play with the platform visualization. Recommanded options:
340     ``--cfg=tracing:yes --cfg=tracing/categorized:yes``
341     |br| `examples/s4u/trace-platform/s4u-trace-platform.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/trace-platform/s4u-trace-platform.cpp>`_
342
343 ========================
344 Larger SimGrid Examplars
345 ========================
346
347 This section contains application examples that are somewhat larger
348 than the previous examples.
349
350   - **Ping Pong:**
351     This simple example just sends one message back and forth.
352     The tesh file laying in the directory show how to start the simulator binary, highlighting how to pass options to 
353     the simulators (as detailed in Section :ref:`options`). 
354     |br| `examples/s4u/app-pingpong/s4u-app-pingpong.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/app-pingpong/s4u-app-pingpong.cpp>`_
355
356   - **Token ring:**
357     Shows how to implement a classical communication pattern, where a
358     token is exchanged along a ring to reach every participant.
359     |br| `examples/s4u/app-token-ring/s4u-app-token-ring.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/app-token-ring/s4u-app-token-ring.cpp>`_
360
361   - **Master Workers:**
362     Another good old example, where one Master process has a bunch of task to dispatch to a set of several Worker 
363     processes. This example comes in two equivalent variants, one
364     where the actors are specified as simple functions (which is easier to
365     understand for newcomers) and one where the actors are specified
366     as classes (which is more powerful for the users wanting to build
367     their own projects upon the example).
368     |br| `examples/s4u/app-masterworkers/s4u-app-masterworkers-class.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/app-masterworkers/s4u-app-masterworkers-class.cpp>`_
369     |br| `examples/s4u/app-masterworkers/s4u-app-masterworkers-fun.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/app-masterworkers/s4u-app-masterworkers-fun.cpp>`_
370     
371 Data diffusion
372 --------------
373
374   - **Bit Torrent:** 
375     Classical protocol for Peer-to-Peer data diffusion.
376     |br| `examples/s4u/app-bittorrent/s4u-bittorrent.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/app-bittorrent/s4u-bittorrent.cpp>`_
377     
378   - **Chained Send:** 
379     Data broadcast over a ring of processes.
380     |br| `examples/s4u/app-chainsend/s4u-app-chainsend.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/app-chainsend/s4u-app-chainsend.cpp>`_
381
382 Distributed Hash Tables (DHT)
383 -----------------------------
384
385   - **Chord Protocol** 
386     One of the most famous DHT protocol.
387     |br| `examples/s4u/dht-chord/s4u-dht-chord.cpp <https://framagit.org/simgrid/simgrid/tree/master/examples/s4u/dht-chord/s4u-dht-chord.cpp>`_
388
389 .. TODO:: document here the examples about plugins
390
391 .. |br| raw:: html
392
393    <br />
394
395 .. |cpp| image:: /img/lang_cpp.png
396    :align: middle
397    :width: 12
398
399 .. |py| image:: /img/lang_python.png
400    :align: middle
401    :width: 12