1 /*! \page tracing Tracing Simulations for Visualization
3 \htmlinclude .tracing.doc.toc
5 \section tracing_tracing Tracing Simulations for Visualization
7 The trace visualization is widely used to observe and understand the behavior
8 of parallel applications and distributed algorithms. Usually, this is done in a
9 two-step fashion: the user instruments the application and the traces are
10 analyzed after the end of the execution. The visualization itself can highlights
11 unexpected behaviors, bottlenecks and sometimes can be used to correct
12 distributed algorithms. The SimGrid team has instrumented the library
13 in order to let users trace their simulations and analyze them. This part of the
14 user manual explains how the tracing-related features can be enabled and used
15 during the development of simulators using the SimGrid library.
17 \subsection tracing_tracing_howitworks How it works
19 For now, the SimGrid library is instrumented so users can trace the <b>platform
20 utilization</b> using the MSG, SimDAG and SMPI interface. This means that the tracing will
21 register how much power is used for each host and how much bandwidth is used for
22 each link of the platform. The idea with this type of tracing is to observe the
23 overall view of resources utilization in the first place, especially the
24 identification of bottlenecks, load-balancing among hosts, and so on.
26 The idea of the tracing facilities is to give SimGrid users to possibility to
27 classify MSG and SimDAG tasks by category, tracing the platform utilization
28 (hosts and links) for each of the categories. For that,
29 the tracing interface enables the declaration of categories and a function to
30 mark a task with a previously declared category. <em>The tasks that are not
31 classified according to a category are not traced</em>. Even if the user
32 does not specify any category, the simulations can still be traced in terms
33 of resource utilization by using a special parameter that is detailed below.
35 \subsection tracing_tracing_enabling Enabling using CMake
37 With the sources of SimGrid, it is possible to enable the tracing
38 using the parameter <b>-Denable_tracing=ON</b> when the cmake is executed.
39 The section \ref tracing_tracing_functions describes all the functions available
40 when this Cmake options is activated. These functions will have no effect
41 if SimGrid is configured without this option (they are wiped-out by the
45 $ cmake -Denable_tracing=ON .
49 \subsection tracing_tracing_functions Tracing Functions
51 \li <b>\c TRACE_category (const char *category)</b>: This function should be used
52 to define a user category. The category can be used to differentiate the tasks
53 that are created during the simulation (for example, tasks from server1,
54 server2, or request tasks, computation tasks, communication tasks).
55 All resource utilization (host power and link bandwidth) will be
56 classified according to the task category. Tasks that do not belong to a
57 category are not traced. The color for the category that is being declared
58 is random (use next function to specify a color).
60 \li <b>\c TRACE_category_with_color (const char *category, const char *color)</b>: Same
61 as TRACE_category, but let user specify a color encoded as a RGB-like string with
62 three floats from 0 to 1. So, to specify a red color, the user can pass "1 0 0" as
63 color parameter. A light-gray color can be specified using "0.7 0.7 0.7" as color.
65 \li <b>\c TRACE_msg_set_task_category (m_task_t task, const char *category)</b>:
66 This function should be called after the creation of a MSG task, to define the
67 category of that task. The first parameter \c task must contain a task that was
68 created with the function \c MSG_task_create. The second parameter
69 \c category must contain a category that was previously defined by the function
72 \li <b>\c TRACE_sd_set_task_category (SD_task_t task, const char *category)</b>:
73 This function should be called after the creation of a SimDAG task, to define the
74 category of that task. The first parameter \c task must contain a task that was
75 created with the function \c MSG_task_create. The second parameter
76 \c category must contain a category that was previously defined by the function
79 \li <b>\c TRACE_declare_mark(const char *mark_type)</b>: This function
80 declares a new Paje event type in the trace file that can be used by
81 simulators to declare application-level marks. This function is
82 independent of which API is used in SimGrid.
84 \li <b>\c TRACE_mark(const char *mark_type, const char *mark_value)</b>:
85 This function creates a mark in the trace file. The first parameter
86 had to be previously declared using \c TRACE_declare_mark, the second
87 is the identifier for this mark instance. We recommend that the \c
88 mark_value (the second parameter) is a unique value for the whole
89 trace file (the whole simulation). Nevertheless, this is not a strong
90 requirement: the trace will be valid if there are multiple mark
91 identifiers for the same trace.
93 \li <b>\c TRACE_[host|link]_variable_declare (const char *variable)</b>:
94 Declare a user variable that will be associated to host/link. A variable can
95 be used to trace user variables such as the number of tasks in a server,
96 the number of clients in an application (for hosts), and so on.
98 \li <b>\c TRACE_[host|link]_variable_declare_with_color (const char
99 *var, const char *color)</b>: Same as \c
100 TRACE_[host|link]_variable_declare, but user decides which color will
101 be assigned to the variable. The color needs to be a string with three
102 numbers separated by spaces in the range [0,1]. A light-gray color can
103 be specified using "0.7 0.7 0.7" as color.
105 \li <b>\c TRACE_[host|link]_variable_[set|add|sub] (const char *[host|link], const char *variable, double value)</b>:
106 Set the value of a given user variable for a given host/link. The value
107 of this variable is always associated to the host/link. The host/link
108 parameters should be its name as the one listed in the platform file.
110 \li <b>\c TRACE_[host|link]_variable_[set|add|sub]_with_time (double time, const char *[host|link], const char *variable, double value)</b>:
111 Same as TRACE_[host|link]_variable_[set|add|sub], but let user specify
112 the time used to trace it. Users can specify a time that is not the
113 simulated clock time as defined by the core simulator. This allows
114 a fine-grain control of time definition, but should be used with
115 caution since the trace can be inconsistent if resource utilization
116 traces are also traced.
118 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub] (const char *src, const char *dst, const char *variable, double value)</b>:
119 Same as TRACE_link_variable_[set|add|sub], but now users specify a source and
120 destination hosts (as the names from the platform file). The tracing library
121 will get the corresponding route that connects those two hosts (src and dst) and
122 [set|add|sub] the value's variable for all the links of the route.
124 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub]_with_time (double time, const char *src, const char *dst, const char *variable, double value)</b>:
125 Same as TRACE_link_srcdst_variable_[set|add|sub], but user specify a time different from the simulated time.
127 \subsection tracing_tracing_options Tracing configuration Options
129 To check which tracing options are available for your simulator, you
130 can just run it with the option <b>--help-tracing</b>. These are the
131 options accepted by the tracing system of SimGrid as of today, you
132 can use them by running your simulator with the <b>--cfg=</b> switch:
137 Safe switch. It activates (or deactivates) the tracing system.
138 No other tracing options take effect if this one is not activated.
146 It activates the categorized resource utilization tracing. It should
147 be enabled if tracing categories are used by this simulator.
149 --cfg=tracing/categorized:1
153 tracing/uncategorized
155 It activates the uncategorized resource utilization tracing. Use it if
156 this simulator do not use tracing categories and resource use have to be
159 --cfg=tracing/uncategorized:1
165 A file with this name will be created to register the simulation. The file
166 is in the Paje format and can be analyzed using Triva or Paje visualization
167 tools. More information can be found in these webpages:
168 <a href="http://triva.gforge.inria.fr/">http://triva.gforge.inria.fr/</a>
169 <a href="http://paje.sourceforge.net/">http://paje.sourceforge.net/</a>
171 --cfg=tracing/filename:mytracefile.trace
173 If you do not provide this parameter, the trace file will be named simgrid.trace.
178 By default, the tracing system uses all routes in the platform file
179 to re-create a "graph" of the platform and register it in the trace file.
180 This option let the user tell the tracing system to use only the routes
181 that are composed with just one link.
183 --cfg=tracing/onelink_only:1
189 This option only has effect if this simulator is SMPI-based. Traces the MPI
190 interface and generates a trace that can be analyzed using Gantt-like
191 visualizations. Every MPI function (implemented by SMPI) is transformed in a
192 state, and point-to-point communications can be analyzed with arrows.
200 This option only has effect if this simulator is SMPI-based. The processes
201 are grouped by the hosts where they were executed.
203 --cfg=tracing/smpi/group:1
209 This option only has effect if this simulator is MSG-based. It traces the
210 behavior of all categorized MSG tasks, grouping them by hosts.
212 --cfg=tracing/msg/task:1
218 This option only has effect if this simulator is MSG-based. It traces the
219 behavior of all categorized MSG processes, grouping them by hosts. This option
220 can be used to track process location if this simulator has process migration.
222 --cfg=tracing/msg/process:1
228 This option generates a graph configuration file for Triva considering
229 categorized resource utilization.
231 --cfg=triva/categorized:graph_categorized.plist
237 This option generates a graph configuration file for Triva considering
238 uncategorized resource utilization.
240 --cfg=triva/categorized:graph_uncategorized.plist
243 \subsection tracing_tracing_example_parameters Case studies
245 Some scenarios that might help you decide which tracing options
246 you should use to analyze your simulator.
248 \li I want to trace the resource utilization of all hosts
249 and links of the platform, and my simulator <b>does not</b> use
250 the tracing API. For that, you can run a uncategorized trace
251 with the following parameters (it will work with <b>any</b> Simgrid
256 --cfg=tracing/uncategorized:1 \
257 --cfg=tracing/filename:mytracefile.trace \
258 --cfg=triva/uncategorized:uncat.plist
261 \li I want to trace only a subset of my MSG (or SimDAG) tasks.
262 For that, you will need to create tracing categories using the
263 <b>TRACE_category (...)</b> function (as explained above),
264 and then classify your tasks to a previously declared category
265 using the <b>TRACE_msg_set_task_category (...)</b>
266 (or <b>TRACE_sd_set_task_category (...)</b> for SimDAG tasks). After
267 recompiling, run your simulator with the following parameters:
271 --cfg=tracing/categorized:1 \
272 --cfg=tracing/filename:mytracefile.trace \
273 --cfg=triva/categorized:cat.plist
277 \subsection tracing_tracing_example Example of Instrumentation
279 A simplified example using the tracing mandatory functions.
282 int main (int argc, char **argv)
284 MSG_global_init (&argc, &argv);
286 //(... after deployment ...)
288 //note that category declaration must be called after MSG_create_environment
289 TRACE_category_with_color ("request", "1 0 0");
290 TRACE_category_with_color ("computation", "0.3 1 0.4");
291 TRACE_category ("finalize");
293 m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
294 m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
295 m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
296 m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
297 TRACE_msg_set_task_category (req1, "request");
298 TRACE_msg_set_task_category (req2, "request");
299 TRACE_msg_set_task_category (req3, "request");
300 TRACE_msg_set_task_category (req4, "request");
302 m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
303 TRACE_msg_set_task_category (comp, "computation");
305 m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
306 TRACE_msg_set_task_category (finalize, "finalize");
315 \subsection tracing_tracing_analyzing Analyzing the SimGrid Traces
317 The SimGrid library, during an instrumented simulation, creates a trace file in
318 the Paje file format that contains the platform utilization for the simulation
319 that was executed. The visualization analysis of this file is performed with the
320 visualization tool <a href="http://triva.gforge.inria.fr">Triva</a>, with
321 special configurations tunned to SimGrid needs. This part of the documentation
322 explains how to configure and use Triva to analyse a SimGrid trace file.
324 - <b>Installing Triva</b>: the tool is available in the INRIAGforge,
325 at <a href="http://triva.gforge.inria.fr">http://triva.gforge.inria.fr</a>.
326 Use the following command to get the sources, and then check the file
327 <i>INSTALL</i>. This file contains instructions to install
328 the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
329 be compiled in MacOSes natively, check <i>INSTALL.mac</i> file.
331 $ svn checkout svn://scm.gforge.inria.fr/svn/triva
336 - <b>Executing Triva</b>: a binary called <i>Triva</i> is available after the
337 installation (you can execute it passing <em>--help</em> to check its
338 options). If the triva binary is not available after following the
339 installation instructions, you may want to execute the following command to
340 initialize the GNUstep environment variables. We strongly recommend that you
341 use the latest GNUstep packages, and not the packages available through apt-get
342 in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
343 available packages, you can execute this command:
345 $ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
347 You should be able to see this output after the installation of triva:
349 $ ./Triva.app/Triva --help
350 Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
351 Trace Analysis through Visualization
354 --ti_frequency {double} Animation: frequency of updates
355 --ti_hide Hide the TimeInterval window
356 --ti_forward {double} Animation: value to move time-slice
357 --ti_apply Apply the configuration
358 --ti_update Update on slider change
359 --ti_animate Start animation
360 --ti_start {double} Start of time slice
361 --ti_size {double} Size of time slice
363 --comparison Compare Trace Files (Experimental)
364 --graph Configurable Graph
365 --list Print Trace Type Hierarchy
366 --hierarchy Export Trace Type Hierarchy (dot)
367 --stat Trace Statistics and Memory Utilization
368 --instances List All Trace Entities
369 --linkview Link View (Experimental)
370 --treemap Squarified Treemap
371 --merge Merge Trace Files (Experimental)
372 --check Check Trace File Integrity
374 --gc_conf {file} Graph Configuration in Property List Format
375 --gc_apply Apply the configuration
376 --gc_hide Hide the GraphConfiguration window
378 Triva expects that the user choose one of the available options
379 (currently <em>--graph</em> or <em>--treemap</em> for a visualization analysis)
380 and the trace file from the simulation.
382 - <b>Understanding Triva - time-slice</b>: the analysis of a trace file using
383 the tool always takes into account the concept of the <em>time-slice</em>.
384 This concept means that what is being visualized in the screen is always
385 calculated considering a specific time frame, with its beggining and end
386 timestamp. The time-slice is configured by the user and can be changed
387 dynamically through the window called <em>Time Interval</em> that is opened
388 whenever a trace file is being analyzed. The next figure depicts the time-slice
389 configuration window.
390 In the top of the window, in the space named <i>Trace Time</i>,
391 the two fields show the beggining of the trace (which usually starts in 0) and
392 the end (that depends on the time simulated by SimGrid). The middle of the
393 window, in the square named <i>Time Slice Configuration</i>, contains the
394 aspects related to the time-slice, including its <i>start</i> and its
395 <i>size</i>. The gray rectangle in the bottom of this part indicates the
396 <i>current time-slice</i> that is considered for the drawings. If the checkbox
397 <i>Update Drawings on Sliders Change</i> is not selected, the button
398 <i>Apply</i> must be clicked in order to inform triva that the
399 new time-slice must be considered. The bottom part of the window, in the space
400 indicated by the square <i>Time Slice Animation</i> can be used to advance
401 the time-frame automatically. The user configures the amount of time that the
402 time-frame will forward and how frequent this update will happen. Once this is
403 configured, the user clicks the <i>Play</i> button in order to see the dynamic
404 changes on the drawings.
407 <a href="triva-time_interval.png" border=0><img src="triva-time_interval.png" width="50%" border=0></a>
410 <b>Remarks:</b> when the trace has too many hosts or links, the computation to
411 take into account a new time-slice can be expensive. When this happens, the
412 <i>Frequency</i> parameter, but also updates caused by change on configurations
413 when the checkbox <i>Update Drawings on Sliders
414 Change</i> is selected will not be followed.
416 - <b>Understanding Triva - graph</b>: this part of the documention explains how
417 to analyze the traces using the graph view of Triva, when the user executes
418 the tool passing <em>--graph</em> as parameter. Triva opens three windows when
419 this parameter is used: the <i>Time Interval</i> window (previously described),
420 the <i>Graph Representation</i> window, and the <em>Graph Configuration</em>
421 window. The Graph Representation is the window where drawings take place.
422 Initially, it is completely white waiting for a proper graph configuration input
423 by the user. We start the description of this type of analysis by describing the
424 <i>Graph Configuration</i> window (depicted below). By using a particular
426 can be used to customize the graph drawing according to
427 the SimGrid trace that was created with user-specific categories. Before delving
428 into the details of this customization, let us first explain the major parts of
429 the graph configuration window. The buttons located in the top-right corner can
430 be used to delete, copy and create a new configuration. The checkbox in the
431 top-middle part of the window indicates if the configuration typed in the
432 textfield is syntactically correct (we are using the non-XML
433 <a href="http://en.wikipedia.org/wiki/Property_list">Property List Format</a> to
434 describe the configuration). The pop-up button located on the top-left corner
435 indicates the selected configuration (the user can have multiple graph
436 configurations). The bottom-left text field contains the name of the current
437 configuration (updates on this field must be followed by typing enter on the
438 keyboard to take into account the name change). The bottom-right <em>Apply</em>
439 button activates the current configuration, resulting on an update on the graph
443 <a href="triva-graph_configuration.png" border=0><img src="triva-graph_configuration.png" width="50%" border=0></a>
446 <b>Basic SimGrid Configuration</b>: The figure shows in the big textfield the
447 basic configuration that should be used during the analysis of a SimGrid trace
448 file. The basic logic of the configuration is as follows:
454 The nodes of the graph will be created based on the <i>node</i> parameter, which
455 in this case is the different <em>"HOST"</em>s of the platform
456 used to simulate. The <i>edge</i> parameter indicates that the edges of the
457 graph will be created based on the <em>"LINK"</em>s of the platform. After the
458 definition of these two parameters, the configuration must detail how
459 <em>HOST</em>s and <em>LINK</em>s should be drawn. For that, the configuration
460 must have an entry for each of the types used. For <em>HOST</em>, as basic
461 configuration, we have:
468 The parameter <em>size</em> indicates which variable from the trace file will be
469 used to define the size of the node HOST in the visualization. If the simulation
470 was executed with availability traces, the size of the nodes will be changed
471 according to these traces. The parameter <em>scale</em> indicates if the value
472 of the variable is <em>global</em> or <em>local</em>. If it is global, the value
473 will be relative to the power of all other hosts, if it is local, the value will
475 For <em>LINK</em> we have:
485 For the types specified in the <em>edge</em> parameter (such as <em>LINK</em>),
486 the configuration must contain two additional parameters: <em>src</em> and
487 <em>dst</em> that are used to properly identify which nodes this edge is
488 connecting. The values <em>source</em> and <em>destination</em> are always present
489 in the SimGrid trace file and should not be changed in the configuration. The
490 parameter <em>size</em> for the LINK, in this case, is configured as the
491 variable <em>bandwidth</em>, with a <em>global</em> scale. The scale meaning
492 here is exactly the same used for nodes. The last parameter is the GraphViz
493 algorithm used to calculate the position of the nodes in the graph
496 graphviz-algorithm = neato;
499 <b>Customizing the Graph Representation</b>: triva is capable to handle
500 a customized graph representation based on the variables present in the trace
501 file. In the case of SimGrid, every time a category is created for tasks, two
502 variables in the trace file are defined: one to indicate node utilization (how
503 much power was used by that task category), and another to indicate link
504 utilization (how much bandwidth was used by that category). For instance, if the
505 user declares a category named <i>request</i>, there will be variables named
506 <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b> for power and
507 <b>b</b> for bandwidth). It is important to notice that the variable
508 <i>prequest</i> in this case is only available for HOST, and
509 <i>brequest</i> is only available for LINK. <b>Example</b>: suppose there are
510 two categories for tasks: request and compute. To create a customized graph
511 representation with a proportional separation of host and link utilization, use
512 as configuration for HOST and LINK this:
521 values = (prequest, pcomputation);
534 values = (brequest, bcomputation);
538 Where <i>sep_host</i> contains a composition of type <i>separation</i> where
539 its max size is the <i>power</i> of the host and the variables <i>prequest</i>
540 and <i>pcomputation</i> are drawn proportionally to the size of the HOST. And
541 <i>sep_link</i> is also a separation where max is defined as the
542 <i>bandwidth</i> of the link, and the variables <i>brequest</i> and
543 <i>bcomputation</i> are drawn proportionally within a LINK.
544 <i>This configuration enables the analysis of resource utilization by MSG tasks,
545 and the identification of load-balancing issues, network bottlenecks, for
547 <b>Other compositions</b>: besides <i>separation</i>, it is possible to use
548 other types of compositions, such as gradients, and colors, like this:
553 values = (numberOfTasks);
557 values = (is_server);
560 Where <i>gra_host</i> creates a gradient within a node of the graph, using a
561 global scale and using as value a variable called <i>numberOfTasks</i>, that
562 could be declared by the user using the optional tracing functions of SimGrid.
563 If scale is global, the max and min value for the gradient will be equal to the
564 max and min numberOfTasks among all hosts, and if scale is local, the max and
565 min value based on the value of numberOfTasks locally in each host.
566 And <i>color_host</i> composition draws a square based on a positive value of
567 the variable <i>is_server</i>, that could also be defined by the user using the
568 SimGrid tracing functions. \n
569 <b>The Graph Visualization</b>: The next figure shows a graph visualization of a
570 given time-slice of the masterslave_forwarder example (present in the SimGrid
571 sources). The red color indicates tasks from the <i>compute</i> category. This
572 visualization was generated with the following configuration:
585 values = (pcompute, pfinalize);
591 size = bandwidth;\section tracing_tracing Tracing Simulations for Visualization
593 The trace visualization is widely used to observe and understand the behavior
594 of parallel applications and distributed algorithms. Usually, this is done in a
595 two-step fashion: the user instruments the application and the traces are
596 analyzed after the end of the execution. The visualization itself can highlights
597 unexpected behaviors, bottlenecks and sometimes can be used to correct
598 distributed algorithms. The SimGrid team has instrumented the library
599 in order to let users trace their simulations and analyze them. This part of the
600 user manual explains how the tracing-related features can be enabled and used
601 during the development of simulators using the SimGrid library.
603 \subsection tracing_tracing_howitworks How it works
605 For now, the SimGrid library is instrumented so users can trace the <b>platform
606 utilization</b> using the MSG, SimDAG and SMPI interface. This means that the tracing will
607 register how much power is used for each host and how much bandwidth is used for
608 each link of the platform. The idea with this type of tracing is to observe the
609 overall view of resources utilization in the first place, especially the
610 identification of bottlenecks, load-balancing among hosts, and so on.
612 The idea of the tracing facilities is to give SimGrid users to possibility to
613 classify MSG and SimDAG tasks by category, tracing the platform utilization
614 (hosts and links) for each of the categories. For that,
615 the tracing interface enables the declaration of categories and a function to
616 mark a task with a previously declared category. <em>The tasks that are not
617 classified according to a category are not traced</em>. Even if the user
618 does not specify any category, the simulations can still be traced in terms
619 of resource utilization by using a special parameter that is detailed below.
621 \subsection tracing_tracing_enabling Enabling using CMake
623 With the sources of SimGrid, it is possible to enable the tracing
624 using the parameter <b>-Denable_tracing=ON</b> when the cmake is executed.
625 The section \ref tracing_tracing_functions describes all the functions available
626 when this Cmake options is activated. These functions will have no effect
627 if SimGrid is configured without this option (they are wiped-out by the
631 $ cmake -Denable_tracing=ON .
635 \subsection tracing_tracing_functions Tracing Functions
637 \li <b>\c TRACE_category (const char *category)</b>: This function should be used
638 to define a user category. The category can be used to differentiate the tasks
639 that are created during the simulation (for example, tasks from server1,
640 server2, or request tasks, computation tasks, communication tasks).
641 All resource utilization (host power and link bandwidth) will be
642 classified according to the task category. Tasks that do not belong to a
643 category are not traced. The color for the category that is being declared
644 is random (use next function to specify a color).
646 \li <b>\c TRACE_category_with_color (const char *category, const char *color)</b>: Same
647 as TRACE_category, but let user specify a color encoded as a RGB-like string with
648 three floats from 0 to 1. So, to specify a red color, the user can pass "1 0 0" as
649 color parameter. A light-gray color can be specified using "0.7 0.7 0.7" as color.
651 \li <b>\c TRACE_msg_set_task_category (m_task_t task, const char *category)</b>:
652 This function should be called after the creation of a MSG task, to define the
653 category of that task. The first parameter \c task must contain a task that was
654 created with the function \c MSG_task_create. The second parameter
655 \c category must contain a category that was previously defined by the function
658 \li <b>\c TRACE_sd_set_task_category (SD_task_t task, const char *category)</b>:
659 This function should be called after the creation of a SimDAG task, to define the
660 category of that task. The first parameter \c task must contain a task that was
661 created with the function \c MSG_task_create. The second parameter
662 \c category must contain a category that was previously defined by the function
665 \li <b>\c TRACE_[host|link]_variable_declare (const char *variable)</b>:
666 Declare a user variable that will be associated to host/link. A variable can
667 be used to trace user variables such as the number of tasks in a server,
668 the number of clients in an application (for hosts), and so on.
670 \li <b>\c TRACE_[host|link]_variable_[set|add|sub] (const char *[host|link], const char *variable, double value)</b>:
671 Set the value of a given user variable for a given host/link. The value
672 of this variable is always associated to the host/link. The host/link
673 parameters should be its name as the one listed in the platform file.
675 \li <b>\c TRACE_[host|link]_variable_[set|add|sub]_with_time (double time, const char *[host|link], const char *variable, double value)</b>:
676 Same as TRACE_[host|link]_variable_[set|add|sub], but let user specify
677 the time used to trace it. Users can specify a time that is not the
678 simulated clock time as defined by the core simulator. This allows
679 a fine-grain control of time definition, but should be used with
680 caution since the trace can be inconsistent if resource utilization
681 traces are also traced.
683 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub] (const char *src, const char *dst, const char *variable, double value)</b>:
684 Same as TRACE_link_variable_[set|add|sub], but now users specify a source and
685 destination hosts (as the names from the platform file). The tracing library
686 will get the corresponding route that connects those two hosts (src and dst) and
687 [set|add|sub] the value's variable for all the links of the route.
689 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub]_with_time (double time, const char *src, const char *dst, const char *variable, double value)</b>:
690 Same as TRACE_link_srcdst_variable_[set|add|sub], but user specify a time different from the simulated time.
692 \subsection tracing_tracing_options Tracing configuration Options
694 These are the options accepted by the tracing system of SimGrid:
699 Safe switch. It activates (or deactivates) the tracing system.
700 No other tracing options take effect if this one is not activated.
705 Register the simulation platform in the trace file.
710 By default, the tracing system uses all routes in the platform file
711 to re-create a "graph" of the platform and register it in the trace file.
712 This option let the user tell the tracing system to use only the routes
713 that are composed with just one link.
718 It activates the categorized resource utilization tracing. It should
719 be enabled if tracing categories are used by this simulator.
722 tracing/uncategorized
724 It activates the uncategorized resource utilization tracing. Use it if
725 this simulator do not use tracing categories and resource use have to be
731 A file with this name will be created to register the simulation. The file
732 is in the Paje format and can be analyzed using Triva or Paje visualization
733 tools. More information can be found in these webpages:
734 <a href="http://triva.gforge.inria.fr/">http://triva.gforge.inria.fr/</a>
735 <a href="http://paje.sourceforge.net/">http://paje.sourceforge.net/</a>
740 This option only has effect if this simulator is SMPI-based. Traces the MPI
741 interface and generates a trace that can be analyzed using Gantt-like
742 visualizations. Every MPI function (implemented by SMPI) is transformed in a
743 state, and point-to-point communications can be analyzed with arrows.
748 This option only has effect if this simulator is SMPI-based. The processes
749 are grouped by the hosts where they were executed.
754 This option only has effect if this simulator is MSG-based. It traces the
755 behavior of all categorized MSG tasks, grouping them by hosts.
760 This option only has effect if this simulator is MSG-based. It traces the
761 behavior of all categorized MSG processes, grouping them by hosts. This option
762 can be used to track process location if this simulator has process migration.
766 triva/categorized:graph_categorized.plist
768 This option generates a graph configuration file for Triva considering
769 categorized resource utilization.
772 triva/uncategorized:graph_uncategorized.plist
774 This option generates a graph configuration file for Triva considering
775 uncategorized resource utilization.
777 \subsection tracing_tracing_example Example of Instrumentation
779 A simplified example using the tracing mandatory functions.
782 int main (int argc, char **argv)
784 MSG_global_init (&argc, &argv);
786 //(... after deployment ...)
788 //note that category declaration must be called after MSG_create_environment
789 TRACE_category_with_color ("request", "1 0 0");
790 TRACE_category_with_color ("computation", "0.3 1 0.4");
791 TRACE_category ("finalize");
793 m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
794 m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
795 m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
796 m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
797 TRACE_msg_set_task_category (req1, "request");
798 TRACE_msg_set_task_category (req2, "request");
799 TRACE_msg_set_task_category (req3, "request");
800 TRACE_msg_set_task_category (req4, "request");
802 m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
803 TRACE_msg_set_task_category (comp, "computation");
805 m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
806 TRACE_msg_set_task_category (finalize, "finalize");
815 \subsection tracing_tracing_analyzing Analyzing the SimGrid Traces
817 The SimGrid library, during an instrumented simulation, creates a trace file in
818 the Paje file format that contains the platform utilization for the simulation
819 that was executed. The visualization analysis of this file is performed with the
820 visualization tool <a href="http://triva.gforge.inria.fr">Triva</a>, with
821 special configurations tunned to SimGrid needs. This part of the documentation
822 explains how to configure and use Triva to analyse a SimGrid trace file.
824 - <b>Installing Triva</b>: the tool is available in the INRIAGforge,
825 at <a href="http://triva.gforge.inria.fr">http://triva.gforge.inria.fr</a>.
826 Use the following command to get the sources, and then check the file
827 <i>INSTALL</i>. This file contains instructions to install
828 the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
829 be compiled in MacOSes natively, check <i>INSTALL.mac</i> file.
831 $ svn checkout svn://scm.gforge.inria.fr/svn/triva
836 - <b>Executing Triva</b>: a binary called <i>Triva</i> is available after the
837 installation (you can execute it passing <em>--help</em> to check its
838 options). If the triva binary is not available after following the
839 installation instructions, you may want to execute the following command to
840 initialize the GNUstep environment variables. We strongly recommend that you
841 use the latest GNUstep packages, and not the packages available through apt-get
842 in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
843 available packages, you can execute this command:
845 $ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
847 You should be able to see this output after the installation of triva:
849 $ ./Triva.app/Triva --help
850 Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
851 Trace Analysis through Visualization
854 --ti_frequency {double} Animation: frequency of updates
855 --ti_hide Hide the TimeInterval window
856 --ti_forward {double} Animation: value to move time-slice
857 --ti_apply Apply the configuration
858 --ti_update Update on slider change
859 --ti_animate Start animation
860 --ti_start {double} Start of time slice
861 --ti_size {double} Size of time slice
863 --comparison Compare Trace Files (Experimental)
864 --graph Configurable Graph
865 --list Print Trace Type Hierarchy
866 --hierarchy Export Trace Type Hierarchy (dot)
867 --stat Trace Statistics and Memory Utilization
868 --instances List All Trace Entities
869 --linkview Link View (Experimental)
870 --treemap Squarified Treemap
871 --merge Merge Trace Files (Experimental)
872 --check Check Trace File Integrity
874 --gc_conf {file} Graph Configuration in Property List Format
875 --gc_apply Apply the configuration
876 --gc_hide Hide the GraphConfiguration window
878 Triva expects that the user choose one of the available options
879 (currently <em>--graph</em> or <em>--treemap</em> for a visualization analysis)
880 and the trace file from the simulation.
882 - <b>Understanding Triva - time-slice</b>: the analysis of a trace file using
883 the tool always takes into account the concept of the <em>time-slice</em>.
884 This concept means that what is being visualized in the screen is always
885 calculated considering a specific time frame, with its beggining and end
886 timestamp. The time-slice is configured by the user and can be changed
887 dynamically through the window called <em>Time Interval</em> that is opened
888 whenever a trace file is being analyzed. The next figure depicts the time-slice
889 configuration window.
890 In the top of the window, in the space named <i>Trace Time</i>,
891 the two fields show the beggining of the trace (which usually starts in 0) and
892 the end (that depends on the time simulated by SimGrid). The middle of the
893 window, in the square named <i>Time Slice Configuration</i>, contains the
894 aspects related to the time-slice, including its <i>start</i> and its
895 <i>size</i>. The gray rectangle in the bottom of this part indicates the
896 <i>current time-slice</i> that is considered for the drawings. If the checkbox
897 <i>Update Drawings on Sliders Change</i> is not selected, the button
898 <i>Apply</i> must be clicked in order to inform triva that the
899 new time-slice must be considered. The bottom part of the window, in the space
900 indicated by the square <i>Time Slice Animation</i> can be used to advance
901 the time-frame automatically. The user configures the amount of time that the
902 time-frame will forward and how frequent this update will happen. Once this is
903 configured, the user clicks the <i>Play</i> button in order to see the dynamic
904 changes on the drawings.
907 <a href="triva-time_interval.png" border=0><img src="triva-time_interval.png" width="50%" border=0></a>
910 <b>Remarks:</b> when the trace has too many hosts or links, the computation to
911 take into account a new time-slice can be expensive. When this happens, the
912 <i>Frequency</i> parameter, but also updates caused by change on configurations
913 when the checkbox <i>Update Drawings on Sliders
914 Change</i> is selected will not be followed.
916 - <b>Understanding Triva - graph</b>: this part of the documention explains how
917 to analyze the traces using the graph view of Triva, when the user executes
918 the tool passing <em>--graph</em> as parameter. Triva opens three windows when
919 this parameter is used: the <i>Time Interval</i> window (previously described),
920 the <i>Graph Representation</i> window, and the <em>Graph Configuration</em>
921 window. The Graph Representation is the window where drawings take place.
922 Initially, it is completely white waiting for a proper graph configuration input
923 by the user. We start the description of this type of analysis by describing the
924 <i>Graph Configuration</i> window (depicted below). By using a particular
926 can be used to customize the graph drawing according to
927 the SimGrid trace that was created with user-specific categories. Before delving
928 into the details of this customization, let us first explain the major parts of
929 the graph configuration window. The buttons located in the top-right corner can
930 be used to delete, copy and create a new configuration. The checkbox in the
931 top-middle part of the window indicates if the configuration typed in the
932 textfield is syntactically correct (we are using the non-XML
933 <a href="http://en.wikipedia.org/wiki/Property_list">Property List Format</a> to
934 describe the configuration). The pop-up button located on the top-left corner
935 indicates the selected configuration (the user can have multiple graph
936 configurations). The bottom-left text field contains the name of the current
937 configuration (updates on this field must be followed by typing enter on the
938 keyboard to take into account the name change). The bottom-right <em>Apply</em>
939 button activates the current configuration, resulting on an update on the graph
943 <a href="triva-graph_configuration.png" border=0><img src="triva-graph_configuration.png" width="50%" border=0></a>
946 <b>Basic SimGrid Configuration</b>: The figure shows in the big textfield the
947 basic configuration that should be used during the analysis of a SimGrid trace
948 file. The basic logic of the configuration is as follows:
954 The nodes of the graph will be created based on the <i>node</i> parameter, which
955 in this case is the different <em>"HOST"</em>s of the platform
956 used to simulate. The <i>edge</i> parameter indicates that the edges of the
957 graph will be created based on the <em>"LINK"</em>s of the platform. After the
958 definition of these two parameters, the configuration must detail how
959 <em>HOST</em>s and <em>LINK</em>s should be drawn. For that, the configuration
960 must have an entry for each of the types used. For <em>HOST</em>, as basic
961 configuration, we have:
968 The parameter <em>size</em> indicates which variable from the trace file will be
969 used to define the size of the node HOST in the visualization. If the simulation
970 was executed with availability traces, the size of the nodes will be changed
971 according to these traces. The parameter <em>scale</em> indicates if the value
972 of the variable is <em>global</em> or <em>local</em>. If it is global, the value
973 will be relative to the power of all other hosts, if it is local, the value will
975 For <em>LINK</em> we have:
985 For the types specified in the <em>edge</em> parameter (such as <em>LINK</em>),
986 the configuration must contain two additional parameters: <em>src</em> and
987 <em>dst</em> that are used to properly identify which nodes this edge is
988 connecting. The values <em>source</em> and <em>destination</em> are always present
989 in the SimGrid trace file and should not be changed in the configuration. The
990 parameter <em>size</em> for the LINK, in this case, is configured as the
991 variable <em>bandwidth</em>, with a <em>global</em> scale. The scale meaning
992 here is exactly the same used for nodes. The last parameter is the GraphViz
993 algorithm used to calculate the position of the nodes in the graph
996 graphviz-algorithm = neato;
999 <b>Customizing the Graph Representation</b>: triva is capable to handle
1000 a customized graph representation based on the variables present in the trace
1001 file. In the case of SimGrid, every time a category is created for tasks, two
1002 variables in the trace file are defined: one to indicate node utilization (how
1003 much power was used by that task category), and another to indicate link
1004 utilization (how much bandwidth was used by that category). For instance, if the
1005 user declares a category named <i>request</i>, there will be variables named
1006 <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b> for power and
1007 <b>b</b> for bandwidth). It is important to notice that the variable
1008 <i>prequest</i> in this case is only available for HOST, and
1009 <i>brequest</i> is only available for LINK. <b>Example</b>: suppose there are
1010 two categories for tasks: request and compute. To create a customized graph
1011 representation with a proportional separation of host and link utilization, use
1012 as configuration for HOST and LINK this:
1021 values = (prequest, pcomputation);
1034 values = (brequest, bcomputation);
1038 Where <i>sep_host</i> contains a composition of type <i>separation</i> where
1039 its max size is the <i>power</i> of the host and the variables <i>prequest</i>
1040 and <i>pcomputation</i> are drawn proportionally to the size of the HOST. And
1041 <i>sep_link</i> is also a separation where max is defined as the
1042 <i>bandwidth</i> of the link, and the variables <i>brequest</i> and
1043 <i>bcomputation</i> are drawn proportionally within a LINK.
1044 <i>This configuration enables the analysis of resource utilization by MSG tasks,
1045 and the identification of load-balancing issues, network bottlenecks, for
1047 <b>Other compositions</b>: besides <i>separation</i>, it is possible to use
1048 other types of compositions, such as gradients, and colors, like this:
1053 values = (numberOfTasks);
1057 values = (is_server);
1060 Where <i>gra_host</i> creates a gradient within a node of the graph, using a
1061 global scale and using as value a variable called <i>numberOfTasks</i>, that
1062 could be declared by the user using the optional tracing functions of SimGrid.
1063 If scale is global, the max and min value for the gradient will be equal to the
1064 max and min numberOfTasks among all hosts, and if scale is local, the max and
1065 min value based on the value of numberOfTasks locally in each host.
1066 And <i>color_host</i> composition draws a square based on a positive value of
1067 the variable <i>is_server</i>, that could also be defined by the user using the
1068 SimGrid tracing functions. \n
1069 <b>The Graph Visualization</b>: The next figure shows a graph visualization of a
1070 given time-slice of the masterslave_forwarder example (present in the SimGrid
1071 sources). The red color indicates tasks from the <i>compute</i> category. This
1072 visualization was generated with the following configuration:
1085 values = (pcompute, pfinalize);
1097 values = (bcompute, bfinalize);
1100 graphviz-algorithm = neato;
1105 <a href="triva-graph_visualization.png" border=0><img src="triva-graph_visualization.png" width="50%" border=0></a>
1109 - <b>Understading Triva - colors</b>: An important issue when using Triva is how
1110 to define colors. To do that, we have to know which variables are defined in
1111 the trace file generated by the SimGrid library. The parameter <em>--list</em>
1112 lists the variables for a given trace file:
1114 $ Triva -l masterslave_forwarder.trace
1132 We can see that HOST has seven variables (from power to pfinalize) and LINK has
1133 four (from bandwidth to bfinalize). To define a red color for the
1134 <i>pcompute</i> and <i>bcompute</i> (which are defined based on user category
1135 <i>compute</i>), execute:
1137 $ defaults write Triva 'pcompute Color' '1 0 0'
1138 $ defaults write Triva 'bcompute Color' '1 0 0'
1140 Where the three numbers in each line are the RGB color with values from 0 to 1.
1147 values = (bcompute, bfinalize);
1150 graphviz-algorithm = neato;
1155 <a href="triva-graph_visualization.png" border=0><img src="triva-graph_visualization.png" width="50%" border=0></a>
1159 - <b>Understading Triva - colors</b>: An important issue when using Triva is how
1160 to define colors. To do that, we have to know which variables are defined in
1161 the trace file generated by the SimGrid library. The parameter <em>--list</em>
1162 lists the variables for a given trace file:
1164 $ Triva -l masterslave_forwarder.trace
1182 We can see that HOST has seven variables (from power to pfinalize) and LINK has
1183 four (from bandwidth to bfinalize). To define a red color for the
1184 <i>pcompute</i> and <i>bcompute</i> (which are defined based on user category
1185 <i>compute</i>), execute:
1187 $ defaults write Triva 'pcompute Color' '1 0 0'
1188 $ defaults write Triva 'bcompute Color' '1 0 0'
1190 Where the three numbers in each line are the RGB color with values from 0 to 1.