1 /*! \page tracing Tracing Simulations for Visualization
3 \htmlinclude .tracing.doc.toc
5 \section tracing_tracing Tracing Simulations for Visualization
7 The trace visualization is widely used to observe and understand the behavior
8 of parallel applications and distributed algorithms. Usually, this is done in a
9 two-step fashion: the user instruments the application and the traces are
10 analyzed after the end of the execution. The visualization itself can highlights
11 unexpected behaviors, bottlenecks and sometimes can be used to correct
12 distributed algorithms. The SimGrid team has instrumented the library
13 in order to let users trace their simulations and analyze them. This part of the
14 user manual explains how the tracing-related features can be enabled and used
15 during the development of simulators using the SimGrid library.
17 \subsection tracing_tracing_howitworks How it works
19 For now, the SimGrid library is instrumented so users can trace the <b>platform
20 utilization</b> using the MSG, SimDAG and SMPI interface. This means that the tracing will
21 register how much power is used for each host and how much bandwidth is used for
22 each link of the platform. The idea with this type of tracing is to observe the
23 overall view of resources utilization in the first place, especially the
24 identification of bottlenecks, load-balancing among hosts, and so on.
26 The idea of the tracing facilities is to give SimGrid users to possibility to
27 classify MSG and SimDAG tasks by category, tracing the platform utilization
28 (hosts and links) for each of the categories. For that,
29 the tracing interface enables the declaration of categories and a function to
30 mark a task with a previously declared category. <em>The tasks that are not
31 classified according to a category are not traced</em>. Even if the user
32 does not specify any category, the simulations can still be traced in terms
33 of resource utilization by using a special parameter that is detailed below.
35 \subsection tracing_tracing_enabling Enabling using CMake
37 With the sources of SimGrid, it is possible to enable the tracing
38 using the parameter <b>-Denable_tracing=ON</b> when the cmake is executed.
39 The section \ref tracing_tracing_functions describes all the functions available
40 when this Cmake options is activated. These functions will have no effect
41 if SimGrid is configured without this option (they are wiped-out by the
45 $ cmake -Denable_tracing=ON .
49 \subsection tracing_tracing_functions Tracing Functions
51 \li <b>\c TRACE_category (const char *category)</b>: This function should be used
52 to define a user category. The category can be used to differentiate the tasks
53 that are created during the simulation (for example, tasks from server1,
54 server2, or request tasks, computation tasks, communication tasks).
55 All resource utilization (host power and link bandwidth) will be
56 classified according to the task category. Tasks that do not belong to a
57 category are not traced. The color for the category that is being declared
58 is random (use next function to specify a color).
60 \li <b>\c TRACE_category_with_color (const char *category, const char *color)</b>: Same
61 as TRACE_category, but let user specify a color encoded as a RGB-like string with
62 three floats from 0 to 1. So, to specify a red color, the user can pass "1 0 0" as
63 color parameter. A light-gray color can be specified using "0.7 0.7 0.7" as color.
65 \li <b>\c TRACE_msg_set_task_category (m_task_t task, const char *category)</b>:
66 This function should be called after the creation of a MSG task, to define the
67 category of that task. The first parameter \c task must contain a task that was
68 created with the function \c MSG_task_create. The second parameter
69 \c category must contain a category that was previously defined by the function
72 \li <b>\c TRACE_sd_set_task_category (SD_task_t task, const char *category)</b>:
73 This function should be called after the creation of a SimDAG task, to define the
74 category of that task. The first parameter \c task must contain a task that was
75 created with the function \c MSG_task_create. The second parameter
76 \c category must contain a category that was previously defined by the function
79 \li <b>\c TRACE_[host|link]_variable_declare (const char *variable)</b>:
80 Declare a user variable that will be associated to host/link. A variable can
81 be used to trace user variables such as the number of tasks in a server,
82 the number of clients in an application (for hosts), and so on.
84 \li <b>\c TRACE_[host|link]_variable_[set|add|sub] (const char *[host|link], const char *variable, double value)</b>:
85 Set the value of a given user variable for a given host/link. The value
86 of this variable is always associated to the host/link. The host/link
87 parameters should be its name as the one listed in the platform file.
89 \li <b>\c TRACE_[host|link]_variable_[set|add|sub]_with_time (double time, const char *[host|link], const char *variable, double value)</b>:
90 Same as TRACE_[host|link]_variable_[set|add|sub], but let user specify
91 the time used to trace it. Users can specify a time that is not the
92 simulated clock time as defined by the core simulator. This allows
93 a fine-grain control of time definition, but should be used with
94 caution since the trace can be inconsistent if resource utilization
95 traces are also traced.
97 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub] (const char *src, const char *dst, const char *variable, double value)</b>:
98 Same as TRACE_link_variable_[set|add|sub], but now users specify a source and
99 destination hosts (as the names from the platform file). The tracing library
100 will get the corresponding route that connects those two hosts (src and dst) and
101 [set|add|sub] the value's variable for all the links of the route.
103 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub]_with_time (double time, const char *src, const char *dst, const char *variable, double value)</b>:
104 Same as TRACE_link_srcdst_variable_[set|add|sub], but user specify a time different from the simulated time.
106 \subsection tracing_tracing_options Tracing configuration Options
108 To check which tracing options are available for your simulator, you
109 can just run it with the option <b>--help-tracing</b>. These are the
110 options accepted by the tracing system of SimGrid as of today, you
111 can use them by running your simulator with the <b>--cfg=</b> switch:
116 Safe switch. It activates (or deactivates) the tracing system.
117 No other tracing options take effect if this one is not activated.
125 It activates the categorized resource utilization tracing. It should
126 be enabled if tracing categories are used by this simulator.
128 --cfg=tracing/categorized:1
132 tracing/uncategorized
134 It activates the uncategorized resource utilization tracing. Use it if
135 this simulator do not use tracing categories and resource use have to be
138 --cfg=tracing/uncategorized:1
144 A file with this name will be created to register the simulation. The file
145 is in the Paje format and can be analyzed using Triva or Paje visualization
146 tools. More information can be found in these webpages:
147 <a href="http://triva.gforge.inria.fr/">http://triva.gforge.inria.fr/</a>
148 <a href="http://paje.sourceforge.net/">http://paje.sourceforge.net/</a>
150 --cfg=tracing/filename:mytracefile.trace
152 If you do not provide this parameter, the trace file will be named simgrid.trace.
157 By default, the tracing system uses all routes in the platform file
158 to re-create a "graph" of the platform and register it in the trace file.
159 This option let the user tell the tracing system to use only the routes
160 that are composed with just one link.
162 --cfg=tracing/onelink_only:1
168 This option only has effect if this simulator is SMPI-based. Traces the MPI
169 interface and generates a trace that can be analyzed using Gantt-like
170 visualizations. Every MPI function (implemented by SMPI) is transformed in a
171 state, and point-to-point communications can be analyzed with arrows.
179 This option only has effect if this simulator is SMPI-based. The processes
180 are grouped by the hosts where they were executed.
182 --cfg=tracing/smpi/group:1
188 This option only has effect if this simulator is MSG-based. It traces the
189 behavior of all categorized MSG tasks, grouping them by hosts.
191 --cfg=tracing/msg/task:1
197 This option only has effect if this simulator is MSG-based. It traces the
198 behavior of all categorized MSG processes, grouping them by hosts. This option
199 can be used to track process location if this simulator has process migration.
201 --cfg=tracing/msg/process:1
207 This option generates a graph configuration file for Triva considering
208 categorized resource utilization.
210 --cfg=triva/categorized:graph_categorized.plist
216 This option generates a graph configuration file for Triva considering
217 uncategorized resource utilization.
219 --cfg=triva/categorized:graph_uncategorized.plist
222 \subsection tracing_tracing_example Example of Instrumentation
224 A simplified example using the tracing mandatory functions.
227 int main (int argc, char **argv)
229 MSG_global_init (&argc, &argv);
231 //(... after deployment ...)
233 //note that category declaration must be called after MSG_create_environment
234 TRACE_category_with_color ("request", "1 0 0");
235 TRACE_category_with_color ("computation", "0.3 1 0.4");
236 TRACE_category ("finalize");
238 m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
239 m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
240 m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
241 m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
242 TRACE_msg_set_task_category (req1, "request");
243 TRACE_msg_set_task_category (req2, "request");
244 TRACE_msg_set_task_category (req3, "request");
245 TRACE_msg_set_task_category (req4, "request");
247 m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
248 TRACE_msg_set_task_category (comp, "computation");
250 m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
251 TRACE_msg_set_task_category (finalize, "finalize");
260 \subsection tracing_tracing_analyzing Analyzing the SimGrid Traces
262 The SimGrid library, during an instrumented simulation, creates a trace file in
263 the Paje file format that contains the platform utilization for the simulation
264 that was executed. The visualization analysis of this file is performed with the
265 visualization tool <a href="http://triva.gforge.inria.fr">Triva</a>, with
266 special configurations tunned to SimGrid needs. This part of the documentation
267 explains how to configure and use Triva to analyse a SimGrid trace file.
269 - <b>Installing Triva</b>: the tool is available in the INRIAGforge,
270 at <a href="http://triva.gforge.inria.fr">http://triva.gforge.inria.fr</a>.
271 Use the following command to get the sources, and then check the file
272 <i>INSTALL</i>. This file contains instructions to install
273 the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
274 be compiled in MacOSes natively, check <i>INSTALL.mac</i> file.
276 $ svn checkout svn://scm.gforge.inria.fr/svn/triva
281 - <b>Executing Triva</b>: a binary called <i>Triva</i> is available after the
282 installation (you can execute it passing <em>--help</em> to check its
283 options). If the triva binary is not available after following the
284 installation instructions, you may want to execute the following command to
285 initialize the GNUstep environment variables. We strongly recommend that you
286 use the latest GNUstep packages, and not the packages available through apt-get
287 in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
288 available packages, you can execute this command:
290 $ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
292 You should be able to see this output after the installation of triva:
294 $ ./Triva.app/Triva --help
295 Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
296 Trace Analysis through Visualization
299 --ti_frequency {double} Animation: frequency of updates
300 --ti_hide Hide the TimeInterval window
301 --ti_forward {double} Animation: value to move time-slice
302 --ti_apply Apply the configuration
303 --ti_update Update on slider change
304 --ti_animate Start animation
305 --ti_start {double} Start of time slice
306 --ti_size {double} Size of time slice
308 --comparison Compare Trace Files (Experimental)
309 --graph Configurable Graph
310 --list Print Trace Type Hierarchy
311 --hierarchy Export Trace Type Hierarchy (dot)
312 --stat Trace Statistics and Memory Utilization
313 --instances List All Trace Entities
314 --linkview Link View (Experimental)
315 --treemap Squarified Treemap
316 --merge Merge Trace Files (Experimental)
317 --check Check Trace File Integrity
319 --gc_conf {file} Graph Configuration in Property List Format
320 --gc_apply Apply the configuration
321 --gc_hide Hide the GraphConfiguration window
323 Triva expects that the user choose one of the available options
324 (currently <em>--graph</em> or <em>--treemap</em> for a visualization analysis)
325 and the trace file from the simulation.
327 - <b>Understanding Triva - time-slice</b>: the analysis of a trace file using
328 the tool always takes into account the concept of the <em>time-slice</em>.
329 This concept means that what is being visualized in the screen is always
330 calculated considering a specific time frame, with its beggining and end
331 timestamp. The time-slice is configured by the user and can be changed
332 dynamically through the window called <em>Time Interval</em> that is opened
333 whenever a trace file is being analyzed. The next figure depicts the time-slice
334 configuration window.
335 In the top of the window, in the space named <i>Trace Time</i>,
336 the two fields show the beggining of the trace (which usually starts in 0) and
337 the end (that depends on the time simulated by SimGrid). The middle of the
338 window, in the square named <i>Time Slice Configuration</i>, contains the
339 aspects related to the time-slice, including its <i>start</i> and its
340 <i>size</i>. The gray rectangle in the bottom of this part indicates the
341 <i>current time-slice</i> that is considered for the drawings. If the checkbox
342 <i>Update Drawings on Sliders Change</i> is not selected, the button
343 <i>Apply</i> must be clicked in order to inform triva that the
344 new time-slice must be considered. The bottom part of the window, in the space
345 indicated by the square <i>Time Slice Animation</i> can be used to advance
346 the time-frame automatically. The user configures the amount of time that the
347 time-frame will forward and how frequent this update will happen. Once this is
348 configured, the user clicks the <i>Play</i> button in order to see the dynamic
349 changes on the drawings.
352 <a href="triva-time_interval.png" border=0><img src="triva-time_interval.png" width="50%" border=0></a>
355 <b>Remarks:</b> when the trace has too many hosts or links, the computation to
356 take into account a new time-slice can be expensive. When this happens, the
357 <i>Frequency</i> parameter, but also updates caused by change on configurations
358 when the checkbox <i>Update Drawings on Sliders
359 Change</i> is selected will not be followed.
361 - <b>Understanding Triva - graph</b>: this part of the documention explains how
362 to analyze the traces using the graph view of Triva, when the user executes
363 the tool passing <em>--graph</em> as parameter. Triva opens three windows when
364 this parameter is used: the <i>Time Interval</i> window (previously described),
365 the <i>Graph Representation</i> window, and the <em>Graph Configuration</em>
366 window. The Graph Representation is the window where drawings take place.
367 Initially, it is completely white waiting for a proper graph configuration input
368 by the user. We start the description of this type of analysis by describing the
369 <i>Graph Configuration</i> window (depicted below). By using a particular
371 can be used to customize the graph drawing according to
372 the SimGrid trace that was created with user-specific categories. Before delving
373 into the details of this customization, let us first explain the major parts of
374 the graph configuration window. The buttons located in the top-right corner can
375 be used to delete, copy and create a new configuration. The checkbox in the
376 top-middle part of the window indicates if the configuration typed in the
377 textfield is syntactically correct (we are using the non-XML
378 <a href="http://en.wikipedia.org/wiki/Property_list">Property List Format</a> to
379 describe the configuration). The pop-up button located on the top-left corner
380 indicates the selected configuration (the user can have multiple graph
381 configurations). The bottom-left text field contains the name of the current
382 configuration (updates on this field must be followed by typing enter on the
383 keyboard to take into account the name change). The bottom-right <em>Apply</em>
384 button activates the current configuration, resulting on an update on the graph
388 <a href="triva-graph_configuration.png" border=0><img src="triva-graph_configuration.png" width="50%" border=0></a>
391 <b>Basic SimGrid Configuration</b>: The figure shows in the big textfield the
392 basic configuration that should be used during the analysis of a SimGrid trace
393 file. The basic logic of the configuration is as follows:
399 The nodes of the graph will be created based on the <i>node</i> parameter, which
400 in this case is the different <em>"HOST"</em>s of the platform
401 used to simulate. The <i>edge</i> parameter indicates that the edges of the
402 graph will be created based on the <em>"LINK"</em>s of the platform. After the
403 definition of these two parameters, the configuration must detail how
404 <em>HOST</em>s and <em>LINK</em>s should be drawn. For that, the configuration
405 must have an entry for each of the types used. For <em>HOST</em>, as basic
406 configuration, we have:
413 The parameter <em>size</em> indicates which variable from the trace file will be
414 used to define the size of the node HOST in the visualization. If the simulation
415 was executed with availability traces, the size of the nodes will be changed
416 according to these traces. The parameter <em>scale</em> indicates if the value
417 of the variable is <em>global</em> or <em>local</em>. If it is global, the value
418 will be relative to the power of all other hosts, if it is local, the value will
420 For <em>LINK</em> we have:
430 For the types specified in the <em>edge</em> parameter (such as <em>LINK</em>),
431 the configuration must contain two additional parameters: <em>src</em> and
432 <em>dst</em> that are used to properly identify which nodes this edge is
433 connecting. The values <em>source</em> and <em>destination</em> are always present
434 in the SimGrid trace file and should not be changed in the configuration. The
435 parameter <em>size</em> for the LINK, in this case, is configured as the
436 variable <em>bandwidth</em>, with a <em>global</em> scale. The scale meaning
437 here is exactly the same used for nodes. The last parameter is the GraphViz
438 algorithm used to calculate the position of the nodes in the graph
441 graphviz-algorithm = neato;
444 <b>Customizing the Graph Representation</b>: triva is capable to handle
445 a customized graph representation based on the variables present in the trace
446 file. In the case of SimGrid, every time a category is created for tasks, two
447 variables in the trace file are defined: one to indicate node utilization (how
448 much power was used by that task category), and another to indicate link
449 utilization (how much bandwidth was used by that category). For instance, if the
450 user declares a category named <i>request</i>, there will be variables named
451 <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b> for power and
452 <b>b</b> for bandwidth). It is important to notice that the variable
453 <i>prequest</i> in this case is only available for HOST, and
454 <i>brequest</i> is only available for LINK. <b>Example</b>: suppose there are
455 two categories for tasks: request and compute. To create a customized graph
456 representation with a proportional separation of host and link utilization, use
457 as configuration for HOST and LINK this:
466 values = (prequest, pcomputation);
479 values = (brequest, bcomputation);
483 Where <i>sep_host</i> contains a composition of type <i>separation</i> where
484 its max size is the <i>power</i> of the host and the variables <i>prequest</i>
485 and <i>pcomputation</i> are drawn proportionally to the size of the HOST. And
486 <i>sep_link</i> is also a separation where max is defined as the
487 <i>bandwidth</i> of the link, and the variables <i>brequest</i> and
488 <i>bcomputation</i> are drawn proportionally within a LINK.
489 <i>This configuration enables the analysis of resource utilization by MSG tasks,
490 and the identification of load-balancing issues, network bottlenecks, for
492 <b>Other compositions</b>: besides <i>separation</i>, it is possible to use
493 other types of compositions, such as gradients, and colors, like this:
498 values = (numberOfTasks);
502 values = (is_server);
505 Where <i>gra_host</i> creates a gradient within a node of the graph, using a
506 global scale and using as value a variable called <i>numberOfTasks</i>, that
507 could be declared by the user using the optional tracing functions of SimGrid.
508 If scale is global, the max and min value for the gradient will be equal to the
509 max and min numberOfTasks among all hosts, and if scale is local, the max and
510 min value based on the value of numberOfTasks locally in each host.
511 And <i>color_host</i> composition draws a square based on a positive value of
512 the variable <i>is_server</i>, that could also be defined by the user using the
513 SimGrid tracing functions. \n
514 <b>The Graph Visualization</b>: The next figure shows a graph visualization of a
515 given time-slice of the masterslave_forwarder example (present in the SimGrid
516 sources). The red color indicates tasks from the <i>compute</i> category. This
517 visualization was generated with the following configuration:
530 values = (pcompute, pfinalize);
536 size = bandwidth;\section tracing_tracing Tracing Simulations for Visualization
538 The trace visualization is widely used to observe and understand the behavior
539 of parallel applications and distributed algorithms. Usually, this is done in a
540 two-step fashion: the user instruments the application and the traces are
541 analyzed after the end of the execution. The visualization itself can highlights
542 unexpected behaviors, bottlenecks and sometimes can be used to correct
543 distributed algorithms. The SimGrid team has instrumented the library
544 in order to let users trace their simulations and analyze them. This part of the
545 user manual explains how the tracing-related features can be enabled and used
546 during the development of simulators using the SimGrid library.
548 \subsection tracing_tracing_howitworks How it works
550 For now, the SimGrid library is instrumented so users can trace the <b>platform
551 utilization</b> using the MSG, SimDAG and SMPI interface. This means that the tracing will
552 register how much power is used for each host and how much bandwidth is used for
553 each link of the platform. The idea with this type of tracing is to observe the
554 overall view of resources utilization in the first place, especially the
555 identification of bottlenecks, load-balancing among hosts, and so on.
557 The idea of the tracing facilities is to give SimGrid users to possibility to
558 classify MSG and SimDAG tasks by category, tracing the platform utilization
559 (hosts and links) for each of the categories. For that,
560 the tracing interface enables the declaration of categories and a function to
561 mark a task with a previously declared category. <em>The tasks that are not
562 classified according to a category are not traced</em>. Even if the user
563 does not specify any category, the simulations can still be traced in terms
564 of resource utilization by using a special parameter that is detailed below.
566 \subsection tracing_tracing_enabling Enabling using CMake
568 With the sources of SimGrid, it is possible to enable the tracing
569 using the parameter <b>-Denable_tracing=ON</b> when the cmake is executed.
570 The section \ref tracing_tracing_functions describes all the functions available
571 when this Cmake options is activated. These functions will have no effect
572 if SimGrid is configured without this option (they are wiped-out by the
576 $ cmake -Denable_tracing=ON .
580 \subsection tracing_tracing_functions Tracing Functions
582 \li <b>\c TRACE_category (const char *category)</b>: This function should be used
583 to define a user category. The category can be used to differentiate the tasks
584 that are created during the simulation (for example, tasks from server1,
585 server2, or request tasks, computation tasks, communication tasks).
586 All resource utilization (host power and link bandwidth) will be
587 classified according to the task category. Tasks that do not belong to a
588 category are not traced. The color for the category that is being declared
589 is random (use next function to specify a color).
591 \li <b>\c TRACE_category_with_color (const char *category, const char *color)</b>: Same
592 as TRACE_category, but let user specify a color encoded as a RGB-like string with
593 three floats from 0 to 1. So, to specify a red color, the user can pass "1 0 0" as
594 color parameter. A light-gray color can be specified using "0.7 0.7 0.7" as color.
596 \li <b>\c TRACE_msg_set_task_category (m_task_t task, const char *category)</b>:
597 This function should be called after the creation of a MSG task, to define the
598 category of that task. The first parameter \c task must contain a task that was
599 created with the function \c MSG_task_create. The second parameter
600 \c category must contain a category that was previously defined by the function
603 \li <b>\c TRACE_sd_set_task_category (SD_task_t task, const char *category)</b>:
604 This function should be called after the creation of a SimDAG task, to define the
605 category of that task. The first parameter \c task must contain a task that was
606 created with the function \c MSG_task_create. The second parameter
607 \c category must contain a category that was previously defined by the function
610 \li <b>\c TRACE_[host|link]_variable_declare (const char *variable)</b>:
611 Declare a user variable that will be associated to host/link. A variable can
612 be used to trace user variables such as the number of tasks in a server,
613 the number of clients in an application (for hosts), and so on.
615 \li <b>\c TRACE_[host|link]_variable_[set|add|sub] (const char *[host|link], const char *variable, double value)</b>:
616 Set the value of a given user variable for a given host/link. The value
617 of this variable is always associated to the host/link. The host/link
618 parameters should be its name as the one listed in the platform file.
620 \li <b>\c TRACE_[host|link]_variable_[set|add|sub]_with_time (double time, const char *[host|link], const char *variable, double value)</b>:
621 Same as TRACE_[host|link]_variable_[set|add|sub], but let user specify
622 the time used to trace it. Users can specify a time that is not the
623 simulated clock time as defined by the core simulator. This allows
624 a fine-grain control of time definition, but should be used with
625 caution since the trace can be inconsistent if resource utilization
626 traces are also traced.
628 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub] (const char *src, const char *dst, const char *variable, double value)</b>:
629 Same as TRACE_link_variable_[set|add|sub], but now users specify a source and
630 destination hosts (as the names from the platform file). The tracing library
631 will get the corresponding route that connects those two hosts (src and dst) and
632 [set|add|sub] the value's variable for all the links of the route.
634 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub]_with_time (double time, const char *src, const char *dst, const char *variable, double value)</b>:
635 Same as TRACE_link_srcdst_variable_[set|add|sub], but user specify a time different from the simulated time.
637 \subsection tracing_tracing_options Tracing configuration Options
639 These are the options accepted by the tracing system of SimGrid:
644 Safe switch. It activates (or deactivates) the tracing system.
645 No other tracing options take effect if this one is not activated.
650 Register the simulation platform in the trace file.
655 By default, the tracing system uses all routes in the platform file
656 to re-create a "graph" of the platform and register it in the trace file.
657 This option let the user tell the tracing system to use only the routes
658 that are composed with just one link.
663 It activates the categorized resource utilization tracing. It should
664 be enabled if tracing categories are used by this simulator.
667 tracing/uncategorized
669 It activates the uncategorized resource utilization tracing. Use it if
670 this simulator do not use tracing categories and resource use have to be
676 A file with this name will be created to register the simulation. The file
677 is in the Paje format and can be analyzed using Triva or Paje visualization
678 tools. More information can be found in these webpages:
679 <a href="http://triva.gforge.inria.fr/">http://triva.gforge.inria.fr/</a>
680 <a href="http://paje.sourceforge.net/">http://paje.sourceforge.net/</a>
685 This option only has effect if this simulator is SMPI-based. Traces the MPI
686 interface and generates a trace that can be analyzed using Gantt-like
687 visualizations. Every MPI function (implemented by SMPI) is transformed in a
688 state, and point-to-point communications can be analyzed with arrows.
693 This option only has effect if this simulator is SMPI-based. The processes
694 are grouped by the hosts where they were executed.
699 This option only has effect if this simulator is MSG-based. It traces the
700 behavior of all categorized MSG tasks, grouping them by hosts.
705 This option only has effect if this simulator is MSG-based. It traces the
706 behavior of all categorized MSG processes, grouping them by hosts. This option
707 can be used to track process location if this simulator has process migration.
711 triva/categorized:graph_categorized.plist
713 This option generates a graph configuration file for Triva considering
714 categorized resource utilization.
717 triva/uncategorized:graph_uncategorized.plist
719 This option generates a graph configuration file for Triva considering
720 uncategorized resource utilization.
722 \subsection tracing_tracing_example Example of Instrumentation
724 A simplified example using the tracing mandatory functions.
727 int main (int argc, char **argv)
729 MSG_global_init (&argc, &argv);
731 //(... after deployment ...)
733 //note that category declaration must be called after MSG_create_environment
734 TRACE_category_with_color ("request", "1 0 0");
735 TRACE_category_with_color ("computation", "0.3 1 0.4");
736 TRACE_category ("finalize");
738 m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
739 m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
740 m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
741 m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
742 TRACE_msg_set_task_category (req1, "request");
743 TRACE_msg_set_task_category (req2, "request");
744 TRACE_msg_set_task_category (req3, "request");
745 TRACE_msg_set_task_category (req4, "request");
747 m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
748 TRACE_msg_set_task_category (comp, "computation");
750 m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
751 TRACE_msg_set_task_category (finalize, "finalize");
760 \subsection tracing_tracing_analyzing Analyzing the SimGrid Traces
762 The SimGrid library, during an instrumented simulation, creates a trace file in
763 the Paje file format that contains the platform utilization for the simulation
764 that was executed. The visualization analysis of this file is performed with the
765 visualization tool <a href="http://triva.gforge.inria.fr">Triva</a>, with
766 special configurations tunned to SimGrid needs. This part of the documentation
767 explains how to configure and use Triva to analyse a SimGrid trace file.
769 - <b>Installing Triva</b>: the tool is available in the INRIAGforge,
770 at <a href="http://triva.gforge.inria.fr">http://triva.gforge.inria.fr</a>.
771 Use the following command to get the sources, and then check the file
772 <i>INSTALL</i>. This file contains instructions to install
773 the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
774 be compiled in MacOSes natively, check <i>INSTALL.mac</i> file.
776 $ svn checkout svn://scm.gforge.inria.fr/svn/triva
781 - <b>Executing Triva</b>: a binary called <i>Triva</i> is available after the
782 installation (you can execute it passing <em>--help</em> to check its
783 options). If the triva binary is not available after following the
784 installation instructions, you may want to execute the following command to
785 initialize the GNUstep environment variables. We strongly recommend that you
786 use the latest GNUstep packages, and not the packages available through apt-get
787 in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
788 available packages, you can execute this command:
790 $ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
792 You should be able to see this output after the installation of triva:
794 $ ./Triva.app/Triva --help
795 Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
796 Trace Analysis through Visualization
799 --ti_frequency {double} Animation: frequency of updates
800 --ti_hide Hide the TimeInterval window
801 --ti_forward {double} Animation: value to move time-slice
802 --ti_apply Apply the configuration
803 --ti_update Update on slider change
804 --ti_animate Start animation
805 --ti_start {double} Start of time slice
806 --ti_size {double} Size of time slice
808 --comparison Compare Trace Files (Experimental)
809 --graph Configurable Graph
810 --list Print Trace Type Hierarchy
811 --hierarchy Export Trace Type Hierarchy (dot)
812 --stat Trace Statistics and Memory Utilization
813 --instances List All Trace Entities
814 --linkview Link View (Experimental)
815 --treemap Squarified Treemap
816 --merge Merge Trace Files (Experimental)
817 --check Check Trace File Integrity
819 --gc_conf {file} Graph Configuration in Property List Format
820 --gc_apply Apply the configuration
821 --gc_hide Hide the GraphConfiguration window
823 Triva expects that the user choose one of the available options
824 (currently <em>--graph</em> or <em>--treemap</em> for a visualization analysis)
825 and the trace file from the simulation.
827 - <b>Understanding Triva - time-slice</b>: the analysis of a trace file using
828 the tool always takes into account the concept of the <em>time-slice</em>.
829 This concept means that what is being visualized in the screen is always
830 calculated considering a specific time frame, with its beggining and end
831 timestamp. The time-slice is configured by the user and can be changed
832 dynamically through the window called <em>Time Interval</em> that is opened
833 whenever a trace file is being analyzed. The next figure depicts the time-slice
834 configuration window.
835 In the top of the window, in the space named <i>Trace Time</i>,
836 the two fields show the beggining of the trace (which usually starts in 0) and
837 the end (that depends on the time simulated by SimGrid). The middle of the
838 window, in the square named <i>Time Slice Configuration</i>, contains the
839 aspects related to the time-slice, including its <i>start</i> and its
840 <i>size</i>. The gray rectangle in the bottom of this part indicates the
841 <i>current time-slice</i> that is considered for the drawings. If the checkbox
842 <i>Update Drawings on Sliders Change</i> is not selected, the button
843 <i>Apply</i> must be clicked in order to inform triva that the
844 new time-slice must be considered. The bottom part of the window, in the space
845 indicated by the square <i>Time Slice Animation</i> can be used to advance
846 the time-frame automatically. The user configures the amount of time that the
847 time-frame will forward and how frequent this update will happen. Once this is
848 configured, the user clicks the <i>Play</i> button in order to see the dynamic
849 changes on the drawings.
852 <a href="triva-time_interval.png" border=0><img src="triva-time_interval.png" width="50%" border=0></a>
855 <b>Remarks:</b> when the trace has too many hosts or links, the computation to
856 take into account a new time-slice can be expensive. When this happens, the
857 <i>Frequency</i> parameter, but also updates caused by change on configurations
858 when the checkbox <i>Update Drawings on Sliders
859 Change</i> is selected will not be followed.
861 - <b>Understanding Triva - graph</b>: this part of the documention explains how
862 to analyze the traces using the graph view of Triva, when the user executes
863 the tool passing <em>--graph</em> as parameter. Triva opens three windows when
864 this parameter is used: the <i>Time Interval</i> window (previously described),
865 the <i>Graph Representation</i> window, and the <em>Graph Configuration</em>
866 window. The Graph Representation is the window where drawings take place.
867 Initially, it is completely white waiting for a proper graph configuration input
868 by the user. We start the description of this type of analysis by describing the
869 <i>Graph Configuration</i> window (depicted below). By using a particular
871 can be used to customize the graph drawing according to
872 the SimGrid trace that was created with user-specific categories. Before delving
873 into the details of this customization, let us first explain the major parts of
874 the graph configuration window. The buttons located in the top-right corner can
875 be used to delete, copy and create a new configuration. The checkbox in the
876 top-middle part of the window indicates if the configuration typed in the
877 textfield is syntactically correct (we are using the non-XML
878 <a href="http://en.wikipedia.org/wiki/Property_list">Property List Format</a> to
879 describe the configuration). The pop-up button located on the top-left corner
880 indicates the selected configuration (the user can have multiple graph
881 configurations). The bottom-left text field contains the name of the current
882 configuration (updates on this field must be followed by typing enter on the
883 keyboard to take into account the name change). The bottom-right <em>Apply</em>
884 button activates the current configuration, resulting on an update on the graph
888 <a href="triva-graph_configuration.png" border=0><img src="triva-graph_configuration.png" width="50%" border=0></a>
891 <b>Basic SimGrid Configuration</b>: The figure shows in the big textfield the
892 basic configuration that should be used during the analysis of a SimGrid trace
893 file. The basic logic of the configuration is as follows:
899 The nodes of the graph will be created based on the <i>node</i> parameter, which
900 in this case is the different <em>"HOST"</em>s of the platform
901 used to simulate. The <i>edge</i> parameter indicates that the edges of the
902 graph will be created based on the <em>"LINK"</em>s of the platform. After the
903 definition of these two parameters, the configuration must detail how
904 <em>HOST</em>s and <em>LINK</em>s should be drawn. For that, the configuration
905 must have an entry for each of the types used. For <em>HOST</em>, as basic
906 configuration, we have:
913 The parameter <em>size</em> indicates which variable from the trace file will be
914 used to define the size of the node HOST in the visualization. If the simulation
915 was executed with availability traces, the size of the nodes will be changed
916 according to these traces. The parameter <em>scale</em> indicates if the value
917 of the variable is <em>global</em> or <em>local</em>. If it is global, the value
918 will be relative to the power of all other hosts, if it is local, the value will
920 For <em>LINK</em> we have:
930 For the types specified in the <em>edge</em> parameter (such as <em>LINK</em>),
931 the configuration must contain two additional parameters: <em>src</em> and
932 <em>dst</em> that are used to properly identify which nodes this edge is
933 connecting. The values <em>source</em> and <em>destination</em> are always present
934 in the SimGrid trace file and should not be changed in the configuration. The
935 parameter <em>size</em> for the LINK, in this case, is configured as the
936 variable <em>bandwidth</em>, with a <em>global</em> scale. The scale meaning
937 here is exactly the same used for nodes. The last parameter is the GraphViz
938 algorithm used to calculate the position of the nodes in the graph
941 graphviz-algorithm = neato;
944 <b>Customizing the Graph Representation</b>: triva is capable to handle
945 a customized graph representation based on the variables present in the trace
946 file. In the case of SimGrid, every time a category is created for tasks, two
947 variables in the trace file are defined: one to indicate node utilization (how
948 much power was used by that task category), and another to indicate link
949 utilization (how much bandwidth was used by that category). For instance, if the
950 user declares a category named <i>request</i>, there will be variables named
951 <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b> for power and
952 <b>b</b> for bandwidth). It is important to notice that the variable
953 <i>prequest</i> in this case is only available for HOST, and
954 <i>brequest</i> is only available for LINK. <b>Example</b>: suppose there are
955 two categories for tasks: request and compute. To create a customized graph
956 representation with a proportional separation of host and link utilization, use
957 as configuration for HOST and LINK this:
966 values = (prequest, pcomputation);
979 values = (brequest, bcomputation);
983 Where <i>sep_host</i> contains a composition of type <i>separation</i> where
984 its max size is the <i>power</i> of the host and the variables <i>prequest</i>
985 and <i>pcomputation</i> are drawn proportionally to the size of the HOST. And
986 <i>sep_link</i> is also a separation where max is defined as the
987 <i>bandwidth</i> of the link, and the variables <i>brequest</i> and
988 <i>bcomputation</i> are drawn proportionally within a LINK.
989 <i>This configuration enables the analysis of resource utilization by MSG tasks,
990 and the identification of load-balancing issues, network bottlenecks, for
992 <b>Other compositions</b>: besides <i>separation</i>, it is possible to use
993 other types of compositions, such as gradients, and colors, like this:
998 values = (numberOfTasks);
1002 values = (is_server);
1005 Where <i>gra_host</i> creates a gradient within a node of the graph, using a
1006 global scale and using as value a variable called <i>numberOfTasks</i>, that
1007 could be declared by the user using the optional tracing functions of SimGrid.
1008 If scale is global, the max and min value for the gradient will be equal to the
1009 max and min numberOfTasks among all hosts, and if scale is local, the max and
1010 min value based on the value of numberOfTasks locally in each host.
1011 And <i>color_host</i> composition draws a square based on a positive value of
1012 the variable <i>is_server</i>, that could also be defined by the user using the
1013 SimGrid tracing functions. \n
1014 <b>The Graph Visualization</b>: The next figure shows a graph visualization of a
1015 given time-slice of the masterslave_forwarder example (present in the SimGrid
1016 sources). The red color indicates tasks from the <i>compute</i> category. This
1017 visualization was generated with the following configuration:
1030 values = (pcompute, pfinalize);
1042 values = (bcompute, bfinalize);
1045 graphviz-algorithm = neato;
1050 <a href="triva-graph_visualization.png" border=0><img src="triva-graph_visualization.png" width="50%" border=0></a>
1054 - <b>Understading Triva - colors</b>: An important issue when using Triva is how
1055 to define colors. To do that, we have to know which variables are defined in
1056 the trace file generated by the SimGrid library. The parameter <em>--list</em>
1057 lists the variables for a given trace file:
1059 $ Triva -l masterslave_forwarder.trace
1077 We can see that HOST has seven variables (from power to pfinalize) and LINK has
1078 four (from bandwidth to bfinalize). To define a red color for the
1079 <i>pcompute</i> and <i>bcompute</i> (which are defined based on user category
1080 <i>compute</i>), execute:
1082 $ defaults write Triva 'pcompute Color' '1 0 0'
1083 $ defaults write Triva 'bcompute Color' '1 0 0'
1085 Where the three numbers in each line are the RGB color with values from 0 to 1.
1092 values = (bcompute, bfinalize);
1095 graphviz-algorithm = neato;
1100 <a href="triva-graph_visualization.png" border=0><img src="triva-graph_visualization.png" width="50%" border=0></a>
1104 - <b>Understading Triva - colors</b>: An important issue when using Triva is how
1105 to define colors. To do that, we have to know which variables are defined in
1106 the trace file generated by the SimGrid library. The parameter <em>--list</em>
1107 lists the variables for a given trace file:
1109 $ Triva -l masterslave_forwarder.trace
1127 We can see that HOST has seven variables (from power to pfinalize) and LINK has
1128 four (from bandwidth to bfinalize). To define a red color for the
1129 <i>pcompute</i> and <i>bcompute</i> (which are defined based on user category
1130 <i>compute</i>), execute:
1132 $ defaults write Triva 'pcompute Color' '1 0 0'
1133 $ defaults write Triva 'bcompute Color' '1 0 0'
1135 Where the three numbers in each line are the RGB color with values from 0 to 1.