1 /*! \page tracing Tracing Simulations for Visualization
3 \htmlinclude .tracing.doc.toc
5 \section tracing_tracing Tracing Simulations for Visualization
7 The trace visualization is widely used to observe and understand the behavior
8 of parallel applications and distributed algorithms. Usually, this is done in a
9 two-step fashion: the user instruments the application and the traces are
10 analyzed after the end of the execution. The visualization itself can highlights
11 unexpected behaviors, bottlenecks and sometimes can be used to correct
12 distributed algorithms. The SimGrid team has instrumented the library
13 in order to let users trace their simulations and analyze them. This part of the
14 user manual explains how the tracing-related features can be enabled and used
15 during the development of simulators using the SimGrid library.
17 \subsection tracing_tracing_howitworks How it works
19 For now, the SimGrid library is instrumented so users can trace the <b>platform
20 utilization</b> using the MSG, SimDAG and SMPI interface. This means that the tracing will
21 register how much power is used for each host and how much bandwidth is used for
22 each link of the platform. The idea with this type of tracing is to observe the
23 overall view of resources utilization in the first place, especially the
24 identification of bottlenecks, load-balancing among hosts, and so on.
26 The idea of the tracing facilities is to give SimGrid users to possibility to
27 classify MSG and SimDAG tasks by category, tracing the platform utilization
28 (hosts and links) for each of the categories. For that,
29 the tracing interface enables the declaration of categories and a function to
30 mark a task with a previously declared category. <em>The tasks that are not
31 classified according to a category are not traced</em>. Even if the user
32 does not specify any category, the simulations can still be traced in terms
33 of resource utilization by using a special parameter that is detailed below.
35 \subsection tracing_tracing_enabling Enabling using CMake
37 With the sources of SimGrid, it is possible to enable the tracing
38 using the parameter <b>-Denable_tracing=ON</b> when the cmake is executed.
39 The section \ref tracing_tracing_functions describes all the functions available
40 when this Cmake options is activated. These functions will have no effect
41 if SimGrid is configured without this option (they are wiped-out by the
45 $ cmake -Denable_tracing=ON .
49 \subsection tracing_tracing_functions Tracing Functions
51 \li <b>\c TRACE_category (const char *category)</b>: This function should be used
52 to define a user category. The category can be used to differentiate the tasks
53 that are created during the simulation (for example, tasks from server1,
54 server2, or request tasks, computation tasks, communication tasks).
55 All resource utilization (host power and link bandwidth) will be
56 classified according to the task category. Tasks that do not belong to a
57 category are not traced. The color for the category that is being declared
58 is random (use next function to specify a color).
60 \li <b>\c TRACE_category_with_color (const char *category, const char *color)</b>: Same
61 as TRACE_category, but let user specify a color encoded as a RGB-like string with
62 three floats from 0 to 1. So, to specify a red color, the user can pass "1 0 0" as
63 color parameter. A light-gray color can be specified using "0.7 0.7 0.7" as color.
65 \li <b>\c TRACE_msg_set_task_category (m_task_t task, const char *category)</b>:
66 This function should be called after the creation of a MSG task, to define the
67 category of that task. The first parameter \c task must contain a task that was
68 created with the function \c MSG_task_create. The second parameter
69 \c category must contain a category that was previously defined by the function
72 \li <b>\c TRACE_sd_set_task_category (SD_task_t task, const char *category)</b>:
73 This function should be called after the creation of a SimDAG task, to define the
74 category of that task. The first parameter \c task must contain a task that was
75 created with the function \c MSG_task_create. The second parameter
76 \c category must contain a category that was previously defined by the function
79 \li <b>\c TRACE_declare_mark(const char *mark_type)</b>: This function
80 declares a new Paje event type in the trace file that can be used by
81 simulators to declare application-level marks. This function is
82 independent of which API is used in SimGrid.
84 \li <b>\c TRACE_mark(const char *mark_type, const char *mark_value)</b>:
85 This function creates a mark in the trace file. The first parameter
86 had to be previously declared using \c TRACE_declare_mark, the second
87 is the identifier for this mark instance. We recommend that the \c
88 mark_value (the second parameter) is a unique value for the whole
89 trace file (the whole simulation). Nevertheless, this is not a strong
90 requirement: the trace will be valid if there are multiple mark
91 identifiers for the same trace.
93 \li <b>\c TRACE_[host|link]_variable_declare (const char *variable)</b>:
94 Declare a user variable that will be associated to host/link. A variable can
95 be used to trace user variables such as the number of tasks in a server,
96 the number of clients in an application (for hosts), and so on.
98 \li <b>\c TRACE_[host|link]_variable_declare_with_color (const char
99 *var, const char *color)</b>: Same as \c
100 TRACE_[host|link]_variable_declare, but user decides which color will
101 be assigned to the variable. The color needs to be a string with three
102 numbers separated by spaces in the range [0,1]. A light-gray color can
103 be specified using "0.7 0.7 0.7" as color.
105 \li <b>\c TRACE_[host|link]_variable_[set|add|sub] (const char *[host|link], const char *variable, double value)</b>:
106 Set the value of a given user variable for a given host/link. The value
107 of this variable is always associated to the host/link. The host/link
108 parameters should be its name as the one listed in the platform file.
110 \li <b>\c TRACE_[host|link]_variable_[set|add|sub]_with_time (double time, const char *[host|link], const char *variable, double value)</b>:
111 Same as TRACE_[host|link]_variable_[set|add|sub], but let user specify
112 the time used to trace it. Users can specify a time that is not the
113 simulated clock time as defined by the core simulator. This allows
114 a fine-grain control of time definition, but should be used with
115 caution since the trace can be inconsistent if resource utilization
116 traces are also traced.
118 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub] (const char *src, const char *dst, const char *variable, double value)</b>:
119 Same as TRACE_link_variable_[set|add|sub], but now users specify a source and
120 destination hosts (as the names from the platform file). The tracing library
121 will get the corresponding route that connects those two hosts (src and dst) and
122 [set|add|sub] the value's variable for all the links of the route.
124 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub]_with_time (double time, const char *src, const char *dst, const char *variable, double value)</b>:
125 Same as TRACE_link_srcdst_variable_[set|add|sub], but user specify a time different from the simulated time.
127 \subsection tracing_tracing_options Tracing configuration Options
129 To check which tracing options are available for your simulator, you
130 can just run it with the option <b>--help-tracing</b>. These are the
131 options accepted by the tracing system of SimGrid as of today, you
132 can use them by running your simulator with the <b>--cfg=</b> switch:
137 Safe switch. It activates (or deactivates) the tracing system.
138 No other tracing options take effect if this one is not activated.
146 It activates the categorized resource utilization tracing. It should
147 be enabled if tracing categories are used by this simulator.
149 --cfg=tracing/categorized:1
153 tracing/uncategorized
155 It activates the uncategorized resource utilization tracing. Use it if
156 this simulator do not use tracing categories and resource use have to be
159 --cfg=tracing/uncategorized:1
165 A file with this name will be created to register the simulation. The file
166 is in the Paje format and can be analyzed using Triva or Paje visualization
167 tools. More information can be found in these webpages:
168 <a href="http://triva.gforge.inria.fr/">http://triva.gforge.inria.fr/</a>
169 <a href="http://paje.sourceforge.net/">http://paje.sourceforge.net/</a>
171 --cfg=tracing/filename:mytracefile.trace
173 If you do not provide this parameter, the trace file will be named simgrid.trace.
178 By default, the tracing system uses all routes in the platform file
179 to re-create a "graph" of the platform and register it in the trace file.
180 This option let the user tell the tracing system to use only the routes
181 that are composed with just one link.
183 --cfg=tracing/onelink_only:1
189 This option only has effect if this simulator is SMPI-based. Traces the MPI
190 interface and generates a trace that can be analyzed using Gantt-like
191 visualizations. Every MPI function (implemented by SMPI) is transformed in a
192 state, and point-to-point communications can be analyzed with arrows.
200 This option only has effect if this simulator is SMPI-based. The processes
201 are grouped by the hosts where they were executed.
203 --cfg=tracing/smpi/group:1
209 This option only has effect if this simulator is MSG-based. It traces the
210 behavior of all categorized MSG tasks, grouping them by hosts.
212 --cfg=tracing/msg/task:1
218 This option only has effect if this simulator is MSG-based. It traces the
219 behavior of all categorized MSG processes, grouping them by hosts. This option
220 can be used to track process location if this simulator has process migration.
222 --cfg=tracing/msg/process:1
228 This option put some events in a time-ordered buffer using the
229 insertion sort algorithm. The process of acquiring and releasing
230 locks to access this buffer and the cost of the sorting algorithm
231 make this process slow. The simulator performance can be severely
232 impacted if this option is activated, but you are sure to get a trace
233 file with events sorted.
235 --cfg=tracing/buffer:1
241 This option changes the way SimGrid register its platform on the trace
242 file. Normally, the tracing considers all routes (no matter their
243 size) on the platform file to re-create the resource topology. If this
244 option is activated, only the routes with one link are used to
245 register the topology within an AS. Routes among AS continue to be
248 --cfg=tracing/onelink_only:1
252 tracing/disable_destroy
254 Disable the destruction of containers at the end of simulation. This
255 can be used with simulators that have a different notion of time
256 (different from the simulated time).
258 --cfg=tracing/disable_destroy:1
264 This option generates a graph configuration file for Triva considering
265 categorized resource utilization.
267 --cfg=triva/categorized:graph_categorized.plist
273 This option generates a graph configuration file for Triva considering
274 uncategorized resource utilization.
276 --cfg=triva/uncategorized:graph_uncategorized.plist
279 \subsection tracing_tracing_example_parameters Case studies
281 Some scenarios that might help you decide which tracing options
282 you should use to analyze your simulator.
284 \li I want to trace the resource utilization of all hosts
285 and links of the platform, and my simulator <b>does not</b> use
286 the tracing API. For that, you can run a uncategorized trace
287 with the following parameters (it will work with <b>any</b> Simgrid
292 --cfg=tracing/uncategorized:1 \
293 --cfg=tracing/filename:mytracefile.trace \
294 --cfg=triva/uncategorized:uncat.plist
297 \li I want to trace only a subset of my MSG (or SimDAG) tasks.
298 For that, you will need to create tracing categories using the
299 <b>TRACE_category (...)</b> function (as explained above),
300 and then classify your tasks to a previously declared category
301 using the <b>TRACE_msg_set_task_category (...)</b>
302 (or <b>TRACE_sd_set_task_category (...)</b> for SimDAG tasks). After
303 recompiling, run your simulator with the following parameters:
307 --cfg=tracing/categorized:1 \
308 --cfg=tracing/filename:mytracefile.trace \
309 --cfg=triva/categorized:cat.plist
313 \subsection tracing_tracing_example Example of Instrumentation
315 A simplified example using the tracing mandatory functions.
318 int main (int argc, char **argv)
320 MSG_global_init (&argc, &argv);
322 //(... after deployment ...)
324 //note that category declaration must be called after MSG_create_environment
325 TRACE_category_with_color ("request", "1 0 0");
326 TRACE_category_with_color ("computation", "0.3 1 0.4");
327 TRACE_category ("finalize");
329 m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
330 m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
331 m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
332 m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
333 TRACE_msg_set_task_category (req1, "request");
334 TRACE_msg_set_task_category (req2, "request");
335 TRACE_msg_set_task_category (req3, "request");
336 TRACE_msg_set_task_category (req4, "request");
338 m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
339 TRACE_msg_set_task_category (comp, "computation");
341 m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
342 TRACE_msg_set_task_category (finalize, "finalize");
351 \subsection tracing_tracing_analyzing Analyzing the SimGrid Traces
353 The SimGrid library, during an instrumented simulation, creates a trace file in
354 the Paje file format that contains the platform utilization for the simulation
355 that was executed. The visualization analysis of this file is performed with the
356 visualization tool <a href="http://triva.gforge.inria.fr">Triva</a>, with
357 special configurations tunned to SimGrid needs. This part of the documentation
358 explains how to configure and use Triva to analyse a SimGrid trace file.
360 - <b>Installing Triva</b>: the tool is available in the Inria's Forge,
361 at <a href="http://triva.gforge.inria.fr">http://triva.gforge.inria.fr</a>.
362 Use the following command to get the sources, and then check the file
363 <i>INSTALL</i>. This file contains instructions to install
364 the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
365 be compiled in MacOSX natively, check <i>INSTALL.mac</i> file.
367 $ git clone git://scm.gforge.inria.fr/triva/triva.git
372 - <b>Executing Triva</b>: a binary called <i>Triva</i> is available after the
373 installation (you can execute it passing <em>--help</em> to check its
374 options). If the triva binary is not available after following the
375 installation instructions, you may want to execute the following command to
376 initialize the GNUstep environment variables. We strongly recommend that you
377 use the latest GNUstep packages, and not the packages available through apt-get
378 in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
379 available packages, you can execute this command:
381 $ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
383 You should be able to see this output after the installation of triva:
385 $ ./Triva.app/Triva --help
386 Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
387 Trace Analysis through Visualization
390 --ti_frequency {double} Animation: frequency of updates
391 --ti_hide Hide the TimeInterval window
392 --ti_forward {double} Animation: value to move time-slice
393 --ti_apply Apply the configuration
394 --ti_update Update on slider change
395 --ti_animate Start animation
396 --ti_start {double} Start of time slice
397 --ti_size {double} Size of time slice
399 --comparison Compare Trace Files (Experimental)
400 --graph Configurable Graph
401 --list Print Trace Type Hierarchy
402 --hierarchy Export Trace Type Hierarchy (dot)
403 --stat Trace Statistics and Memory Utilization
404 --instances List All Trace Entities
405 --linkview Link View (Experimental)
406 --treemap Squarified Treemap
407 --merge Merge Trace Files (Experimental)
408 --check Check Trace File Integrity
410 --gc_conf {file} Graph Configuration in Property List Format
411 --gc_apply Apply the configuration
412 --gc_hide Hide the GraphConfiguration window
414 Triva expects that the user choose one of the available options
415 (currently <em>--graph</em> or <em>--treemap</em> for a visualization analysis)
416 and the trace file from the simulation.
418 - <b>Understanding Triva - time-slice</b>: the analysis of a trace file using
419 the tool always takes into account the concept of the <em>time-slice</em>.
420 This concept means that what is being visualized in the screen is always
421 calculated considering a specific time frame, with its beggining and end
422 timestamp. The time-slice is configured by the user and can be changed
423 dynamically through the window called <em>Time Interval</em> that is opened
424 whenever a trace file is being analyzed. The next figure depicts the time-slice
425 configuration window.
426 In the top of the window, in the space named <i>Trace Time</i>,
427 the two fields show the beggining of the trace (which usually starts in 0) and
428 the end (that depends on the time simulated by SimGrid). The middle of the
429 window, in the square named <i>Time Slice Configuration</i>, contains the
430 aspects related to the time-slice, including its <i>start</i> and its
431 <i>size</i>. The gray rectangle in the bottom of this part indicates the
432 <i>current time-slice</i> that is considered for the drawings. If the checkbox
433 <i>Update Drawings on Sliders Change</i> is not selected, the button
434 <i>Apply</i> must be clicked in order to inform triva that the
435 new time-slice must be considered. The bottom part of the window, in the space
436 indicated by the square <i>Time Slice Animation</i> can be used to advance
437 the time-frame automatically. The user configures the amount of time that the
438 time-frame will forward and how frequent this update will happen. Once this is
439 configured, the user clicks the <i>Play</i> button in order to see the dynamic
440 changes on the drawings.
443 <a href="triva-time_interval.png" border=0><img src="triva-time_interval.png" width="50%" border=0></a>
446 <b>Remarks:</b> when the trace has too many hosts or links, the computation to
447 take into account a new time-slice can be expensive. When this happens, the
448 <i>Frequency</i> parameter, but also updates caused by change on configurations
449 when the checkbox <i>Update Drawings on Sliders
450 Change</i> is selected will not be followed.
452 - <b>Understanding Triva - graph</b>: this part of the documention explains how
453 to analyze the traces using the graph view of Triva, when the user executes
454 the tool passing <em>--graph</em> as parameter. Triva opens three windows when
455 this parameter is used: the <i>Time Interval</i> window (previously described),
456 the <i>Graph Representation</i> window, and the <em>Graph Configuration</em>
457 window. The Graph Representation is the window where drawings take place.
458 Initially, it is completely white waiting for a proper graph configuration input
459 by the user. We start the description of this type of analysis by describing the
460 <i>Graph Configuration</i> window (depicted below). By using a particular
462 can be used to customize the graph drawing according to
463 the SimGrid trace that was created with user-specific categories. Before delving
464 into the details of this customization, let us first explain the major parts of
465 the graph configuration window. The buttons located in the top-right corner can
466 be used to delete, copy and create a new configuration. The checkbox in the
467 top-middle part of the window indicates if the configuration typed in the
468 textfield is syntactically correct (we are using the non-XML
469 <a href="http://en.wikipedia.org/wiki/Property_list">Property List Format</a> to
470 describe the configuration). The pop-up button located on the top-left corner
471 indicates the selected configuration (the user can have multiple graph
472 configurations). The bottom-left text field contains the name of the current
473 configuration (updates on this field must be followed by typing enter on the
474 keyboard to take into account the name change). The bottom-right <em>Apply</em>
475 button activates the current configuration, resulting on an update on the graph
479 <a href="triva-graph_configuration.png" border=0><img src="triva-graph_configuration.png" width="50%" border=0></a>
482 <b>Basic SimGrid Configuration</b>: The figure shows in the big textfield the
483 basic configuration that should be used during the analysis of a SimGrid trace
484 file. The basic logic of the configuration is as follows:
490 The nodes of the graph will be created based on the <i>node</i> parameter, which
491 in this case is the different <em>"HOST"</em>s of the platform
492 used to simulate. The <i>edge</i> parameter indicates that the edges of the
493 graph will be created based on the <em>"LINK"</em>s of the platform. After the
494 definition of these two parameters, the configuration must detail how
495 <em>HOST</em>s and <em>LINK</em>s should be drawn. For that, the configuration
496 must have an entry for each of the types used. For <em>HOST</em>, as basic
497 configuration, we have:
504 The parameter <em>size</em> indicates which variable from the trace file will be
505 used to define the size of the node HOST in the visualization. If the simulation
506 was executed with availability traces, the size of the nodes will be changed
507 according to these traces. The parameter <em>scale</em> indicates if the value
508 of the variable is <em>global</em> or <em>local</em>. If it is global, the value
509 will be relative to the power of all other hosts, if it is local, the value will
511 For <em>LINK</em> we have:
521 For the types specified in the <em>edge</em> parameter (such as <em>LINK</em>),
522 the configuration must contain two additional parameters: <em>src</em> and
523 <em>dst</em> that are used to properly identify which nodes this edge is
524 connecting. The values <em>source</em> and <em>destination</em> are always present
525 in the SimGrid trace file and should not be changed in the configuration. The
526 parameter <em>size</em> for the LINK, in this case, is configured as the
527 variable <em>bandwidth</em>, with a <em>global</em> scale. The scale meaning
528 here is exactly the same used for nodes. The last parameter is the GraphViz
529 algorithm used to calculate the position of the nodes in the graph
532 graphviz-algorithm = neato;
535 <b>Customizing the Graph Representation</b>: triva is capable to handle
536 a customized graph representation based on the variables present in the trace
537 file. In the case of SimGrid, every time a category is created for tasks, two
538 variables in the trace file are defined: one to indicate node utilization (how
539 much power was used by that task category), and another to indicate link
540 utilization (how much bandwidth was used by that category). For instance, if the
541 user declares a category named <i>request</i>, there will be variables named
542 <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b> for power and
543 <b>b</b> for bandwidth). It is important to notice that the variable
544 <i>prequest</i> in this case is only available for HOST, and
545 <i>brequest</i> is only available for LINK. <b>Example</b>: suppose there are
546 two categories for tasks: request and compute. To create a customized graph
547 representation with a proportional separation of host and link utilization, use
548 as configuration for HOST and LINK this:
557 values = (prequest, pcomputation);
570 values = (brequest, bcomputation);
574 Where <i>sep_host</i> contains a composition of type <i>separation</i> where
575 its max size is the <i>power</i> of the host and the variables <i>prequest</i>
576 and <i>pcomputation</i> are drawn proportionally to the size of the HOST. And
577 <i>sep_link</i> is also a separation where max is defined as the
578 <i>bandwidth</i> of the link, and the variables <i>brequest</i> and
579 <i>bcomputation</i> are drawn proportionally within a LINK.
580 <i>This configuration enables the analysis of resource utilization by MSG tasks,
581 and the identification of load-balancing issues, network bottlenecks, for
583 <b>Other compositions</b>: besides <i>separation</i>, it is possible to use
584 other types of compositions, such as gradients, and colors, like this:
589 values = (numberOfTasks);
593 values = (is_server);
596 Where <i>gra_host</i> creates a gradient within a node of the graph, using a
597 global scale and using as value a variable called <i>numberOfTasks</i>, that
598 could be declared by the user using the optional tracing functions of SimGrid.
599 If scale is global, the max and min value for the gradient will be equal to the
600 max and min numberOfTasks among all hosts, and if scale is local, the max and
601 min value based on the value of numberOfTasks locally in each host.
602 And <i>color_host</i> composition draws a square based on a positive value of
603 the variable <i>is_server</i>, that could also be defined by the user using the
604 SimGrid tracing functions. \n
605 <b>The Graph Visualization</b>: The next figure shows a graph visualization of a
606 given time-slice of the masterslave_forwarder example (present in the SimGrid
607 sources). The red color indicates tasks from the <i>compute</i> category. This
608 visualization was generated with the following configuration:
621 values = (pcompute, pfinalize);
633 values = (bcompute, bfinalize);
636 graphviz-algorithm = neato;
641 <a href="triva-graph_visualization.png" border=0><img src="triva-graph_visualization.png" width="50%" border=0></a>
645 - <b>Understading Triva - colors</b>: An important issue when using Triva is how
646 to define colors. To do that, we have to know which variables are defined in
647 the trace file generated by the SimGrid library. The parameter <em>--list</em>
648 lists the variables for a given trace file:
650 $ Triva -l masterslave_forwarder.trace
668 We can see that HOST has seven variables (from power to pfinalize) and LINK has
669 four (from bandwidth to bfinalize). To define a red color for the
670 <i>pcompute</i> and <i>bcompute</i> (which are defined based on user category
671 <i>compute</i>), execute:
673 $ defaults write Triva 'pcompute Color' '1 0 0'
674 $ defaults write Triva 'bcompute Color' '1 0 0'
676 Where the three numbers in each line are the RGB color with values from 0 to 1.