Merge branch 'master' of git+ssh://scm.gforge.inria.fr//gitroot/simgrid/simgrid

[simgrid.git] / doc / tracing.doc

/*! \page tracing Tracing Simulations for Visualization

\htmlinclude .tracing.doc.toc

\section tracing_tracing Tracing Simulations for Visualization

The trace visualization is widely used to observe and understand the behavior
of parallel applications and distributed algorithms. Usually, this is done in a
two-step fashion: the user instruments the application and the traces are
analyzed after the end of the execution. The visualization itself can highlights
unexpected behaviors, bottlenecks and sometimes can be used to correct
distributed algorithms. The SimGrid team has instrumented the library
in order to let users trace their simulations and analyze them. This part of the
user manual explains how the tracing-related features can be enabled and used
during the development of simulators using the SimGrid library.

\subsection tracing_tracing_howitworks How it works

For now, the SimGrid library is instrumented so users can trace the <b>platform
utilization</b> using the MSG, SimDAG and SMPI interface. This means that the tracing will
register how much power is used for each host and how much bandwidth is used for
each link of the platform. The idea with this type of tracing is to observe the
overall view of resources utilization in the first place, especially the
identification of bottlenecks, load-balancing among hosts, and so on.

The idea of the tracing facilities is to give SimGrid users to possibility to
classify MSG and SimDAG tasks by category, tracing the platform utilization
(hosts and links) for each of the categories. For that,
the tracing interface enables the declaration of categories and a function to
mark a task with a previously declared category. <em>The tasks that are not
classified according to a category are not traced</em>. Even if the user
does not specify any category, the simulations can still be traced in terms
of resource utilization by using a special parameter that is detailed below.

\subsection tracing_tracing_enabling Enabling using CMake

With the sources of SimGrid, it is possible to enable the tracing
using the parameter <b>-Denable_tracing=ON</b> when the cmake is
executed.  The sections \ref instr_category_functions, \ref
instr_mark_functions, and \ref instr_uservariables_functions describe
all the functions available when this Cmake options is
activated. These functions will have no effect if SimGrid is
configured without this option (they are wiped-out by the
C-preprocessor).

\verbatim
$ cmake -Denable_tracing=ON .
$ make
\endverbatim

\subsection instr_category_functions Tracing categories functions
\li \c TRACE_category(const char *category)
\li \c TRACE_category_with_color(const char *category, const char *color)
\li \c MSG_task_set_category(m_task_t task, const char *category)
\li \c MSG_task_get_category(m_task_t task)
\li \c SD_task_set_category(SD_task_t task, const char *category)
\li \c SD_task_get_category(SD_task_t task)

\subsection instr_mark_functions Tracing marks functions
\li \c TRACE_declare_mark(const char *mark_type)
\li \c TRACE_mark(const char *mark_type, const char *mark_value)

\subsection instr_uservariables_functions Tracing user variables functions

For hosts:

\li \c TRACE_host_variable_declare(const char *variable)
\li \c TRACE_host_variable_declare_with_color(const char *variable, const char *color)
\li \c TRACE_host_variable_set(const char *host, const char *variable, double value)
\li \c TRACE_host_variable_add(const char *host, const char *variable, double value)
\li \c TRACE_host_variable_sub(const char *host, const char *variable, double value)
\li \c TRACE_host_variable_set_with_time(double time, const char *host, const char *variable, double value)
\li \c TRACE_host_variable_add_with_time(double time, const char *host, const char *variable, double value)
\li \c TRACE_host_variable_sub_with_time(double time, const char *host, const char *variable, double value)

For links:

\li \c TRACE_link_variable_declare(const char *variable)
\li \c TRACE_link_variable_declare_with_color(const char *variable, const char *color)
\li \c TRACE_link_variable_set(const char *link, const char *variable, double value)
\li \c TRACE_link_variable_add(const char *link, const char *variable, double value)
\li \c TRACE_link_variable_sub(const char *link, const char *variable, double value)
\li \c TRACE_link_variable_set_with_time(double time, const char *link, const char *variable, double value)
\li \c TRACE_link_variable_add_with_time(double time, const char *link, const char *variable, double value)
\li \c TRACE_link_variable_sub_with_time(double time, const char *link, const char *variable, double value)

For links, but use source and destination to get route:

\li \c TRACE_link_srcdst_variable_set(const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_add(const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_sub(const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_set_with_time(double time, const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_add_with_time(double time, const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_sub_with_time(double time, const char *src, const char *dst, const char *variable, double value)

\subsection tracing_tracing_options Tracing configuration Options

To check which tracing options are available for your simulator, you
can just run it with the option <b>--help-tracing</b>. These are the
options accepted by the tracing system of SimGrid as of today, you
can use them by running your simulator with the <b>--cfg=</b> switch:

\li <b>\c 
tracing
</b>:
  Safe switch. It activates (or deactivates) the tracing system.
  No other tracing options take effect if this one is not activated.
\verbatim
--cfg=tracing:1
\endverbatim

\li <b>\c 
tracing/categorized
</b>:
  It activates the categorized resource utilization tracing. It should
  be enabled if tracing categories are used by this simulator.
\verbatim
--cfg=tracing/categorized:1
\endverbatim

\li <b>\c 
tracing/uncategorized
</b>:
  It activates the uncategorized resource utilization tracing. Use it if
  this simulator do not use tracing categories and resource use have to be
  traced.
\verbatim
--cfg=tracing/uncategorized:1
\endverbatim

\li <b>\c 
tracing/filename
</b>:
  A file with this name will be created to register the simulation. The file
  is in the Paje format and can be analyzed using Triva or Paje visualization
  tools. More information can be found in these webpages:
     <a href="http://triva.gforge.inria.fr/">http://triva.gforge.inria.fr/</a>
     <a href="http://paje.sourceforge.net/">http://paje.sourceforge.net/</a>
\verbatim
--cfg=tracing/filename:mytracefile.trace
\endverbatim
  If you do not provide this parameter, the trace file will be named simgrid.trace.

\li <b>\c
tracing/onelink_only
</b>:
  By default, the tracing system uses all routes in the platform file
  to re-create a "graph" of the platform and register it in the trace file.
  This option let the user tell the tracing system to use only the routes
  that are composed with just one link.
\verbatim
--cfg=tracing/onelink_only:1
\endverbatim

\li <b>\c 
tracing/smpi
</b>:
  This option only has effect if this simulator is SMPI-based. Traces the MPI
  interface and generates a trace that can be analyzed using Gantt-like
  visualizations. Every MPI function (implemented by SMPI) is transformed in a
  state, and point-to-point communications can be analyzed with arrows.
\verbatim
--cfg=tracing/smpi:1
\endverbatim

\li <b>\c 
tracing/smpi/group
</b>:
  This option only has effect if this simulator is SMPI-based. The processes
  are grouped by the hosts where they were executed.
\verbatim
--cfg=tracing/smpi/group:1
\endverbatim

\li <b>\c 
tracing/msg/process
</b>:
  This option only has effect if this simulator is MSG-based. It traces the
  behavior of all categorized MSG processes, grouping them by hosts. This option
  can be used to track process location if this simulator has process migration.
\verbatim
--cfg=tracing/msg/process:1
\endverbatim

\li <b>\c
tracing/buffer
</b>:
 This option put some events in a time-ordered buffer using the
 insertion sort algorithm. The process of acquiring and releasing
 locks to access this buffer and the cost of the sorting algorithm
 make this process slow. The simulator performance can be severely
 impacted if this option is activated, but you are sure to get a trace
 file with events sorted.
\verbatim
--cfg=tracing/buffer:1
\endverbatim

\li <b>\c
tracing/onelink_only
</b>:
This option changes the way SimGrid register its platform on the trace
file. Normally, the tracing considers all routes (no matter their
size) on the platform file to re-create the resource topology. If this
option is activated, only the routes with one link are used to
register the topology within an AS.  Routes among AS continue to be
traced as usual.
\verbatim
--cfg=tracing/onelink_only:1
\endverbatim

\li <b>\c
tracing/disable_destroy
</b>:
Disable the destruction of containers at the end of simulation. This
can be used with simulators that have a different notion of time
(different from the simulated time).
\verbatim
--cfg=tracing/disable_destroy:1
\endverbatim

\li <b>\c 
triva/categorized
</b>:
  This option generates a graph configuration file for Triva considering
  categorized resource utilization.
\verbatim
--cfg=triva/categorized:graph_categorized.plist
\endverbatim

\li <b>\c 
triva/uncategorized
</b>:
  This option generates a graph configuration file for Triva considering
  uncategorized resource utilization.
\verbatim
--cfg=triva/uncategorized:graph_uncategorized.plist
\endverbatim

\subsection tracing_tracing_example_parameters Case studies

Some scenarios that might help you decide which tracing options
you should use to analyze your simulator.

\li I want to trace the resource utilization of all hosts
and links of the platform, and my simulator <b>does not</b> use
the tracing API. For that, you can run a uncategorized trace
with the following parameters (it will work with <b>any</b> Simgrid
simulator):
\verbatim
./your_simulator \
          --cfg=tracing:1 \
          --cfg=tracing/uncategorized:1 \
          --cfg=tracing/filename:mytracefile.trace \
          --cfg=triva/uncategorized:uncat.plist
\endverbatim

\li I want to trace only a subset of my MSG (or SimDAG) tasks.
For that, you will need to create tracing categories using the
<b>TRACE_category (...)</b> function (as explained above),
and then classify your tasks to a previously declared category
using the <b>MSG_task_set_category (...)</b>
(or <b>SD_task_set_category (...)</b> for SimDAG tasks). After
recompiling, run your simulator with the following parameters:
\verbatim
./your_simulator \
          --cfg=tracing:1 \
          --cfg=tracing/categorized:1 \
          --cfg=tracing/filename:mytracefile.trace \
          --cfg=triva/categorized:cat.plist
\endverbatim


\subsection tracing_tracing_example Example of Instrumentation

A simplified example using the tracing mandatory functions.

\verbatim
int main (int argc, char **argv)
{
  MSG_global_init (&argc, &argv);

  //(... after deployment ...)

  //note that category declaration must be called after MSG_create_environment
  TRACE_category_with_color ("request", "1 0 0");
  TRACE_category_with_color ("computation", "0.3 1 0.4");
  TRACE_category ("finalize");

  m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
  m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
  m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
  m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
  MSG_task_set_category (req1, "request");
  MSG_task_set_category (req2, "request");
  MSG_task_set_category (req3, "request");
  MSG_task_set_category (req4, "request");

  m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
  MSG_task_set_category (comp, "computation");

  m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
  MSG_task_set_category (finalize, "finalize");

  //(...)

  MSG_clean();
  return 0;
}
\endverbatim

\subsection tracing_tracing_analyzing Analyzing the SimGrid Traces

The SimGrid library, during an instrumented simulation, creates a trace file in
the Paje file format that contains the platform utilization for the simulation
that was executed. The visualization analysis of this file is performed with the
visualization tool <a href="http://triva.gforge.inria.fr">Triva</a>, with
special configurations tunned to SimGrid needs. This part of the documentation
explains how to configure and use Triva to analyse a SimGrid trace file.

- <b>Installing Triva</b>: the tool is available in the Inria's Forge, 
at <a href="http://triva.gforge.inria.fr">http://triva.gforge.inria.fr</a>.
Use the following command to get the sources, and then check the file
<i>INSTALL</i>. This file contains instructions to install
the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
be compiled in MacOSX natively, check <i>INSTALL.mac</i> file.
\verbatim
$ git clone git://scm.gforge.inria.fr/triva/triva.git
$ cd triva
$ cat INSTALL
\endverbatim

- <b>Executing Triva</b>: a binary called <i>Triva</i> is available after the
  installation (you can execute it passing <em>--help</em> to check its
options). If the triva binary is not available after following the
installation instructions, you may want to execute the following command to
initialize the GNUstep environment variables. We strongly recommend that you
use the latest GNUstep packages, and not the packages available through apt-get
in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
available packages, you can execute this command:
\verbatim
$ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
\endverbatim
You should be able to see this output after the installation of triva:
\verbatim
$ ./Triva.app/Triva --help
Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
Trace Analysis through Visualization

TimeInterval
    --ti_frequency {double}    Animation: frequency of updates
    --ti_hide                  Hide the TimeInterval window
    --ti_forward {double}      Animation: value to move time-slice
    --ti_apply                 Apply the configuration
    --ti_update                Update on slider change
    --ti_animate               Start animation
    --ti_start {double}        Start of time slice
    --ti_size {double}         Size of time slice
Triva
    --comparison               Compare Trace Files (Experimental)
    --graph                    Configurable Graph
    --list                     Print Trace Type Hierarchy
    --hierarchy                Export Trace Type Hierarchy (dot)
    --stat                     Trace Statistics and Memory Utilization
    --instances                List All Trace Entities
    --linkview                 Link View (Experimental)
    --treemap                  Squarified Treemap
    --merge                    Merge Trace Files (Experimental)
    --check                    Check Trace File Integrity
GraphConfiguration
    --gc_conf {file}           Graph Configuration in Property List Format
    --gc_apply                 Apply the configuration
    --gc_hide                  Hide the GraphConfiguration window
\endverbatim
Triva expects that the user choose one of the available options 
(currently <em>--graph</em> or <em>--treemap</em> for a visualization analysis)
and the trace file from the simulation.

- <b>Understanding Triva - time-slice</b>: the analysis of a trace file using
  the tool always takes into account the concept of the <em>time-slice</em>.
This concept means that what is being visualized in the screen is always
calculated considering a specific time frame, with its beggining and end
timestamp. The time-slice is configured by the user and can be changed
dynamically through the window called <em>Time Interval</em> that is opened
whenever a trace file is being analyzed. The next figure depicts the time-slice
configuration window.
In the top of the window, in the space named <i>Trace Time</i>,
the two fields show the beggining of the trace (which usually starts in 0) and
the end (that depends on the time simulated by SimGrid). The middle of the
window, in the square named <i>Time Slice Configuration</i>, contains the
aspects related to the time-slice, including its <i>start</i> and its
<i>size</i>. The gray rectangle in the bottom of this part indicates the 
<i>current time-slice</i> that is considered for the drawings. If the checkbox 
<i>Update Drawings on Sliders Change</i> is not selected, the button
<i>Apply</i> must be clicked in order to inform triva that the
new time-slice must be considered. The bottom part of the window, in the space
indicated by the square <i>Time Slice Animation</i> can be used to advance
the time-frame automatically. The user configures the amount of time that the
time-frame will forward and how frequent this update will happen. Once this is
configured, the user clicks the <i>Play</i> button in order to see the dynamic
changes on the drawings.
<center>
\htmlonly
<a href="triva-time_interval.png" border=0><img src="triva-time_interval.png" width="50%" border=0></a>
\endhtmlonly
</center>
<b>Remarks:</b> when the trace has too many hosts or links, the computation to
take into account a new time-slice can be expensive. When this happens, the
<i>Frequency</i> parameter, but also updates caused by change on configurations
when the checkbox <i>Update Drawings on Sliders
Change</i> is selected will not be followed.

- <b>Understanding Triva - graph</b>: one possibility to analyze
  SimGrid traces is to use Triva's graph view, using the
  <em>--graph</em> parameter to activate this view, and
  <em>--gc_conf</em> with a graph configuration to customize the graph
  according to the traces. A valid graph configuration (we are using
  the non-XML <a
  href="http://en.wikipedia.org/wiki/Property_list">Property List
  Format</a> to describe the configuration) can be created for any
  SimGrid-based simulator using the
  <em>--cfg=triva/uncategorized:graph_uncategorized.plist</em> or
  <em>--cfg=triva/categorized:graph_categorized.plist</em> (if the
  simulator defines resource utilization categories) when executing
  the simulation.

<b>Basic SimGrid Configuration</b>: The basic description of the configuration
is as follows:
\verbatim
{
  node = (LINK, HOST, );
  edge = (HOST-LINK, LINK-HOST, LINK-LINK, );
\endverbatim

The nodes of the graph will be created based on the <i>node</i>
parameter, which in this case is the different <em>"HOST"</em>s and
<em>"LINK"</em>s of the platform used to simulate. The <i>edge</i>
parameter indicates that the edges of the graph will be created based
on the <em>"HOST-LINK"</em>s, <em>"LINK-HOST"</em>s, and
<em>"LINK-LINK"</em>s of the platform.  After the definition of these
two parameters, the configuration must detail how the nodes
(<em>HOST</em>s and <em>LINK</em>s) should be drawn.

For that, the configuration must have an entry for each of
the types used. For <em>HOST</em>, as basic configuration, we have:

\verbatim
  HOST = {
    type = square;
    size = power;
    values = (power_used);
  };
\endverbatim

The parameter <em>size</em> indicates which variable from the trace
file will be used to define the size of the node HOST in the
visualization. If the simulation was executed with availability
traces, the size of the nodes will be changed according to these
traces. The parameter <em>type</em> indicates which geometrical shape
will be used to represent HOST, and the <em>values</em> parameter
indicates which values from the trace will be used to fill the shape.

For <em>LINK</em> we have:

\verbatim 
  LINK = {
    type = rhombus;
    size = bandwidth;
    values = (bandwidth_used);
  };
}
\endverbatim

The same configuration parameters are used here: <em>type</em> (with a
rhombus), the <em>size</em> (whose value is from trace's bandwidth
variable) and the <em>values</em>.

<b>Customizing the Graph Representation</b>: triva is capable to handle
a customized graph representation based on the variables present in the trace
file. In the case of SimGrid, every time a category is created for tasks, two
variables in the trace file are defined: one to indicate node utilization (how
much power was used by that task category), and another to indicate link
utilization (how much bandwidth was used by that category). For instance, if the
user declares a category named <i>request</i>, there will be variables named
<b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b> for power and
<b>b</b> for bandwidth). It is important to notice that the variable
<i>prequest</i> in this case is only available for HOST, and
<i>brequest</i> is only available for LINK. <b>Example</b>: suppose there are
two categories for tasks: request and compute. To create a customized graph
representation with a proportional separation of host and link utilization, use
as configuration for HOST and LINK this:

\verbatim
  HOST = {
    type = square;
    size = power;
    values = (prequest, pcomputation);
  };
  LINK = {
    type = rhombus;
    size = bandwidth;
    values = (brequest, bcomputation);
  };
\endverbatim

This configuration enables the analysis of resource utilization by MSG tasks,
and the identification of load-balancing issues, network bottlenecks, for
instance. \n

<b>The Graph Visualization</b>: The next figure shows a graph visualization of a
given time-slice of the masterslave_forwarder example (present in the SimGrid
sources). The red color indicates tasks from the <i>compute</i> category. This
visualization was generated with the following configuration:

\verbatim
{
  node = (LINK, HOST, );
  edge = (HOST-LINK, LINK-HOST, LINK-LINK, );

  host = {
    type = square;
    size = power;
    values = (pcompute, pfinalize);
  };
  link = {
    type = rhombus;
    size = bandwidth;
    values = (bcompute, bfinalize);
  };
}
\endverbatim

<center>
\htmlonly
<a href="triva-graph_visualization.png" border=0><img src="triva-graph_visualization.png" width="50%" border=0></a>
\endhtmlonly
</center>

- <b>Understading Triva - colors</b>: Colors are now registered in
trace files. See the tracing API to how to define them for your
simulator.

*/