Fix smpi_execute_benched.

[simgrid.git] / doc / doxygen / outcomes_vizu.doc

/*! \page outcomes_vizu Visualization and Statistical Analysis

SimGrid comes with an extensive support to trace and register what
happens during the simulation, so that it can be either visualized or
statistically analysed after the simulation. 

This tracing is widely used to observe and understand the behavior of
parallel applications and distributed algorithms. Usually, this is
done in a two-step fashion: the user instruments the application and
the traces are analyzed after the end of the execution. The analysis
can highlights unexpected behaviors, bottlenecks and sometimes can be
used to correct distributed algorithms. The SimGrid team has
instrumented the library in order to let users trace their simulations
and analyze them. This part of the user manual explains how the
tracing-related features can be enabled and used during the
development of simulators using the SimGrid library.

\section instr_category_functions Tracing categories functions

The SimGrid library is instrumented so users can trace the platform
utilization using MSG, SimDAG and SMPI interfaces. It registers how
much power is used for each host and how much bandwidth is used for
each link of the platform. The idea with this type of tracing is to
observe the overall view of resources utilization in the first place,
especially the identification of bottlenecks, load-balancing among
hosts, and so on.

Another possibility is to trace resource utilization by
categories. Categorized resource utilization tracing gives SimGrid
users to possibility to classify MSG and SimDAG tasks by category,
tracing resource utilization for each of the categories. The functions
below let the user declare a category and apply it to tasks. <em>The
tasks that are not classified according to a category are not
traced</em>. Even if the user does not specify any category, the
simulations can still be traced in terms of resource utilization by
using a special parameter that is detailed below (see section \ref
tracing_tracing_options).

\li \c TRACE_category(const char *category)
\li \c TRACE_category_with_color(const char *category, const char *color)
\li \c MSG_task_set_category(msg_task_t task, const char *category)
\li \c MSG_task_get_category(msg_task_t task)
\li \c SD_task_set_category(SD_task_t task, const char *category)
\li \c SD_task_get_category(SD_task_t task)

\section instr_mark_functions Tracing marks functions
\li \c TRACE_declare_mark(const char *mark_type)
\li \c TRACE_mark(const char *mark_type, const char *mark_value)

\section instr_uservariables_functions Tracing user variables functions

For hosts:

\li \c TRACE_host_variable_declare(const char *variable)
\li \c TRACE_host_variable_declare_with_color(const char *variable, const char *color)
\li \c TRACE_host_variable_set(const char *host, const char *variable, double value)
\li \c TRACE_host_variable_add(const char *host, const char *variable, double value)
\li \c TRACE_host_variable_sub(const char *host, const char *variable, double value)
\li \c TRACE_host_variable_set_with_time(double time, const char *host, const char *variable, double value)
\li \c TRACE_host_variable_add_with_time(double time, const char *host, const char *variable, double value)
\li \c TRACE_host_variable_sub_with_time(double time, const char *host, const char *variable, double value)

For links:

\li \c TRACE_link_variable_declare(const char *variable)
\li \c TRACE_link_variable_declare_with_color(const char *variable, const char *color)
\li \c TRACE_link_variable_set(const char *link, const char *variable, double value)
\li \c TRACE_link_variable_add(const char *link, const char *variable, double value)
\li \c TRACE_link_variable_sub(const char *link, const char *variable, double value)
\li \c TRACE_link_variable_set_with_time(double time, const char *link, const char *variable, double value)
\li \c TRACE_link_variable_add_with_time(double time, const char *link, const char *variable, double value)
\li \c TRACE_link_variable_sub_with_time(double time, const char *link, const char *variable, double value)

For links, but use source and destination to get route:

\li \c TRACE_link_srcdst_variable_set(const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_add(const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_sub(const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_set_with_time(double time, const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_add_with_time(double time, const char *src, const char *dst, const char *variable, double value)
\li \c TRACE_link_srcdst_variable_sub_with_time(double time, const char *src, const char *dst, const char *variable, double value)

\section tracing_tracing_options Tracing configuration Options

To check which tracing options are available for your simulator, you
can just run it with the option \verbatim --help-tracing \endverbatim
to get a very detailed and updated explanation of each tracing
parameter. These are some of the options accepted by the tracing
system of SimGrid, you can use them by running your simulator with the
<b>--cfg=</b> switch:

\li <b>\c
tracing
</b>:
  Safe switch. It activates (or deactivates) the tracing system.
  No other tracing options take effect if this one is not activated.
\verbatim
--cfg=tracing:yes
\endverbatim

\li <b>\c
tracing/categorized
</b>:
  It activates the categorized resource utilization tracing. It should
  be enabled if tracing categories are used by this simulator.
\verbatim
--cfg=tracing/categorized:yes
\endverbatim

\li <b>\c
tracing/uncategorized
</b>:
  It activates the uncategorized resource utilization tracing. Use it if
  this simulator do not use tracing categories and resource use have to be
  traced.
\verbatim
--cfg=tracing/uncategorized:yes
\endverbatim

\li <b>\c
tracing/filename
</b>:
  A file with this name will be created to register the simulation. The file
  is in the Paje format and can be analyzed using Viva or Paje visualization
  tools. More information can be found in these webpages:
     <a href="http://github.com/schnorr/viva/">http://github.com/schnorr/viva/</a>
     <a href="http://github.com/schnorr/pajeng/">http://github.com/schnorr/pajeng/</a>
\verbatim
--cfg=tracing/filename:mytracefile.trace
\endverbatim
  If you do not provide this parameter, the trace file will be named simgrid.trace.

\li <b>\c
tracing/smpi
</b>:
  This option only has effect if this simulator is SMPI-based. Traces the MPI
  interface and generates a trace that can be analyzed using Gantt-like
  visualizations. Every MPI function (implemented by SMPI) is transformed in a
  state, and point-to-point communications can be analyzed with arrows.
\verbatim
--cfg=tracing/smpi:yes
\endverbatim

\li <b>\c
tracing/smpi/group
</b>:
  This option only has effect if this simulator is SMPI-based. The processes
  are grouped by the hosts where they were executed.
\verbatim
--cfg=tracing/smpi/group:yes
\endverbatim

\li <b>\c
tracing/smpi/computing
</b>:
  This option only has effect if this simulator is SMPI-based. The parts external
to SMPI are also outputted to the trace. Provides better way to analyze the data automatically.
\verbatim
--cfg=tracing/smpi/computing:yes
\endverbatim

\li <b>\c
tracing/smpi/internals
</b>:
  This option only has effect if this simulator is SMPI-based. Display internal communications
happening during a collective MPI call.
\verbatim
--cfg=tracing/smpi/internals:yes
\endverbatim

\li <b>\c
tracing/smpi/display-sizes
</b>:
  This option only has effect if this simulator is SMPI-based. Display the sizes of the messages
exchanged in the trace, both in the links and on the states. For collective, size means the global size of data sent by the process in general.
\verbatim
--cfg=tracing/smpi/display-sizes:yes
\endverbatim

\li <b>\c
tracing/smpi/sleeping
</b>:
TODO
\verbatim
TODO
\endverbatim

\li <b>\c
tracing/smpi/format
</b>:
TODO
\verbatim
TODO
\endverbatim

\li <b>\c
tracing/smpi/format/ti-one-file
</b>:
TODO
\verbatim
TODO
\endverbatim

\li <b>\c
tracing/msg/vm
</b>:
TODO
\verbatim
TODO
\endverbatim

\li <b>\c
tracing/msg/process
</b>:
  This option only has effect if this simulator is MSG-based. It traces the
  behavior of all categorized MSG processes, grouping them by hosts. This option
  can be used to track process location if this simulator has process migration.
\verbatim
--cfg=tracing/msg/process:yes
\endverbatim

\li <b>\c
tracing/buffer
</b>:
 This option put some events in a time-ordered buffer using the
 insertion sort algorithm. The process of acquiring and releasing
 locks to access this buffer and the cost of the sorting algorithm
 make this process slow. The simulator performance can be severely
 impacted if this option is activated, but you are sure to get a trace
 file with events sorted.
\verbatim
--cfg=tracing/buffer:yes
\endverbatim

\li <b>\c
tracing/onelink-only
</b>:
This option changes the way SimGrid register its platform on the trace
file. Normally, the tracing considers all routes (no matter their
size) on the platform file to re-create the resource topology. If this
option is activated, only the routes with one link are used to
register the topology within a netzone. Routes among netzones continue to be
traced as usual.
\verbatim
--cfg=tracing/onelink-only:yes
\endverbatim

\li <b>\c
tracing/disable-link
</b>:
TODO
\verbatim
TODO
\endverbatim

\li <b>\c
tracing/disable-power
</b>:
TODO
\verbatim
TODO
\endverbatim

\li <b>\c
tracing/disable-destroy
</b>:
Disable the destruction of containers at the end of simulation. This
can be used with simulators that have a different notion of time
(different from the simulated time).
\verbatim
--cfg=tracing/disable-destroy:yes
\endverbatim

\li <b>\c
tracing/basic
</b>:
Some visualization tools are not able to parse correctly the Paje file format.
Use this option if you are using one of these tools to visualize the simulation
trace. Keep in mind that the trace might be incomplete, without all the
information that would be registered otherwise.
\verbatim
--cfg=tracing/basic:yes
\endverbatim

\li <b>\c
tracing/comment
</b>:
Use this to add a comment line to the top of the trace file.
\verbatim
--cfg=tracing/comment:my_string
\endverbatim

\li <b>\c
tracing/comment-file
</b>:
Use this to add the contents of a file to the top of the trace file as comment.
\verbatim
--cfg=tracing/comment-file:textual_file.txt
\endverbatim

\li <b>\c
tracing/precision
</b>:
This option determines the precision of timings stored in the trace file. Make sure
you set \ref options_model_precision to at least the same value as this option! (Traces
cannot be more accurate than the simulation; they can be less accurate, though.)

The following example will give you a precision of E-10 in the trace file:
\verbatim
--cfg=tracing/precision:10
\endverbatim

\li <b>\c
tracing/platform
</b>:
TODO
\verbatim
TODO
\endverbatim

\li <b>\c
tracing/platform/topology
</b>:
TODO
\verbatim
TODO
\endverbatim

\li <b>\c
viva/categorized
</b>:
  This option generates a graph configuration file for Viva considering
  categorized resource utilization.
\verbatim
--cfg=viva/categorized:graph_categorized.plist
\endverbatim

\li <b>\c
viva/uncategorized
</b>:
  This option generates a graph configuration file for Viva considering
  uncategorized resource utilization.
\verbatim
--cfg=viva/uncategorized:graph_uncategorized.plist
\endverbatim

Please pass \verbatim --help-tracing \endverbatim to your simulator
for the updated list of tracing options.

\section tracing_tracing_example_parameters Case studies

Some scenarios that might help you decide which tracing options
you should use to analyze your simulator.

\li I want to trace the resource utilization of all hosts
and links of the platform, and my simulator <b>does not</b> use
the tracing API. For that, you can run a uncategorized trace
with the following parameters (it will work with <b>any</b> Simgrid
simulator):
\verbatim
./your_simulator \
          --cfg=tracing:yes \
          --cfg=tracing/uncategorized:yes \
          --cfg=tracing/filename:mytracefile.trace \
          --cfg=viva/uncategorized:uncat.plist
\endverbatim

\li I want to trace only a subset of my MSG (or SimDAG) tasks.
For that, you will need to create tracing categories using the
<b>TRACE_category (...)</b> function (as explained above),
and then classify your tasks to a previously declared category
using the <b>MSG_task_set_category (...)</b>
(or <b>SD_task_set_category (...)</b> for SimDAG tasks). After
recompiling, run your simulator with the following parameters:
\verbatim
./your_simulator \
          --cfg=tracing:yes \
          --cfg=tracing/categorized:yes \
          --cfg=tracing/filename:mytracefile.trace \
          --cfg=viva/categorized:cat.plist
\endverbatim


\section tracing_tracing_example Example of Instrumentation

A simplified example using the tracing mandatory functions.

\verbatim
int main (int argc, char **argv)
{
  MSG_init (&argc, &argv);

  //(... after deployment ...)

  //note that category declaration must be called after MSG_create_environment
  TRACE_category_with_color ("request", "1 0 0");
  TRACE_category_with_color ("computation", "0.3 1 0.4");
  TRACE_category ("finalize");

  msg_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
  msg_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
  msg_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
  msg_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
  MSG_task_set_category (req1, "request");
  MSG_task_set_category (req2, "request");
  MSG_task_set_category (req3, "request");
  MSG_task_set_category (req4, "request");

  msg_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
  MSG_task_set_category (comp, "computation");

  msg_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
  MSG_task_set_category (finalize, "finalize");

  //(...)

  MSG_clean();
  return 0;
}
\endverbatim

\section tracing_tracing_analyzing Analyzing SimGrid Simulation Traces

A SimGrid-based simulator, when executed with the correct parameters
(see above) creates a trace file in the Paje file format holding the
simulated behavior of the application or the platform. You have
several options to analyze this trace file:

- Dump its contents to a CSV-like format using `pj_dump` (see <a
  href="https://github.com/schnorr/pajeng/wiki/pj_dump">PajeNG's wiki
  on pj_dump</a> and more generally the <a
  href="https://github.com/schnorr/pajeng/">PajeNG suite</a>) and use
  gnuplot to plot resource usage, time spent on blocking/executing
  functions, and so on. Filtering capabilities are at your hand by
  doing `grep`, with the best regular expression you can provide, to
  get only parts of the trace (for instance, only a subset of
  resources or processes).

- Derive statistics from trace metrics (the ones built-in with any
  SimGrid simulation, but also those metrics you injected in the trace
  using the TRACE module) using the <a
  href="http://www.r-project.org/">R project</a> and all its
  modules. You can also combine R with <a
  href="http://ggplot2.org/">ggplot2</a> to get a number of high
  quality plots from your simulation metrics. You need to `pj_dump`
  the contents of the SimGrid trace file to use R.

- Visualize the behavior of your simulation using classic space/time
  views (gantt-charts) provided by the <a
  href="https://github.com/schnorr/pajeng/">PajeNG suite</a> and any
  other tool that supports the <a
  href="http://paje.sourceforge.net/download/publication/lang-paje.pdf">Paje
  file format</a>. Consider this option if you need to understand the
  causality of your distributed simulation.

- Visualize the behavior of your simulation with treemaps (specially
  if your simulation has a platform with several thousand resources),
  provided by the <a href="http://github.com/schnorr/viva/">Viva</a>
  visualization tool. See <a
  href="https://github.com/schnorr/viva/wiki">Viva's wiki</a> for
  further details on what is a treemap and how to use it.

- Correlate the behavior of your simulator with the platform topology
  with an interactive, force-directed, and hierarchical graph
  visualization, provided by <a
  href="http://github.com/schnorr/viva/">Viva</a>.  Check <a
  href="https://github.com/schnorr/viva/wiki">Viva's wiki</a> for
  further details. This <a
  href="http://hal.inria.fr/hal-00738321/">research report</a>,
  published at ISPASS 2013, has a detailed description of this
  visualization technique.

- You can also check our online <a
  href="http://simgrid.gforge.inria.fr/tutorials.html"> tutorial
  section</a> that contains a dedicated tutorial with several
  suggestions on how to use the tracing infrastructure. Look for the
  SimGrid User::Visualization 101 tutorial.

- Ask for help on the <a
  href="mailto:simgrid-user@lists.gforge.inria.fr">simgrid-user@lists.gforge.inria.fr</a>
  mailing list, giving us a detailed explanation on what your
  simulator does and what kind of information you want to trace. You
  can also check the <a
  href="http://lists.gforge.inria.fr/pipermail/simgrid-user/">mailing
  list archive</a> for old messages regarding tracing and analysis.

\subsection tracing_viva_analysis Viva Visualization Tool

This subsection describe some of the concepts regarding the <a
href="http://github.com/schnorr/viva/">Viva Visualization Tool</a> and
its relation with SimGrid traces. You should refer to Viva's website
for further details on all its visualization techniques.

\subsubsection tracing_viva_time_slice Time Slice

The analysis of a trace file using the tool always takes into account
the concept of the <em>time-slice</em>.  This concept means that what
is being visualized in the screen is always calculated considering a
specific time frame, with its beggining and end timestamp. The
time-slice is configured by the user and can be changed dynamically
through the window called <em>Time Interval</em> that is opened
whenever a trace file is being analyzed. Users are capable to select
the beggining and size of the time slice.

\subsubsection tracing_viva_graph Hierarchical Graph View

As stated above (see section \ref tracing_tracing_analyzing), one
possibility to analyze SimGrid traces is to use Viva's graph view with
a graph configuration to customize the graph according to the
traces. A valid graph configuration (we are using the non-XML <a
href="http://en.wikipedia.org/wiki/Property_list">Property List
Format</a> to describe the configuration) can be created for any
SimGrid-based simulator using the
<em>--cfg=viva/uncategorized:graph_uncategorized.plist</em> or
<em>--cfg=viva/categorized:graph_categorized.plist</em> (if the
simulator defines resource utilization categories) when executing the
simulation.

\subsubsection basic_conf Basic Graph Configuration

The basic description of the configuration is as follows:
\verbatim
{
  node = (LINK, HOST, );
  edge = (HOST-LINK, LINK-HOST, LINK-LINK, );
\endverbatim

The nodes of the graph will be created based on the <i>node</i>
parameter, which in this case is the different <em>"HOST"</em>s and
<em>"LINK"</em>s of the platform used to simulate. The <i>edge</i>
parameter indicates that the edges of the graph will be created based
on the <em>"HOST-LINK"</em>s, <em>"LINK-HOST"</em>s, and
<em>"LINK-LINK"</em>s of the platform.  After the definition of these
two parameters, the configuration must detail how the nodes
(<em>HOST</em>s and <em>LINK</em>s) should be drawn.

For that, the configuration must have an entry for each of
the types used. For <em>HOST</em>, as basic configuration, we have:

\verbatim
  HOST = {
    type = square;
    size = power;
    values = (power_used);
  };
\endverbatim

The parameter <em>size</em> indicates which variable from the trace
file will be used to define the size of the node HOST in the
visualization. If the simulation was executed with availability
traces, the size of the nodes will be changed according to these
traces. The parameter <em>type</em> indicates which geometrical shape
will be used to represent HOST, and the <em>values</em> parameter
indicates which values from the trace will be used to fill the shape.

For <em>LINK</em> we have:

\verbatim
  LINK = {
    type = rhombus;
    size = bandwidth;
    values = (bandwidth_used);
  };
}
\endverbatim

The same configuration parameters are used here: <em>type</em> (with a
rhombus), the <em>size</em> (whose value is from trace's bandwidth
variable) and the <em>values</em>.

\subsubsection custom_graph Customizing the Graph Representation

Viva is capable to handle a customized graph representation based on
the variables present in the trace file. In the case of SimGrid, every
time a category is created for tasks, two variables in the trace file
are defined: one to indicate node utilization (how much power was used
by that task category), and another to indicate link utilization (how
much bandwidth was used by that category). For instance, if the user
declares a category named <i>request</i>, there will be variables
named <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b>
for power and <b>b</b> for bandwidth). It is important to notice that
the variable <i>prequest</i> in this case is only available for HOST,
and <i>brequest</i> is only available for LINK. <b>Example</b>:
suppose there are two categories for tasks: request and compute. To
create a customized graph representation with a proportional
separation of host and link utilization, use as configuration for HOST
and LINK this:

\verbatim
  HOST = {
    type = square;
    size = power;
    values = (prequest, pcomputation);
  };
  LINK = {
    type = rhombus;
    size = bandwidth;
    values = (brequest, bcomputation);
  };
\endverbatim

This configuration enables the analysis of resource utilization by MSG
tasks through the identification of load-balancing issues and network
bottlenecks, for instance.

*/