+++ /dev/null
-/*! \page tracing Tracing Simulations
-
-
-Tracing is widely used to observe and understand the behavior of
-parallel applications and distributed algorithms. Usually, this is
-done in a two-step fashion: the user instruments the application and
-the traces are analyzed after the end of the execution. The analysis
-can highlights unexpected behaviors, bottlenecks and sometimes can be
-used to correct distributed algorithms. The SimGrid team has
-instrumented the library in order to let users trace their simulations
-and analyze them. This part of the user manual explains how the
-tracing-related features can be enabled and used during the
-development of simulators using the SimGrid library.
-
-\section instr_category_functions Tracing categories functions
-
-The SimGrid library is instrumented so users can trace the platform
-utilization using MSG, SimDAG and SMPI interfaces. It registers how
-much power is used for each host and how much bandwidth is used for
-each link of the platform. The idea with this type of tracing is to
-observe the overall view of resources utilization in the first place,
-especially the identification of bottlenecks, load-balancing among
-hosts, and so on.
-
-Another possibility is to trace resource utilization by
-categories. Categorized resource utilization tracing gives SimGrid
-users to possibility to classify MSG and SimDAG tasks by category,
-tracing resource utilization for each of the categories. The functions
-below let the user declare a category and apply it to tasks. <em>The
-tasks that are not classified according to a category are not
-traced</em>. Even if the user does not specify any category, the
-simulations can still be traced in terms of resource utilization by
-using a special parameter that is detailed below (see section \ref
-tracing_tracing_options).
-
-\li \c TRACE_category(const char *category)
-\li \c TRACE_category_with_color(const char *category, const char *color)
-\li \c MSG_task_set_category(msg_task_t task, const char *category)
-\li \c MSG_task_get_category(msg_task_t task)
-\li \c SD_task_set_category(SD_task_t task, const char *category)
-\li \c SD_task_get_category(SD_task_t task)
-
-\section instr_mark_functions Tracing marks functions
-\li \c TRACE_declare_mark(const char *mark_type)
-\li \c TRACE_mark(const char *mark_type, const char *mark_value)
-
-\section instr_uservariables_functions Tracing user variables functions
-
-For hosts:
-
-\li \c TRACE_host_variable_declare(const char *variable)
-\li \c TRACE_host_variable_declare_with_color(const char *variable, const char *color)
-\li \c TRACE_host_variable_set(const char *host, const char *variable, double value)
-\li \c TRACE_host_variable_add(const char *host, const char *variable, double value)
-\li \c TRACE_host_variable_sub(const char *host, const char *variable, double value)
-\li \c TRACE_host_variable_set_with_time(double time, const char *host, const char *variable, double value)
-\li \c TRACE_host_variable_add_with_time(double time, const char *host, const char *variable, double value)
-\li \c TRACE_host_variable_sub_with_time(double time, const char *host, const char *variable, double value)
-
-For links:
-
-\li \c TRACE_link_variable_declare(const char *variable)
-\li \c TRACE_link_variable_declare_with_color(const char *variable, const char *color)
-\li \c TRACE_link_variable_set(const char *link, const char *variable, double value)
-\li \c TRACE_link_variable_add(const char *link, const char *variable, double value)
-\li \c TRACE_link_variable_sub(const char *link, const char *variable, double value)
-\li \c TRACE_link_variable_set_with_time(double time, const char *link, const char *variable, double value)
-\li \c TRACE_link_variable_add_with_time(double time, const char *link, const char *variable, double value)
-\li \c TRACE_link_variable_sub_with_time(double time, const char *link, const char *variable, double value)
-
-For links, but use source and destination to get route:
-
-\li \c TRACE_link_srcdst_variable_set(const char *src, const char *dst, const char *variable, double value)
-\li \c TRACE_link_srcdst_variable_add(const char *src, const char *dst, const char *variable, double value)
-\li \c TRACE_link_srcdst_variable_sub(const char *src, const char *dst, const char *variable, double value)
-\li \c TRACE_link_srcdst_variable_set_with_time(double time, const char *src, const char *dst, const char *variable, double value)
-\li \c TRACE_link_srcdst_variable_add_with_time(double time, const char *src, const char *dst, const char *variable, double value)
-\li \c TRACE_link_srcdst_variable_sub_with_time(double time, const char *src, const char *dst, const char *variable, double value)
-
-\section tracing_tracing_options Tracing configuration Options
-
-To check which tracing options are available for your simulator, you
-can just run it with the option \verbatim --help-tracing \endverbatim
-to get a very detailed and updated explanation of each tracing
-parameter. These are some of the options accepted by the tracing
-system of SimGrid, you can use them by running your simulator with the
-<b>--cfg=</b> switch:
-
-\li <b>\c
-tracing
-</b>:
- Safe switch. It activates (or deactivates) the tracing system.
- No other tracing options take effect if this one is not activated.
-\verbatim
---cfg=tracing:yes
-\endverbatim
-
-\li <b>\c
-tracing/categorized
-</b>:
- It activates the categorized resource utilization tracing. It should
- be enabled if tracing categories are used by this simulator.
-\verbatim
---cfg=tracing/categorized:yes
-\endverbatim
-
-\li <b>\c
-tracing/uncategorized
-</b>:
- It activates the uncategorized resource utilization tracing. Use it if
- this simulator do not use tracing categories and resource use have to be
- traced.
-\verbatim
---cfg=tracing/uncategorized:yes
-\endverbatim
-
-\li <b>\c
-tracing/filename
-</b>:
- A file with this name will be created to register the simulation. The file
- is in the Paje format and can be analyzed using Viva or Paje visualization
- tools. More information can be found in these webpages:
- <a href="http://github.com/schnorr/viva/">http://github.com/schnorr/viva/</a>
- <a href="http://github.com/schnorr/pajeng/">http://github.com/schnorr/pajeng/</a>
-\verbatim
---cfg=tracing/filename:mytracefile.trace
-\endverbatim
- If you do not provide this parameter, the trace file will be named simgrid.trace.
-
-\li <b>\c
-tracing/smpi
-</b>:
- This option only has effect if this simulator is SMPI-based. Traces the MPI
- interface and generates a trace that can be analyzed using Gantt-like
- visualizations. Every MPI function (implemented by SMPI) is transformed in a
- state, and point-to-point communications can be analyzed with arrows.
-\verbatim
---cfg=tracing/smpi:yes
-\endverbatim
-
-\li <b>\c
-tracing/smpi/group
-</b>:
- This option only has effect if this simulator is SMPI-based. The processes
- are grouped by the hosts where they were executed.
-\verbatim
---cfg=tracing/smpi/group:yes
-\endverbatim
-
-\li <b>\c
-tracing/smpi/computing
-</b>:
- This option only has effect if this simulator is SMPI-based. The parts external
-to SMPI are also outputted to the trace. Provides better way to analyze the data automatically.
-\verbatim
---cfg=tracing/smpi/computing:yes
-\endverbatim
-
-\li <b>\c
-tracing/smpi/internals
-</b>:
- This option only has effect if this simulator is SMPI-based. Display internal communications
-happening during a collective MPI call.
-\verbatim
---cfg=tracing/smpi/internals:yes
-\endverbatim
-
-\li <b>\c
-tracing/smpi/display-sizes
-</b>:
- This option only has effect if this simulator is SMPI-based. Display the sizes of the messages
-exchanged in the trace, both in the links and on the states. For collective, size means the global size of data sent by the process in general.
-\verbatim
---cfg=tracing/smpi/display-sizes:yes
-\endverbatim
-
-\li <b>\c
-tracing/smpi/sleeping
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-tracing/smpi/format
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-tracing/smpi/format/ti-one-file
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-tracing/msg/vm
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-tracing/msg/process
-</b>:
- This option only has effect if this simulator is MSG-based. It traces the
- behavior of all categorized MSG processes, grouping them by hosts. This option
- can be used to track process location if this simulator has process migration.
-\verbatim
---cfg=tracing/msg/process:yes
-\endverbatim
-
-\li <b>\c
-tracing/buffer
-</b>:
- This option put some events in a time-ordered buffer using the
- insertion sort algorithm. The process of acquiring and releasing
- locks to access this buffer and the cost of the sorting algorithm
- make this process slow. The simulator performance can be severely
- impacted if this option is activated, but you are sure to get a trace
- file with events sorted.
-\verbatim
---cfg=tracing/buffer:yes
-\endverbatim
-
-\li <b>\c
-tracing/onelink-only
-</b>:
-This option changes the way SimGrid register its platform on the trace
-file. Normally, the tracing considers all routes (no matter their
-size) on the platform file to re-create the resource topology. If this
-option is activated, only the routes with one link are used to
-register the topology within an AS. Routes among AS continue to be
-traced as usual.
-\verbatim
---cfg=tracing/onelink-only:yes
-\endverbatim
-
-\li <b>\c
-tracing/disable-link
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-tracing/disable-power
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-tracing/disable-destroy
-</b>:
-Disable the destruction of containers at the end of simulation. This
-can be used with simulators that have a different notion of time
-(different from the simulated time).
-\verbatim
---cfg=tracing/disable-destroy:yes
-\endverbatim
-
-\li <b>\c
-tracing/basic
-</b>:
-Some visualization tools are not able to parse correctly the Paje file format.
-Use this option if you are using one of these tools to visualize the simulation
-trace. Keep in mind that the trace might be incomplete, without all the
-information that would be registered otherwise.
-\verbatim
---cfg=tracing/basic:yes
-\endverbatim
-
-\li <b>\c
-tracing/comment
-</b>:
-Use this to add a comment line to the top of the trace file.
-\verbatim
---cfg=tracing/comment:my_string
-\endverbatim
-
-\li <b>\c
-tracing/comment-file
-</b>:
-Use this to add the contents of a file to the top of the trace file as comment.
-\verbatim
---cfg=tracing/comment-file:textual_file.txt
-\endverbatim
-
-\li <b>\c
-tracing/precision
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-tracing/platform
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-tracing/platform/topology
-</b>:
-TODO
-\verbatim
-TODO
-\endverbatim
-
-\li <b>\c
-viva/categorized
-</b>:
- This option generates a graph configuration file for Viva considering
- categorized resource utilization.
-\verbatim
---cfg=viva/categorized:graph_categorized.plist
-\endverbatim
-
-\li <b>\c
-viva/uncategorized
-</b>:
- This option generates a graph configuration file for Viva considering
- uncategorized resource utilization.
-\verbatim
---cfg=viva/uncategorized:graph_uncategorized.plist
-\endverbatim
-
-Please pass \verbatim --help-tracing \endverbatim to your simulator
-for the updated list of tracing options.
-
-\section tracing_tracing_example_parameters Case studies
-
-Some scenarios that might help you decide which tracing options
-you should use to analyze your simulator.
-
-\li I want to trace the resource utilization of all hosts
-and links of the platform, and my simulator <b>does not</b> use
-the tracing API. For that, you can run a uncategorized trace
-with the following parameters (it will work with <b>any</b> Simgrid
-simulator):
-\verbatim
-./your_simulator \
- --cfg=tracing:yes \
- --cfg=tracing/uncategorized:yes \
- --cfg=tracing/filename:mytracefile.trace \
- --cfg=viva/uncategorized:uncat.plist
-\endverbatim
-
-\li I want to trace only a subset of my MSG (or SimDAG) tasks.
-For that, you will need to create tracing categories using the
-<b>TRACE_category (...)</b> function (as explained above),
-and then classify your tasks to a previously declared category
-using the <b>MSG_task_set_category (...)</b>
-(or <b>SD_task_set_category (...)</b> for SimDAG tasks). After
-recompiling, run your simulator with the following parameters:
-\verbatim
-./your_simulator \
- --cfg=tracing:yes \
- --cfg=tracing/categorized:yes \
- --cfg=tracing/filename:mytracefile.trace \
- --cfg=viva/categorized:cat.plist
-\endverbatim
-
-
-\section tracing_tracing_example Example of Instrumentation
-
-A simplified example using the tracing mandatory functions.
-
-\verbatim
-int main (int argc, char **argv)
-{
- MSG_init (&argc, &argv);
-
- //(... after deployment ...)
-
- //note that category declaration must be called after MSG_create_environment
- TRACE_category_with_color ("request", "1 0 0");
- TRACE_category_with_color ("computation", "0.3 1 0.4");
- TRACE_category ("finalize");
-
- msg_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
- msg_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
- msg_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
- msg_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
- MSG_task_set_category (req1, "request");
- MSG_task_set_category (req2, "request");
- MSG_task_set_category (req3, "request");
- MSG_task_set_category (req4, "request");
-
- msg_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
- MSG_task_set_category (comp, "computation");
-
- msg_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
- MSG_task_set_category (finalize, "finalize");
-
- //(...)
-
- MSG_clean();
- return 0;
-}
-\endverbatim
-
-\section tracing_tracing_analyzing Analyzing SimGrid Simulation Traces
-
-A SimGrid-based simulator, when executed with the correct parameters
-(see above) creates a trace file in the Paje file format holding the
-simulated behavior of the application or the platform. You have
-several options to analyze this trace file:
-
-- Dump its contents to a CSV-like format using `pj_dump` (see <a
- href="https://github.com/schnorr/pajeng/wiki/pj_dump">PajeNG's wiki
- on pj_dump</a> and more generally the <a
- href="https://github.com/schnorr/pajeng/">PajeNG suite</a>) and use
- gnuplot to plot resource usage, time spent on blocking/executing
- functions, and so on. Filtering capabilities are at your hand by
- doing `grep`, with the best regular expression you can provide, to
- get only parts of the trace (for instance, only a subset of
- resources or processes).
-
-- Derive statistics from trace metrics (the ones built-in with any
- SimGrid simulation, but also those metrics you injected in the trace
- using the TRACE module) using the <a
- href="http://www.r-project.org/">R project</a> and all its
- modules. You can also combine R with <a
- href="http://ggplot2.org/">ggplot2</a> to get a number of high
- quality plots from your simulation metrics. You need to `pj_dump`
- the contents of the SimGrid trace file to use R.
-
-- Visualize the behavior of your simulation using classic space/time
- views (gantt-charts) provided by the <a
- href="https://github.com/schnorr/pajeng/">PajeNG suite</a> and any
- other tool that supports the <a
- href="http://paje.sourceforge.net/download/publication/lang-paje.pdf">Paje
- file format</a>. Consider this option if you need to understand the
- causality of your distributed simulation.
-
-- Visualize the behavior of your simulation with treemaps (specially
- if your simulation has a platform with several thousand resources),
- provided by the <a href="http://github.com/schnorr/viva/">Viva</a>
- visualization tool. See <a
- href="https://github.com/schnorr/viva/wiki">Viva's wiki</a> for
- further details on what is a treemap and how to use it.
-
-- Correlate the behavior of your simulator with the platform topology
- with an interactive, force-directed, and hierarchical graph
- visualization, provided by <a
- href="http://github.com/schnorr/viva/">Viva</a>. Check <a
- href="https://github.com/schnorr/viva/wiki">Viva's wiki</a> for
- further details. This <a
- href="http://hal.inria.fr/hal-00738321/">research report</a>,
- published at ISPASS 2013, has a detailed description of this
- visualization technique.
-
-- You can also check our online <a
- href="http://simgrid.gforge.inria.fr/tutorials.html"> tutorial
- section</a> that contains a dedicated tutorial with several
- suggestions on how to use the tracing infrastructure. Look for the
- SimGrid User::Visualization 101 tutorial.
-
-- Ask for help on the <a
- href="mailto:simgrid-user@lists.gforge.inria.fr">simgrid-user@lists.gforge.inria.fr</a>
- mailing list, giving us a detailed explanation on what your
- simulator does and what kind of information you want to trace. You
- can also check the <a
- href="http://lists.gforge.inria.fr/pipermail/simgrid-user/">mailing
- list archive</a> for old messages regarding tracing and analysis.
-
-\subsection tracing_viva_analysis Viva Visualization Tool
-
-This subsection describe some of the concepts regarding the <a
-href="http://github.com/schnorr/viva/">Viva Visualization Tool</a> and
-its relation with SimGrid traces. You should refer to Viva's website
-for further details on all its visualization techniques.
-
-\subsubsection tracing_viva_time_slice Time Slice
-
-The analysis of a trace file using the tool always takes into account
-the concept of the <em>time-slice</em>. This concept means that what
-is being visualized in the screen is always calculated considering a
-specific time frame, with its beggining and end timestamp. The
-time-slice is configured by the user and can be changed dynamically
-through the window called <em>Time Interval</em> that is opened
-whenever a trace file is being analyzed. Users are capable to select
-the beggining and size of the time slice.
-
-\subsubsection tracing_viva_graph Hierarchical Graph View
-
-%As stated above (see section \ref tracing_tracing_analyzing), one
-possibility to analyze SimGrid traces is to use Viva's graph view with
-a graph configuration to customize the graph according to the
-traces. A valid graph configuration (we are using the non-XML <a
-href="http://en.wikipedia.org/wiki/Property_list">Property List
-Format</a> to describe the configuration) can be created for any
-SimGrid-based simulator using the
-<em>--cfg=viva/uncategorized:graph_uncategorized.plist</em> or
-<em>--cfg=viva/categorized:graph_categorized.plist</em> (if the
-simulator defines resource utilization categories) when executing the
-simulation.
-
-\subsubsection basic_conf Basic Graph Configuration
-
-The basic description of the configuration is as follows:
-\verbatim
-{
- node = (LINK, HOST, );
- edge = (HOST-LINK, LINK-HOST, LINK-LINK, );
-\endverbatim
-
-The nodes of the graph will be created based on the <i>node</i>
-parameter, which in this case is the different <em>"HOST"</em>s and
-<em>"LINK"</em>s of the platform used to simulate. The <i>edge</i>
-parameter indicates that the edges of the graph will be created based
-on the <em>"HOST-LINK"</em>s, <em>"LINK-HOST"</em>s, and
-<em>"LINK-LINK"</em>s of the platform. After the definition of these
-two parameters, the configuration must detail how the nodes
-(<em>HOST</em>s and <em>LINK</em>s) should be drawn.
-
-For that, the configuration must have an entry for each of
-the types used. For <em>HOST</em>, as basic configuration, we have:
-
-\verbatim
- HOST = {
- type = square;
- size = power;
- values = (power_used);
- };
-\endverbatim
-
-The parameter <em>size</em> indicates which variable from the trace
-file will be used to define the size of the node HOST in the
-visualization. If the simulation was executed with availability
-traces, the size of the nodes will be changed according to these
-traces. The parameter <em>type</em> indicates which geometrical shape
-will be used to represent HOST, and the <em>values</em> parameter
-indicates which values from the trace will be used to fill the shape.
-
-For <em>LINK</em> we have:
-
-\verbatim
- LINK = {
- type = rhombus;
- size = bandwidth;
- values = (bandwidth_used);
- };
-}
-\endverbatim
-
-The same configuration parameters are used here: <em>type</em> (with a
-rhombus), the <em>size</em> (whose value is from trace's bandwidth
-variable) and the <em>values</em>.
-
-\subsubsection custom_graph Customizing the Graph Representation
-
-Viva is capable to handle a customized graph representation based on
-the variables present in the trace file. In the case of SimGrid, every
-time a category is created for tasks, two variables in the trace file
-are defined: one to indicate node utilization (how much power was used
-by that task category), and another to indicate link utilization (how
-much bandwidth was used by that category). For instance, if the user
-declares a category named <i>request</i>, there will be variables
-named <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b>
-for power and <b>b</b> for bandwidth). It is important to notice that
-the variable <i>prequest</i> in this case is only available for HOST,
-and <i>brequest</i> is only available for LINK. <b>Example</b>:
-suppose there are two categories for tasks: request and compute. To
-create a customized graph representation with a proportional
-separation of host and link utilization, use as configuration for HOST
-and LINK this:
-
-\verbatim
- HOST = {
- type = square;
- size = power;
- values = (prequest, pcomputation);
- };
- LINK = {
- type = rhombus;
- size = bandwidth;
- values = (brequest, bcomputation);
- };
-\endverbatim
-
-This configuration enables the analysis of resource utilization by MSG
-tasks through the identification of load-balancing issues and network
-bottlenecks, for instance.
-
-*/
