X-Git-Url: http://info.iut-bm.univ-fcomte.fr/pub/gitweb/simgrid.git/blobdiff_plain/9e68ca10e951fb61e944c99c7774b1e415ae9f6d..bb0a82c6a53abd8c2c09cc556781fca5928bb6b9:/doc/doxygen/tracing.doc
diff --git a/doc/doxygen/tracing.doc b/doc/doxygen/tracing.doc
index a21a0492e1..b4b203eeeb 100644
--- a/doc/doxygen/tracing.doc
+++ b/doc/doxygen/tracing.doc
@@ -1,33 +1,16 @@
-/*! \page tracing Tracing Simulations for Visualization
-
-
-The trace visualization is widely used to observe and understand the behavior
-of parallel applications and distributed algorithms. Usually, this is done in a
-two-step fashion: the user instruments the application and the traces are
-analyzed after the end of the execution. The visualization itself can highlights
-unexpected behaviors, bottlenecks and sometimes can be used to correct
-distributed algorithms. The SimGrid team has instrumented the library
-in order to let users trace their simulations and analyze them. This part of the
-user manual explains how the tracing-related features can be enabled and used
-during the development of simulators using the SimGrid library.
-
-\section tracing_tracing_howitworks How it works
-
-For now, the SimGrid library is instrumented so users can trace the platform
-utilization using the MSG, SimDAG and SMPI interface. This means that the tracing will
-register how much power is used for each host and how much bandwidth is used for
-each link of the platform. The idea with this type of tracing is to observe the
-overall view of resources utilization in the first place, especially the
-identification of bottlenecks, load-balancing among hosts, and so on.
-
-The idea of the tracing facilities is to give SimGrid users to possibility to
-classify MSG and SimDAG tasks by category, tracing the platform utilization
-(hosts and links) for each of the categories. For that,
-the tracing interface enables the declaration of categories and a function to
-mark a task with a previously declared category. The tasks that are not
-classified according to a category are not traced. Even if the user
-does not specify any category, the simulations can still be traced in terms
-of resource utilization by using a special parameter that is detailed below.
+/*! \page tracing Tracing Simulations
+
+
+Tracing is widely used to observe and understand the behavior of
+parallel applications and distributed algorithms. Usually, this is
+done in a two-step fashion: the user instruments the application and
+the traces are analyzed after the end of the execution. The analysis
+can highlights unexpected behaviors, bottlenecks and sometimes can be
+used to correct distributed algorithms. The SimGrid team has
+instrumented the library in order to let users trace their simulations
+and analyze them. This part of the user manual explains how the
+tracing-related features can be enabled and used during the
+development of simulators using the SimGrid library.
\section tracing_tracing_enabling Enabling using CMake
@@ -46,6 +29,26 @@ $ make
\endverbatim
\section instr_category_functions Tracing categories functions
+
+The SimGrid library is instrumented so users can trace the platform
+utilization using MSG, SimDAG and SMPI interfaces. It registers how
+much power is used for each host and how much bandwidth is used for
+each link of the platform. The idea with this type of tracing is to
+observe the overall view of resources utilization in the first place,
+especially the identification of bottlenecks, load-balancing among
+hosts, and so on.
+
+Another possibility is to trace resource utilization by
+categories. Categorized resource utilization tracing gives SimGrid
+users to possibility to classify MSG and SimDAG tasks by category,
+tracing resource utilization for each of the categories. The functions
+below let the user declare a category and apply it to tasks. The
+tasks that are not classified according to a category are not
+traced. Even if the user does not specify any category, the
+simulations can still be traced in terms of resource utilization by
+using a special parameter that is detailed below (see section \ref
+tracing_tracing_options).
+
\li \c TRACE_category(const char *category)
\li \c TRACE_category_with_color(const char *category, const char *color)
\li \c MSG_task_set_category(msg_task_t task, const char *category)
@@ -93,9 +96,11 @@ For links, but use source and destination to get route:
\section tracing_tracing_options Tracing configuration Options
To check which tracing options are available for your simulator, you
-can just run it with the option --help-tracing. These are the
-options accepted by the tracing system of SimGrid as of today, you
-can use them by running your simulator with the --cfg= switch:
+can just run it with the option \verbatim --help-tracing \endverbatim
+to get a very detailed and updated explanation of each tracing
+parameter. These are some of the options accepted by the tracing
+system of SimGrid, you can use them by running your simulator with the
+--cfg= switch:
\li \c
tracing
@@ -103,7 +108,7 @@ tracing
Safe switch. It activates (or deactivates) the tracing system.
No other tracing options take effect if this one is not activated.
\verbatim
---cfg=tracing:1
+--cfg=tracing:yes
\endverbatim
\li \c
@@ -112,7 +117,7 @@ tracing/categorized
It activates the categorized resource utilization tracing. It should
be enabled if tracing categories are used by this simulator.
\verbatim
---cfg=tracing/categorized:1
+--cfg=tracing/categorized:yes
\endverbatim
\li \c
@@ -122,17 +127,17 @@ tracing/uncategorized
this simulator do not use tracing categories and resource use have to be
traced.
\verbatim
---cfg=tracing/uncategorized:1
+--cfg=tracing/uncategorized:yes
\endverbatim
\li \c
tracing/filename
:
A file with this name will be created to register the simulation. The file
- is in the Paje format and can be analyzed using Triva or Paje visualization
+ is in the Paje format and can be analyzed using Viva or Paje visualization
tools. More information can be found in these webpages:
- http://triva.gforge.inria.fr/
- http://paje.sourceforge.net/
+ http://github.com/schnorr/viva/
+ http://github.com/schnorr/pajeng/
\verbatim
--cfg=tracing/filename:mytracefile.trace
\endverbatim
@@ -146,7 +151,7 @@ tracing/onelink_only
This option let the user tell the tracing system to use only the routes
that are composed with just one link.
\verbatim
---cfg=tracing/onelink_only:1
+--cfg=tracing/onelink_only:yes
\endverbatim
\li \c
@@ -157,7 +162,7 @@ tracing/smpi
visualizations. Every MPI function (implemented by SMPI) is transformed in a
state, and point-to-point communications can be analyzed with arrows.
\verbatim
---cfg=tracing/smpi:1
+--cfg=tracing/smpi:yes
\endverbatim
\li \c
@@ -166,7 +171,34 @@ tracing/smpi/group
This option only has effect if this simulator is SMPI-based. The processes
are grouped by the hosts where they were executed.
\verbatim
---cfg=tracing/smpi/group:1
+--cfg=tracing/smpi/group:yes
+\endverbatim
+
+\li \c
+tracing/smpi/computing
+:
+ This option only has effect if this simulator is SMPI-based. The parts external
+to SMPI are also outputted to the trace. Provides better way to analyze the data automatically.
+\verbatim
+--cfg=tracing/smpi/computing:yes
+\endverbatim
+
+\li \c
+tracing/smpi/internals
+:
+ This option only has effect if this simulator is SMPI-based. Display internal communications
+happening during a collective MPI call.
+\verbatim
+--cfg=tracing/smpi/internals:yes
+\endverbatim
+
+\li \c
+tracing/smpi/display_sizes
+:
+ This option only has effect if this simulator is SMPI-based. Display the sizes of the messages
+exchanged in the trace, both in the links and on the states. For collective, size means the global size of data sent by the process in general.
+\verbatim
+--cfg=tracing/smpi/display_sizes:yes
\endverbatim
\li \c
@@ -176,7 +208,7 @@ tracing/msg/process
behavior of all categorized MSG processes, grouping them by hosts. This option
can be used to track process location if this simulator has process migration.
\verbatim
---cfg=tracing/msg/process:1
+--cfg=tracing/msg/process:yes
\endverbatim
\li \c
@@ -189,7 +221,7 @@ tracing/buffer
impacted if this option is activated, but you are sure to get a trace
file with events sorted.
\verbatim
---cfg=tracing/buffer:1
+--cfg=tracing/buffer:yes
\endverbatim
\li \c
@@ -202,7 +234,7 @@ option is activated, only the routes with one link are used to
register the topology within an AS. Routes among AS continue to be
traced as usual.
\verbatim
---cfg=tracing/onelink_only:1
+--cfg=tracing/onelink_only:yes
\endverbatim
\li \c
@@ -212,7 +244,7 @@ Disable the destruction of containers at the end of simulation. This
can be used with simulators that have a different notion of time
(different from the simulated time).
\verbatim
---cfg=tracing/disable_destroy:1
+--cfg=tracing/disable_destroy:yes
\endverbatim
\li \c
@@ -223,7 +255,7 @@ Use this option if you are using one of these tools to visualize the simulation
trace. Keep in mind that the trace might be incomplete, without all the
information that would be registered otherwise.
\verbatim
---cfg=tracing/basic:1
+--cfg=tracing/basic:yes
\endverbatim
\li \c
@@ -243,23 +275,26 @@ Use this to add the contents of a file to the top of the trace file as comment.
\endverbatim
\li \c
-triva/categorized
+viva/categorized
:
- This option generates a graph configuration file for Triva considering
+ This option generates a graph configuration file for Viva considering
categorized resource utilization.
\verbatim
---cfg=triva/categorized:graph_categorized.plist
+--cfg=viva/categorized:graph_categorized.plist
\endverbatim
\li \c
-triva/uncategorized
+viva/uncategorized
:
- This option generates a graph configuration file for Triva considering
+ This option generates a graph configuration file for Viva considering
uncategorized resource utilization.
\verbatim
---cfg=triva/uncategorized:graph_uncategorized.plist
+--cfg=viva/uncategorized:graph_uncategorized.plist
\endverbatim
+Please pass \verbatim --help-tracing \endverbatim to your simulator
+for the updated list of tracing options.
+
\section tracing_tracing_example_parameters Case studies
Some scenarios that might help you decide which tracing options
@@ -272,10 +307,10 @@ with the following parameters (it will work with any Simgrid
simulator):
\verbatim
./your_simulator \
- --cfg=tracing:1 \
- --cfg=tracing/uncategorized:1 \
+ --cfg=tracing:yes \
+ --cfg=tracing/uncategorized:yes \
--cfg=tracing/filename:mytracefile.trace \
- --cfg=triva/uncategorized:uncat.plist
+ --cfg=viva/uncategorized:uncat.plist
\endverbatim
\li I want to trace only a subset of my MSG (or SimDAG) tasks.
@@ -287,10 +322,10 @@ using the MSG_task_set_category (...)
recompiling, run your simulator with the following parameters:
\verbatim
./your_simulator \
- --cfg=tracing:1 \
- --cfg=tracing/categorized:1 \
+ --cfg=tracing:yes \
+ --cfg=tracing/categorized:yes \
--cfg=tracing/filename:mytracefile.trace \
- --cfg=triva/categorized:cat.plist
+ --cfg=viva/categorized:cat.plist
\endverbatim
@@ -332,123 +367,106 @@ int main (int argc, char **argv)
}
\endverbatim
-\section tracing_tracing_analyzing Analyzing the SimGrid Traces
-
-The SimGrid library, during an instrumented simulation, creates a trace file in
-the Paje file format that contains the platform utilization for the simulation
-that was executed. The visualization analysis of this file is performed with the
-visualization tool Triva, with
-special configurations tunned to SimGrid needs. This part of the documentation
-explains how to configure and use Triva to analyse a SimGrid trace file.
-
-- Installing Triva: the tool is available in the Inria's Forge,
-at http://triva.gforge.inria.fr.
-Use the following command to get the sources, and then check the file
-INSTALL. This file contains instructions to install
-the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
-be compiled in MacOSX natively, check INSTALL.mac file.
-\verbatim
-$ git clone git://scm.gforge.inria.fr/triva/triva.git
-$ cd triva
-$ cat INSTALL
-\endverbatim
-
-- Executing Triva: a binary called Triva is available after the
- installation (you can execute it passing --help to check its
-options). If the triva binary is not available after following the
-installation instructions, you may want to execute the following command to
-initialize the GNUstep environment variables. We strongly recommend that you
-use the latest GNUstep packages, and not the packages available through apt-get
-in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
-available packages, you can execute this command:
-\verbatim
-$ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
-\endverbatim
-You should be able to see this output after the installation of triva:
-\verbatim
-$ ./Triva.app/Triva --help
-Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
-Trace Analysis through Visualization
-
-TimeInterval
- --ti_frequency {double} Animation: frequency of updates
- --ti_hide Hide the TimeInterval window
- --ti_forward {double} Animation: value to move time-slice
- --ti_apply Apply the configuration
- --ti_update Update on slider change
- --ti_animate Start animation
- --ti_start {double} Start of time slice
- --ti_size {double} Size of time slice
-Triva
- --comparison Compare Trace Files (Experimental)
- --graph Configurable Graph
- --list Print Trace Type Hierarchy
- --hierarchy Export Trace Type Hierarchy (dot)
- --stat Trace Statistics and Memory Utilization
- --instances List All Trace Entities
- --linkview Link View (Experimental)
- --treemap Squarified Treemap
- --merge Merge Trace Files (Experimental)
- --check Check Trace File Integrity
-GraphConfiguration
- --gc_conf {file} Graph Configuration in Property List Format
- --gc_apply Apply the configuration
- --gc_hide Hide the GraphConfiguration window
-\endverbatim
-Triva expects that the user choose one of the available options
-(currently --graph or --treemap for a visualization analysis)
-and the trace file from the simulation.
-
-- Understanding Triva - time-slice: the analysis of a trace file using
- the tool always takes into account the concept of the time-slice.
-This concept means that what is being visualized in the screen is always
-calculated considering a specific time frame, with its beggining and end
-timestamp. The time-slice is configured by the user and can be changed
-dynamically through the window called Time Interval that is opened
-whenever a trace file is being analyzed. The next figure depicts the time-slice
-configuration window.
-In the top of the window, in the space named Trace Time,
-the two fields show the beggining of the trace (which usually starts in 0) and
-the end (that depends on the time simulated by SimGrid). The middle of the
-window, in the square named Time Slice Configuration, contains the
-aspects related to the time-slice, including its start and its
-size. The gray rectangle in the bottom of this part indicates the
-current time-slice that is considered for the drawings. If the checkbox
-Update Drawings on Sliders Change is not selected, the button
-Apply must be clicked in order to inform triva that the
-new time-slice must be considered. The bottom part of the window, in the space
-indicated by the square Time Slice Animation can be used to advance
-the time-frame automatically. The user configures the amount of time that the
-time-frame will forward and how frequent this update will happen. Once this is
-configured, the user clicks the Play button in order to see the dynamic
-changes on the drawings.
-
-\htmlonly
-
-\endhtmlonly
-
-Remarks: when the trace has too many hosts or links, the computation to
-take into account a new time-slice can be expensive. When this happens, the
-Frequency parameter, but also updates caused by change on configurations
-when the checkbox Update Drawings on Sliders
-Change is selected will not be followed.
-
-- Understanding Triva - graph: one possibility to analyze
- SimGrid traces is to use Triva's graph view, using the
- --graph parameter to activate this view, and
- --gc_conf with a graph configuration to customize the graph
- according to the traces. A valid graph configuration (we are using
- the non-XML Property List
- Format to describe the configuration) can be created for any
- SimGrid-based simulator using the
- --cfg=triva/uncategorized:graph_uncategorized.plist or
- --cfg=triva/categorized:graph_categorized.plist (if the
- simulator defines resource utilization categories) when executing
- the simulation.
-
-Basic SimGrid Configuration: The basic description of the configuration
-is as follows:
+\section tracing_tracing_analyzing Analyzing SimGrid Simulation Traces
+
+A SimGrid-based simulator, when executed with the correct parameters
+(see above) creates a trace file in the Paje file format holding the
+simulated behavior of the application or the platform. You have
+several options to analyze this trace file:
+
+- Dump its contents to a CSV-like format using `pj_dump` (see PajeNG's wiki
+ on pj_dump and more generally the PajeNG suite) and use
+ gnuplot to plot resource usage, time spent on blocking/executing
+ functions, and so on. Filtering capabilities are at your hand by
+ doing `grep`, with the best regular expression you can provide, to
+ get only parts of the trace (for instance, only a subset of
+ resources or processes).
+
+- Derive statistics from trace metrics (the ones built-in with any
+ SimGrid simulation, but also those metrics you injected in the trace
+ using the TRACE module) using the R project and all its
+ modules. You can also combine R with ggplot2 to get a number of high
+ quality plots from your simulation metrics. You need to `pj_dump`
+ the contents of the SimGrid trace file to use R.
+
+- Visualize the behavior of your simulation using classic space/time
+ views (gantt-charts) provided by the PajeNG suite and any
+ other tool that supports the Paje
+ file format. Consider this option if you need to understand the
+ causality of your distributed simulation.
+
+- Visualize the behavior of your simulation with treemaps (specially
+ if your simulation has a platform with several thousand resources),
+ provided by the Viva
+ visualization tool. See Viva's wiki for
+ further details on what is a treemap and how to use it.
+
+- Correlate the behavior of your simulator with the platform topology
+ with an interactive, force-directed, and hierarchical graph
+ visualization, provided by Viva. Check Viva's wiki for
+ further details. This research report,
+ published at ISPASS 2013, has a detailed description of this
+ visualization technique.
+
+- You can also check our online tutorial
+ section that contains a dedicated tutorial with several
+ suggestions on how to use the tracing infrastructure. Look for the
+ SimGrid User::Visualization 101 tutorial.
+
+- Ask for help on the simgrid-user@lists.gforge.inria.fr
+ mailing list, giving us a detailed explanation on what your
+ simulator does and what kind of information you want to trace. You
+ can also check the mailing
+ list archive for old messages regarding tracing and analysis.
+
+\subsection tracing_viva_analysis Viva Visualization Tool
+
+This subsection describe some of the concepts regarding the Viva Visualization Tool and
+its relation with SimGrid traces. You should refer to Viva's website
+for further details on all its visualization techniques.
+
+\subsubsection tracing_viva_time_slice Time Slice
+
+The analysis of a trace file using the tool always takes into account
+the concept of the time-slice. This concept means that what
+is being visualized in the screen is always calculated considering a
+specific time frame, with its beggining and end timestamp. The
+time-slice is configured by the user and can be changed dynamically
+through the window called Time Interval that is opened
+whenever a trace file is being analyzed. Users are capable to select
+the beggining and size of the time slice.
+
+\subsubsection tracing_viva_graph Hierarchical Graph View
+
+%As stated above (see section \ref tracing_tracing_analyzing), one
+possibility to analyze SimGrid traces is to use Viva's graph view with
+a graph configuration to customize the graph according to the
+traces. A valid graph configuration (we are using the non-XML Property List
+Format to describe the configuration) can be created for any
+SimGrid-based simulator using the
+--cfg=viva/uncategorized:graph_uncategorized.plist or
+--cfg=viva/categorized:graph_categorized.plist (if the
+simulator defines resource utilization categories) when executing the
+simulation.
+
+\subsubsection basic_conf Basic Graph Configuration
+
+The basic description of the configuration is as follows:
\verbatim
{
node = (LINK, HOST, );
@@ -498,20 +516,23 @@ The same configuration parameters are used here: type (with a
rhombus), the size (whose value is from trace's bandwidth
variable) and the values.
-Customizing the Graph Representation: triva is capable to handle
-a customized graph representation based on the variables present in the trace
-file. In the case of SimGrid, every time a category is created for tasks, two
-variables in the trace file are defined: one to indicate node utilization (how
-much power was used by that task category), and another to indicate link
-utilization (how much bandwidth was used by that category). For instance, if the
-user declares a category named request, there will be variables named
-prequest and a brequest (p for power and
-b for bandwidth). It is important to notice that the variable
-prequest in this case is only available for HOST, and
-brequest is only available for LINK. Example: suppose there are
-two categories for tasks: request and compute. To create a customized graph
-representation with a proportional separation of host and link utilization, use
-as configuration for HOST and LINK this:
+\subsubsection custom_graph Customizing the Graph Representation
+
+Viva is capable to handle a customized graph representation based on
+the variables present in the trace file. In the case of SimGrid, every
+time a category is created for tasks, two variables in the trace file
+are defined: one to indicate node utilization (how much power was used
+by that task category), and another to indicate link utilization (how
+much bandwidth was used by that category). For instance, if the user
+declares a category named request, there will be variables
+named prequest and a brequest (p
+for power and b for bandwidth). It is important to notice that
+the variable prequest in this case is only available for HOST,
+and brequest is only available for LINK. Example:
+suppose there are two categories for tasks: request and compute. To
+create a customized graph representation with a proportional
+separation of host and link utilization, use as configuration for HOST
+and LINK this:
\verbatim
HOST = {
@@ -526,41 +547,8 @@ as configuration for HOST and LINK this:
};
\endverbatim
-This configuration enables the analysis of resource utilization by MSG tasks,
-and the identification of load-balancing issues, network bottlenecks, for
-instance. \n
-
-The Graph Visualization: The next figure shows a graph visualization of a
-given time-slice of the masterslave_forwarder example (present in the SimGrid
-sources). The red color indicates tasks from the compute category. This
-visualization was generated with the following configuration:
-
-\verbatim
-{
- node = (LINK, HOST, );
- edge = (HOST-LINK, LINK-HOST, LINK-LINK, );
-
- host = {
- type = square;
- size = power;
- values = (pcompute, pfinalize);
- };
- link = {
- type = rhombus;
- size = bandwidth;
- values = (bcompute, bfinalize);
- };
-}
-\endverbatim
-
-
-\htmlonly
-
-\endhtmlonly
-
-
-- Understading Triva - colors: Colors are now registered in
-trace files. See the tracing API to how to define them for your
-simulator.
+This configuration enables the analysis of resource utilization by MSG
+tasks through the identification of load-balancing issues and network
+bottlenecks, for instance.
*/