X-Git-Url: http://info.iut-bm.univ-fcomte.fr/pub/gitweb/simgrid.git/blobdiff_plain/dfe09e5c06ea550bdd21bc6ec3ea1a26d58569ec..d8eb62b207b566949a0d9ce649a7b21e226b9168:/doc/doxygen/outcomes_vizu.doc diff --git a/doc/doxygen/outcomes_vizu.doc b/doc/doxygen/outcomes_vizu.doc index d8a7ad3267..06fdfd09ee 100644 --- a/doc/doxygen/outcomes_vizu.doc +++ b/doc/doxygen/outcomes_vizu.doc @@ -1,4 +1,4 @@ -/*! \page outcomes_vizu Visualization and Statistical Analysis +/*! @page outcomes_vizu Visualization and Statistical Analysis SimGrid comes with an extensive support to trace and register what happens during the simulation, so that it can be either visualized or @@ -15,7 +15,7 @@ and analyze them. This part of the user manual explains how the tracing-related features can be enabled and used during the development of simulators using the SimGrid library. -\section instr_category_functions Tracing categories functions +@section instr_category_functions Tracing categories functions The SimGrid library is instrumented so users can trace the platform utilization using MSG, SimDAG and SMPI interfaces. It registers how @@ -33,193 +33,192 @@ below let the user declare a category and apply it to tasks. The tasks that are not classified according to a category are not traced. Even if the user does not specify any category, the simulations can still be traced in terms of resource utilization by -using a special parameter that is detailed below (see section \ref +using a special parameter that is detailed below (see section @ref tracing_tracing_options). -\li \c TRACE_category(const char *category) -\li \c TRACE_category_with_color(const char *category, const char *color) -\li \c MSG_task_set_category(msg_task_t task, const char *category) -\li \c MSG_task_get_category(msg_task_t task) -\li \c SD_task_set_category(SD_task_t task, const char *category) -\li \c SD_task_get_category(SD_task_t task) +@li @c TRACE_category(const char *category) +@li @c TRACE_category_with_color(const char *category, const char *color) +@li @c MSG_task_set_category(msg_task_t task, const char *category) +@li @c MSG_task_get_category(msg_task_t task) +@li @c SD_task_set_category(SD_task_t task, const char *category) +@li @c SD_task_get_category(SD_task_t task) -\section instr_mark_functions Tracing marks functions -\li \c TRACE_declare_mark(const char *mark_type) -\li \c TRACE_mark(const char *mark_type, const char *mark_value) +@section instr_mark_functions Tracing marks functions +@li @c TRACE_declare_mark(const char *mark_type) +@li @c TRACE_mark(const char *mark_type, const char *mark_value) -\section instr_uservariables_functions Tracing user variables functions +@section instr_uservariables_functions Tracing user variables functions For hosts: -\li \c TRACE_host_variable_declare(const char *variable) -\li \c TRACE_host_variable_declare_with_color(const char *variable, const char *color) -\li \c TRACE_host_variable_set(const char *host, const char *variable, double value) -\li \c TRACE_host_variable_add(const char *host, const char *variable, double value) -\li \c TRACE_host_variable_sub(const char *host, const char *variable, double value) -\li \c TRACE_host_variable_set_with_time(double time, const char *host, const char *variable, double value) -\li \c TRACE_host_variable_add_with_time(double time, const char *host, const char *variable, double value) -\li \c TRACE_host_variable_sub_with_time(double time, const char *host, const char *variable, double value) +@li @c TRACE_host_variable_declare(const char *variable) +@li @c TRACE_host_variable_declare_with_color(const char *variable, const char *color) +@li @c TRACE_host_variable_set(const char *host, const char *variable, double value) +@li @c TRACE_host_variable_add(const char *host, const char *variable, double value) +@li @c TRACE_host_variable_sub(const char *host, const char *variable, double value) +@li @c TRACE_host_variable_set_with_time(double time, const char *host, const char *variable, double value) +@li @c TRACE_host_variable_add_with_time(double time, const char *host, const char *variable, double value) +@li @c TRACE_host_variable_sub_with_time(double time, const char *host, const char *variable, double value) For links: -\li \c TRACE_link_variable_declare(const char *variable) -\li \c TRACE_link_variable_declare_with_color(const char *variable, const char *color) -\li \c TRACE_link_variable_set(const char *link, const char *variable, double value) -\li \c TRACE_link_variable_add(const char *link, const char *variable, double value) -\li \c TRACE_link_variable_sub(const char *link, const char *variable, double value) -\li \c TRACE_link_variable_set_with_time(double time, const char *link, const char *variable, double value) -\li \c TRACE_link_variable_add_with_time(double time, const char *link, const char *variable, double value) -\li \c TRACE_link_variable_sub_with_time(double time, const char *link, const char *variable, double value) +@li @c TRACE_link_variable_declare(const char *variable) +@li @c TRACE_link_variable_declare_with_color(const char *variable, const char *color) +@li @c TRACE_link_variable_set(const char *link, const char *variable, double value) +@li @c TRACE_link_variable_add(const char *link, const char *variable, double value) +@li @c TRACE_link_variable_sub(const char *link, const char *variable, double value) +@li @c TRACE_link_variable_set_with_time(double time, const char *link, const char *variable, double value) +@li @c TRACE_link_variable_add_with_time(double time, const char *link, const char *variable, double value) +@li @c TRACE_link_variable_sub_with_time(double time, const char *link, const char *variable, double value) For links, but use source and destination to get route: -\li \c TRACE_link_srcdst_variable_set(const char *src, const char *dst, const char *variable, double value) -\li \c TRACE_link_srcdst_variable_add(const char *src, const char *dst, const char *variable, double value) -\li \c TRACE_link_srcdst_variable_sub(const char *src, const char *dst, const char *variable, double value) -\li \c TRACE_link_srcdst_variable_set_with_time(double time, const char *src, const char *dst, const char *variable, double value) -\li \c TRACE_link_srcdst_variable_add_with_time(double time, const char *src, const char *dst, const char *variable, double value) -\li \c TRACE_link_srcdst_variable_sub_with_time(double time, const char *src, const char *dst, const char *variable, double value) +@li @c TRACE_link_srcdst_variable_set(const char *src, const char *dst, const char *variable, double value) +@li @c TRACE_link_srcdst_variable_add(const char *src, const char *dst, const char *variable, double value) +@li @c TRACE_link_srcdst_variable_sub(const char *src, const char *dst, const char *variable, double value) +@li @c TRACE_link_srcdst_variable_set_with_time(double time, const char *src, const char *dst, const char *variable, double value) +@li @c TRACE_link_srcdst_variable_add_with_time(double time, const char *src, const char *dst, const char *variable, double value) +@li @c TRACE_link_srcdst_variable_sub_with_time(double time, const char *src, const char *dst, const char *variable, double value) -\section tracing_tracing_options Tracing configuration Options +@section tracing_tracing_options Tracing configuration Options To check which tracing options are available for your simulator, you -can just run it with the option \verbatim --help-tracing \endverbatim +can just run it with the option @verbatim --help-tracing @endverbatim to get a very detailed and updated explanation of each tracing parameter. These are some of the options accepted by the tracing system of SimGrid, you can use them by running your simulator with the --cfg= switch: -\li \c +@li @c tracing : Safe switch. It activates (or deactivates) the tracing system. No other tracing options take effect if this one is not activated. -\verbatim +@verbatim --cfg=tracing:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/categorized : It activates the categorized resource utilization tracing. It should be enabled if tracing categories are used by this simulator. -\verbatim +@verbatim --cfg=tracing/categorized:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/uncategorized : It activates the uncategorized resource utilization tracing. Use it if this simulator do not use tracing categories and resource use have to be traced. -\verbatim +@verbatim --cfg=tracing/uncategorized:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/filename : A file with this name will be created to register the simulation. The file - is in the Paje format and can be analyzed using Viva or Paje visualization + is in the Paje format and can be analyzed using Paje visualization tools. More information can be found in these webpages: - http://github.com/schnorr/viva/ http://github.com/schnorr/pajeng/ -\verbatim +@verbatim --cfg=tracing/filename:mytracefile.trace -\endverbatim +@endverbatim If you do not provide this parameter, the trace file will be named simgrid.trace. -\li \c +@li @c tracing/smpi : This option only has effect if this simulator is SMPI-based. Traces the MPI interface and generates a trace that can be analyzed using Gantt-like visualizations. Every MPI function (implemented by SMPI) is transformed in a state, and point-to-point communications can be analyzed with arrows. -\verbatim +@verbatim --cfg=tracing/smpi:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/smpi/group : This option only has effect if this simulator is SMPI-based. The processes are grouped by the hosts where they were executed. -\verbatim +@verbatim --cfg=tracing/smpi/group:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/smpi/computing : This option only has effect if this simulator is SMPI-based. The parts external to SMPI are also outputted to the trace. Provides better way to analyze the data automatically. -\verbatim +@verbatim --cfg=tracing/smpi/computing:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/smpi/internals : This option only has effect if this simulator is SMPI-based. Display internal communications happening during a collective MPI call. -\verbatim +@verbatim --cfg=tracing/smpi/internals:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/smpi/display-sizes : This option only has effect if this simulator is SMPI-based. Display the sizes of the messages exchanged in the trace, both in the links and on the states. For collective, size means the global size of data sent by the process in general. -\verbatim +@verbatim --cfg=tracing/smpi/display-sizes:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/smpi/sleeping : TODO -\verbatim +@verbatim TODO -\endverbatim +@endverbatim -\li \c +@li @c tracing/smpi/format : TODO -\verbatim +@verbatim TODO -\endverbatim +@endverbatim -\li \c +@li @c tracing/smpi/format/ti-one-file : TODO -\verbatim +@verbatim TODO -\endverbatim +@endverbatim -\li \c -tracing/msg/vm +@li @c +tracing/vm : TODO -\verbatim +@verbatim TODO -\endverbatim +@endverbatim -\li \c +@li @c tracing/msg/process : This option only has effect if this simulator is MSG-based. It traces the behavior of all categorized MSG processes, grouping them by hosts. This option can be used to track process location if this simulator has process migration. -\verbatim +@verbatim --cfg=tracing/msg/process:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/buffer : This option put some events in a time-ordered buffer using the @@ -228,11 +227,11 @@ tracing/buffer make this process slow. The simulator performance can be severely impacted if this option is activated, but you are sure to get a trace file with events sorted. -\verbatim +@verbatim --cfg=tracing/buffer:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/onelink-only : This option changes the way SimGrid register its platform on the trace @@ -241,151 +240,131 @@ size) on the platform file to re-create the resource topology. If this option is activated, only the routes with one link are used to register the topology within a netzone. Routes among netzones continue to be traced as usual. -\verbatim +@verbatim --cfg=tracing/onelink-only:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/disable-link : TODO -\verbatim +@verbatim TODO -\endverbatim +@endverbatim -\li \c +@li @c tracing/disable-power : TODO -\verbatim +@verbatim TODO -\endverbatim +@endverbatim -\li \c +@li @c tracing/disable-destroy : Disable the destruction of containers at the end of simulation. This can be used with simulators that have a different notion of time (different from the simulated time). -\verbatim +@verbatim --cfg=tracing/disable-destroy:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/basic : Some visualization tools are not able to parse correctly the Paje file format. Use this option if you are using one of these tools to visualize the simulation trace. Keep in mind that the trace might be incomplete, without all the information that would be registered otherwise. -\verbatim +@verbatim --cfg=tracing/basic:yes -\endverbatim +@endverbatim -\li \c +@li @c tracing/comment : Use this to add a comment line to the top of the trace file. -\verbatim +@verbatim --cfg=tracing/comment:my_string -\endverbatim +@endverbatim -\li \c +@li @c tracing/comment-file : Use this to add the contents of a file to the top of the trace file as comment. -\verbatim +@verbatim --cfg=tracing/comment-file:textual_file.txt -\endverbatim +@endverbatim -\li \c +@li @c tracing/precision : This option determines the precision of timings stored in the trace file. Make sure -you set \ref options_model_precision to at least the same value as this option! (Traces +you set @ref options_model_precision to at least the same value as this option! (Traces cannot be more accurate than the simulation; they can be less accurate, though.) The following example will give you a precision of E-10 in the trace file: -\verbatim +@verbatim --cfg=tracing/precision:10 -\endverbatim +@endverbatim -\li \c +@li @c tracing/platform : TODO -\verbatim +@verbatim TODO -\endverbatim +@endverbatim -\li \c +@li @c tracing/platform/topology : TODO -\verbatim +@verbatim TODO -\endverbatim +@endverbatim -\li \c -viva/categorized -: - This option generates a graph configuration file for Viva considering - categorized resource utilization. -\verbatim ---cfg=viva/categorized:graph_categorized.plist -\endverbatim - -\li \c -viva/uncategorized -: - This option generates a graph configuration file for Viva considering - uncategorized resource utilization. -\verbatim ---cfg=viva/uncategorized:graph_uncategorized.plist -\endverbatim - -Please pass \verbatim --help-tracing \endverbatim to your simulator +Please pass @verbatim --help-tracing @endverbatim to your simulator for the updated list of tracing options. -\section tracing_tracing_example_parameters Case studies +@section tracing_tracing_example_parameters Case studies Some scenarios that might help you decide which tracing options you should use to analyze your simulator. -\li I want to trace the resource utilization of all hosts +@li I want to trace the resource utilization of all hosts and links of the platform, and my simulator does not use the tracing API. For that, you can run a uncategorized trace with the following parameters (it will work with any Simgrid simulator): -\verbatim -./your_simulator \ - --cfg=tracing:yes \ - --cfg=tracing/uncategorized:yes \ - --cfg=tracing/filename:mytracefile.trace \ - --cfg=viva/uncategorized:uncat.plist -\endverbatim - -\li I want to trace only a subset of my MSG (or SimDAG) tasks. +@verbatim +./your_simulator @ + --cfg=tracing:yes @ + --cfg=tracing/uncategorized:yes @ + --cfg=tracing/filename:mytracefile.trace @ +@endverbatim + +@li I want to trace only a subset of my MSG (or SimDAG) tasks. For that, you will need to create tracing categories using the TRACE_category (...) function (as explained above), and then classify your tasks to a previously declared category using the MSG_task_set_category (...) (or SD_task_set_category (...) for SimDAG tasks). After recompiling, run your simulator with the following parameters: -\verbatim -./your_simulator \ - --cfg=tracing:yes \ - --cfg=tracing/categorized:yes \ - --cfg=tracing/filename:mytracefile.trace \ - --cfg=viva/categorized:cat.plist -\endverbatim +@verbatim +./your_simulator @ + --cfg=tracing:yes @ + --cfg=tracing/categorized:yes @ + --cfg=tracing/filename:mytracefile.trace @ +@endverbatim -\section tracing_tracing_example Example of Instrumentation +@section tracing_tracing_example Example of Instrumentation A simplified example using the tracing mandatory functions. -\verbatim +@verbatim int main (int argc, char **argv) { MSG_init (&argc, &argv); @@ -417,9 +396,9 @@ int main (int argc, char **argv) MSG_clean(); return 0; } -\endverbatim +@endverbatim -\section tracing_tracing_analyzing Analyzing SimGrid Simulation Traces +@section tracing_tracing_analyzing Analyzing SimGrid Simulation Traces A SimGrid-based simulator, when executed with the correct parameters (see above) creates a trace file in the Paje file format holding the @@ -453,23 +432,6 @@ several options to analyze this trace file: file format. Consider this option if you need to understand the causality of your distributed simulation. -- Visualize the behavior of your simulation with treemaps (specially - if your simulation has a platform with several thousand resources), - provided by the Viva - visualization tool. See Viva's wiki for - further details on what is a treemap and how to use it. - -- Correlate the behavior of your simulator with the platform topology - with an interactive, force-directed, and hierarchical graph - visualization, provided by Viva. Check Viva's wiki for - further details. This research report, - published at ISPASS 2013, has a detailed description of this - visualization technique. - - You can also check our online tutorial section that contains a dedicated tutorial with several @@ -484,123 +446,4 @@ several options to analyze this trace file: href="http://lists.gforge.inria.fr/pipermail/simgrid-user/">mailing list archive for old messages regarding tracing and analysis. -\subsection tracing_viva_analysis Viva Visualization Tool - -This subsection describe some of the concepts regarding the Viva Visualization Tool and -its relation with SimGrid traces. You should refer to Viva's website -for further details on all its visualization techniques. - -\subsubsection tracing_viva_time_slice Time Slice - -The analysis of a trace file using the tool always takes into account -the concept of the time-slice. This concept means that what -is being visualized in the screen is always calculated considering a -specific time frame, with its beggining and end timestamp. The -time-slice is configured by the user and can be changed dynamically -through the window called Time Interval that is opened -whenever a trace file is being analyzed. Users are capable to select -the beggining and size of the time slice. - -\subsubsection tracing_viva_graph Hierarchical Graph View - -As stated above (see section \ref tracing_tracing_analyzing), one -possibility to analyze SimGrid traces is to use Viva's graph view with -a graph configuration to customize the graph according to the -traces. A valid graph configuration (we are using the non-XML Property List -Format to describe the configuration) can be created for any -SimGrid-based simulator using the ---cfg=viva/uncategorized:graph_uncategorized.plist or ---cfg=viva/categorized:graph_categorized.plist (if the -simulator defines resource utilization categories) when executing the -simulation. - -\subsubsection basic_conf Basic Graph Configuration - -The basic description of the configuration is as follows: -\verbatim -{ - node = (LINK, HOST, ); - edge = (HOST-LINK, LINK-HOST, LINK-LINK, ); -\endverbatim - -The nodes of the graph will be created based on the node -parameter, which in this case is the different "HOST"s and -"LINK"s of the platform used to simulate. The edge -parameter indicates that the edges of the graph will be created based -on the "HOST-LINK"s, "LINK-HOST"s, and -"LINK-LINK"s of the platform. After the definition of these -two parameters, the configuration must detail how the nodes -(HOSTs and LINKs) should be drawn. - -For that, the configuration must have an entry for each of -the types used. For HOST, as basic configuration, we have: - -\verbatim - HOST = { - type = square; - size = power; - values = (power_used); - }; -\endverbatim - -The parameter size indicates which variable from the trace -file will be used to define the size of the node HOST in the -visualization. If the simulation was executed with availability -traces, the size of the nodes will be changed according to these -traces. The parameter type indicates which geometrical shape -will be used to represent HOST, and the values parameter -indicates which values from the trace will be used to fill the shape. - -For LINK we have: - -\verbatim - LINK = { - type = rhombus; - size = bandwidth; - values = (bandwidth_used); - }; -} -\endverbatim - -The same configuration parameters are used here: type (with a -rhombus), the size (whose value is from trace's bandwidth -variable) and the values. - -\subsubsection custom_graph Customizing the Graph Representation - -Viva is capable to handle a customized graph representation based on -the variables present in the trace file. In the case of SimGrid, every -time a category is created for tasks, two variables in the trace file -are defined: one to indicate node utilization (how much power was used -by that task category), and another to indicate link utilization (how -much bandwidth was used by that category). For instance, if the user -declares a category named request, there will be variables -named prequest and a brequest (p -for power and b for bandwidth). It is important to notice that -the variable prequest in this case is only available for HOST, -and brequest is only available for LINK. Example: -suppose there are two categories for tasks: request and compute. To -create a customized graph representation with a proportional -separation of host and link utilization, use as configuration for HOST -and LINK this: - -\verbatim - HOST = { - type = square; - size = power; - values = (prequest, pcomputation); - }; - LINK = { - type = rhombus; - size = bandwidth; - values = (brequest, bcomputation); - }; -\endverbatim - -This configuration enables the analysis of resource utilization by MSG -tasks through the identification of load-balancing issues and network -bottlenecks, for instance. - */