/*! \page tracing Tracing Simulations for Visualization \htmlinclude .tracing.doc.toc \section tracing_tracing Tracing Simulations for Visualization The trace visualization is widely used to observe and understand the behavior of parallel applications and distributed algorithms. Usually, this is done in a two-step fashion: the user instruments the application and the traces are analyzed after the end of the execution. The visualization itself can highlights unexpected behaviors, bottlenecks and sometimes can be used to correct distributed algorithms. The SimGrid team has instrumented the library in order to let users trace their simulations and analyze them. This part of the user manual explains how the tracing-related features can be enabled and used during the development of simulators using the SimGrid library. \subsection tracing_tracing_howitworks How it works For now, the SimGrid library is instrumented so users can trace the platform utilization using the MSG, SimDAG and SMPI interface. This means that the tracing will register how much power is used for each host and how much bandwidth is used for each link of the platform. The idea with this type of tracing is to observe the overall view of resources utilization in the first place, especially the identification of bottlenecks, load-balancing among hosts, and so on. The idea of the tracing facilities is to give SimGrid users to possibility to classify MSG and SimDAG tasks by category, tracing the platform utilization (hosts and links) for each of the categories. For that, the tracing interface enables the declaration of categories and a function to mark a task with a previously declared category. The tasks that are not classified according to a category are not traced. Even if the user does not specify any category, the simulations can still be traced in terms of resource utilization by using a special parameter that is detailed below. \subsection tracing_tracing_enabling Enabling using CMake With the sources of SimGrid, it is possible to enable the tracing using the parameter -Denable_tracing=ON when the cmake is executed. The sections \ref instr_category_functions, \ref instr_mark_functions, and \ref instr_uservariables_functions describe all the functions available when this Cmake options is activated. These functions will have no effect if SimGrid is configured without this option (they are wiped-out by the C-preprocessor). \verbatim $ cmake -Denable_tracing=ON . $ make \endverbatim \subsection instr_category_functions Tracing categories functions \li \c TRACE_category(const char *category) \li \c TRACE_category_with_color(const char *category, const char *color) \li \c MSG_task_set_category(m_task_t task, const char *category) \li \c MSG_task_get_category(m_task_t task) \li \c SD_task_set_category(SD_task_t task, const char *category) \li \c SD_task_get_category(SD_task_t task) \subsection instr_mark_functions Tracing marks functions \li \c TRACE_declare_mark(const char *mark_type) \li \c TRACE_mark(const char *mark_type, const char *mark_value) \subsection instr_uservariables_functions Tracing user variables functions For hosts: \li \c TRACE_host_variable_declare(const char *variable) \li \c TRACE_host_variable_declare_with_color(const char *variable, const char *color) \li \c TRACE_host_variable_set(const char *host, const char *variable, double value) \li \c TRACE_host_variable_add(const char *host, const char *variable, double value) \li \c TRACE_host_variable_sub(const char *host, const char *variable, double value) \li \c TRACE_host_variable_set_with_time(double time, const char *host, const char *variable, double value) \li \c TRACE_host_variable_add_with_time(double time, const char *host, const char *variable, double value) \li \c TRACE_host_variable_sub_with_time(double time, const char *host, const char *variable, double value) For links: \li \c TRACE_link_variable_declare(const char *variable) \li \c TRACE_link_variable_declare_with_color(const char *variable, const char *color) \li \c TRACE_link_variable_set(const char *link, const char *variable, double value) \li \c TRACE_link_variable_add(const char *link, const char *variable, double value) \li \c TRACE_link_variable_sub(const char *link, const char *variable, double value) \li \c TRACE_link_variable_set_with_time(double time, const char *link, const char *variable, double value) \li \c TRACE_link_variable_add_with_time(double time, const char *link, const char *variable, double value) \li \c TRACE_link_variable_sub_with_time(double time, const char *link, const char *variable, double value) For links, but use source and destination to get route: \li \c TRACE_link_srcdst_variable_set(const char *src, const char *dst, const char *variable, double value) \li \c TRACE_link_srcdst_variable_add(const char *src, const char *dst, const char *variable, double value) \li \c TRACE_link_srcdst_variable_sub(const char *src, const char *dst, const char *variable, double value) \li \c TRACE_link_srcdst_variable_set_with_time(double time, const char *src, const char *dst, const char *variable, double value) \li \c TRACE_link_srcdst_variable_add_with_time(double time, const char *src, const char *dst, const char *variable, double value) \li \c TRACE_link_srcdst_variable_sub_with_time(double time, const char *src, const char *dst, const char *variable, double value) \subsection tracing_tracing_options Tracing configuration Options To check which tracing options are available for your simulator, you can just run it with the option --help-tracing. These are the options accepted by the tracing system of SimGrid as of today, you can use them by running your simulator with the --cfg= switch: \li \c tracing : Safe switch. It activates (or deactivates) the tracing system. No other tracing options take effect if this one is not activated. \verbatim --cfg=tracing:1 \endverbatim \li \c tracing/categorized : It activates the categorized resource utilization tracing. It should be enabled if tracing categories are used by this simulator. \verbatim --cfg=tracing/categorized:1 \endverbatim \li \c tracing/uncategorized : It activates the uncategorized resource utilization tracing. Use it if this simulator do not use tracing categories and resource use have to be traced. \verbatim --cfg=tracing/uncategorized:1 \endverbatim \li \c tracing/filename : A file with this name will be created to register the simulation. The file is in the Paje format and can be analyzed using Triva or Paje visualization tools. More information can be found in these webpages: http://triva.gforge.inria.fr/ http://paje.sourceforge.net/ \verbatim --cfg=tracing/filename:mytracefile.trace \endverbatim If you do not provide this parameter, the trace file will be named simgrid.trace. \li \c tracing/onelink_only : By default, the tracing system uses all routes in the platform file to re-create a "graph" of the platform and register it in the trace file. This option let the user tell the tracing system to use only the routes that are composed with just one link. \verbatim --cfg=tracing/onelink_only:1 \endverbatim \li \c tracing/smpi : This option only has effect if this simulator is SMPI-based. Traces the MPI interface and generates a trace that can be analyzed using Gantt-like visualizations. Every MPI function (implemented by SMPI) is transformed in a state, and point-to-point communications can be analyzed with arrows. \verbatim --cfg=tracing/smpi:1 \endverbatim \li \c tracing/smpi/group : This option only has effect if this simulator is SMPI-based. The processes are grouped by the hosts where they were executed. \verbatim --cfg=tracing/smpi/group:1 \endverbatim \li \c tracing/msg/process : This option only has effect if this simulator is MSG-based. It traces the behavior of all categorized MSG processes, grouping them by hosts. This option can be used to track process location if this simulator has process migration. \verbatim --cfg=tracing/msg/process:1 \endverbatim \li \c tracing/buffer : This option put some events in a time-ordered buffer using the insertion sort algorithm. The process of acquiring and releasing locks to access this buffer and the cost of the sorting algorithm make this process slow. The simulator performance can be severely impacted if this option is activated, but you are sure to get a trace file with events sorted. \verbatim --cfg=tracing/buffer:1 \endverbatim \li \c tracing/onelink_only : This option changes the way SimGrid register its platform on the trace file. Normally, the tracing considers all routes (no matter their size) on the platform file to re-create the resource topology. If this option is activated, only the routes with one link are used to register the topology within an AS. Routes among AS continue to be traced as usual. \verbatim --cfg=tracing/onelink_only:1 \endverbatim \li \c tracing/disable_destroy : Disable the destruction of containers at the end of simulation. This can be used with simulators that have a different notion of time (different from the simulated time). \verbatim --cfg=tracing/disable_destroy:1 \endverbatim \li \c triva/categorized : This option generates a graph configuration file for Triva considering categorized resource utilization. \verbatim --cfg=triva/categorized:graph_categorized.plist \endverbatim \li \c triva/uncategorized : This option generates a graph configuration file for Triva considering uncategorized resource utilization. \verbatim --cfg=triva/uncategorized:graph_uncategorized.plist \endverbatim \subsection tracing_tracing_example_parameters Case studies Some scenarios that might help you decide which tracing options you should use to analyze your simulator. \li I want to trace the resource utilization of all hosts and links of the platform, and my simulator does not use the tracing API. For that, you can run a uncategorized trace with the following parameters (it will work with any Simgrid simulator): \verbatim ./your_simulator \ --cfg=tracing:1 \ --cfg=tracing/uncategorized:1 \ --cfg=tracing/filename:mytracefile.trace \ --cfg=triva/uncategorized:uncat.plist \endverbatim \li I want to trace only a subset of my MSG (or SimDAG) tasks. For that, you will need to create tracing categories using the TRACE_category (...) function (as explained above), and then classify your tasks to a previously declared category using the MSG_task_set_category (...) (or SD_task_set_category (...) for SimDAG tasks). After recompiling, run your simulator with the following parameters: \verbatim ./your_simulator \ --cfg=tracing:1 \ --cfg=tracing/categorized:1 \ --cfg=tracing/filename:mytracefile.trace \ --cfg=triva/categorized:cat.plist \endverbatim \subsection tracing_tracing_example Example of Instrumentation A simplified example using the tracing mandatory functions. \verbatim int main (int argc, char **argv) { MSG_global_init (&argc, &argv); //(... after deployment ...) //note that category declaration must be called after MSG_create_environment TRACE_category_with_color ("request", "1 0 0"); TRACE_category_with_color ("computation", "0.3 1 0.4"); TRACE_category ("finalize"); m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL); m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL); m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL); m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL); MSG_task_set_category (req1, "request"); MSG_task_set_category (req2, "request"); MSG_task_set_category (req3, "request"); MSG_task_set_category (req4, "request"); m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL); MSG_task_set_category (comp, "computation"); m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL); MSG_task_set_category (finalize, "finalize"); //(...) MSG_clean(); return 0; } \endverbatim \subsection tracing_tracing_analyzing Analyzing the SimGrid Traces The SimGrid library, during an instrumented simulation, creates a trace file in the Paje file format that contains the platform utilization for the simulation that was executed. The visualization analysis of this file is performed with the visualization tool Triva, with special configurations tunned to SimGrid needs. This part of the documentation explains how to configure and use Triva to analyse a SimGrid trace file. - Installing Triva: the tool is available in the Inria's Forge, at http://triva.gforge.inria.fr. Use the following command to get the sources, and then check the file INSTALL. This file contains instructions to install the tool's dependencies in a Ubuntu/Debian Linux. The tool can also be compiled in MacOSX natively, check INSTALL.mac file. \verbatim $ git clone git://scm.gforge.inria.fr/triva/triva.git $ cd triva $ cat INSTALL \endverbatim - Executing Triva: a binary called Triva is available after the installation (you can execute it passing --help to check its options). If the triva binary is not available after following the installation instructions, you may want to execute the following command to initialize the GNUstep environment variables. We strongly recommend that you use the latest GNUstep packages, and not the packages available through apt-get in Ubuntu/Debian packaging systems. If you install GNUstep using the latest available packages, you can execute this command: \verbatim $ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh \endverbatim You should be able to see this output after the installation of triva: \verbatim $ ./Triva.app/Triva --help Usage: Triva [OPTIONS...] TRACE0 [TRACE1] Trace Analysis through Visualization TimeInterval --ti_frequency {double} Animation: frequency of updates --ti_hide Hide the TimeInterval window --ti_forward {double} Animation: value to move time-slice --ti_apply Apply the configuration --ti_update Update on slider change --ti_animate Start animation --ti_start {double} Start of time slice --ti_size {double} Size of time slice Triva --comparison Compare Trace Files (Experimental) --graph Configurable Graph --list Print Trace Type Hierarchy --hierarchy Export Trace Type Hierarchy (dot) --stat Trace Statistics and Memory Utilization --instances List All Trace Entities --linkview Link View (Experimental) --treemap Squarified Treemap --merge Merge Trace Files (Experimental) --check Check Trace File Integrity GraphConfiguration --gc_conf {file} Graph Configuration in Property List Format --gc_apply Apply the configuration --gc_hide Hide the GraphConfiguration window \endverbatim Triva expects that the user choose one of the available options (currently --graph or --treemap for a visualization analysis) and the trace file from the simulation. - Understanding Triva - time-slice: the analysis of a trace file using the tool always takes into account the concept of the time-slice. This concept means that what is being visualized in the screen is always calculated considering a specific time frame, with its beggining and end timestamp. The time-slice is configured by the user and can be changed dynamically through the window called Time Interval that is opened whenever a trace file is being analyzed. The next figure depicts the time-slice configuration window. In the top of the window, in the space named Trace Time, the two fields show the beggining of the trace (which usually starts in 0) and the end (that depends on the time simulated by SimGrid). The middle of the window, in the square named Time Slice Configuration, contains the aspects related to the time-slice, including its start and its size. The gray rectangle in the bottom of this part indicates the current time-slice that is considered for the drawings. If the checkbox Update Drawings on Sliders Change is not selected, the button Apply must be clicked in order to inform triva that the new time-slice must be considered. The bottom part of the window, in the space indicated by the square Time Slice Animation can be used to advance the time-frame automatically. The user configures the amount of time that the time-frame will forward and how frequent this update will happen. Once this is configured, the user clicks the Play button in order to see the dynamic changes on the drawings.