[simgrid.git] / doc / module-msg.doc

/** \addtogroup MSG_API

      MSG was the first distributed programming environment provided within
      SimGrid. While almost realistic, it remains quite simple (simplistic?).
      This describes the native to MSG.

      \section jMSG_who Who should use this (and who shouldn't)
      
      You should use MSG if you want to study some heuristics for a
      given problem you don't really want to implement. If you want to
      use the C programming language, your are in the right
      section. To use the Java or Ruby programming interfaces, please refer to
      the documentation provided in the relevant packages.

  \section MSG_funct Offered functionnalities
   - \ref msg_simulation
   - \ref m_process_management
   - \ref m_host_management
   - \ref m_task_management
   - \ref msg_file_management
   - \ref msg_task_usage
   - \ref msg_VMs
   - \ref msg_trace_driven
   - \ref msg_deprecated_functions


  Also make sure to visit the page @ref MSG_examples.
*/

/**
@defgroup MSG_examples MSG Examples
@ingroup MSG_API
 
@htmlonly <!-- DOXYGEN_NAVBAR_LABEL="Examples" --> @endhtmlonly

MSG comes with an extensive set of examples. It is sometimes difficult
to find the one you need. This list aims at helping you finding the
example from which you can learn what you want to.

@section MSG_ex_basics Basic examples and features

*/

/** 
@defgroup msg_simulation   Main MSG simulation Functions
@ingroup MSG_API
@brief Describes how to setup and control your simulation.

The basic workflow is the following (check the \ref MSG_examples for
details).

 -# Initialize the library with #MSG_global_init
 -# Create a platform (usually by parsing a file with
    #MSG_create_environment)
 -# Register the functions that your processes are supposed to run with
    #MSG_function_register (and maybe #MSG_function_register_default)
 -# Launch your processes from a deployment file with #MSG_launch_application
 -# Run the simulation with #MSG_main
 -# Cleanup the library with #MSG_clean before ending your program
    (optional).

@htmlonly <!-- DOXYGEN_NAVBAR_LABEL="Simulation Control" --> @endhtmlonly
*/

/** @defgroup m_process_management Process Management Functions 
 *  @ingroup MSG_API
 *  @brief This section describes the process structure of MSG
 *         (#m_process_t) and the functions for managing it.
 */
   
/** @defgroup m_host_management Host Management Functions
 *  @ingroup MSG_API
 *  @brief This section describes the host structure of MSG
 */
  
/** @defgroup m_task_management Task Management Functions
 *  @ingroup MSG_API
 *  @brief This section describes the task structure of MSG
 *         (#m_task_t) and the functions for managing it. See
 *         \ref msg_task_usage to see how to put the tasks in action.
 *
 * \htmlonly <!-- DOXYGEN_NAVBAR_LABEL="Tasks" --> \endhtmlonly
 */
 
/** @defgroup msg_task_usage Task Actions
 *  @ingroup MSG_API
 *  @brief This section describes the functions that can be used
 *         by a process to execute, communicate or otherwise handle some task.
 */

/** @defgroup msg_VMs VMs
 *  @ingroup MSG_API
 *  @brief This section describes the interface created to mimick IaaS clouds.
 *
 *  With it, you can create virtual machines to put your processes
 *  into, and interact directly with the VMs to manage groups of
 *  processes. 
 *
 *  This interface is highly experimental at this point. Testing is
 *  welcomed, but do not expect too much of it right now. Even the
 *  interfaces may be changed in future releases of SimGrid (although
 *  things are expected to stabilize nicely before SimGrid v3.8).
 *  There is no guaranty on the rest of SimGrid, and there is less
 *  than that on this part.
 *
 */

/** @defgroup msg_file_management File Management Functions
 *  @ingroup MSG_API
 *  @brief This section describes the file structure of MSG
 *         (#msg_file_t) and the functions for managing it. It
 *   is based on POSIX functions.
 */ 


/** 
@defgroup msg_trace_driven Trace-driven simulations
@ingroup MSG_API
@brief This section describes the functions allowing to build trace-driven simulations.

\htmlonly <!-- DOXYGEN_NAVBAR_LABEL="Trace-Driven" --> \endhtmlonly

This is very handy when you want to test an algorithm or protocol that
does nothing unless it receives some events from outside. For example,
a P2P protocol reacts to requests from the user, but does nothing if
there is no such event. 
      
In such situations, SimGrid allows to write your protocol in your C
file, and the events to react to in a separate text file. Declare a
function handling each of the events that you want to accept in your
trace files, register them using #MSG_action_register in your main,
and then use #MSG_action_trace_run to launch the simulation. You can
either have one trace file containing all your events, or a file per
simulated process. 
         
Check the examples in <b>examples/msg/actions/actions.c</b> for details.
 
 */ 

 
 
/**
@defgroup MSG_LUA      Lua bindings
@ingroup MSG_API
@brief Lua bindings to MSG (\ref MSG_API)

@htmlonly <!--  DOXYGEN_NAVBAR_LABEL="LUA bindings" --> @endhtmlonly 
          
This is the lua bindings of the \ref MSG_API interface.

\section lMSG_who Who should use this (and who shouldn't)
      
If you want to use MSG to study your algorithm, but you don't want to
use the C language (using \ref MSG_API), then you should use some
bindings such as this one. The advantage of the lua bindings is that
they are distributed directly with the main archive (in contrary to
Java and Ruby bindings, for example, that are distributed separately).
Another advantage of lua is that there is almost no performance loss
with regard to the C version (at least there shouln't be any -- it is
still to be precisely assessed).

\section MSG_Lua_funct  Lua offered functionnalities in MSG

Almost all important features of the MSG interface are available from
the lua bindings. Unfortunately, since doxygen does not support the
lua modules implemented directly in C as we are using, there is no
ready to use reference documentation for this module. Even more than
for the other modules, you will have to dig into the source code of
the examples to learn how to use it. 

\section Lua_examples Examples of lua MSG
 
  - \ref MSG_ex_master_slave_lua
  - \ref MSG_ex_master_slave_lua_bypass
  - Also, the lua version of the Chord example (in the source tree)
    is a working non-trivial example of use of the lua bindings
*/

/**
@defgroup msg_deprecated_functions MSG Deprecated
@ingroup MSG_API
@brief This section describes the deprecated functions. PLEASE STOP USING THEM.

We don't remove them because the ability to run old scientific
code is something important to us. But these functionalities are
not actively supported anymore. 

To access these functions, you should define the relevant option
at configuration time in ccmake.
 */


/** 
@defgroup MSG_ex_asynchronous_communications Asynchronous communications
@ingroup MSG_examples

Simulation of asynchronous communications between a sender and a receiver using a realistic platform and
an external description of the deployment.

\section MSG_ex_ms_TOC Table of contents:
 - \ref MSG_ext_icomms_code
   - \ref MSG_ext_icomms_preliminary
   - \ref MSG_ext_icomms_Sender
   - \ref MSG_ext_icomms_Receiver
   - \ref MSG_ext_icomms_core
   - \ref MSG_ext_icomms_Main
 - \ref MSG_ext_icomms_fct_Waitall
 - \ref MSG_ext_icomms_fct_Waitany

<hr> 

\don'tinclude msg/icomms/peer.c

\section MSG_ext_icomms_code Code of the application

\subsection MSG_ext_icomms_preliminary Preliminary declarations
\skip include
\until Sender function

\subsection MSG_ext_icomms_Sender Sender function

The sender send to a receiver an asynchronous message with the function "MSG_task_isend()". Cause this function is non-blocking 
we have to make "MSG_comm_test()" to know   if the communication is finished for finally destroy it with function "MSG_comm_destroy()".
It also available to "make MSG_comm_wait()" which make both of them.  

  C style arguments (argc/argv) are interpreted as:
   - the number of tasks to distribute
   - the computation size of each task
   - the size of the files associated to each task
   - a list of host that will accept those tasks.
   - the time to sleep at the beginning of the function
   - This time defined the process sleep time
                        if time = 0 use of MSG_comm_wait()
                        if time > 0 use of MSG_comm_test()


\until Receiver function

\subsection MSG_ext_icomms_Receiver Receiver function

This function executes tasks when it receives them. As the receiving is asynchronous we have to test the communication to know
if it is completed or not with "MSG_comm_test()" or wait for the completion "MSG_comm_wait()".

  C style arguments (argc/argv) are interpreted as:
   - the id to use for received the communication.
   - the time to sleep at the beginning of the function
   - This time defined the process sleep time
                        if time = 0 use of MSG_comm_wait()
                        if time > 0 use of MSG_comm_test()

\until Test function

\subsection MSG_ext_icomms_core Simulation core

  This function is the core of the simulation and is divided only into 3 parts
  thanks to MSG_create_environment() and MSG_launch_application().
     -# Simulation settings : MSG_create_environment() creates a realistic 
        environment
     -# Application deployment : create the processes on the right locations with  
        MSG_launch_application()
     -# The simulation is run with #MSG_main()
         
  Its arguments are:
        - <i>platform_file</i>: the name of a file containing an valid surfxml platform description.
        - <i>application_file</i>: the name of a file containing a valid surfxml application description

\until Main function

\subsection MSG_ext_icomms_Main Main function

This initializes MSG, runs a simulation, and free all data-structures created by MSG.

\until end_of_main

\don'tinclude msg/icomms/peer2.c

\section MSG_ext_icomms_fct_Waitall Waitall function for sender

The use of this function permit to send all messages and wait for the completion of all in one time.

\skipline Sender function
\until end_of_sender

\section MSG_ext_icomms_fct_Waitany Waitany function

The MSG_comm_waitany() function return the place of the first message send or receive from a xbt_dynar_t table.

\subsection MSG_ext_icomms_fct_Waitany_sender From a sender
We can use this function to wait all sent messages.
\don'tinclude msg/icomms/peer3.c
\skipline Sender function
\until end_of_sender

\subsection MSG_ext_icomms_fct_Waitany_receiver From a receiver
We can also wait for the arrival of all messages.
\don'tinclude msg/icomms/peer3.c
\skipline Receiver function
\until end_of_receiver

*/

/** 
@defgroup MSG_ex_master_slave Basic Master/Slaves
@ingroup MSG_examples
    
Simulation of a master-slave application using a realistic platform
and an external description of the deployment. 

\section MSG_ex_ms_TOC Table of contents:
    
 - \ref MSG_ext_ms_code
   - \ref MSG_ext_ms_preliminary
   - \ref MSG_ext_ms_master
   - \ref MSG_ext_ms_slave
   - \ref MSG_ext_ms_forwarder
   - \ref MSG_ext_ms_core
   - \ref MSG_ext_ms_main
 - \ref MSG_ext_ms_helping
   - \ref MSG_ext_ms_application 
   - \ref MSG_ext_ms_platform
 
<hr> 

\don'tinclude msg/masterslave/masterslave_forwarder.c

\section MSG_ext_ms_code Code of the application

\subsection MSG_ext_ms_preliminary Preliminary declarations

\skip include
\until printf
\until }

\subsection MSG_ext_ms_master Master code
    
This function has to be assigned to a m_process_t that will behave as
the master. It should not be called directly but either given as a
parameter to #MSG_process_create() or registered as a public function
through #MSG_function_register() and then automatically assigned to a
process through #MSG_launch_application().
 
C style arguments (argc/argv) are interpreted as:
   - the number of tasks to distribute
   - the computation size of each task
   - the size of the files associated to each task
   - a list of host that will accept those tasks.

Tasks are dumbly sent in a round-robin style.
      
\until end_of_master
    
\subsection MSG_ext_ms_slave Slave code
    
This function has to be assigned to a #m_process_t that has to behave
as a slave. Just like the master fuction (described in \ref
MSG_ext_ms_master), it should not be called directly.

This function keeps waiting for tasks and executes them as it receives them.
      
\until end_of_slave

\subsection MSG_ext_ms_forwarder Forwarder code
   
This function has to be assigned to a #m_process_t that has to behave
as a forwarder. Just like the master function (described in \ref
MSG_ext_ms_master), it should not be called directly.

C style arguments (argc/argv) are interpreted as a list of host that
will accept those tasks.

This function keeps waiting for tasks and dispathes them to its slaves.

\until end_of_forwarder

\subsection MSG_ext_ms_core Simulation core

This function is the core of the simulation and is divided only into 3 parts
thanks to MSG_create_environment() and MSG_launch_application().
   -# Simulation settings : MSG_create_environment() creates a realistic 
      environment
   -# Application deployment : create the processes on the right locations with  
      MSG_launch_application()
   -# The simulation is run with #MSG_main()
         
Its arguments are:
        - <i>platform_file</i>: the name of a file containing an valid surfxml platform description.
        - <i>application_file</i>: the name of a file containing a valid surfxml application description
        
\until end_of_test_all
      
\subsection MSG_ext_ms_main Main() function
   
This initializes MSG, runs a simulation, and free all data-structures created by MSG.
      
\until end_of_main

\section MSG_ext_ms_helping Helping files

\subsection MSG_ext_ms_application Example of application file

\include msg/masterslave/deployment_masterslave.xml

\subsection MSG_ext_ms_platform Example of platform file

\include msg/small_platform.xml
   
*/

/** \page MSG_ex_master_slave_lua Master/slave Lua application
    
    Simulation of a master-slave application using lua bindings
       - \ref MSG_ext_ms_code_lua
       - \ref MSG_ext_ms_master_lua
       - \ref MSG_ext_ms_slave_lua
       - \ref MSG_ext_ms_core_lua

     - \ref MSG_ext_ms_helping
       - \ref MSG_ext_ms_application 
       - \ref MSG_ext_ms_platform
       
       
      \don'tinclude lua/masterslave/master_slave.lua
      
      \section MSG_ext_ms_code_lua Code of the application
      
      \subsection MSG_ext_ms_master_lua Master code
      
            as described ine the C native master/Slave exmaple , this function has to be assigned to a m_process_t that will behave as the master.
 
      Lua style arguments (...) in for the master are interpreted as:
       - the number of tasks to distribute
       - the computation size of each task
       - the size of the files associated to each task
       - a list of host that will accept those tasks.

      Tasks are dumbly sent in a round-robin style.
      
      \until end_of_master
      
      
      \subsection MSG_ext_ms_slave_lua Slave code
    
      This function has to be assigned to a #m_process_t that has to behave as a slave.
      This function keeps waiting for tasks and executes them as it receives them.
      
      \until end_of_slave
         \subsection MSG_ext_ms_core_lua Simulation core

      in this section the core of the simulation which start by including the simgrid lib for bindings
      : <i>require "simgrid" </i>
      
         -# Simulation settings : <i>simgrid.platform</i> creates a realistic 
            environment 
         -# Application deployment : create the processes on the right locations with  
            <i>simgrid.application</i>
         -# The simulation is run with <i>simgrid.run</i>
         
      Its arguments are:
        - <i>platform_file</i>: the name of a file containing an valid surfxml platform description.( first command line argument)
        - <i>application_file</i>: the name of a file containing a valid surfxml application description ( second commande line argument )
        
      \until simgrid.clean()
      
*/

/** \page MSG_ex_master_slave_lua_bypass Master/slave Bypass Lua application
    
    Simulation of a master-slave application using lua bindings, Bypassing the XML parser
       - \ref MSG_ext_ms_code_lua
       - \ref MSG_ext_ms_master_lua
       - \ref MSG_ext_ms_slave_lua
       - \ref MSG_ext_ms_core_lua
       
       
      \don'tinclude lua/console/master_slave_bypass.lua
      
      \section MSG_ext_ms_code_lua Code of the application
      
      \subsection MSG_ext_ms_master_lua Master code
      
            as described ine the C native master/Slave exmaple , this function has to be assigned to a m_process_t that will behave as the master.
 
      Lua style arguments (...) in for the master are interpreted as:
       - the number of tasks to distribute
       - the computation size of each task
       - the size of the files associated to each task
       - a list of host that will accept those tasks.

      Tasks are dumbly sent in a round-robin style.
      
      \until end_of_master
      
      
      \subsection MSG_ext_ms_slave_lua Slave code
    
      This function has to be assigned to a #m_process_t that has to behave as a slave.
      This function keeps waiting for tasks and executes them as it receives them.
      
      \until end_of_slave
         \subsection MSG_ext_ms_core_lua Simulation core

      in this section the core of the simulation which start by including the simgrid lib for bindings, then create the resources we need to set up our environment bypassing the XML parser.
      : <i>require "simgrid" </i>
      
         -# Hosts : <i>simgrid.Host.new</i> instanciate a new host with an id, and power.
         -# Links : <i>simgrid.Link.new</i> instanictae a new link that will require an id, bandwith and latency values.
         -# Route : <i>simgrid.Route.new</i> define a route between two hosts specifying the links to use.
         -# Simulation settings : <i>simgrid.register_platform();</i> register own platform without using the XML SURF parser.

         we can also bypass the XML deployment file, and associate functions for each of defined hosts.
        - <i>simgrid.Host.setFunction</i>: associate a function to a host, specifying arguments if needed.
        - <i>simgrid.register_application()</i>: saving the deployment settings before running the simualtion.

      \until simgrid.clean()
      
*/