X-Git-Url: http://info.iut-bm.univ-fcomte.fr/pub/gitweb/simgrid.git/blobdiff_plain/0a21b9f790a054594bc96362331582333216a8c7..HEAD:/doc/doxygen/platform.doc diff --git a/doc/doxygen/platform.doc b/doc/doxygen/platform.doc deleted file mode 100644 index 021adbe2bb..0000000000 --- a/doc/doxygen/platform.doc +++ /dev/null @@ -1,1561 +0,0 @@ -/*! \page platform Platform Description - -@tableofcontents - -In order to run any simulation, SimGrid must be provided with three things: -something to run (i.e., your code), a description of the platform on which you -want to simulate your application and lastly information about the deployment -process: Which process should be deployed to which processor/core? - -For the last two items, there are essentially two possible ways you can provide -this information as an input: -\li You can program it, either by using the Lua console ( - \ref MSG_Lua_funct) or, if you're using MSG, some of MSG's platform and - deployment functions (\ref msg_simulation). If you want to use this, - check the particular documentation. (You can also check the section - \ref pf_flexml_bypassing, however, this documentation is deprecated; - there is a new, but undocumented, way to do it properly). -\li You can use two XML files: one contains the platform description while - the other contains the deployment instructions. - -For more information on SimGrid's deployment features, please refer to -the \ref deployment documentation. - -The platform description may be intricate. This documentation is all -about how to write this file: The basic concepts are introduced. Furthermore, -advanced options are explained. Additionally, some hints and tips on how to -write a good platform description are given. - -\section pf_overview Some words about XML and DTD - -We chose to use XML not only because it's extensible but also because many -tools (and plugins for existing tools) are available that facilitate editing and -validating XML files. Furthermore, libraries that parse XML are often already -available and very well tested. - -The XML checking is done based on the Document Type Definition (DTD) file, -available at -http://simgrid.gforge.inria.fr/simgrid.dtd. - -If you read the DTD, you should notice the following: -\li The platform tags contain a version attribute; the current version is 3. - This property might be used in the future to provide backwards - compatibility. -\li The DTD contains definitions for the two files used by SimGrid (i.e., - platform description and deployment). - -\section pf_basics Basic concepts - -Nowadays, the Internet is composed of a bunch of independently managed -networks. Within each of those networks, there are entry and exit -points (most of the time, you can both enter and exit through the same -point); this allows to leave the current network and reach other -networks, possibly even in other locations. -At the upper level, such a network is called -Autonomous System (AS), while at the lower level it is named -sub-network, or LAN (local area network). -They are indeed autonomous: routing is defined -(within the limits of his network) by the administrator, and so, those -networks can operate without a connection to other -networks. So-called gateways allow you to go from one network to -another, if such a (physical) connection exists. Every node in one network -that can be directly reached (i.e., without traversing other nodes) from -another network is called a gateway. -Each autonomous system consists of equipment such as cables (network links), -routers and switches as well as computers. - -The structure of the SimGrid platform description relies exactly on the same -concept as a real-life platform (see above). Every resource (computers, -network equipment etc.) belongs to an AS, which can be defined by using the -\ tag. Within an AS, the routing between its elements can be defined -abitrarily. There are several modes for routing, and exactly one mode must be -selected by specifying the routing attribute in the AS tag: - -\verbatim - -\endverbatim - -\remark -Other supported values for the routing attribute can be found below, Section -\ref pf_raf. -\endremark - -There is also the ```` tag; this tag takes two attributes, ``src`` (source) -and ``dst`` (destination). Both source and destination must be valid identifiers -for routers (these will be introduced later). Contained by the ```` are -network links; these links must be used in order to communicate from the source -to the destination specified in the tag. Hence, a route merely describes -how to reach a router from another router. - -\remark -More information and (code-)examples can be found in the Section \ref pf_rm. -\endremark - -An AS can also contain one or more AS; this allows you to -define the hierarchy of your platform. - -Within each AS, the following types of resources exist: -\li host: a hostmachine; contains processors/cores etc. -\li router: a router or a gateway. -\li link: a link that defines a connection between two (or - more) resources. Every link has a bandwidth and a latency. -\li cluster: like a real cluster, contains many hosts - interconnected by some dedicated network. Each cluster is itself an AS. - -Between these elements, a routing has to be defined. The AS is -supposed to be Autonomous, hence this has to be done at the AS level. The AS -handles two different types of entities (host/router and -AS); you are responsible to define routes between those elements, -otherwise entities will be unconnected and therefore unreachable from other -entities. Although several algorithms for routing are built into SimGrid (see -\ref pf_rm), you might encounter a case where you want to define -routes manually (for instance, due to specific requirements of your -platform). - -There are three tags to use: -\li ASroute: to define routes between two AS -\li route: to define routes between two host/router -\li bypassRoute: to define routes between two AS that - will bypass default routing (as specified by the ``routing`` attribute - supplied to ````, see above). - -Here is an illustration of these concepts: - -![An illustration of an AS hierarchy. Here, AS1 contains 5 other AS' who in turn may contain other AS' as well.](AS_hierarchy.png) - Circles represent processing units and squares represent network routers. Bold - lines represent communication links. AS2 models the core of a national - network interconnecting a small flat cluster (AS4) and a larger - hierarchical cluster (AS5), a subset of a LAN (AS6), and a set of peers - scattered around the world (AS7). - -\section pf_pftags Resource description - -\subsection pf_As Platform: The \ tag - -The concept of an AS was already outlined above (Section \ref pf_basics); -recall that the AS is so important because it groups other resources (such -as routers/hosts) together (in fact, these resources must be contained by -an AS). - -Available attributes : - -Attribute name | Mandatory | Values | Description ---------------- | --------- | ------ | ----------- -id | yes | String | The identifier of an AS; facilitates referring to this AS. ID must be unique. -routing | yes | Full\| Floyd\| Dijkstra\| DijkstraCache\| none\| Vivaldi\| Cluster | See Section \ref pf_rm for details. - - -Example: -\verbatim - - - - - - -\endverbatim - -In this example, AS0 contains two hosts (host1 and host2). The route -between the hosts goes through link1. - -\subsection pf_Cr Computing resources: hosts, clusters and peers. - -\subsubsection pf_host The tag - -A host represents a computer/node card. Every host is able to execute -code and it can send and receive data to/from other hosts. Most importantly, -a host can contain more than 1 core. - -### Attributes: ### - -Attribute name | Mandatory | Values | Description ---------------- | --------- | ------ | ----------- -id | yes | String | The identifier of the host. facilitates referring to this AS. -power | yes | double (must be > 0.0) | Computational power of every core of this host in FLOPS. Must be larger than 0.0. -core | no | int (Default: 1) | The number of cores of this host. If more than one core is specified, the "power" parameter refers to every core, i.e., the total computational power is #cores*power.
If 6 cores are specified, up to 6 tasks can be executed without sharing the computational power; if more than 6 tasks are executed, computational power will be shared among these tasks.
Warning: Although functional, this model was never scientifically assessed. -availability | no | int | Specify if the percentage of power available. (What? TODO) -availability_file| no | string | (Relative or absolute) filename to use as input; must contain availability traces for this host. The syntax of this file is defined below.
Note: The filename must be specified with your system's format. -state | no | ON\|OFF
(Default: ON) | Is this host running or not? -state_file | no | string | Same mechanism as availability_file.
Note: The filename must be specified with your system's format. -coordinates | no | string | Must be provided when choosing the Vivaldi, coordinate-based routing model for the AS the host belongs to. More details can be found in the Section \ref pf_P2P_tags. - -### Possible children: ### - -Tag name | Description | Documentation ------------- | ----------- | ------------- - | Defines mounting points between some storage resource and the host. | \ref pf_sto_mo - | The prop tag allows you to define additional information on this host following the attribute/value schema. You may want to use it to give information to the tool you use for rendering your simulation, for example. | N/A - -### Example ### - -\verbatim - - - - - -\endverbatim - - -\anchor pf_host_dynamism -### Expressing dynamism ### - -SimGrid provides mechanisms to change a hosts' availability over -time, using the ``availability_file`` attribute to the ```` tag -and a separate text file whose syntax is exemplified below. - -#### Adding a trace file #### - -\verbatim - - - -\endverbatim - -#### Example of "bob.trace" file #### - -~~~~~~~~~~~~~~{.py} -PERIODICITY 1.0 - 0.0 1.0 - 11.0 0.5 - 20.0 0.8 -~~~~~~~~~~~~~~ - -Let us begin to explain this example by looking at line 2. (Line 1 will become clear soon). -The first column describes points in time, in this case, time 0. The second column -describes the relative amount of power this host is able to deliver (relative -to the maximum performance specified in the ```` tag). (Clearly, the -second column needs to contain values that are not smaller than 0 and not larger than 1). -In this example, our host will deliver 500 Mflop/s at time 0, as 500 Mflop/s is the -maximum performance of this host. At time 11.0, it will -deliver half of its maximum performance, i.e., 250 Mflop/s until time 20.0 when it will -will start delivering 80\% of its power. In this example, this amounts to 400 Mflop/s. - -Since the periodicity in line 1 was set to be 1.0, i.e., 1 timestep, this host will -continue to provide 500 Mflop/s from time 21. From time 32 it will provide 250 MFlop/s and so on. - -### Changing initial state ### - -It is also possible to specify whether the host is up or down by setting the -``state`` attribute to either ON (default value) or OFF. - -#### Example: Expliciting the default value "ON" #### - -\verbatim - - - -\endverbatim - -If you want this host to be unavailable, simply substitute ON with OFF. - -### Expressing churn ### - -To express the fact that a host can change state over time (as in P2P -systems, for instance), it is possible to use a file describing the time -at which the host is turned on or off. An example of the content -of such a file is presented below. - -#### Adding a state file #### - -\verbatim - - - -\endverbatim - -#### Example of "bob.fail" file #### - -~~~{.py} - PERIODICITY 10.0 - 1.0 -1.0 - 2.0 1.0 -~~~ - -A negative value means down (i.e., OFF) while a positive one means up and - running (i.e., ON). From time 0.0 to time 1.0, the host is on. At time 1.0, it is -turned off and at time 2.0, it is turned on again until time 12 (2.0 plus the -periodicity 10.0). It will be turned on again at time 13.0 until time 23.0, and -so on. - - - -\subsubsection pf_cluster - -```` represents a machine-cluster. It is most commonly used -when one wants to define many hosts and a network quickly. Technically, -``cluster`` is a meta-tag: from the inner SimGrid point of -view, a cluster is an AS where some optimized routing is defined. -The default inner organization of the cluster is as follow: - -\verbatim - __________ - | | - | router | - ____________|__________|_____________ backbone - | | | | | | - l0| l1| l2| l97| l96 | | l99 - | | | ........ | | | - | | - c-0.me c-99.me -\endverbatim - -Here, a set of hosts is defined. Each of them has a link -to a central backbone (backbone is a link itself, as a link can -be used to represent a switch, see the switch / link section -below for more details about it). A router allows to connect a -cluster to the outside world. Internally, -SimGrid treats a cluster as an AS containing all hosts: the router is the default -gateway for the cluster. - -There is an alternative organization, which is as follows: -\verbatim - __________ - | | - | router | - |__________| - / | \ - / | \ - l0 / l1| \l2 - / | \ - / | \ - host0 host1 host2 -\endverbatim - -The principle is the same, except that there is no backbone. This representation -can be obtained easily: just do not set the bb_* attributes. - - -Attribute name | Mandatory | Values | Description ---------------- | --------- | ------ | ----------- -id | yes | string | The identifier of the cluster. Facilitates referring to this cluster. -prefix | yes | string | Each node of the cluster has to have a name. This name will be prefixed with this prefix. -suffix | yes | string | Each node of the cluster will be suffixed with this suffix -radical | yes | string | Regexp used to generate cluster nodes name. Syntax: "10-20" will give you 11 machines numbered from 10 to 20, "10-20;2" will give you 12 machines, one with the number 2, others numbered as before. The produced number is concatenated between prefix and suffix to form machine names. -power | yes | int | Same as the ``power`` attribute of the ```` tag. -core | no | int (default: 1) | Same as the ``core`` attribute of the ```` tag. -bw | yes | int | Bandwidth for the links between nodes and backbone (if any). See link section for syntax/details. -lat | yes | int | Latency for the links between nodes and backbone (if any). See link section for syntax/details. -sharing_policy | no | string | Sharing policy for the links between nodes and backbone (if any). See link section for syntax/details. -bb_bw | no | int | Bandwidth for backbone (if any). See link section for syntax/details. If bb_bw and bb_lat (see below) attributes are omitted, no backbone is created (alternative cluster architecture described before). -bb_lat | no | int | Latency for backbone (if any). See link section for syntax/details. If bb_lat and bb_bw (see above) attributes are omitted, no backbone is created (alternative cluster architecture described before). -bb_sharing_policy | no | string | Sharing policy for the backbone (if any). See link section for syntax/details. -availability_file | no | string | Allows you to use a file as input for availability. Similar to hosts attribute. -state_file | no | string | Allows you to use a file as input for states. Similar to hosts attribute. -loopback_bw | no | int | Bandwidth for loopback (if any). See link section for syntax/details. If loopback_bw and loopback_lat (see below) attributes are omitted, no loopback link is created and all intra-node communication will use the main network link of the node. Loopback link is a FATPIPE. -loopback_lat | no | int | Latency for loopback (if any). See link section for syntax/details. See loopback_bw for more info. -topology | no | FLAT\|TORUS\|FAT_TREE (default: FLAT) | Network topology to use. SimGrid currently supports FLAT (with or without backbone, as described before), TORUS and FAT_TREE attributes for this tag. -topo_parameters | no | string | Specific parameters to pass for the topology defined in the topology tag. For torus networks, comma-separated list of the number of nodes in each dimension of the torus. For fat trees, refer to \ref AsClusterFatTree "AsClusterFatTree documentation". - -TODO - -the router name is defined as the resulting String in the following -java line of code: - -@verbatim -router_name = prefix + clusterId + _router + suffix; -@endverbatim - - -#### Cluster example #### - -Consider the following two (and independent) uses of the ``cluster`` tag: - -\verbatim - - - -\endverbatim - -The second example creates one router and 100 machines with the following names: -\verbatim -c-my_cluster_2_router.me -c-0.me -c-1.me -c-2.me -... -c-99.me -\endverbatim - -\subsubsection pf_peer - -This tag represents a peer, as in Peer-to-Peer (P2P) networks. However, internally, -SimGrid transforms a peer into an AS (similar to Cluster). Hence, this tag -is virtually only a shortcut that comes with some pre-defined resources -and values. These are: - -\li A tiny AS whose routing type is cluster is created -\li A host -\li Two links: One for download and one for upload. This is - convenient to use and simulate stuff under the last mile model (e.g., ADSL peers). -\li It connects the two links to the host -\li It creates a router (a gateway) that serves as an entry point for this peer zone. - This router has coordinates. - -#### Attributes #### - -Attribute name | Mandatory | Values | Description ---------------- | --------- | ------ | ----------- -id | yes | string | The identifier of the peer. Facilitates referring to this peer. -power | yes | int | See the description of the ``host`` tag for this attribute -bw_in | yes | int | Bandwidth downstream -bw_out | yes | int | Bandwidth upstream -lat | yes | double | Latency for both up- and downstream, in seconds. -coordinates | no | string | Coordinates of the gateway for this peer. Example value: 12.8 14.4 6.4 -sharing_policy | no | SHARED\|FULLDUPLEX (default: FULLDUPLEX) | Sharing policy for links. See link description for details. -availability_file| no | string | Availability file for the peer. Same as host availability file. See host description for details. -state_file | no | string | State file for the peer. Same as host state file. See host description for details. - -Internally, SimGrid transforms any ```` construct such as -\verbatim - -\endverbatim -into an ```` (see Sections \ref pf_basics and \ref pf_As). In fact, this example of the ```` tag -is completely equivalent to the following declaration: - -\verbatim - - - - - - - -\endverbatim - - -\subsection pf_ne Network equipments: links and routers - -There are two tags available to represent network entities: -1. ````: Represents a entity that has a limited bandwidth, a - latency, and that can be shared according to TCP way to share this - bandwidth. -\remark -The concept of links in SimGrid may not be intuitive, as links are not limited -to connecting (exactly) two entities; in fact, you can have more than two equipments -connected to it. (In graph theoretical terms: A link in SimGrid is not an edge, -but a hyperedge) -\endremark - -2. ````: Represents an entity that a message can be routed - to, but that is unable to execute any code. In SimGrid, routers have also - no impact on the performance: Routers do not limit any bandwidth nor - do they increase latency. As a matter of fact, routers are (almost) ignored - by the simulator when the simulation has begun. - -\remark - If you want to represent an entity like a switch, you must use ```` (see section). Routers are used - to run some routing algorithm and determine routes (see Section \ref pf_routing for details). -\endremark - -\subsubsection pf_router - -%As said before, router is used only to give some information -for routing algorithms. So, it does not have any attributes except : - -#### Attributes #### - -Attribute name | Mandatory | Values | Description ---------------- | --------- | ------ | ----------- -id | yes | string | The identifier of the router to be used when referring to it. -coordinates | yes | string | Must be provided when choosing the Vivaldi, coordinate-based routing model for the AS the router belongs to. More details can be found in the Section \ref pf_P2P_tags. - -#### Example #### - -\verbatim - -\endverbatim - -\subsubsection pf_link - -Network links can represent one-hop network connections. They are -characterized by their id and their bandwidth; links can (but may not) be subject -to latency. - -#### Attributes #### - -Attribute name | Mandatory | Values | Description ---------------- | --------- | ------ | ----------- -id | yes | string | The identifier of the link to be used when referring to it. -bandwidth | yes | int | Maximum bandwidth for this link, given in bytes/s -latency | no | double (default: 0.0) | Latency for this link. -sharing_policy | no | SHARED\|FATPIPE\|FULLDUPLEX (default: SHARED) | Sharing policy for the link. -state | no | ON\|OFF (default: ON) | Allows you to to turn this link on or off (working / not working) -bandwidth_file | no | string | Allows you to use a file as input for bandwidth. -latency_file | no | string | Allows you to use a file as input for latency. -state_file | no | string | Allows you to use a file as input for states. - - -#### Possible shortcuts for ``latency`` #### - -When using the latency attribute, you can specify the latency by using the scientific -notation or by using common abbreviations. For instance, the following three tags -are equivalent: - -\verbatim - - - -\endverbatim - -Here, the second tag uses "us", meaning "microseconds". Other shortcuts are: - -Name | Abbreviation | Time (in seconds) ----- | ------------ | ----------------- -Week | w | 7 * 24 * 60 * 60 -Day | d | 24 * 60 * 60 -Hour | h | 60 * 60 -Minute | m | 60 -Second | s | 1 -Millisecond | ms | 0.001 = 10^(-3) -Microsecond | us | 0.000001 = 10^(-6) -Nanosecond | ns | 0.000000001 = 10^(-9) -Picosecond | ps | 0.000000000001 = 10^(-12) - -#### Sharing policy #### - -By default a network link is SHARED, i.e., if two or more data flows go -through a link, the bandwidth is shared fairly among the data flows. This -is similar to the sharing policy TCP uses. - -On the other hand, if a link is defined as a FATPIPE, -each flow going through this link will be allocated the complete bandwidth, -i.e., no sharing occurs and the bandwidth is only limiting single flows: -The complete bandwidth provided by this link in this mode -is ``#flows*bandwidth``. - -The last mode available is FULLDUPLEX. This means that SimGrid will -automatically generate two links (one carrying the suffix _UP and the other the -suffix _DOWN) for each ```` tag. This models situations when the direction -of traffic is important. - -\remark -Transfers from one side to the other will interact similarly as -TCP when ACK returning packets circulate on the other direction. More -discussion about it is available in the description of link_ctn description. -\endremark - -In other words: The SHARED policy defines a physical limit for the bandwidth. -The FATPIPE mode defines a limit for each application, -with no upper total limit. - -\remark -Tip: By using the FATPIPE mode, you can model big backbones that -won't affect performance (except latency). -\endremark - -#### Example #### - -\verbatim - -\endverbatim - -#### Expressing dynamism and failures #### - -Similar to hosts, it is possible to declare links whose state, bandwidth -or latency changes over time (see Section \ref pf_hosts_dynamism for details). - -In the case of network links, the ``bandwidth`` and ``latency`` attributes are -replaced by the ``bandwidth_file`` and ``latency_file`` attributes. -The following XML snippet demonstrates how to use this feature in the platform -file. The structure of the files "link1.bw" and "link1.lat" is shown below. - -\verbatim - -\endverbatim - -\note -Even if the syntax is the same, the semantic of bandwidth and latency -trace files differs from that of host availability files. For bandwidth and -latency, the corresponding files do not -express availability as a fraction of the available capacity but directly in - bytes per seconds for the bandwidth and in seconds for the latency. This is -because most tools allowing to capture traces on real platforms (such as NWS) - express their results this way. - \endnote - -##### Example of "link1.bw" file ##### - -~~~{.py} -PERIODICITY 12.0 -4.0 40000000 -8.0 60000000 -~~~ - -In this example, the bandwidth changes repeatedly, with all changes -being repeated every 12 seconds. - -At the beginning of the the simulation, the link's bandwidth is 80,000,000 -B/s (i.e., 80 Mb/s); this value was defined in the XML snippet above. -After four seconds, it drops to 40 Mb/s (line 2), and climbs -back to 60 Mb/s after another 4 seconds (line 3). The value does not change any -more until the end of the period, that is, after 12 seconds have been simulated). -At this point, periodicity kicks in and this behavior is repeated: Seconds -12-16 will experience 80 Mb/s, 16-20 40 Mb/s etc.). - -##### Example of "link1.lat" file ##### - -~~~{.py} -PERIODICITY 5.0 -1.0 0.001 -2.0 0.01 -3.0 0.001 -~~~ - -In this example, the latency varies with a period of 5 seconds. -In the xml snippet above, the latency is initialized to be 0.0001s (100µs). This -value will be kept during the first second, since the latency_file contains -changes to this value at second one, two and three. -At second one, the value will be 0.001, i.e., 1ms. One second later it will -be adjusted to 0.01 (or 10ms) and one second later it will be set again to 1ms. The -value will not change until second 5, when the periodicity defined in line 1 -kicks in. It then loops back, starting at 100µs (the initial value) for one second. - - -#### The ```` tag #### - -Similar to ````, the link may also contain the ```` tag; see the host -documentation (Section \ref pf_host) for an example. - - -TODO - -\subsection pf_storage Storage - -\note -This is a prototype version that should evolve quickly, this -is just some doc valuable only at the time of writing this doc -This section describes the storage management under SimGrid ; nowadays -it's only usable with MSG. It relies basically on linux-like concepts. -You also may want to have a look to its corresponding section in \ref -msg_file_management ; functions access are organized as a POSIX-like -interface. -\endnote - -\subsubsection pf_sto_conc Storage Main concepts -Basically there is 3 different entities to know : -\li the storage_type: here you define some kind of storage that - you will instantiate many type on your platform. Think of it like - a definition of throughput of a specific disk. -\li the storage: instance of a storage_type. Defines a - new storage of storage_type -\li the mount: says that the storage is located into this - specific resource. - -the content of a storage has to be defined in a content file that -contains the content. The path to this file has to be passed within -the content attribute . Here is a way to generate it: - -\verbatim -find /path/you/want -type f -exec ls -l {} \; 2>/dev/null > ./content.txt -\endverbatim - -\subsubsection pf_sto_sttp storage_type - - -storage_type attributes : -\li id (mandatory): the identifier of the storage_type to be - used when referring to it. -\li model (mandatory): Unused for now by the simulator (but - mandatory, ok) -\li content: default value 0. The file containing the disk - content. (may be moved soon or later to storage tag. - -The tag must contains some predefined model prop, as may do some other -resources tags. -storage_type mandatory model_prop : -\li Bwrite: value in B/s. Write throughput -\li Bread: value in B/s. Read throughput -\li Bconnexion: value in B/s. Connection throughput (i.e. the - throughput of the storage connector). - -A storage_type can also contain the prop tag. The prop tag allows you -to define additional information on this storage_type following the -attribute/value schema. You may want to use it to give information to -the tool you use for rendering your simulation, for example. - -\verbatim - - - - - - -\endverbatim - -\subsubsection pf_sto_st storage - -storage_type attributes : -\li id (mandatory): the identifier of the storage to be used - when referring to it. -\li typeId (mandatory): the identifier of the storage_type that - this storage belongs to. -\li attach (mandatory): the host (name) to which the storage is - attached to. - -\subsubsection pf_sto_mo mount - -mount attributes : -\li id (mandatory): the id of the storage that must be - mounted on that computer. -\li name (mandatory): the name that will be the logical - reference to this disk (the mount point). - -\subsubsection pf_sto_mst mstorage -Note : unused for now -mstorage attributes : -\li typeId (mandatory): the id of the storage that must - be mounted on that computer. -\li name (mandatory): the name that will be the logical - reference to this disk (the mount point). - -\section pf_routing Routing - -To achieve high performance, the routing tables used within SimGrid are -static. This means that routing between two nodes is calculated once -and will not change during execution. The SimGrid team chose to use this -approach as it is rare to have a real deficiency of a resource; -most of the time, a communication fails because the links experience too much -congestion and hence, your connection stops before the timeout or -because the computer designated to be the destination of that message -is not responding. - -We also chose to use shortest paths algorithms in order to emulate -routing. Doing so is consistent with the reality: RIP, OSPF, BGP are -all calculating shortest paths. They have some convergence time, but -at the end, so when the platform is stable (and this should be the -moment you want to simulate something using SimGrid) your packets will -follow the shortest paths. - -\subsection pf_rm Routing models - -Within each AS, you have to define a routing model to use. You have -basically 3 main kind of routing models : - -\li Shortest-path based models: you let SimGrid calculates shortest - paths and manage it. Behaves more or less as most real life - routing. -\li Manually-entered route models: you'll have to define all routes - manually by yourself into the platform description file. - Consistent with some manually managed real life routing. -\li Simple/fast models: those models offers fast, low memory routing - algorithms. You should consider to use it if you can make some - assumptions about your AS. Routing in this case is more or less - ignored - -\subsubsection pf_raf The router affair - -Expressing routers becomes mandatory when using shortest-path based -models or when using ns-3 or the bindings to the GTNetS packet-level -simulator instead of the native analytical network model implemented -in SimGrid. - -For graph-based shortest path algorithms, routers are mandatory, -because both algorithms need a graph, and so we need to have source -and destination for each edge. - -Routers are naturally an important concept in GTNetS or ns-3 since the -way they run the packet routing algorithms is actually simulated. -Instead, the SimGrid’s analytical models aggregate the routing time -with the transfer time. Rebuilding a graph representation only from -the route information turns to be a very difficult task, because of -the missing information about how routes intersect. That is why we -introduced a \ tag, which is simply used to express these -intersection points. The only attribute accepted by this tag an id. It -is important to understand that the \ tag is only used to -provide topological information. - -To express those topological information, some route have to be -defined saying which link is between which routers. Description or the -route syntax is given below, as well as example for the different -models. - -\subsubsection pf_rm_sh Shortest-path based models - -Here is the complete list of such models, that computes routes using -classic shortest-paths algorithms. How to choose the best suited -algorithm is discussed later in the section devoted to it. - -\li Floyd: Floyd routing data. Pre-calculates all routes once. -\li Dijkstra: Dijkstra routing data ,calculating routes when - necessary. -\li DijkstraCache: Dijkstra routing data. Handle some cache for - already calculated routes. - -All those shortest-path models are instanciated the same way. Here are -some example of it: - -Floyd example : -\verbatim - - - - - - - - - - - - - - - -\endverbatim - -ASroute given at the end gives a topological information: link1 is -between router1 and host1. - -Dijsktra example : -\verbatim - - - - - - - - - - - - - - - - -\endverbatim - -DijsktraCache example : -\verbatim - - - ... -(platform unchanged compared to upper example) -\endverbatim - -\subsubsection pf_rm_me Manually-entered route models - -\li Full: You have to enter all necessary routes manually - -Full example : -\verbatim - - - - - - -\endverbatim - -\subsubsection pf_rm_sf Simple/fast models - -\li none: No routing (Unless you know what you are doing, avoid -using this mode in combination with a non Constant network model). -None Example : -\verbatim - - -\endverbatim - -\li Vivaldi: Vivaldi routing, so when you want to use - coordinates. See the corresponding section P2P below for details. -\li Cluster: Cluster routing, specific to cluster tag, should - not be used, except internally. - -\subsection ps_dec Defining routes - -The principle of route definition is the same for the 4 available tags -for doing it. Those for tags are: - -\li route: to define route between host/router -\li ASroute: to define route between AS -\li bypassRoute: to bypass normal routes as calculated by the - network model between host/router -\li bypassASroute: same as bypassRoute, but for AS - -Basically all those tags will contain an (ordered) list of references -to link that compose the route you want to define. - -Consider the example below: - -\verbatim - - - - - -\endverbatim - -The route here from host Alice to Bob will be first link1, then link2, -and finally link3. What about the reverse route ? route and -ASroute have an optional attribute symmetrical, that can -be either YES or NO. YES means that the reverse route is the same -route in the inverse order, and is set to YES by default. Note that -this is not the case for bypass*Route, as it is more probable that you -want to bypass only one default route. - -For an ASroute, things are just slightly more complicated, as you have -to give the id of the gateway which is inside the AS you're talking -about you want to access ... So it looks like this : - - -\verbatim - - - -\endverbatim - -gw == gateway, so when any message are trying to go from AS1 to AS2, -it means that it must pass through router1 to get out of the AS, then -pass through link1, and get into AS2 by being received by router2. -router1 must belong to AS1 and router2 must belong to AS2. - -\subsubsection pf_linkctn link_ctn - -a link_ctn is the tag that is used in order to reference a -link in a route. Its id is the link id it refers to. - -link_ctn attributes : -\li id (mandatory): Id of the link this tag refers to -\li direction: if the link referenced by id has been - declared as FULLDUPLEX, this is used to indicate in which - direction the route you're defining is going through this link. - Possible values "UP" or "DOWN". - -\subsubsection pf_asro ASroute - -ASroute tag purpose is to let people write manually their routes -between AS. It's useful when you're in Full model. - -ASroute attributes : -\li src (mandatory): the source AS id. -\li dst (mandatory): the destination AS id. -\li gw_src (mandatory): the gateway to be used within the AS. - Can be any host or \b router defined into the \b src AS or - into one of the AS it includes. -\li gw_dst (mandatory): the gateway to be used within the AS. - Can be any host or \b router defined into the \b dst AS or - into one of the AS it includes. -\li symmetrical: if the route is symmetric, the reverse route - will be the opposite of the one defined. Can be either YES or NO, - default is YES. - -Example of ASroute with Full -\verbatim - - - - - - - - - - - - - - -\endverbatim - -\subsubsection pf_ro route -The principle is the same as ASroute : route contains list of -links that are in the path between src and dst, except that it is for -routes between a src that can be either host or \b router and a -dst that can be either host or \b router. Useful for Full -as well as for the shortest-paths based models, where you -have to give topological information. - - -route attributes : -\li src (mandatory): the source id. -\li dst (mandatory): the destination id. -\li symmetrical: if the route is symmetric, the reverse route - will be the opposite of the one defined. Can be either YES or NO, - default is YES. - -route example in Full -\verbatim - - - -\endverbatim - -route example in a shortest-path model -\verbatim - - - -\endverbatim -Note that when using route to give topological information, you have -to give routes with one link only in it, as SimGrid needs to know -which host are at the end of the link. - -\subsubsection pf_byASro bypassASroute - -Note : bypassASroute and bypassRoute are under rewriting to perform -better ; so you may not use it yet As said before, once you choose -a model, it (if so) calculates routes for you. But maybe you want to -define some of your routes, which will be specific. You may also want -to bypass some routes defined in lower level AS at an upper stage : -bypassASroute is the tag you're looking for. It allows to -bypass routes defined between already defined between AS (if you want -to bypass route for a specific host, you should just use byPassRoute). -The principle is the same as ASroute : bypassASroute contains -list of links that are in the path between src and dst. - -bypassASroute attributes : -\li src (mandatory): the source AS id. -\li dst (mandatory): the destination AS id. -\li gw_src (mandatory): the gateway to be used within the AS. - Can be any host or \b router defined into the \b src AS or - into one of the AS it includes. -\li gw_dst (mandatory): the gateway to be used within the AS. - Can be any host or \b router defined into the \b dst AS or - into one of the AS it includes. -\li symmetrical: if the route is symmetric, the reverse route - will be the opposite of the one defined. Can be either YES or NO, - default is YES. - -bypassASroute Example -\verbatim - - - -\endverbatim - -\subsubsection pf_byro bypassRoute -Note : bypassASRoute and bypassRoute are under rewriting to perform -better ; so you may not use it yet As said before, once you choose -a model, it (if so) calculates routes for you. But maybe you want to -define some of your routes, which will be specific. You may also want -to bypass some routes defined in lower level AS at an upper stage : -bypassRoute is the tag you're looking for. It allows to bypass -routes defined between host/router. The principle is the same -as route : bypassRoute contains list of links references of -links that are in the path between src and dst. - -bypassRoute attributes : -\li src (mandatory): the source AS id. -\li dst (mandatory): the destination AS id. -\li symmetrical: if the route is symmetric, the reverse route - will be the opposite of the one defined. Can be either YES or NO, - default is YES. - -bypassRoute Example -\verbatim - - - -\endverbatim - - -\subsection pb_baroex Basic Routing Example - -Let's say you have an AS named AS_Big that contains two other AS, AS_1 -and AS_2. If you want to make a host (h1) from AS_1 with another one -(h2) from AS_2 then you'll have to proceed as follows: -\li First, you have to ensure that a route is defined from h1 to the - AS_1's exit gateway and from h2 to AS_2's exit gateway. -\li Then, you'll have to define a route between AS_1 to AS_2. As those - AS are both resources belonging to AS_Big, then it has to be done - at AS_big level. To define such a route, you have to give the - source AS (AS_1), the destination AS (AS_2), and their respective - gateway (as the route is effectively defined between those two - entry/exit points). Elements of this route can only be elements - belonging to AS_Big, so links and routers in this route should be - defined inside AS_Big. If you choose some shortest-path model, - this route will be computed automatically. - -As said before, there are mainly 2 tags for routing : -\li ASroute: to define routes between two AS -\li route: to define routes between two host/router - -As we are dealing with routes between AS, it means that those we'll -have some definition at AS_Big level. Let consider AS_1 contains 1 -host, 1 link and one router and AS_2 3 hosts, 4 links and one router. -There will be a central router, and a cross-like topology. At the end -of the crosses arms, you'll find the 3 hosts and the router that will -act as a gateway. We have to define routes inside those two AS. Let -say that AS_1 contains full routes, and AS_2 contains some Floyd -routing (as we don't want to bother with defining all routes). As -we're using some shortest path algorithms to route into AS_2, we'll -then have to define some route to gives some topological -information to SimGrid. Here is a file doing it all : - -\verbatim - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\endverbatim - -\section pf_other_tags Tags not (directly) describing the platform - -There are 3 tags, that you can use inside a \ tag that are -not describing the platform: -\li random: it allows you to define random generators you want to use - for your simulation. -\li config: it allows you to pass some configuration stuff like, for - example, the network model and so on. It follows the -\li include: simply allows you to include another file into the - current one. - -\subsection pf_conf config -config attributes : -\li id (mandatory): the identifier of the config to be used - when referring to it. - - -config tag only purpose is to include prop tags. Valid -id are basically the same as the list of possible parameters you can -use by command line, except that "/" are used for namespace -definition. See the \ref options config and options page for more -information. - - -config example -\verbatim - - - - - - - - - - - - - -... -\endverbatim - - -\subsection pf_rand random -Not yet in use, and possibly subject to huge modifications. - -\subsection pf_incl include -include tag allows to import into a file platform parts located -in another file. This is done with the intention to help people -combine their different AS and provide new platforms. Those files -should contains XML part that contains either -include,cluster,peer,AS,trace,trace_connect tags. - -include attributes : -\li file (mandatory): filename of the file to include. Possible - values: absolute or relative path, syntax similar to the one in - use on your system. - -Note: due to some obscure technical reasons, you have to open -and close tag in order to let it work. -include Example -\verbatim - - - - - - - - -\endverbatim - -\subsection pf_tra trace and trace_connect -Both tags are an alternate way to passe availability, state, and so on -files to entity. Instead of referring to the file directly in the host, -link, or cluster tag, you proceed by defining a trace with an id -corresponding to a file, later a host/link/cluster, and finally using -trace_connect you say that the file trace must be used by the entity. -Get it ? Let's have a look at an example : - -\verbatim - - - - - -\endverbatim - -All constraints you have is that trace_connect is after -trace and host definitions. - - -trace attributes : -\li id (mandatory): the identifier of the trace to be used when - referring to it. -\li file: filename of the file to include. Possible values : - absolute or relative path, syntax similar to the one in use on - your system. If omitted, the system expects that you provide the - trace values inside the trace tags (see below). -\li trace periodicity (mandatory): trace periodicity, same - definition as in hosts (see upper for details). - -Here is an example of trace when no file name is provided: - -\verbatim - - 0.0 1.0 - 11.0 0.5 - 20.0 0.8 - -\endverbatim - -trace_connect attributes : -\li kind: the type of trace, possible values - HOST_AVAIL|POWER|LINK_AVAIL|BANDWIDTH|LATENCY, default: - HOST_AVAIL -\li trace (mandatory): the identifier of the trace referenced. -\li element (mandatory): the identifier of the entity referenced. - - - -\section pf_hints Hints and tips, or how to write a platform efficiently - -Now you should know at least the syntax and be able to create a -platform by your own. However, after having ourselves wrote some platforms, there -are some best practices you should pay attention to in order to -produce good platform and some choices you can make in order to have -faster simulations. Here's some hints and tips, then. - -\subsection pf_as_h AS Hierarchy -The AS design allows SimGrid to go fast, because computing route is -done only for the set of resources defined in this AS. If you're using -only a big AS containing all resource with no AS into it and you're -using Full model, then ... you'll loose all interest into it. On the -other hand, designing a binary tree of AS with, at the lower level, -only one host, then you'll also loose all the good AS hierarchy can -give you. Remind you should always be "reasonable" in your platform -definition when choosing the hierarchy. A good choice if you try to -describe a real life platform is to follow the AS described in -reality, since this kind of trade-off works well for real life -platforms. - -\subsection pf_exit_as Exit AS: why and how -Users that have looked at some of our platforms may have notice a -non-intuitive schema ... Something like that : - - -\verbatim - - - - - - - - - - - - - - - - - - - - - - - -\endverbatim - -In the AS_4, you have an exitAS_4 defined, containing only one router, -and routes defined to that AS from all other AS (as cluster is only a -shortcut for an AS, see cluster description for details). If there was -an upper AS, it would define routes to and from AS_4 with the gateway -router_4. It's just because, as we did not allowed (for performances -issues) to have routes from an AS to a single host/router, you have to -enclose your gateway, when you have AS included in your AS, within an -AS to define routes to it. - -\subsection pf_P2P_tags P2P or how to use coordinates -SimGrid allows you to use some coordinated-based system, like vivaldi, -to describe a platform. The main concept is that you have some peers -that are located somewhere: this is the function of the -coordinates of the \ or \ tag. There's nothing -complicated in using it, here is an example of it: - -\verbatim - - - - - - - - - - - ... - - - -\endverbatim - -Coordinates are then used to calculate latency between two hosts by -calculating the euclidean distance between the two hosts coordinates. -The results express the latency in ms. - -Note that the previous example defines a routing directly between hosts but it could be also used to define a routing between AS. -That is for example what is commonly done when using peers (see Section \ref pf_peer). -\verbatim - - - - - - - - - - - - - -\endverbatim -In such a case though, we connect the AS created by the peer tag with the Vivaldi routing mechanism. -This means that to route between AS1 and AS2, it will use the coordinates of router_AS1 and router_AS2. -This is currently a convention and we may offer to change this convention in the DTD later if needed. -You may have noted that conveniently, a peer named FOO defines an AS named FOO and a router named router_FOO, which is why it works seamlessly with the peer tag. - - -\subsection pf_wisely Choosing wisely the routing model to use - - -Choosing wisely the routing model to use can significantly fasten your -simulation/save your time when writing the platform/save tremendous -disk space. Here is the list of available model and their -characteristics (lookup : time to resolve a route): - -\li Full: Full routing data (fast, large memory requirements, - fully expressive) -\li Floyd: Floyd routing data (slow initialization, fast - lookup, lesser memory requirements, shortest path routing only). - Calculates all routes at once at the beginning. -\li Dijkstra: Dijkstra routing data (fast initialization, slow - lookup, small memory requirements, shortest path routing only). - Calculates a route when necessary. -\li DijkstraCache: Dijkstra routing data (fast initialization, - fast lookup, small memory requirements, shortest path routing - only). Same as Dijkstra, except it handles a cache for latest used - routes. -\li none: No routing (usable with Constant network only). - Defines that there is no routes, so if you try to determine a - route without constant network within this AS, SimGrid will raise - an exception. -\li Vivaldi: Vivaldi routing, so when you want to use coordinates -\li Cluster: Cluster routing, specific to cluster tag, should - not be used. - -\subsection pf_switch Hey, I want to describe a switch but there is no switch tag ! - -Actually we did not include switch tag, ok. But when you're trying to -simulate a switch, the only major impact it has when you're using -fluid model (and SimGrid uses fluid model unless you activate GTNetS, -ns-3, or constant network mode) is the impact of the upper limit of -the switch motherboard speed that will eventually be reached if you're -using intensively your switch. So, the switch impact is similar to a -link one. That's why we are used to describe a switch using a link tag -(as a link is not an edge by a hyperedge, you can connect more than 2 -other links to it). - -\subsection pf_platform_multipath How to express multipath routing in platform files? - -It is unfortunately impossible to express the fact that there is more -than one routing path between two given hosts. Let's consider the -following platform file: - -\verbatim - - - - - - - - - -\endverbatim - -Although it is perfectly valid, it does not mean that data traveling -from A to C can either go directly (using link 3) or through B (using -links 1 and 2). It simply means that the routing on the graph is not -trivial, and that data do not following the shortest path in number of -hops on this graph. Another way to say it is that there is no implicit -in these routing descriptions. The system will only use the routes you -declare (such as <route src="A" dst="C"><link_ctn -id="3"/></route>), without trying to build new routes by aggregating -the provided ones. - -You are also free to declare platform where the routing is not -symmetric. For example, add the following to the previous file: - -\verbatim - - - - -\endverbatim - -This makes sure that data from C to A go through B where data from A -to C go directly. Don't worry about realism of such settings since -we've seen ways more weird situation in real settings (in fact, that's -the realism of very regular platforms which is questionable, but -that's another story). - -\section pf_flexml_bypassing Bypassing the XML parser with your own C functions -NOTE THAT THIS DOCUMENTATION, WHILE STILL WORKING, IS STRONGLY DEPRECATED - -So you want to bypass the XML files parser, uh? Maybe doing some parameter -sweep experiments on your simulations or so? This is possible, and -it's not even really difficult (well. Such a brutal idea could be -harder to implement). Here is how it goes. - -For this, you have to first remember that the XML parsing in SimGrid is done -using a tool called FleXML. Given a DTD, this gives a flex-based parser. If -you want to bypass the parser, you need to provide some code mimicking what -it does and replacing it in its interactions with the SURF code. So, let's -have a look at these interactions. - -FleXML parser are close to classical SAX parsers. It means that a -well-formed SimGrid platform XML file might result in the following -"events": - - - start "platform_description" with attribute version="2" - - start "host" with attributes id="host1" power="1.0" - - end "host" - - start "host" with attributes id="host2" power="2.0" - - end "host" - - start "link" with ... - - end "link" - - start "route" with ... - - start "link_ctn" with ... - - end "link_ctn" - - end "route" - - end "platform_description" - -The communication from the parser to the SURF code uses two means: -Attributes get copied into some global variables, and a surf-provided -function gets called by the parser for each event. For example, the event - - start "host" with attributes id="host1" power="1.0" - -let the parser do something roughly equivalent to: -\verbatim - strcpy(A_host_id,"host1"); - A_host_power = 1.0; - STag_host(); -\endverbatim - -In SURF, we attach callbacks to the different events by initializing the -pointer functions to some the right surf functions. Since there can be -more than one callback attached to the same event (if more than one -model is in use, for example), they are stored in a dynar. Example in -workstation_ptask_L07.c: -\verbatim - /* Adding callback functions */ - surf_parse_reset_parser(); - surfxml_add_callback(STag_surfxml_host_cb_list, &parse_cpu_init); - surfxml_add_callback(STag_surfxml_prop_cb_list, &parse_properties); - surfxml_add_callback(STag_surfxml_link_cb_list, &parse_link_init); - surfxml_add_callback(STag_surfxml_route_cb_list, &parse_route_set_endpoints); - surfxml_add_callback(ETag_surfxml_link_c_ctn_cb_list, &parse_route_elem); - surfxml_add_callback(ETag_surfxml_route_cb_list, &parse_route_set_route); - - /* Parse the file */ - surf_parse_open(file); - xbt_assert(!surf_parse(), "Parse error in %s", file); - surf_parse_close(); -\endverbatim - -So, to bypass the FleXML parser, you need to write your own version of the -surf_parse function, which should do the following: - - Fill the A__ variables with the wanted values - - Call the corresponding STag__fun function to simulate tag start - - Call the corresponding ETag__fun function to simulate tag end - - (do the same for the next set of values, and loop) - -Then, tell SimGrid that you want to use your own "parser" instead of the stock one: -\verbatim - surf_parse = surf_parse_bypass_environment; - MSG_create_environment(NULL); - surf_parse = surf_parse_bypass_application; - MSG_launch_application(NULL); -\endverbatim - -A set of macros are provided at the end of -include/surf/surfxml_parse.h to ease the writing of the bypass -functions. An example of this trick is distributed in the file -examples/msg/masterslave/masterslave_bypass.c - - -*/