3 <object id="TOC" data="graphical-toc.svg" type="image/svg+xml"></object>
5 window.onload=function() { // Wait for the SVG to be loaded before changing it
6 var elem=document.querySelector("#TOC").contentDocument.getElementById("RoutingBox")
7 elem.style="opacity:0.93999999;fill:#ff0000;fill-opacity:0.1;stroke:#000000;stroke-width:0.35277778;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1";
15 Demystifying the routing
16 ########################
18 When describing a platform, routing is certainly the most complex
19 and error-prone part. This section explains the basics of SimGrid's
20 routing mechanism which allows you to easily compose and scale your
25 .. |flat_img| image:: img/zone_hierarchy.png
28 .. |tree_img| image:: img/zone_tree.svg
30 Circles represent processing units and squares represent network
31 routers. Bold lines represent communication links. The zone "AS2" models the core of a national network interconnecting a
32 small flat cluster (AS4) and a larger hierarchical cluster (AS5), a
33 subset of a LAN (AS6), and a set of peers scattered around the world
36 Networking zones (:ref:`pf_tag_zone`) are an advanced concept used to factorize the description
37 to reduce the size of your platform on disk and in memory.
38 Any zone may contain sub-zones, allowing for a hierarchical
39 decomposition of the platform (as you can see in the tree representation on the left).
40 Routing can be made more efficient (as the
41 inter-zone routing gets factored with :ref:`pf_tag_zoneroute`) and
42 allows you to have more than one routing model in your platform. For
43 example, you can have a coordinate-based routing for the WAN parts
44 of your platforms, a full routing within each datacenter, and a highly
45 optimized routing within each cluster of the datacenter. In this
46 case, determining the route between two given hosts gets
47 "somewhat more complex" but SimGrid still computes
48 these routes for you in a time- and space-efficient manner.
51 Routing basic elements: hosts and links
52 ***************************************
54 A platform is composed of a set of resources, namely hosts, links and disks.
55 On these resources you may run activities that will require some capacity and
56 will make the time advance.
58 Given a look at this example of some hosts and links being declared
62 <zone id="AS5-4" routing="Full">
63 <host id="host0" speed="1Gf"/>
64 <host id="host1" speed="2Gf"/>
65 <link id="link0" bandwidth="125MBps" latency="100us"/>
68 It describes a simple FullZone with 2 hosts inside connected through
69 a link. Note that the ``link0`` just represents a resource with a
70 certain bandwidth capacity and latency. It's only when you add
71 a route between ``host0`` and ``host1`` that this link will be used by
72 SimGrid in the communications.
76 <zone id="AS5-4" routing="Full">
78 <route src="host0" dst="host1"><link_ctn id="link0"/></route>
81 Note that no verification is performed concerning the links you use in a route.
82 This is quite flexible and enables interesting features. However, it also allows you
83 to do some strange topologies, such as having a single link used by a pair
84 of hosts from different zone:
88 <zone id="Nonsense" routing="Full">
89 <host id="host3" speed="1Gf"/>
90 <host id="host4" speed="2Gf"/>
91 <route src="host3" dst="host4"><link_ctn id="link0"/></route>
94 Probably you do not want to do this, but it's your responsibility to write
95 your platform file properly. SimGrid will not try to be smarter than you!
97 Describing routes: intra vs inter
98 *********************************
103 TLDR: use :ref:`pf_tag_route`
105 The routing mechanism inside a given zone is defined by ``routing=`` parameter
106 in the :ref:`pf_tag_zone` (see options in :ref:`intra-zone section <intra_zone>`). For example, in a *Full* zone, the user must declare
107 a :ref:`pf_tag_route` for each pair of hosts inside the zone. Other zones, such as *Floyd*
108 or *Dijkstra* will calculate the shortest path, while *DragonFly* and *Fat-Tree* uses
109 specialized routing algorithms to improve performance.
111 When adding a route inside a zone, keep in mind that you need 3 main parameters:
112 - src: Host (or router) source
113 - dst: Host (or router) destination
114 - links: list of resources (links in this case) used in the communication
119 TLDR: use :ref:`pf_tag_zoneroute`
121 When describing complex topologies, such as the one depicted in the beginning
122 of this page, you will need to connected not only hosts but zones too. The rationale
123 behind a route between zone is exactly the same as for hosts. The only difference is
124 the 2 new gateway parameters in the syntax of :ref:`pf_tag_zoneroute`.
126 A zone is not a physical resource, just a collection of resources (including other zones).
127 Consequently, you need to describe the gateway, i.e. the physical resource inside the zone used for the route.
128 It gives you 4 parameters to describe a zoneRoute:
129 - src: The object of source zone
130 - dst: The object of destination zone
131 - gw_src: Gateway inside src zone. A Host (or router) belonging to src zone.
132 - gw_dst: Gateway inside dst zone. A Host (or router) belonging to dst zone.
133 - links: Links that connect gw_src to gw_dst.
135 .. note:: The gateways must be a component of the zone (either directly or member of some child sub-zone). SimGrid will verify these parameters when adding a route.
137 .. warning:: SimGrid does not have the concept of default gateway/router. Each zoneRoute must describe the appropriate gateways which may be different for each route.
139 Calculating the routes
140 **********************
142 This section is not mandatory for a normal SimGrid user. However, if you want
143 to know a little more of we calculate the route
144 between nodes inside SimGrid, keep reading it.
149 Intra-zone communications
150 =========================
152 This is the easy, happy case. When
153 a host wants to communicate with another host belonging to the same
154 zone, it is the zone's duty to find the list of links that are
155 involved in the communication.
157 As we stated earlier, each zone implements a different strategy, defined
158 through the ``routing=`` parameter.
159 - **Full**: all routes must be explicitly given using the
160 :ref:`pf_tag_route` and :ref:`pf_tag_link_ctn` tags (this :ref:`routing
161 model <pf_rm>` is both simple and inefficient :). It is OK to not
162 specify each and every route between hosts, as long as you do not try
163 to start a communication on any of the missing routes during your
165 - **Dijkstra/Floyd**: calculates the shortest path between each pair
166 of nodes using the routes described by the user (:ref:`pf_tag_route`).
167 As long as you graph is connected, no problems.
169 - Dijkstra: shortest-path calculated considering the path's latency. As
170 the latency of links can change during simulation, it's recomputed each
171 time a route is necessary.
173 - Floyd: use the number of hops to build shortest path. It's calculated only
174 once at the beginning of the simulation (as the platform is fixed).
176 - **Cluster/Fat-Tree/DragonFly/Torus**: routing is defined by the topology, automatically created.
177 These zones must be defined through the :ref:`pf_tag_cluster` tag in the XML.
178 - **Star**: star-like topology. Users describe routes from/to every host in the zone.
179 - **Vivaldi/Wi-Fi**: "fully-connected" zones with special characteristics.
183 Inter-zone communications
184 =========================
186 .. image:: ./img/zoom_comm.svg
189 Inter-zone communications are a little more complicated since you need to pass
190 through several zones. Let's have a look in more details in a communication
191 within our initial topology.
193 In this case, *Host1* within *AS2* wants to communicate with *Host2* from *AS5-4*.
194 As we can see, they're not part of the same zone nor have direct links connecting
195 them. The routing procedure is as follows:
197 1. **Find common root and ancestors**: As a SimGrid's platform is a tree of zones,
198 it is assured that we have a common zone that includes both hosts. Also, we need
199 the zone within the root zone that contains the hosts. In our case, we have:
201 - **Common root**: *AS1*, it is the root zone that contains all hosts in our example
203 - **Src ancestor**: *AS2*, it is the own *Host1's* zone.
205 - **Dst ancestor**: *AS5*, it's the *AS5* that contains *AS5-4*.
207 2. **Adding route from src to dst ancestor**: Ask *AS1* for the route between *AS2* and *AS5*.
209 This route is defined by the following configuration
213 <zoneRoute> src="AS2" dst="AS5" gw_src="Host1" gw_dst"="gw1">
214 <link_ctn id="Link1">
217 Add *Link1* to our list of links.
219 Also, we can see in this route that the gateway for *AS2* is *Host1* and for *AS5* is *gw1*.
221 Consequently, we need to go from *Host1* to *AS2*'s gateway (*Host1*) and from *Host2* to *AS5*'s
224 3. **Recursively search for route between hosts (Host1/Host2) and ancestors (AS2, AS5)**
226 3.1. **Route from Host1 to AS2's gateway (Host1)**: nothing to do, same zone.
228 3.2. **Route from Host2 to AS5's gateway (gw1)**: start step 1 again, searching
229 for a common root (*AS5* in this case) and the common ancestors (*AS5-4* and *AS5-3*).
231 This route is defined as follows.
235 <zoneRoute> src="AS5-4" dst="AS5-3" gw_src="gw2" gw_dst"="gw1">
236 <link_ctn id="Link3">
239 Add *Link3* to list of links.
241 4. **Add local links in src and dst zones**
243 4.1. **Route from Host1 to AS2's gateway**: same node, no link to add.
245 4.2. **Route from Host2 to AS5-4's gateway**: follow intra-zone and add *Link2*.
247 The last route, as it is an internal route in *AS5-4*, is defined using the :ref:`pf_tag_route` tag.
251 <route> src="Host2" dst="gw2">
252 <link_ctn id="Link2">
256 In the end, our communication from *Host1/AS2* to *Host2/AS5-4* will pass through
257 the links: *Link1, Link3* and *Link2*.
259 Note that a communication between *Host3/AS2* and *Host2/AS5-4* follow the same procedure, except
260 for step 4.1 where we would add the link between *Host3* and *Host1* inside *AS2* zone.
266 The link used of loopback communications has a special treatment in SimGrid. As it can be
267 quite tedious to describe each a loopback link for each host in the platform, SimGrid provides
268 a global **FATPIPE** link which is used by all hosts by default.
270 By default, this link has the following characteristics:
272 - **Bandwidth**: 10GBps. It can be changed through configuration, see :ref:`cfg=network/loopback`.
274 - **Latency**: 0ms. See :ref:`cfg=network/loopback` for more details.
278 These default values are arbitrary chosen and must be carefully configured to reflect
279 your environment if needed.
281 In addition, you can add :ref:`pf_tag_route` from a node to itself to modify the loopback link
282 for a specific node. In this case, SimGrid will get this link (instead of the global one) for
283 the local communications.
287 <link id="loopback" bandwidth="100MBps" latency="0"/>
288 <route src="Tremblay" dst="Tremblay">
289 <link_ctn id="loopback"/>
292 Finally, some zones (e.g. :ref:`pf_tag_cluster`) allow you to describe the characteristics of
293 the loopback nodes inside the zone. These links are equivalent to adding specific routes and
294 have higher priority than the global loopback link.
298 **Loopback links are used only for local communications**.
300 You may have noticed that we didn't include them at step 3.1 in :ref:`inter_zone`.
301 Loopback links will be used only when src and dst are the same, not in the recursive search