- bandwidth. <b>LINKS ARE NOT EDGES BUT HYPEREDGES</b>: it means
- that you can have more than 2 equipments connected to it.
-\li <b>router</b>: represents something that one message can be routed
- to, but does not accept any code, nor have any influence on the
- performances (no bandwidth, no latency, not anything).<b>ROUTERS
- ARE ENTITIES (ALMOST) IGNORED BY THE SIMULATOR WHEN THE SIMULATION
- HAS BEGUN</b>. If you want to represent something like a switch,
- you must use <b>link</b> (see section below). Routers are used in
- order to run some routing algorithm and determine routes (see
- routing section for details).
-
-let's see deeper what those entities hide.
-
-\subsubsection pf_router router
-As said before, <b>router</b> is used only to give some information
-for routing algorithms. So, it does not have any attributes except :
-
-<b>router</b> attributes :
-\li <b>id (mandatory)</b>: the identifier of the router to be used
- when referring to it.
- \li <b>coordinates</b>: you'll have to give it if you choose the
- vivaldi, coordinate-based routing model for the AS the host
- belongs to. More details about it in the P2P coordinates based
- section.
-
-<b>router example</b>
-\verbatim
- <router id="gw_dc1_horizdist"/>
-\endverbatim
-
-\subsubsection pf_link link
-
-Network links can represent one-hop network connections. They are
-characterized by their id and their bandwidth. The latency is optional
-with a default value of 0.0. For instance, we can declare a network
-link named link1 having bandwidth of 1Gb/s and a latency of 50µs.
-Example link:
-
-\verbatim
- <link id="LINK1" bandwidth="125000000" latency="5E-5"/>
-\endverbatim
-<b>Expressing sharing policy</b>
-
-By default a network link is SHARED, that is if more than one flow go
-through a link, each gets a share of the available bandwidth similar
-to the share TCP connections offers.
-
-Conversely if a link is defined as a FATPIPE, each flow going through
-this link will get all the available bandwidth, whatever the number of
-flows. The FATPIPE behavior allows to describe big backbones that
-won't affect performances (except latency). Finally a link can be
-considered as FULLDUPLEX, that means that in the simulator, 2 links
-(one named UP and the other DOWN) will be created for each link, so as
-the transfers from one side to the other will interact similarly as
-TCP when ACK returning packets circulate on the other direction. More
-discussion about it is available in <b>link_ctn</b> description.
-
-\verbatim
- <link id="SWITCH" bandwidth="125000000" latency="5E-5" sharing_policy="FATPIPE" />
-\endverbatim
-
-<b>Expressing dynamicity and failures</b>
-
-As for hosts, it is possible to declare links whose state, bandwidth
-or latency change over the time. In this case, the bandwidth and
-latency attributes are respectively replaced by the bandwidth file and
-latency file attributes and the corresponding text files.
-
-\verbatim
- <link id="LINK1" state_file="link1.fail" bandwidth="80000000" latency=".0001" bandwidth_file="link1.bw" latency_file="link1.lat" />
-\endverbatim
-
-It has to be noted that even if the syntax is the same, the semantic
-of bandwidth and latency trace files differs from that of host
-availability files. Those files do not express availability as a
-fraction of the available capacity but directly in bytes per seconds
-for the bandwidth and in seconds for the latency. This is because most
-tools allowing to capture traces on real platforms (such as NWS)
-express their results this way.
-
-<b>Example of "link1.bw" file</b>
-\verbatim
-
-1 PERIODICITY 12.0
-2 4.0 40000000
-3 8.0 60000000
-\endverbatim
-<b>Example of "link1.lat" file</b>
-\verbatim
- 1 PERIODICITY 5.0
-2 1.0 0.001
-3 2.0 0.01
-4 3.0 0.001
-\endverbatim
-
-In this example, the bandwidth varies with a period of 12 seconds
-while the latency varies with a period of 5 seconds. At the beginning
-of simulation, the link’s bandwidth is of 80,000,000 B/s (i.e., 80
-Mb/s). After four seconds, it drops at 40 Mb/s, and climbs back to 60
-Mb/s after eight seconds. It keeps that way until second 12 (ie, until
-the end of the period), point at which it loops its behavior (seconds
-12-16 will experience 80 Mb/s, 16-20 40 Mb/s and so on). In the same
-time, the latency values are 100µs (initial value) on the [0, 1[ time
-interval, 1ms on [1, 2[, 10ms on [2, 3[, 1ms on [3,5[ (i.e., until the
-end of period). It then loops back, starting at 100µs for one second.
-
-<b>link</b> attributes :
-\li <b>id (mandatory)</b>: the identifier of the link to be used when referring to it.
-\li <b>bandwidth (mandatory)</b>: bandwidth for the link.
-\li <b>lat </b>: latency for the link. Default is 0.0.
-\li <b>sharing_policy</b>: sharing policy for the link.
-\li <b>state</b>: Allow you to to set link as ON or OFF. Default is ON.
-\li <b>bandwidth_file</b>: Allow you to use a file as input for bandwidth.
-\li <b>latency_file</b>: Allow you to use a file as input for latency.
-\li <b>state_file</b>: Allow you to use a file as input for states.
-
-As an host, a <b>link</b> tag can also contain the <b>prop</b> tag.
-
-<b>link example</b>
-\verbatim
- <link id="link1" bandwidth="125000000" latency="0.000100"/>
-\endverbatim
-
-
-\subsection pf_storage Storage
-
-<b>Note : This is a prototype version that should evolve quickly, this
-is just some doc valuable only at the time of writing this doc</b>
-This section describes the storage management under SimGrid ; nowadays
-it's only usable with MSG. It relies basically on linux-like concepts.
-You also may want to have a look to its corresponding section in \ref
-msg_file_management ; functions access are organized as a POSIX-like
-interface.
-
-\subsubsection pf_sto_conc Storage Main concepts
-Basically there is 3 different entities to know :
-\li the <b>storage_type</b>: here you define some kind of storage that
- you will instantiate many type on your platform. Think of it like
- a definition of throughput of a specific disk.
-\li the <b>storage</b>: instance of a <b>storage_type</b>. Defines a
- new storage of <b>storage_type</b>
-\li the <b>mount</b>: says that the storage is located into this
- specific resource.
-
-the content of a storage has to be defined in a content file that
-contains the content. The path to this file has to be passed within
-the <b>content</b> attribute . Here is a way to generate it:
-
-\verbatim
-find /path/you/want -type f -exec ls -l {} \; 2>/dev/null > ./content.txt
-\endverbatim
-
-\subsubsection pf_sto_sttp storage_type
-
-
-<b>storage_type</b> attributes :
-\li <b>id (mandatory)</b>: the identifier of the storage_type to be
- used when referring to it.
-\li <b>model (mandatory)</b>: Unused for now by the simulator (but
- mandatory, ok)
-\li <b>content</b>: default value 0. The file containing the disk
- content. (may be moved soon or later to <b>storage</b> tag.
-
-The tag must contains some predefined prop, as may do some other
-resources tags. This should moved to attributes soon or later.
-<b>storage_type</b> mandatory <b>prop</b> :
-\li <b>Bwrite</b>: value in B/s. Write throughput
-\li <b>Bread</b>: value in B/s. Read throughput
-\li <b>Bconnexion</b>: value in B/s. Connection throughput (i.e. the
- throughput of the storage connector).
-
-\subsubsection pf_sto_st storage
-
-<b>storage_type</b> attributes :
-\li <b>id (mandatory)</b>: the identifier of the storage to be used
- when referring to it.
-\li <b>typeId (mandatory)</b>: the identifier of the storage_type that
- this storage belongs to.
-
-
-\subsubsection pf_sto_mo mount
-
-<b>mount</b> attributes :
-\li <b>id (mandatory)</b>: the id of the <b>storage</b> that must be
- mounted on that computer.
-\li <b>name (mandatory)</b>: the name that will be the logical
- reference to this disk (the mount point).
-
-\subsubsection pf_sto_mst mstorage
-<b>Note : unused for now</b>
-<b>mstorage</b> attributes :
-\li <b>typeId (mandatory)</b>: the id of the <b>storage</b> that must
- be mounted on that computer.
-\li <b>name (mandatory)</b>: the name that will be the logical
- reference to this disk (the mount point).
-
-\section pf_routing Routing
-
-In order to run fast, it has been chosen to use static routing within
-SimGrid. By static, it means that it is calculated once (or almost),
-and will not change during execution. We chose to do that because it
-is rare to have a real deficiency of a resource ; most of the time, a
-communication fails because the links are too overloaded, and so your
-connection stops before the time out, or because the computer at the
-other end is not answering.
+ bandwidth.
+@remark
+ The concept of links in SimGrid may not be intuitive, as links are not
+ limited to connecting (exactly) two entities; in fact, you can have more than
+ two equipments connected to it. (In graph theoretical terms: A link in
+ SimGrid is not an edge, but a hyperedge)
+
+2. ``<router/>``: Represents an entity that a message can be routed
+ to, but that is unable to execute any code. In SimGrid, routers have also
+ no impact on the performance: Routers do not limit any bandwidth nor
+ do they increase latency. As a matter of fact, routers are (almost) ignored
+ by the simulator when the simulation has begun.
+
+3. ``<backbone/>``: This tag is only available when the containing network zone is
+ used as a cluster (i.e., mode="Cluster")
+
+@remark
+ If you want to represent an entity like a switch, you must use ``<link>`` (see section). Routers are used
+ to run some routing algorithm and determine routes (see Section @ref pf_routing for details).
+
+@subsubsection pf_backbone <backbone/>
+
+@note
+ This tag is <b>only available</b> when the containing network zone uses the "Cluster" routing mode!
+
+Using this tag, you can designate an already existing link to be a backbone.
+
+Attribute name | Mandatory | Values | Description
+--------------- | --------- | ------ | -----------
+id | yes | string | Name of the link that is supposed to act as a backbone.
+
+@subsection pf_storage Storage
+
+@note
+ This is a prototype version that should evolve quickly, hence this
+ is just some doc valuable only at the time of writing.
+ This section describes the storage management under SimGrid ; nowadays
+ it's only usable with MSG. It relies basically on linux-like concepts.
+ You also may want to have a look to its corresponding section in
+ @ref msg_file ; access functions are organized as a POSIX-like
+ interface.
+
+@subsubsection pf_sto_conc Storage - Main Concepts
+
+The storage facilities implemented in SimGrid help to model (and account for)
+storage devices, such as tapes, hard-drives, CD or DVD devices etc.
+A typical situation is depicted in the figure below:
+
+@image html ./webcruft/storage_sample_scenario.png
+@image latex ./webcruft/storage_sample_scenario.png "storage_sample_scenario" width=@textwidth
+
+In this figure, two hosts called Bob and Alice are interconnected via a network
+and each host is physically attached to a disk; it is not only possible for each host to
+mount the disk they are attached to directly, but they can also mount disks
+that are in a remote location. In this example, Bob mounts Alice's disk remotely
+and accesses the storage via the network.
+
+SimGrid provides 3 different entities that can be used to model setups
+that include storage facilities:
+
+Entity name | Description
+--------------- | -----------
+@ref pf_storage_entity_storage_type "storage_type" | Defines a template for a particular kind of storage (such as a hard-drive) and specifies important features of the storage, such as capacity, performance (read/write), contents, ... Different models of hard-drives use different storage_types (because the difference between an SSD and an HDD does matter), as they differ in some specifications (e.g., different sizes or read/write performance).
+@ref pf_tag_storage "storage" | Defines an actual instance of a storage type (disk, RAM, ...); uses a ``storage_type`` template (see line above) so that you don't need to re-specify the same details over and over again.
+@ref pf_tag_mount "mount" | Must be wrapped by a @ref pf_tag_host tag; declares which storage(s) this host has mounted and where (i.e., the mountpoint).
+
+
+@anchor pf_storage_content_file
+### %Storage Content File ###
+
+In order to assess exactly how much time is spent reading from the storage,
+SimGrid needs to know what is stored on the storage device (identified by distinct (file-)name, like in a file system)
+and what size this content has.
+
+@note
+ The content file is never changed by the simulation; it is parsed once
+ per simulation and kept in memory afterwards. When the content of the
+ storage changes, only the internal SimGrid data structures change.
+
+@anchor pf_storage_content_file_structure
+#### Structure of a %Storage Content File ####
+
+Here is an excerpt from two storage content file; if you want to see the whole file, check
+the file ``examples/platforms/content/storage_content.txt`` that comes with the
+SimGrid source code.
+
+SimGrid essentially supports two different formats: UNIX-style filepaths should
+follow the well known format:
+
+@verbatim
+/lib/libsimgrid.so.3.6.2 12710497
+/bin/smpicc 918
+/bin/smpirun 7292
+/bin/smpif2c 1990
+/bin/simgrid_update_xml 5018
+/bin/graphicator 66986
+/bin/simgrid-colorizer 2993
+/bin/smpiff 820
+/bin/tesh 356434
+@endverbatim
+
+Windows filepaths, unsurprisingly, use the windows style:
+
+@verbatim
+@Windows@avastSS.scr 41664
+@Windows@bfsvc.exe 75264
+@Windows@bootstat.dat 67584
+@Windows@CoreSingleLanguage.xml 31497
+@Windows@csup.txt 12
+@Windows@dchcfg64.exe 335464
+@Windows@dcmdev64.exe 93288
+@endverbatim
+
+@note
+ The different file formats come at a cost; in version 3.12 (and most likely
+ in later versions, too), copying files from windows-style storages to unix-style
+ storages (and vice versa) is not supported.
+
+@anchor pf_storage_content_file_create
+#### Generate a %Storage Content File ####
+
+If you want to generate a storage content file based on your own filesystem (or at least a filesystem you have access to),
+try running this command (works only on unix systems):
+
+@verbatim
+find . -type f -exec ls -1s --block=1 {} @; 2>/dev/null | awk '{ print $2 " " $1}' > ./content.txt
+@endverbatim
+
+@subsubsection pf_storage_entities The Storage Entities
+
+These are the entities that you can use in your platform files to include
+storage in your model. See also the list of our @ref pf_storage_example_files "example files";
+these might also help you to get started.
+
+@anchor pf_storage_entity_storage_type
+#### @<storage_type@> ####
+
+Attribute name | Mandatory | Values | Description
+--------------- | --------- | ------ | -----------
+id | yes | string | Identifier of this storage_type; used when referring to it
+model | no | string | In the future, this will allow to change the performance model to use
+size | yes | string | Specifies the amount of available storage space; you can specify storage like "500GiB" or "500GB" if you want. (TODO add a link to all the available abbreviations)
+content | yes | string | Path to a @ref pf_storage_content_file "Storage Content File" on your system. This file must exist.
+
+This tag must contain some predefined model properties, specified via the <model_prop> tag. Here is a list,
+see below for an example:
+
+Property id | Mandatory | Values | Description
+--------------- | --------- | ------ | -----------
+Bwrite | yes | string | Bandwidth for write access; in B/s (but you can also specify e.g. "30MBps")
+Bread | yes | string | Bandwidth for read access; in B/s (but you can also specify e.g. "30MBps")
+
+@note
+ A storage_type can also contain the <b><prop></b> tag. The <prop> tag allows you
+ to associate additional information to this <storage_type> and follows the
+ attribute/value schema; see the example below. You may want to use it to give information to
+ the tool you use for rendering your simulation, for example.
+
+Here is a complete example for the ``storage_type`` tag:
+@verbatim
+<storage_type id="single_HDD" size="4000">
+ <model_prop id="Bwrite" value="30MBps" />
+ <model_prop id="Bread" value="100MBps" />
+ <prop id="Brand" value="Western Digital" />
+</storage_type>
+@endverbatim
+
+@subsubsection pf_tag_storage <storage>
+
+Attributes | Mandatory | Values | Description
+-------------- | --------- | ------ | -----------
+id | yes | string | Identifier of this ``storage``; used when referring to it
+typeId | yes | string | Here you need to refer to an already existing @ref pf_storage_entity_storage_type "@<storage_type@>"; the storage entity defined by this tag will then inherit the properties defined there.
+attach | yes | string | Name of a host (see Section @ref pf_tag_host) to which this storage is <i>physically</i> attached to (e.g., a hard drive in a computer)
+content | no | string | When specified, overwrites the content attribute of @ref pf_storage_entity_storage_type "@<storage_type@>"
+
+Here are two examples:
+
+@verbatim
+ <storage id="Disk1" typeId="single_HDD" attach="bob" />
+
+ <storage id="Disk2" typeId="single_SSD"
+ content="content/win_storage_content.txt" />
+@endverbatim
+
+The first example is straightforward: A disk is defined and called "Disk1"; it is
+of type "single_HDD" (shown as an example of @ref pf_storage_entity_storage_type "@<storage_type@>" above) and attached
+to a host called "bob" (the definition of this host is omitted here).
+
+The second storage is called "Disk2", is still of the same type as Disk1 but
+now specifies a new content file (so the contents will be different from Disk1)
+and the filesystem uses the windows style; finally, it is attached to a second host,
+called alice (which is again not defined here).
+
+@subsubsection pf_tag_mount <mount>
+
+| Attribute | Mandatory | Values | Description |
+| ----------- | ----------- | -------- | ------------- |
+| id | yes | string | Refers to a @ref pf_tag_storage "<storage>" entity that will be mounted on that computer |
+| name | yes | string | Path/location to/of the logical reference (mount point) of this disk
+
+This tag must be enclosed by a @ref pf_tag_host tag. It then specifies where the mountpoint of a given storage device (defined by the ``id`` attribute)
+is; this location is specified by the ``name`` attribute.
+
+Here is a simple example, taken from the file ``examples/platform/storage.xml``:
+
+@verbatim
+ <storage_type id="single_SSD" size="500GiB">
+ <model_prop id="Bwrite" value="60MBps" />
+ <model_prop id="Bread" value="200MBps" />
+ </storage_type>
+
+ <storage id="Disk2" typeId="single_SSD"
+ content="content/win_storage_content.txt"
+ attach="alice" />
+ <storage id="Disk4" typeId="single_SSD"
+ content="content/small_content.txt"
+ attach="denise"/>
+
+ <host id="alice" speed="1Gf">
+ <mount storageId="Disk2" name="c:"/>
+ </host>
+
+ <host id="denise" speed="1Gf">
+ <mount storageId="Disk2" name="c:"/>
+ <mount storageId="Disk4" name="/home"/>
+ </host>
+@endverbatim
+
+This example is quite interesting, as the same device, called "Disk2", is mounted by
+two hosts at the same time! Note, however, that the host called ``alice`` is actually
+attached to this storage, as can be seen in the @ref pf_tag_storage "<storage>"
+tag. This means that ``denise`` must access this storage through the network, but SimGrid automatically takes
+care of that for you.
+
+Furthermore, this example shows that ``denise`` has mounted two storages with different
+filesystem types (unix and windows). In general, a host can mount as many storage devices as
+required.
+
+@note
+ Again, the difference between ``attach`` and ``mount`` is simply that
+ an attached storage is always physically inside (or connected to) that machine;
+ for instance, a USB stick is attached to one and only one machine (where it's plugged-in)
+ but it can only be mounted on others, as mounted storage can also be a remote location.
+
+###### Example files #####
+
+@verbinclude example_filelist_xmltag_mount
+
+@subsubsection pf_storage_example_files Example files
+
+Several examples were already discussed above; if you're interested in full examples,
+check the the following platforms:
+
+1. ``examples/platforms/storage.xml``
+2. ``examples/platforms/remote_io.xml``
+
+If you're looking for some examplary C code, you may find the source code
+available in the directory ``examples/msg/io/`` useful.
+
+@subsubsection pf_storage_examples_modelling Modelling different situations
+
+The storage functionality of SimGrid is type-agnostic, that is, the implementation
+does not presume any type of storage, such as HDDs/SSDs, RAM,
+CD/DVD devices, USB sticks etc.
+
+This allows the user to apply the simulator for a wide variety of scenarios; one
+common scenario would be the access of remote RAM.
+
+#### Modelling the access of remote RAM ####
+
+How can this be achieved in SimGrid? Let's assume we have a setup where three hosts
+(HostA, HostB, HostC) need to access remote RAM:
+
+@verbatim
+ Host A
+ /
+RAM -- Host B
+ @
+ Host C
+@endverbatim
+
+An easy way to model this scenario is to setup and define the RAM via the
+@ref pf_tag_storage "storage" and @ref pf_storage_entity_storage_type "storage type"
+entities and attach it to a remote dummy host; then, every host can have their own links
+to this host (modelling for instance certain scenarios, such as PCIe ...)
+
+@verbatim
+ Host A
+ /
+RAM - Dummy -- Host B
+ @
+ Host C
+@endverbatim
+
+Now, if read from this storage, the host that mounts this storage
+communicates to the dummy host which reads from RAM and
+sends the information back.
+
+
+@section pf_routing Routing
+
+To achieve high performance, the routing tables used within SimGrid are
+static. This means that routing between two nodes is calculated once
+and will not change during execution. The SimGrid team chose to use this
+approach as it is rare to have a real deficiency of a resource;
+most of the time, a communication fails because the links experience too much
+congestion and hence, your connection stops before the timeout or
+because the computer designated to be the destination of that message
+is not responding.