From f4a852ede5720bc6cbbd71a1bef959d04718a09f Mon Sep 17 00:00:00 2001 From: Martin Quinson Date: Tue, 16 Oct 2018 22:11:43 +0200 Subject: [PATCH] docs: new section on M&S hints --- doc/doxygen/howtos.doc | 132 ----------------------- docs/source/platform.rst | 6 -- docs/source/platform_howtos.rst | 175 +++++++++++++++++++++++++++++++ tools/cmake/DefinePackages.cmake | 4 +- 4 files changed, 178 insertions(+), 139 deletions(-) delete mode 100644 doc/doxygen/howtos.doc create mode 100644 docs/source/platform_howtos.rst diff --git a/doc/doxygen/howtos.doc b/doc/doxygen/howtos.doc deleted file mode 100644 index a764b7af8f..0000000000 --- a/doc/doxygen/howtos.doc +++ /dev/null @@ -1,132 +0,0 @@ -/** @page howto Use Cases and Howtos - -@tableofcontents - -There is no perfect model, but only models that are adapted to the -specific study that you want to do. SimGrid provide several advanced -mechanisms that you can adapt to model the situation that you are -interested into, and it is often uneasy to see where to start with. - -This page collects several hints and tricks on modeling situations. -Most probably, the exact situation that you want to model will not be -described here (unless we already [scooped -you](http://www.phdcomics.com/comics/archive.php?comicid=789)), but we -hope you will find some ideas and see how to use the modeling options -that SimGrid provides. - -Of course, you should also check the @ref examples page, that shows -more detailed usage examples. As for the rest of the documentation -(and of SimGrid, actually), any contribution is welcome. - -@section howto_churn How to model churn - -One of the biggest challenges in P2P settings is to cope with the -churn, meaning that resources keep appearing and disappearing. In -SimGrid, you can always change the state of each host manually, with -eg simgrid::s4u::Host::on(). To reduce the burden when the churn is -high, you can also attach a **state profile** to the host directly. - -This is not possible from S4U yet (TODO), and you should use the \c -state_file attribute of @ref pf_tag_host or @ref pf_tag_cluster. - -Every lines (but the last) of such files describe timed events with -the form "date value". Example: - -~~~{.py} - 1 0 - 2 1 - LOOPAFTER 8 -~~~ - - - At time t=1, the host is turned off (value 0 means OFF) - - At time t=2, it is turned back on (other values means ON) - - At time t=10, the history is reset (because that's 8 seconds after - the last event). So the host will be turned off again at t=11. - -If your trace does not contain a LOOPAFTER line, then your profile is -only executed once and not repetitively. - -@section howto_multicore How to model multicore machines - -Multicore machines are very complex, and there is many way to model -them. The default models of SimGrid are coarse grain and capture some -elements of this reality. Here is how to declare simple multicore hosts: - -@code{xml} - -@endcode - -It declares a 4-cores host called "mymachine", each core computing 8 -GFlops per second. If you put one activity of 8 GFlop on this host, it -will be computed in 1 second (by default, activities are -single-threaded and cannot leverage the computing power of more than -one core). If you put two of them together, they will still be -computed in one second, and so on up to 4 tasks. If you put 5 tasks, -they will share the total computing resource, and all tasks will be -computed at 5/4 = 1.25 second. That's a very simple model, but that's -all what you will get by default from SimGrid. - -@subsection howto_multicore_pinning Pinning tasks to cores - -This model does not account for any sort of resource reservation such -as task pinning (where you manually select on which core each of the -existing activity should execute). If you want to go that way, the -best is probably to model you 4-core processor as 4 separte hosts, and -assigning the activities to cores by migrating them to the declared -hosts. In some sense, this is takes the whole Network-On-Chip idea -really seriously. - -Some extra complications may arise if you go that way. If you have -more tasks than cores, you'll have to [schedule your -tasks](https://en.wikipedia.org/wiki/Scheduling_%28computing%29#Operating_system_process_scheduler_implementations) -yourself on the cores (so don't do that if you can avoid). Since you -cannot have more than one network model, you will end up with a TCP -connexion between your cores. You probably want to work this around in -your application by never starting any simulated communication between -your cores. Instead, connect equally all cores to the external -network. - -Modeling a multicore CPU as a set of SimGrid hosts may seem strange -and unconvincing, but some users achieved very realistic simulations -of multi-core and GPU machines this way. - -@section howto_bootenergy Modeling machine bootup and shutdown periods - -When a physical host boots up, a lot of things happen. It takes time -during which the machine is not usable but dissipates energy, and -programs actually die and restart during a reboot. Since there is many -ways to model it, SimGrid does not do any modeling choice for you but -the most obvious ones. - -Any actor (or process in MSG) running on an host that is shut down -will be killed and all its activities (tasks in MSG) will be -automatically canceled. If killed the actor was marked as -auto-restartable (with simgrid::s4u::Actor::setAutoRestart() or with -MSG_process_auto_restart_set()), it will start anew with the same -parameters when the host boots back up. - -By default, shutdowns and bootups are instantaneous. If you want to -add an extra delay, you have to do that yourself, for example from an -/controler/ actor that runs on another host. The best way to do so is -to declare a fictionous pstate where the CPU delivers 0 flop per -second (so every activity on that host will be frozen when the host is -in this pstate). When you want to switch the host off, your controler -switches the host to that specific pstate (with -simgrid::s4u::Host::setPstate()), waits for the amount of time that -you decided necessary for your host to shut down, and turns the host -off (with simgrid::s4u::Host::turnOff()). To boot up, switch the host -on, go into the specific pstate, wait a while and go to a more regular -pstate. - -To model the energy dissipation, you need to put the right energy -consumption in your startup/shutdown specific pstate. Remember that -the energy consumed is equal to the instantaneous consumption -multiplied by the time in which the host keeps in that state. Do the -maths, and set the right instantaneous consumption to your pstate, and -you'll get the whole boot period to consume the amount of energy that -you want. You may want to have one fictionous pstate for the bootup -period and another one for the shutdown period. - -Of course, this is only one possible way to model these things. YMMV ;) - -*/ \ No newline at end of file diff --git a/docs/source/platform.rst b/docs/source/platform.rst index b6c1cb368e..761788921a 100644 --- a/docs/source/platform.rst +++ b/docs/source/platform.rst @@ -41,11 +41,5 @@ scripting language ` instead. In the future, it should be possible to describe the platform in python or directly in C++, but this is not possible yet. -As usual, SimGrid is a versatile framework, and you should find the -way of describing your platform that best fits your experimental -practice. - - - .. LocalWords: SimGrid diff --git a/docs/source/platform_howtos.rst b/docs/source/platform_howtos.rst new file mode 100644 index 0000000000..9612964df7 --- /dev/null +++ b/docs/source/platform_howtos.rst @@ -0,0 +1,175 @@ +.. _platform: + +.. raw:: html + + + +
+
+ +Modeling Hints +############## + +There is no perfect model, but only models that are adapted to the +specific study that you want to do. SimGrid provide several advanced +mechanisms that you can adapt to model the situation that you are +interested into, and it is often uneasy to see where to start with. +This page collects several hints and tricks on modeling situations. +Even if you are looking for a very advanced, specific use case, these +examples may help you to design the solution you need. + +Doing Science with SimGrid +************************** + +Many users are using SimGrid as a scientific instrument for their +research. This tool was indeed invented to that extend, and we strive +to streamline this kind of usage. But SimGrid is no magical tool, and +it is of your responsability that the tool actually provides sensible +results. Fortunately, there is a vast literature on how to avoid +Modeling & Simulations pitfalls. We review here two particular works. + +In `An Integrated Approach to Evaluating Simulation Credibility +`_, the authors +provide a methodology enabling the users to increase their confidence +in the simulation tools they use. First of all, you must know what you +actually expect to discover whether the tool actually covers your +needs. Then, as they say, "a fool with a tool is still a fool", so you +need to think about your methodology before you submit your articles. +You really should read this article, that contains many other +interesting advices. + +`Seven Pitfalls in Modeling and Simulation Research +`_ is even more +specific. Here are the listed pitfalls: (1) Don't know whether it's +modeling or simulation, (2) No separation of concerns, (3) No clear +scientific question, (4) Implementing everything from scratch, (5) +Unsupported claims, (6) Toy duck approach, and (7) The tunnel view. As +you can see, this article is a must read. It's a pitty that it's not +freely available, though. + +Modeling Churn in P2P +********************* + +One of the biggest challenges in P2P settings is to cope with the +churn, meaning that resources keep appearing and disappearing. In +SimGrid, you can always change the state of each host manually, with +eg :cpp:func:`simgrid::s4u::Host::turn_on`. To reduce the burden when +the churn is high, you can also attach a **state profile** to the host +directly. + +This is not possible from S4U yet (TODO), and you should use the +``state_file`` attribute of :ref:`pf_tag_host`, :ref:`pf_tag_cluster` +or :ref:`pf_tag_link`. + +Every lines (but the last) of such files describe timed events with +the form "date value". Example: + +.. code-block:: python + + 1 0 + 2 1 + LOOPAFTER 8 + +- At time t=1, the host is turned off (value 0 means OFF) +- At time t=2, it is turned back on (other values means ON) +- At time t=10, the history is reset (because that's 8 seconds after + the last event). So the host will be turned off again at t=11. + +If your trace does not contain a LOOPAFTER line, then your profile is +only executed once and not repetitively. + + +Modeling Multicore Machines +*************************** + +Default Model +============= + +Multicore machines are very complex, and there is many way to model +them. The default models of SimGrid are coarse grain and capture some +elements of this reality. Here is how to declare simple multicore hosts: + +.. code-block:: xml + + + +It declares a 4-cores host called "mymachine", each core computing 8 +GFlops per second. If you put one activity of 8 GFlop on this host, it +will be computed in 1 second (by default, activities are +single-threaded and cannot leverage the computing power of more than +one core). If you put two of them together, they will still be +computed in one second, and so on up to 4 tasks. If you put 5 tasks, +they will share the total computing resource, and all tasks will be +computed at 5/4 = 1.25 second. That's a very simple model, but that's +all what you will get by default from SimGrid. + +Pinning tasks to cores +====================== + +The default model does not account for task pinning, where you +manually select on which core each of the existing activity should +execute. The best solution to model this is probably to model your +4-core processor as 4 separte hosts, and assigning the activities to +cores by migrating them to the declared hosts. In some sense, this +takes the whole Network-On-Chip idea really seriously. + +Some extra complications may arise here. If you have more tasks than +cores, you'll have to `schedule your tasks +`_ +yourself on the cores (so you'd better avoid this complexity). Since +you cannot have more than one network model in a given SimGrid +simulation, you will end up with a TCP connexion between your cores. A +possible work around is to never start any simulated communication +between the cores and have the same routes from each core to the +rest of the external network. + +Modeling a multicore CPU as a set of SimGrid hosts may seem strange +and unconvincing, but some users achieved very realistic simulations +of multi-core and GPU machines this way. + +Modeling machine bootup and shutdown periods +******************************************** + +When a physical host boots up, a lot of things happen. It takes time +during which the machine is not usable but dissipates energy, and +programs actually die and restart during a reboot. Since there is many +ways to model it, SimGrid does not do any modeling choice for you but +the most obvious ones. + +Any actor (or process in MSG) running on an host that is shut down +will be killed and all its activities (tasks in MSG) will be +automatically canceled. If killed the actor was marked as +auto-restartable (with +:cpp:func:`simgrid::s4u::Actor::set_auto_restart` or with +:cpp:func:`MSG_process_auto_restart_set`), it will start anew with the +same parameters when the host boots back up. + +By default, shutdowns and bootups are instantaneous. If you want to +add an extra delay, you have to do that yourself, for example from an +`controler` actor that runs on another host. The best way to do so is +to declare a fictionous pstate where the CPU delivers 0 flop per +second (so every activity on that host will be frozen when the host is +in this pstate). When you want to switch the host off, your controler +switches the host to that specific pstate (with +:cpp:func:`simgrid::s4u::Host::set_pstate`), waits for the amount of +time that you decided necessary for your host to shut down, and turns +the host off (with :cpp:func:`simgrid::s4u::Host::turn_off`). To boot +up, switch the host on, go into the specific pstate, wait a while and +go to a more regular pstate. + +To model the energy dissipation, you need to put the right energy +consumption in your startup/shutdown specific pstate. Remember that +the energy consumed is equal to the instantaneous consumption +multiplied by the time in which the host keeps in that state. Do the +maths, and set the right instantaneous consumption to your pstate, and +you'll get the whole boot period to consume the amount of energy that +you want. You may want to have one fictionous pstate for the bootup +period and another one for the shutdown period. + +Of course, this is only one possible way to model these things. YMMV ;) + diff --git a/tools/cmake/DefinePackages.cmake b/tools/cmake/DefinePackages.cmake index 628e0786ff..188bc2d93c 100644 --- a/tools/cmake/DefinePackages.cmake +++ b/tools/cmake/DefinePackages.cmake @@ -865,7 +865,6 @@ set(DOC_SOURCES doc/doxygen/deployment.doc doc/doxygen/footer.html doc/doxygen/header.html - doc/doxygen/howtos.doc doc/doxygen/index.doc doc/doxygen/inside.doc doc/doxygen/inside_tests.doc @@ -924,6 +923,9 @@ set(DOC_SOURCES docs/source/intro_yours.rst docs/source/models.rst docs/source/platform.rst + docs/source/platform_examples.rst + docs/source/platform_howtos.rst + docs/source/platform_reference.rst docs/source/scenar_config.rst docs/source/scenario.rst -- 2.20.1