These changes should be transparent to the users of the MSG, SMPI and SimDag interfaces, even if some buggy features were reworked
while some new features were added.
-Users are expected to switch to this new version at their own pace, as we do not have the manpower to do any bugfixing on the old releases.
+Users are expected to switch to this new version at their own pace, as we do not have the manpower to do any bug fixing on the old releases.
This version was tested on several Linux distributions, Mac OSX, Windows (restricted to the Java bindings when not using the Ubuntu
subsystem of Win10), FreeBSD and NetBSD.
* Java: Massive memleaks and performance issues fixed.
* Plus the usual bug fixes, cleanups and documentation improvements.
-Users are expected to switch to this new version at their own pace, as we do not have the manpower to do any bugfixing on the old releases.
+Users are expected to switch to this new version at their own pace, as we do not have the manpower to do any bug fixing on the old releases.
This version was tested on several Linux distributions, Mac OSX, Windows (restricted to our Java bindings), FreeBSD and NetBSD.
None of our 600+ integration tests is known to fail on any of these.
On the Cloud side, we implemented multi-core VMs, which naturally acts as containers of processes;
S4U, the future interface of SimGrid 4.0 also progressed quite a bit.
-The Storage is currently cleaned up by Fred, and some API changes are to be expected. We are sorry but the exisiting API is so crappy that
+The Storage is currently cleaned up by Fred, and some API changes are to be expected. We are sorry but the existing API is so crappy that
nobody ever used it, I guess. If you need it, please speak up soon!
We renamed AS into NetZone in the XML files (but the old one still work so no rush to update your platform). Since our platforms are
hierarchical, it makes no sense to name our zones "Autonomous Systems". Some other documentation pages got updated and modified. If
you see a particularly old, misleading or otherwise ugly doc page, please tell us and we'll fix it. Even typo reports are welcome.
-But most of the work on S4U is not directly visible from the user. We rewamped the whole code of activities (comms, execs, mutex, etc) to
+But most of the work on S4U is not directly visible from the user. We revamped the whole code of activities (comms, execs, mutex, etc) to
not mix the applicative logic (calling test() or wait()) with the object refcounting. Now, you can test your comm object as many time as
you want. This change was really intrusive in our internals, and we're not done with restabilizing every bits, but we're on it.
Still on the S4U front, we managed to remove a few more XBT modules. We prefer to use the std or boost libraries nowadays, and switching
-away from the XBT module enable to reduce our maintainance burden. Be warned that XBT will not always remain included in SimGrid.
+away from the XBT module enable to reduce our maintenance burden. Be warned that XBT will not always remain included in SimGrid.
On the infrastructure side, we are trying to setup a regular build task for the main projects using SimGrid, to check that our changes
don't break them. The one of StarPU is close to be working (even if not completely). If you want to have your own code tested regularly
SimGrid4: Existing interfaces remain unchanged, but the new S4U interface is budding and the internals are deeply reorganized.
Since 2015, we work hard to reduce the changes to public APIs. When we need to rename a public library symbol in S4U, we let your compiler
-issue an explicative warning when you use the deprecated function. These messages remain for four releases, i.e. for one full year,
+issue an explicit warning when you use the deprecated function. These messages remain for four releases, i.e. for one full year,
before turning into an error. Starting with v3.15, your can also adapt to API changes with the SIMGRID_VERSION macro, that is defined to
31500 for v3.15, to 31901 for v3.19.1 and so on.
Starting with this v3.19.1, our commitment to reduce the changes to the public interfaces is extended from the API to the ABI: a program
-using only MSG or SimDag and compiled against a given version of simgrid can probably be used with a later version of SimGrid without
-recompilation. We will do our best... but don't expect too much of it, that's a really difficult goal during such profund refactoring.
+using only MSG or SimDag and compiled against a given version of SimGrid can probably be used with a later version of SimGrid without
+recompilation. We will do our best... but don't expect too much of it, that's a really difficult goal during such profound refactoring.
The difference between v3.19 and v3.19.1 is that the former was accidentally breaking the ABI of MSG, while the later is restoring the
previous ABI.
Version 3.22 (April 2. 2019)
----------------------------
-The Easter Christmas Release. It was expected from Chrismas, but I was so late that I even managed to miss the spring deadline.
+The Easter Christmas Release. It was expected from Christmas, but I was so late that I even managed to miss the spring deadline.
This started to be a running joke, so I decided to release it for April 1. But I'm even late for this... Sorry :)
.. rst-class:: compact-list
for the existing users). Their examples are now hidden in deprecated/ Please switch to S4U if you like C++ or to Python if not.
This new version also introduce a heavy load of internal cleanups. Fred converted more internals to real C++, with more classes and less
-procedural cruft. Henri and the other Wrench developpers reported many bugs around activity canceling and resource failures, and we fixed
+procedural cruft. Henri and the other Wrench developers reported many bugs around activity canceling and resource failures, and we fixed
quite a bit of them, but many dark snakes remain in that lake. Fred and Martin converted more doc to the new system (the platform chapter
is not finished, but it's not worse than the old one either) while Augustin completed the tutorial for MPI applications. Augustin also
added several non-blocking collectives to SMPI, even if Martin finally decided to release right before he could complete the last ones
version number: v3.23.1 then 3.23.3 until yesterday, and soon 3.24.1.
As a user, there is no urgency to upgrade, even if you should not wait more than 9 months to upgrade to another stable version: our policy is
-to keep backward compatibility and nice upgrading pathes for 3 stable versions. v3.24 removes symbols that got deprecated in v3.20, last
+to keep backward compatibility and nice upgrading patches for 3 stable versions. v3.24 removes symbols that got deprecated in v3.20, last
year. It deprecates things that will continue to work until v3.27.
Speaking of deprecation, we would like to hear from you if you are using the Java bindings under Windows without the WSL installed.
Version 3.25 (Feb 2. 2020)
--------------------------
-This is the "Palindrom Day" release (today is 02 02 2020).
+This is the "Palindrome Day" release (today is 02 02 2020).
.. rst-class:: compact-list
and there is no heterogeneous wait_any() that would mix Exec/Comm/Io activities. See ``examples/s4u/{io,exec,comm}-dependent`` for what's already there.
Since last fall, we continued to push toward the future SimGrid4 release. This requires to remove MSG and SimDAG once all users have
-migrated to S4U. The two old interfaces are still here, but this release gives another gentle incitative toward the migration. You now
+migrated to S4U. The two old interfaces are still here, but this release gives another gentle incentive toward the migration. You now
have to explicitly ask for MSG to be compiled in (and it may be removed by Q42020 or Q12021 along with the current Java bindings), and
this release proposes a nice S4U replacement for some parts of SimDAG.
is still possible through the regular instrumentation API based on the Paje format.
On the bindings front, we dropped the Lua bindings to create new platforms, as the C++ and Python interfaces are much better to that extend.
-Also, the algorithm tutorial can now be taken in Python, for those of you alergic to C++.
+Also, the algorithm tutorial can now be taken in Python, for those of you allergic to C++.
-Finally, on the SMPI front, we introduced a new documentation section on calibrating the SMPI models from your measurements and fixed some issues
-with the replay mechanism.
+Finally, on the SMPI front, we introduced a :ref:`new documentation section <models_calibration>` on calibrating the SMPI models from your
+measurements and fixed some issues with the replay mechanism.
-Version 3.31 (expected spring 2021)
------------------------------------
+Version 3.31 (not released yet)
+-------------------------------
+
+Expected: spring 2022
-On the model checking front, the long awaited big bang finally occurred, greatly simplifying future evolution.
+**On the model checking front**, the long awaited big bang finally occurred, greatly simplifying future evolution.
A formal verification with Mc SimGrid implies two processes: a verified application that is an almost regular SimGrid simulation, and a checker that
is an external process guiding the verified application to ensure that it explores every possible execution scenario. When formal verification was
-initially introduced in SimGrid 15 years ago, both processes were intertwined in the same system process but the mandated system tricks made it
-impossible to use classical debugging tools such as gdb or valgrind on that Frankenstein process. Having more than one heap in your process is not
-usually supported.
+initially introduced in SimGrid 15 years ago, both processes were intertwined in the same system process, but the mandated system tricks made it
+impossible to use gdb or valgrind on that Frankenstein process. Having two heaps in one process is not usually supported.
-The design was simplified in v3.12 (2015) by splitting the application and the checker in separate system process. But both processes remained tightly
-coupled: when the checker needed some information (for example the mailbox implied in a send operation to compute whether this operation `commutes
+The design was simplified in v3.12 (2015) by splitting the application and the checker in separate system processes. But both processes remained tightly
+coupled: when the checker needed some information (such as the mailbox implied in a send operation, to compute whether this operation `commutes
with another one <https://en.wikipedia.org/wiki/Partial_order_reduction>`_), the checker was directly reading the memory of the other system process.
This was efficient and nice in C, but it prevented us from using C++ features such as opaque ``std::function`` data types. As such, it hindered the
ongoing SimDAG++ code reorganization toward SimGrid4, where all activity classes should be homogeneously written in modern C++.
-This release introduces a new design, where the simcalls are given object-oriented Observers that can serialize the relevant information over the wire.
-This information is used on the checker side to build Transition objects, representing the application simcalls. This explanation may not be crystal
-clear, but the checker code is now much easier to work with as the formal logic is not spoiled with system-level tricks to retrieve the needed information.
+This release introduces a new design, where the simcalls are given object-oriented ``Observers`` that can serialize the relevant information over the wire.
+This information is used on the checker side to build ``Transition`` objects that the application simcalls. The checker code is now much simpler, as the
+formal logic is not spoiled with system-level tricks to retrieve the needed information.
+
+This cleaned design allowed us to finally implement the support for mutexes, semaphores and barriers in the model-checker (condition variables are still
+missing). This enables in particular the verification of RMA primitives with Mc SimGrid, as their implementation in SMPI is based on mutexes and barriers.
+Simix, a central element of the SimGrid 3 design, was also finally removed: the last bits are deprecated and will be removed in 3.35. We also replaced the
+old, non-free ISP test suite by the one from the `MPI Bug Initiative <https://hal.archives-ouvertes.fr/hal-03474762>`_ (not all tests are activated yet).
+This will eventually help improving the robustness of Mc SimGrid.
+
+These changes unlock the future of Mc SimGrid. For the next releases, we plan to implement another exploration algorithm based on event unfoldings (using
+`The Anh Pham's thesis <https://tel.archives-ouvertes.fr/tel-02462074/document>`_), the exploration of scenarios where the actors get killed and/or where
+communications timeout, and the addition of a `wrapper to pthreads <https://hal.inria.fr/hal-02449080>`, opening the path to the verification classical
+multithreaded applications.
+
+
+**On the model front,** we continued our quest for the modeling of parallel tasks (ptasks for short). Parallel tasks are intended to be an extension
+of the max-min fairness model (that computes the sharing of communication flows or computation tasks) to tasks mixing resource kinds (e.g., a MPI
+computationnal kernel with computations and communications, or a video stream with IO read, network transfer and decompression on the CPU). Just
+specify the amount of computation for each involved host, the amount of data to transfer between each host pair, and you're set. The model will
+identify bottleneck resources and fairly share them across activities within a ptask. From a user-level perspective, SimGrid handles ptasks just like
+every other activity except that the usual SimGrid models (LV08 or SMPI) rely on an optimized algorithm that cannot handle ptasks. You must
+activate :ref:`the L07 model <s4u_ex_ptasks>` on :ref:`the command line <options_model_select>`. This "model" remains a sort of hack since its introduction 15 years ago, as
+it has never been well defined. We never succeded to unify L07 and max-min based models: Fairness is still to be defined in this context that mixes
+flops and communicated bytes. The resulting activity rates are then specific to ptasks. Furthermore, unlike our network models, this model were not
+thoroughly validated with respect to real experiments before `the thesis of Adrien Faure <https://tel.archives-ouvertes.fr/tel-03155702>`_ (and the
+outcome was quite disappointing). Recent articles by Bonald and Roberts `properly define <https://hal.inria.fr/hal-01243985>`_ the allocation
+objective we had in mind (under the name Bounded MaxMin Fairness -- BMF) and `study the convergence <https://hal.archives-ouvertes.fr/hal-01552739>`_
+of the microscopic dynamic model to a macroscopic equilibrium, but this convergence could only be proved in rather simple cases. Even worse, there is
+no known algorithm to efficiently compute a BMF!
+
+L07 should still be avoided as we have exhibited simple scenarios where its solution is irrelevant to the BMF one (that is mathematically sound). This
+release thus introduces a new BMF model to finally unify both classical and parallel tasks, but this is still ongoing work. The implemented
+heuristic works very well for most SimGrid tests, but we have found some (not so prevalent) corner cases where our code fails to solve the sharing
+problem in over 10 minutes... So this all should still be considered an ongoing research effort. We expect to have a better understanding of this issue
+by the next release.
+
+On a related topic, this release introduces :cpp:func:`simgrid::s4u::this_actor::thread_execute`, which allows creating a computation that comprises
+several threads, and thus capable of utilizing more cores than a classical :cpp:func:`simgrid::s4u::this_actor::execute` action. The goal is to make
+it straightforward to model multithreaded computational kernels, and it comes with an illustrating example. It can be seen as a simplified ptask, but
+since it does not mix bytes and flops and has a homogeneous consumption over a single CPU, it perfectly fits with the classical SimGrid model.
-Future work on the model checker include: support for mutexes (that are still not handled), implementation of another exploration algorithm based on UDPOR
-(`The Anh Pham's thesis <https://tel.archives-ouvertes.fr/tel-02462074/document>`_ was defended in 2019), and robustness improvement using the `MPI Bug
-Initiative <https://hal.archives-ouvertes.fr/hal-03474762>`_ tests. Many things that were long dreamed of now become technically possible in this code base.
.. |br| raw:: html
