X-Git-Url: http://info.iut-bm.univ-fcomte.fr/pub/gitweb/simgrid.git/blobdiff_plain/ebcf5b5967286b2041615e8d777ac5edd7925b60..refs/heads/master:/ChangeLog?ds=sidebyside diff --git a/ChangeLog b/ChangeLog index 135a86933f..d41a80e34a 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,9 +1,308 @@ -SimGrid (3.30.1) NOT RELEASED YET (v3.31 expected March 20. 2022, 15:33 UTC) +SimGrid (3.35.1) not released (target: Feb 24) + + +---------------------------------------------------------------------------- + +SimGrid (3.35) November 23. 2023 + +The "Thanks Giving up stateful model-checking" release. Stateless model checking remains. + +S4U: + - New class ActivitySet to ease wait_any()/test_any()/wait_all() + - Deprecate {Comm,Io,Exec}::{wait_any,wait_all,test_any} and friends + - Simplify a bit the declaration of multi-zoned platforms from C++ + - New function NetZone::add_route(host1, host2, links) when you don't need gateways + - Also add a variant with s4u::Link, when you don't want to specify the directions + on symmetric routes. + - Zone's gateways can now be controlled directly. + - Add NetZone::add_route(zone1, zone2, links) specifying the route between zones + - Introduce a Mailbox::get_async() with no payload parameter. You can use the new + Comm::get_payload() once the communication is over to retrieve the payload. + - Implement recursive mutexes. Simply pass true to the constructor to get one. + - Simplify the expression of horizontal scaling of Tasks. + - Each Task now consists of a dispatcher, a collector and one or more instances. + - The parallelism degree of each of these can be set. + - Several examples have been added or modified accordingly. + - Update s4u::create_DAG_from_json() to support wfformat 1.4. + - Introduce a new MessageQueue abstraction and associated Mess simulated object. + The behavior of a MessageQueue is similar to that of a Mailbox, but intended for + control messages that do not incur any simulated cost. Information is automagically + transported over thin air between producer and consumer. See examples/cpp/mess-wait + - New function: Mutex::get_owner() + +S4U plugins: + - New: Add a JBOD (just a bunch of disks) concept. It's a sort of host with many disks. + - Revamp the battery plugin: rewrite completely the API, for a better usability. + The examples were updated accordingly. + The battery can now act as a simple connector (see battery-connector example). + - Revamp of the photovoltaic plugin: now called SolarPanel and complete rewrite of the API + - Add chiller plugin: enable the management of chillers consuming electrical energy + to compensate heat generated by hosts. + - Add a battery-chiller-solar example combining several plugins to evaluate the amount + of brown energy (from the electrical grid) and green energy (from the solar panel) + during a given computation. + +SMPI: + - New SMPI_app_instance_join(): wait for the completion of a started MPI instance + - MPI_UNIVERSE_SIZE now initialized to the total amount of hosts in the platform + - Memory usage due to SMPI for non-MPI actors greatly reduced. + - New implemented calls: MPI_Isendrecv, MPI_Isendrecv_replace + +sthread: + - Allow to use on valgrind-observed or gdb-observed processes. + - Install sthread on user's disk. + - Implement recursive pthreads. + - Implement pthread_barrier and pthread_cond (but conditional are not supported by the MC yet). + - Add some McMini codes to test sthread further (controlled with enable_testsuite_McMini). + +Model checking: + - Remove stateful model-checking. This was not used, not really working, and very hard to fix. + Liveness properties cannot be verified anymore. + - More informative backtraces on assertion failure. + - Fix dependency bugs for mutex and other transitions + - Fix some reversible_race definitions, and also the rest of ODPOR. + +Python: + - Make the host_load plugin available from Python. See examples/python/plugin-host-load + - Mailbox::get_async() does not return a pair anymore. Use comm.get_payload() instead. + - Comm::waitall/waitany/testany() are gone. Please use ActivitySet() instead. + - Comm::waitallfor() is gone too. Its semantic was unclear on timeout anyway. + - Io::waitany() and waitanyfor() are gone. Please use ActivitySet() instead. + - Do not export the values of enums. So you need to write e.g. SharingPolicy.LINEAR + while it should have been possible to write LINEAR alone before. This is the advised + behavior for modern C++ code. + +C API: + - Introduce sg_activity_set_t and deprecate wait_all/wait_any/test_any for + Exec, Io and Comm. + +Kernel: + - optimize an internal data structure (replace boost::circular_buffer_space_optimized by + std::deque to store pending and unmatched Comms in Mailboxes). It is actually a revert + to what was used a few years back. The boost structure had a lower memory footprint than + deques, but it appeared that their "space_optimized" character was generating a huge lot + of refcount changes on the stored Comms. + +General: + - Fix errors with ns-3 v3.36+ + - Many other small bug fixes, in particular in MC and sthread. + +---------------------------------------------------------------------------- + +SimGrid (3.34) June 26. 2023 + + Save the planet, skip a release: 3.33 was due 6 months ago, so skip directly to 3.34. + +General: + - SimGrid now requires a compiler with C++17 support for public headers too. + Sibling projects should upgrade their FindSimGrid.cmake + - Remove the MSG API: its EOL was scheduled for 2020. + - Remove the Java bindings: they were limited to the MSG interface. + - On Windows, you now need to install WSL2 as the native builds are now disabled. + It was not really working anyway. + - Support for 32bits architecture is not tested anymore on our CI infrastructure. + It may break in the future, but we think that nobody's using SimGrid on 32 bits. + - Remove the surf module. It was replaced by the kernel/models module, and that + refactoring took almost 10 years to properly complete. + +S4U: + - Activity::set_remaining() is not public anymore. Use for example + Comm::set_payload_size() to change the size of the simulated data. + - New function: Engine::flatify_platform(), to get a fully detailed vision of the + configured platform. + - New Task abstraction: They are designed to represent dataflows, i.e, graphs of repeatable Activities. + See the examples under examples/cpp/task-* and the associated documentation. + - Full simDAG integration: Activity::start() actually starts only when all dependencies + are fulfilled. If it cannot be started right away, it will start as soon as it becomes + possible. + - Allow to set a concurrency limit on disks and hosts, as it was already the case for links. + - Rename Link::get_usage() to Link::get_load() for consistency with Host:: + - Every signal now come with a static version that is invoked for every object of that class, + and an instance version that is invoked for this specific object only. For example, + s4u::Actor::on_suspend_cb() adds a callback that is invoked for the suspend of any actor while + s4u::Actor::on_this_suspend_cb() adds a callback for this specific actor only. + - Activity::on_suspended_cb() is renamed to Activity::on_suspend_cb(), and fired right before the suspend. + - Activity::on_resumed_cb() is renamed to Activity::on_resume_cb(), and fired right before the resume. + - Resource::on_state_change_cb() is renamed to Resource::on_onoff_cb() to distinguish from the + Activity::on_state_change_cb() that is related to the activity state machine, not on/off. + - Activity signals (veto, suspend, resume, completion) are now specialized by activity class. + That is, callbacks registered in Exec::on_suspend_cb will not be fired for Comms nor Ios. + +New S4U plugins: + - Battery: Enable the management of batteries on hosts. + See the examples under examples/cpp/battery-* and the documentation in the Plugins page. + - Photovoltaic: Enable the management of photovoltaic panels on hosts. + See the examples under examples/cpp/photovoltaic-* and the documentation in the Plugins page. + +Kernel: + - optimize an internal data structure (use a set instead of a list for ongoing activities), + leading to a potentially big performance gain, in particular with many detached comms. + +MPI: + - New option smpi/barrier-collectives to add a barrier to some collectives + to detect dangerous code that /may/ work on some MPI implems. + - New function SMPI_app_instance_start() to easily start a MPI instance in your S4U simulation. + +Models: + - Write the section of the manual about models, at least. + - WiFi: the total capacity of a link depends on the amount of flows on that link. + - Use the nonlinear callback feature of LMM to reflect this. + - Calibration values can be changed to match different MCS configurations + - See the example teshsuite/models/wifi_usage_decay/wifi_usage_decay.cpp + - See also "A Flow-Level Wi-Fi Model for Large Scale Network Simulation" + https://hal.archives-ouvertes.fr/hal-03777726 + - Merge parameters network/bandwidth-factor and smpi/bw-factor that serve the same purpose. + - Same for the latency + - Rewrite the corresponding documentation. + - Allow to disable the TCP windowing modeling by setting network/TCP-gamma to 0. + - Finally kill the 'compound' host model. You can change the CPU or network model + with the default host model, as it should. + - Rename option "surf/precision" to "precision/timing" for clarity. + - Rename option "maxmin/precision" to "precision/work-amount" for clarity. + - New function: Engine::flatify_platform() to debug your platform. + +sthread: + - Implement pthread_join in MC mode. + - Implement semaphore functions in sthread. + - Add an intricated way to verify the access to non-reentrant data structures + It requires code annotation, as shown in examples/sthread/stdobject/stdobject.cpp + +Model checking: + - Stateless model-checking is now usable on any system, including Mac OSX and ARM processors. + - The stateless aspects of the MC are now enabled by default in all SimGrid builds. + Liveness and stateful aspects are still controlled by the enabling_model-checking + configuration option. + - Introducing ODPOR and SDPOR reduction strategies + - Introducing guiding heuristics, trying to find bugs faster than DFS in reduced state space. + - Synchronize the MBI tests with upstream. + - Show the full actor backtraces when replaying a MC trace (with model-check/replay) + and the status of all actors on deadlocks in MC mode. + +XBT: + - simgrid::xbt::cmdline and simgrid::xbt::binary_name are gone. + Please use simgrid::s4u::Engine::get_cmdline() instead. + +Documentation: + - New tutorial on simulating DAGs. + - New section in the user guide on the provided performance models. + - New section presenting some technical good practices for (potential) contributors. + - Add a section on errors and exceptions to the API documentation. + - Move the s4u examples to a section on their own to ease navigation. + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#18: Java bindings should be redone or removed + - FG!118: Wi-Fi callback mechanism + - FG!119: SMPI: add option to inject a barrier before every collective call + - GH#383: Segfault when adding a disk after load_platform(xml) + +---------------------------------------------------------------------------- + +SimGrid (3.32) October 3. 2022. + +The Wiedervereinigung release. Germany was reunited 32 years ago. + +General: + - SimGrid now requires a compiler with C++17 support to compile the lib. + Our public headers still allow the user code to be compiled in C++14. + - Support graphviz v3 and ns-3 v3.36 (older versions are still supported). + - Tested with clang (v11, v13, v14 and v16), gcc (v7 to v13) and IntelCC v2022.2 + +S4U: + - API evolutions: + - kill signal Comm::on_completion that was not working anyway. + - Expose signals Activity::on_suspend and Activity::on_resume + - New macro xbt_enforce(): similar to xbt_assert(), but throws an AssertionError + instead of calling abort(). + - New: s4u::Exec::get_thread_count() + - Various cleanups around virtual machines: + - host_by_name() and friends now only return hosts. VMs are now excluded. + - It is now impossible to search a VM by name globally. + You can only search VM by name on a given PM, so either you know + the PM on which your VM runs and you can search by name, or you need + to manually iterate over all PMs to search this VM. + - The s4u::VirtualMachine constructor is now deprecated. + Please use s4u::Host::create_vm() instead. + - Rename s4u::VirtualMachine::on_creation() to on_vm_creation() to + avoid confusion with s4u:Host::on_creation() that is inherited. + Also s4u::VirtualMachine::on_destruction -> on_vm_destruction(). + - Bug fixes: + - One-sided communications (Comm::sendto) can now be detached, + and should now be more resilient to network and host faults. + +Python: + - Added the following bindings / examples: + - Comm (now 100% covers the C++ interface): + - Comm.dst_data_size, Comm.mailbox, Comm.sender, Comm.start_time, Comm.finish_time + - Comm.state_str [examples: examples/python/comm-failure/, examples/python/comm-host2host/] + - Comm.remaining [examples: examples/python/comm-host2host/, examples/python/comm-suspend/] + - Comm.set_payload_size [example: examples/python/comm-host2host/] + - Comm.set_rate [example: examples/python/comm-throttling/] + - Comm.sendto, Comm.sendto_init, Comm.sendto_async [example: examples/python/comm-host2host/] + - Comm.start, Comm.suspend, Comm.resume [example: examples/python/comm-host2host/] + - Comm.test_any [example: examples/python/comm-testany/] + - Comm.wait_until [example: examples/python/comm-waituntil/] + - Engine: + - Engine.host_by_name [example: examples/python/comm-host2host/] + - Engine.mailbox_by_name_or_create [example: examples/python/comm-pingpong/] + - Engine.set_config + - Mailbox: Mailbox.ready [example: examples/python/comm-ready/] + - Ptask [example: examples/python/exec-ptask/]: + - this_actor.exec_init + - this_actor.parallel_execute + - Exec.suspend + - Exec.wait_for + - Added an AssertionError exception that may be thrown in case of error. + For instance, creating two hosts with the same name will now throw this exception + instead of killing the interpreter. + +SMPI: + - Implement MPI_File_get_type_extent(), MPI_File_s/get_atomicity() and + MPI_File_get_byte_offset() + - Intercept getpid() calls to return the simulated ones. + - Fix various bugs in MPI IO. + +Platform description & visualization: + - More robust sanity checks for platforms, to reject forbidden topologies with + a proper error message. + - New platform example: supernode.cpp and supernode.py. + The Python version generates a nice graphical representation of the platform. + - Bug fixes around fat-tree topologies. + - Allow to dump the platform topology as a CSV file representing the graph edges + with platform_graph_export_csv() (similar to the DOT export). + - Fix graphicator for "cluster" topologies (e.g. fat-tree, dragonfly). + +Models: + - Fix a bug when using ptasks with multicores (FG!111). + +Model-Checker: + - First bits of sthread, that intercepts pthread operations at runtime. + The intend is to use it together with simgrid-mc, but it is TBD. + - Sync MBI generators with upstream changes. + - Various cosmetics, small bug fixes and inner refactorings + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#105: "Variable penalty should not be negative!" with in-flight messages and bandwidth profiles + - FG#109: Application time reported by --cfg=smpi/display-timing:yes is wrong + - FG#110: Wait_any does not trigger new model solve when host events occur + - FG#111: Wrong execution time in rare cases when using multicore + - FG!98: Re-enable the tests for legacy stochastic profiles + - FG!109: Trigger new engine solve upon host events such as host on/off + - FG!116: SMPI/replay: Fix issue with recv of size =0 + +---------------------------------------------------------------------------- + +SimGrid (3.31) March 22. 2022. + +The ненасильство release. We stand against war. + +Against the agression by a sick system that forces peoples to take arms against each other. MC: - Rework the internals, for simpler and modern code. This shall unlock many future improvements. - - You can now define plugins onto SafetyChecker (a simple DFS explorer), using the declared signals. - See CommunicationDeterminism for an example. + - You can now define plugins onto the DFS explorer (previously called SafetyChecker), using the + declared signals. See CommunicationDeterminism for an example. - Support mutex, semaphore and barrier in DPOR reduction - Seems to work on Arm64 architectures too. - Display a nice error message when ptrace is not usable. @@ -18,31 +317,81 @@ SMPI: - tracing: ensure that we dump the TI traces continuously during execution and not just at the end, reducing memory cost and performance hit. - Update OpenMPI collectives selection logic to match current one (4.1.2) + - Add a coherence check for collective operation order and root/MPI_Op + coherence. Potentially costly so not activated unless smpi:pendantic is set + or -analyze is given. S4U: - New signal: Engine::on_simulation_start_cb() - - Reimplementation of barriers natively. - Previously, they were implemented on top of s4u::Mutex and s4u::ConditionVariable. + - Introduce a new execution mode with this_actor::thread_execute(). This simulate + the execution of a certain amount of flops by multiple threads ran by a host. Each + thread executes the same number of flops, given as argument. An example of this new + function can be found in examples/cpp/exec-threads. + - Reimplementation of barriers natively. + Previously, they were implemented on top of s4u::Mutex and s4u::ConditionVariable. The new version should be faster (and can be used in the model-checker). + - Actor::get_restart_count(): Returns the number of reboots that this actor did. MSG: - MSG_barrier_destroy now expects a non-const msg_barrier parameter. New plugin: the Chaos Monkey (killing actors at any time) - - Along with the new simgrid-monkey script, it tests whether your simulation - resists resource failures at any possible timestamp in your simulation. - - It is mostly intended to test the simgrid core in extreme conditions, - but users may find it interesting too. + - Along with the new simgrid-monkey script, it tests whether your simulation + resists resource failures at any possible timestamp in your simulation. + - It is mostly intended to test the SimGrid core in extreme conditions, + but some users may find it interesting too. + +Models: + - New solver for parallel task: BMF. + - More realistic sharing of heterogeneous resources compared to the fair + bottleneck solver used by ptask_L07. + - Implement the BMF (Bottleneck max fairness) fairness. + - Improved resource sharing for parallel tasks with sub-flows (parallel + communications between same source and destination inside the ptask). + - Parameters: + - "--cfg=host/model:ptask_L07 --cfg=host/solver:bmf": enable the ptask + model with BMF solver. + - "--cfg=bmf/max-iterations: ": maximum number of iterations performed + by BMF solver (default: 1000). + - "--cfg=bmf/precision: ": numerical precision used when computing + resource sharing (default: 1e-12). + - This model requires Eigen3 library. Make sure Eigen3 is installed to use BMF. + +General: + - Modifications of the Profile mechanism, with some impact on users + - Addition of a new (S4U) method to init profiles from generic functions to improve versatility + - Fix initial behaviour of state_profiles + - Modify periodicity to behave like a period, and not like a loop delay XBT: - Drop xbt_dynar_shrink(). +Python: + - Made the following bindings static (previously member functions): + - Actor: Actor.kill_all(), Actor.by_pid() + - Host: Host.by_name(), Host.current(), Host.on_creation_cb() + - Mailbox: Mailbox.by_name() + - Added the following bindings: + - this_actor.warning() + - Mailbox.put_init() [example: examples/python/comm-waitallfor/] + - Comm.detach() [example: examples/python/comm-waitallfor/] + - Comm.wait_for() [example: examples/python/comm-waitfor/] + - Comm.wait_any_for() + - Comm.wait_all_for() [example: examples/python/comm-waitallfor/] + - Mutex [example: examples/python/synchro-mutex/] + - Barrier [example: examples/python/synchro-barrier/] + - Semaphore [example: examples/python/synchro-semaphore/] + +Build System: + - Remove target "make uninstall" which was incomplete and no longer maintained. + Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) (FG: issues on Framagit; GH: issues on GitHub) - FG#57: Mc SimGrid should test whether ptrace is usable - FG#87: Smpi scripts fail with spaces in paths - FG#100: [SMPI] Order of the message matching is not guaranteed - FG#101: LGPL 2.1 is deprecated license + - FG#104: "make uninstall" not up-to-date - GH#151: Missing mutexes for DPOR. ---------------------------------------------------------------------------- @@ -129,7 +478,6 @@ The "Ask a stupid question" release. We wish that every user ask one question about SimGrid to celebrate. On Mattermost, Stack Overflow or using the issues tracker. - New modeling features: - Non-linear resource sharing, modeling resources whose performance heavily degrades with contention: - The total capacity may be updated dynamically through a callback @@ -428,7 +776,7 @@ The Release release (the French lockdown was eased today). Important user-visible changes: - SimGrid now requires a compiler with C++14 support. - Sibling projects should upgrade their FindSimgrid.cmake + Sibling projects should upgrade their FindSimGrid.cmake - Surf precision default value is now 1e-9, instead of 1e-5. This was changed as several users had difficulties to understand issues when using high bandwidth or small latency events. The new value was already the default for SMPI and @@ -754,7 +1102,7 @@ General: - Network model 'NS3' was renamed into 'ns-3'. Python: - - Simgrid can now hopefully be installed with pip. + - SimGrid can now hopefully be installed with pip. S4U: - wait_any can now be used for asynchronous executions too. @@ -1802,7 +2150,7 @@ SimGrid (3.12) stable; urgency=low - InfiniBand network model added: Based on the works of Jerome Vienne http://mescal.imag.fr/membres/jean-marc.vincent/index.html/PhD/Vienne.pdf - When smpi/display_timing is set, also display global simulation time and application times - - Have smpirun, smpicc and friends display the simgrid git hash version on --git-version + - Have smpirun, smpicc and friends display the SimGrid git hash version on --git-version * Collective communications - SMP-aware algorithms are now dynamically handled. An internal communicator is created for each node, and an external one to handle communications between "leaders" of each node - MVAPICH2 (1.9) collective algorithms selector: normal and SMP algorithms are handled, and selection logic is based on the one used on TACC's Stampede cluster (https://www.tacc.utexas.edu/stampede/). @@ -2543,8 +2891,8 @@ SimGrid (3.6.2) stable; urgency=low Portability * Create an installer for windows with nsis (amd64 and win32) - - Add an hello world project to illustrate simgrid project creation. - - Embed libpcre into the Simgrid installer to avoid + - Add an hello world project to illustrate SimGrid project creation. + - Embed libpcre into the SimGrid installer to avoid its compilation burden * The raw execution contexts should work on Apple now * Port to Windows 64 bits @@ -2577,7 +2925,7 @@ SimGrid (3.6.1) stable; urgency=low SimGrid-java (3.6) unstable; urgency=low * Initial release. - * Split of every thing from simgrid v3.5 into a separate package. + * Split of every thing from SimGrid v3.5 into a separate package. -- 2011-10-05 Da SimGrid team @@ -3091,7 +3439,7 @@ SimGrid (3.4) stable; urgency=low * Greatly improved our cdash/ctest interactions Check http://cdash.inria.fr/CDash/index.php?project=Simgrid * Added memory checking tests with valgrind; lot of memleak fixing. - This may be the first release of simgrid with so few memory issues + This may be the first release of SimGrid with so few memory issues * Added code coverage tests. Our coverage is still improvable, but at least we see it on cdash. @@ -3397,7 +3745,7 @@ SimGrid (3.3.2) stable; urgency=low Timing report of this version: This version seem to be more than 5% faster than 3.3.1 (on linux - 64bits with contextes). The gain is less than expected, we are + 64bits with contexts). The gain is less than expected, we are investigating this for next release. -- Da SimGrid team Wed, 19 Aug 2009 17:07:12 +0200 @@ -3490,7 +3838,7 @@ SimGrid (3.3.1) stable; urgency=low - Linux(debian)/amd64/context - Linux(debian)/amd64/pthreads These targets fail about 1/10 of times on gras/pmm, but we believe - that this is because of the test, not because of simgrid. + that this is because of the test, not because of SimGrid. amok/saturate_sg fails even more rarely, and the test may not be the problem. @@ -3547,7 +3895,7 @@ SimGrid (3.3) stable; urgency=high is really less memory-demanding, which should allow you to use larger files in SimGrid [AL]. - * Inform valgrind about our contextes, so that it becomes usable + * Inform valgrind about our contexts, so that it becomes usable with the default (and more effecient) version of SimGrid [contributed by Sékou Diakite, many thanks]