X-Git-Url: http://info.iut-bm.univ-fcomte.fr/pub/gitweb/simgrid.git/blobdiff_plain/455a9c602ca8bf5bff5c942cbde83eb648318af1..968613a336f2fff832f9085e69b990e7371ca8e5:/ChangeLog diff --git a/ChangeLog b/ChangeLog index 2d7250f5ca..5e238de1f7 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,12 +1,750 @@ +SimGrid (3.32.1) not released yet (target december 22) + +General: + - Remove the MSG API: its EOL was scheduled for 2020. + - Remove the Java bindings: they were limited to the MSG interface. + - On Windows, you now need to install WSL2 as the native builds are now disabled. + It was not really working anyway. + - Support for 32bits architecture is not tested anymore on our CI infrastructure. + It may break in the future, but we think that nobody's using SimGrid on 32 bits. + - Remove the surf module. It was replaced by the kernel/models module, and that + refactoring took almost 10 years to properly complete. + +S4U: + - Activity::set_remaining() is not public anymore. Use for example + Comm::set_payload_size() to change the size of the simulated data. + - New function: Engine::flatify_platform(), to get a fully detailed vision of the + configured platform. + - Full simDAG integration: Activity::start() actually starts only when all dependencies + are fullfiled. If it cannot be started right away, it will start as soon as it becomes + possible. + +Kernel: + - optimize an internal datastructure (use a set instead of a list for ongoing activities), + leading to a potentially big performance gain, in particular with many detached comms. + +MPI: + - New option smpi/barrier-collectives to add a barrier to some collectives + to detect dangerous code that /may/ work on some MPI implems. + - New function SMPI_app_instance_start() to easily start a MPI instance in your S4U simulation. + +Models: + - Write the section of the manual about models, at least. + - WiFi: the total capacity of a link depends on the amout of flows on that link. + - Use the nonlinear callback feature of LMM to reflect this. + - Calibration values can be changed to match different MCS configurations + - See the example teshsuite/models/wifi_usage_decay/wifi_usage_decay.cpp + - See also "A Flow-Level Wi-Fi Model for Large Scale Network Simulation" + https://hal.archives-ouvertes.fr/hal-03777726 + - Merge parameters network/bandwidth-factor and smpi/bw-factor that serve the same purpose. + - Same for the latency + - Rewrite the corresponding documentation. + - Allow to disable the TCP windowing modeling by setting network/TCP-gamma to 0. + - Finally kill the 'compound' host model. You can change the CPU or network model + with the default host model, as it should. + - Rename option "surf/precision" to "precision/timing" for clarity. + - Rename option "maxmin/precision" to "precision/work-amount" for clarity. + - New function: Engine::flatify_platform() to debug your platform. + +sthread: + - Implement pthread_join in MC mode. + - Implement semaphore functions in sthread. + +Model checking: + - Synchronize the MBI tests with upstream. + - Show the full actor bactraces when replaying a MC trace (with model-check/replay) + and the status of all actors on deadlocks in MC mode. + +XBT: + - simgrid::xbt::cmdline and simgrid::xbt::binary_name are gone. + Please use simgrid::s4u::Engine::get_cmdline() instead. + +Documentation: + - New section in the user guide on the provided performance models. + - New section presenting some technical good practices for (potential) contributors. + - Add a section on errors and exceptions to the API documentation. + - Move the s4u examples to a section on their own to ease navigation. + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#18: Java bindings should be redone or removed + - FG!118: Wi-Fi callback mechanism + - FG!119: SMPI: add option to inject a barrier before every collective call + - GH#383: Segfault when adding a disk after load_platform(xml) + ---------------------------------------------------------------------------- -SimGrid (3.25.1) UNRELEASED (target: March 20 2020 or v3.26 on 3/26/20) +SimGrid (3.32) October 3. 2022. -Fixed bugs (FG#.. -> framagit bugs; FG!.. -> framagit merge requests): +The Wiedervereinigung release. Germany was reunited 32 years ago. + +General: + - SimGrid now requires a compiler with C++17 support to compile the lib. + Our public headers still allow the user code to be compiled in C++14. + - Support graphviz v3 and ns-3 v3.36 (older versions are still supported). + - Tested with clang (v11, v13, v14 and v16), gcc (v7 to v13) and IntelCC v2022.2 + +S4U: + - API evolutions: + - kill signal Comm::on_completion that was not working anyway. + - Expose signals Activity::on_suspend and Activity::on_resume + - New macro xbt_enforce(): similar to xbt_assert(), but throws an AssertionError + instead of calling abort(). + - New: s4u::Exec::get_thread_count() + - Various cleanups around virtual machines: + - host_by_name() and friends now only return hosts. VMs are now excluded. + - It is now impossible to search a VM by name globally. + You can only search VM by name on a given PM, so either you know + the PM on which your VM runs and you can search by name, or you need + to manually iterate over all PMs to search this VM. + - The s4u::VirtualMachine constructor is now deprecated. + Please use s4u::Host::create_vm() instead. + - Rename s4u::VirtualMachine::on_creation() to on_vm_creation() to + avoid confusion with s4u:Host::on_creation() that is inherited. + Also s4u::VirtualMachine::on_destruction -> on_vm_destruction(). + - Bug fixes: + - One-sided communications (Comm::sendto) can now be detached, + and should now be more resilient to network and host faults. + +Python: + - Added the following bindings / examples: + - Comm (now 100% covers the C++ interface): + - Comm.dst_data_size, Comm.mailbox, Comm.sender, Comm.start_time, Comm.finish_time + - Comm.state_str [examples: examples/python/comm-failure/, examples/python/comm-host2host/] + - Comm.remaining [examples: examples/python/comm-host2host/, examples/python/comm-suspend/] + - Comm.set_payload_size [example: examples/python/comm-host2host/] + - Comm.set_rate [example: examples/python/comm-throttling/] + - Comm.sendto, Comm.sendto_init, Comm.sendto_async [example: examples/python/comm-host2host/] + - Comm.start, Comm.suspend, Comm.resume [example: examples/python/comm-host2host/] + - Comm.test_any [example: examples/python/comm-testany/] + - Comm.wait_until [example: examples/python/comm-waituntil/] + - Engine: + - Engine.host_by_name [example: examples/python/comm-host2host/] + - Engine.mailbox_by_name_or_create [example: examples/python/comm-pingpong/] + - Engine.set_config + - Mailbox: Mailbox.ready [example: examples/python/comm-ready/] + - Ptask [example: examples/python/exec-ptask/]: + - this_actor.exec_init + - this_actor.parallel_execute + - Exec.suspend + - Exec.wait_for + - Added an AssertionError exception that may be thrown in case of error. + For instance, creating two hosts with the same name will now throw this exception + instead of killing the interpreter. + +SMPI: + - Implement MPI_File_get_type_extent(), MPI_File_s/get_atomicity() and + MPI_File_get_byte_offset() + - Intercept getpid() calls to return the simulated ones. + - Fix various bugs in MPI IO. + +Platform description & visualization: + - More robust sanity checks for platforms, to reject forbidden topologies with + a proper error message. + - New platform example: supernode.cpp and supernode.py. + The Python version generates a nice graphical representation of the platform. + - Bug fixes around fat-tree topologies. + - Allow to dump the platform topology as a CSV file representing the graph edges + with platform_graph_export_csv() (similar to the DOT export). + - Fix graphicator for "cluster" topologies (e.g. fat-tree, dragonfly). + +Models: + - Fix a bug when using ptasks with multicores (FG!111). + +Model-Checker: + - First bits of sthread, that intercepts pthread operations at runtime. + The intend is to use it together with simgrid-mc, but it is TBD. + - Sync MBI generators with upstream changes. + - Various cosmetics, small bug fixes and inner refactorings + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#105: "Variable penalty should not be negative!" with in-flight messages and bandwidth profiles + - FG#109: Application time reported by --cfg=smpi/display-timing:yes is wrong + - FG#110: Wait_any does not trigger new model solve when host events occur + - FG#111: Wrong execution time in rare cases when using multicore + - FG!98: Re-enable the tests for legacy stochastic profiles + - FG!109: Trigger new engine solve upon host events such as host on/off + - FG!116: SMPI/replay: Fix issue with recv of size =0 + +---------------------------------------------------------------------------- + +SimGrid (3.31) March 22. 2022. + +The ненасильство release. We stand against war. + +Against the agression by a sick system that forces peoples to take arms against each other. + +MC: + - Rework the internals, for simpler and modern code. This shall unlock many future improvements. + - You can now define plugins onto the DFS explorer (previously called SafetyChecker), using the + declared signals. See CommunicationDeterminism for an example. + - Support mutex, semaphore and barrier in DPOR reduction + - Seems to work on Arm64 architectures too. + - Display a nice error message when ptrace is not usable. + - New test suite, imported from the MPI Bugs Initiative (MBI). Not all MBI generators are integrated yet. + - Remove the ISP test suite: it's not free software, and it's superseeded by MBI. + +SMPI: + - fix for FG#100 by ensuring small asynchronous messages never overtake larger + ones, conforming to the standard. + - replay: fix waitall behaviour to avoid forgetting requests and leaking + their handles. + - tracing: ensure that we dump the TI traces continuously during execution and + not just at the end, reducing memory cost and performance hit. + - Update OpenMPI collectives selection logic to match current one (4.1.2) + - Add a coherence check for collective operation order and root/MPI_Op + coherence. Potentially costly so not activated unless smpi:pendantic is set + or -analyze is given. + +S4U: + - New signal: Engine::on_simulation_start_cb() + - Introduce a new execution mode with this_actor::thread_execute(). This simulate + the execution of a certain amount of flops by multiple threads ran by a host. Each + thread executes the same number of flops, given as argument. An example of this new + function can be found in examples/cpp/exec-threads. + - Reimplementation of barriers natively. + Previously, they were implemented on top of s4u::Mutex and s4u::ConditionVariable. + The new version should be faster (and can be used in the model-checker). + - Actor::get_restart_count(): Returns the number of reboots that this actor did. + +MSG: + - MSG_barrier_destroy now expects a non-const msg_barrier parameter. + +New plugin: the Chaos Monkey (killing actors at any time) + - Along with the new simgrid-monkey script, it tests whether your simulation + resists resource failures at any possible timestamp in your simulation. + - It is mostly intended to test the simgrid core in extreme conditions, + but some users may find it interesting too. + +Models: + - New solver for parallel task: BMF. + - More realistic sharing of heterogeneous resources compared to the fair + bottleneck solver used by ptask_L07. + - Implement the BMF (Bottleneck max fairness) fairness. + - Improved resource sharing for parallel tasks with sub-flows (parallel + communications between same source and destination inside the ptask). + - Parameters: + - "--cfg=host/model:ptask_L07 --cfg=host/solver:bmf": enable the ptask + model with BMF solver. + - "--cfg=bmf/max-iterations: ": maximum number of iterations performed + by BMF solver (default: 1000). + - "--cfg=bmf/precision: ": numerical precision used when computing + resource sharing (default: 1e-12). + - This model requires Eigen3 library. Make sure Eigen3 is installed to use BMF. + +General: + - Modifications of the Profile mechanism, with some impact on users + - Addition of a new (S4U) method to init profiles from generic functions to improve versatility + - Fix initial behaviour of state_profiles + - Modify periodicity to behave like a period, and not like a loop delay + +XBT: + - Drop xbt_dynar_shrink(). + +Python: + - Made the following bindings static (previously member functions): + - Actor: Actor.kill_all(), Actor.by_pid() + - Host: Host.by_name(), Host.current(), Host.on_creation_cb() + - Mailbox: Mailbox.by_name() + - Added the following bindings: + - this_actor.warning() + - Mailbox.put_init() [example: examples/python/comm-waitallfor/] + - Comm.detach() [example: examples/python/comm-waitallfor/] + - Comm.wait_for() [example: examples/python/comm-waitfor/] + - Comm.wait_any_for() + - Comm.wait_all_for() [example: examples/python/comm-waitallfor/] + - Mutex [example: examples/python/synchro-mutex/] + - Barrier [example: examples/python/synchro-barrier/] + - Semaphore [example: examples/python/synchro-semaphore/] + +Build System: + - Remove target "make uninstall" which was incomplete and no longer maintained. + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#57: Mc SimGrid should test whether ptrace is usable + - FG#87: Smpi scripts fail with spaces in paths + - FG#100: [SMPI] Order of the message matching is not guaranteed + - FG#101: LGPL 2.1 is deprecated license + - FG#104: "make uninstall" not up-to-date + - GH#151: Missing mutexes for DPOR. + +---------------------------------------------------------------------------- + +SimGrid (3.30) January 30. 2022. + +The Sunday Bloody Sunday release. + +Main user-visible changes: + - The SimDag API for the simulation of the scheduling of Directed Acyclic + Graphs has been dropped. It was marked as deprecated for a couple of years. + We finally complete the implementation of what has been called SimDag++ + internally, i.e., porting the different features of SimDag on top of S4U. + The new way to simulate the execution of dependent activities directly by + maestro (without any other actor) is details in the examples/cpp/dag-* series + of examples. + - The removal of SimDag led us to also remove the export to Jedule files that + was tightly coupled to SimDag. The instrumentation of DAG simulation is still + possible through the regular instrumentation API based on the Paje format. + - We also dropped the old and clumsy Lua bindings to create platforms in a + programmatic way. It can be done in C++ in a much cleaner way now, which + motivates this suppression. + +S4U: + - Introduce on_X_cb() functions for all signals, to attach a new + callback to the signal X. The signal variables are now hidden and + only these functions should be used. + Rationale: this enables the usual deprecation scheme where functions + remain for 4 releases if we need to modify the signals, while the + current code with the signal variables directly visible prevents any + smooth transition. + - New function: Engine::run_until(date), to split the simulation. + - New signal: Activity::on_veto, to detect when an activity fails to start. + - Signal change: Comm::on_start(Comm&, bool) has been replaced by + Comm::on_send and Comm::recv. These two signals respectively correspond to + when the sending or receiving side of a Comm is ready. They are raised at + the same locations as the former Comm::on_start signal. + - New function: Engine::track_vetoed_activities() to interrupt run() + when an activity fails to start, and to keep track of such activities. + Please see the corresponding example for more info. + - New functions: s4u::Comm::{sendto_init, set_source, set_destination} to enable + the use of vetoers with direct host-to-host communications. Both source and + destination have to set for a comm to start. Each call to these setters check + if all vetoes are satisfied. When it is the case, the comm starts. A use case of + these functions is given in examples/cpp/dag-scheduling. + - New functions: {Exec, Io}::update_priority allow you to modify the priority of + these kinds of activities during their execution. Behavior is detailed in + examples/cpp/io-priority/. + +SMPI: + - Dynamic costs for MPI operations: New API to allow users to dynamically + change injected costs for MPI_Recv, MPI_Send and MPI_Isend operations. + Alternative for smpi/or, smpi/os and smpi/ois configuration options. + - Fix some issues with the replay mechanism. + +XBT: + - Function xbt::Extendable::get_data() is now templated with the type of the + pointee. Untyped function is deprecated. Use get_data() if you still + want to retrieve void*. + +Documentation: + - New section: "SimGrid MPI calibration of a Grid5000 cluster" + presenting how to properly calibrate MPI communications in SimGrid. + - Complete and reword the platform section, which is now completed. + +Python: + - Thread contexts are used by default with Python bindings. Other kinds of + contexts revealed unstable, specially starting with pybind11 v2.8.0. + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#95: Wrong computation time for multicore execution after pstate change + - FG#97: Wrong computation time for ptask+multicore+pstates + - FG#98: SMPI offline simulation is inconsistent with the online simulation + (deadlocks / message truncation) + - FG#99: Weird segfault when not sealing an host + +---------------------------------------------------------------------------- + +SimGrid (3.29) October 7. 2021 + +The "Ask a stupid question" release. + +We wish that every user ask one question about SimGrid to celebrate. +On Mattermost, Stack Overflow or using the issues tracker. + +New modeling features: + - Non-linear resource sharing, modeling resources whose performance heavily degrades with contention: + - The total capacity may be updated dynamically through a callback + and depends mainly on the number of concurrent flows. + - Examples (both cpp and python): io-degradation, network-nonlinear, exec-cpu-nonlinear + + - Dynamic factors: model variability in the speed of activities + - Each action can now have a factor that affects its progression. + This multiplicative factor is applied when updating the amount of work + remaining, thereby an activity with factor=0.5 only uses half of the + instantaneous power/bandwidth it is allocated and will appear twice + slower than what it actually consumes. + - This can be used to model a overhead (e.g., there is a 20 bytes + header in a 480 bytes TCP packet so the factor 0.9583) but the novelty + is this factor can now easily be adjusted depending on activity's and + resources characteristics. + - This existed for network (e.g., the effective bandwidth depends + on the message in SMPI piecewise-linear network model) but it is now + more general (the factor may depend on the source and destination and + thus account to different behaviors for intra-node communications and + extra-node communications) and is available for CPUs (e.g., if you + want to model an affinity as in the "Unrelated Machines" problem in + scheduling) and disks (e.g., if you want to model a stochastic + capacity) too. + - For that, resources can be provided with a callback that computes + the activity factor when creating the action. + - Example: examples/cpp/exec-cpu-factors + - The same mechanism is also available for the latency, which + allows to easily introduce complex variability patterns. + +Python: + - Added support to programmatic platform creation in Python. + Example: examples/python/clusters-multicpu + +S4U: + - Disk and Host now have a set_sharing_policy() too, for non-linear sharing. + This can only be set through the API, not through XML files. + +SMPI: + - TI Tracing/Replay: + - Multiple fixes to ensure reproducibility of tracing + - scan/excan can now be replayed + - wait action now uses ranks and not pid, as the other ones. + - smpi/init and smpi/finalization-barrier are now valid for replays. + - exit() is now intercepted by SMPI to avoid premature shutdown of + simulation. First non 0 return codes is returned as simulation return + code. + +Documentation: + * New section "Release Notes" documenting recent and current developments. + * New section "Modeling I/O: the realistic way" presenting how to properly model disks in SimGrid. + * Improvements in API Reference for C++ and Python interfaces. + +ns-3 model: + - Make wifi creation compatible with ns-3 version 3.34 too. + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#77: Search feature of doc is broken (update sphinx theme version) + - FG#78: Multiple fixes for SMPI replay: + - TI tracing of allotallv/w was outputting wrong values + - MPI_LOGICAL in fortran is actually 32 bits wide, and not 8. + +---------------------------------------------------------------------------- + +SimGrid (3.28) July 14. 2021 + +The Victoriadagarna Release. + +New features: + - C++ platform interface: Users can now describe their platform directly in C++. + This provides greatly flexibility and performance improvement for complex + platforms. Main features: + - Fat-Tree/DragonFly/Torus composing: allows you to create clusters of + "zones", instead of single hosts. This feature enables the description + of clusters with complex hosts, composed of several CPUs, GPUs, etc. + - StarZone: new zone with a Star-like topology. The routes are defined + as a set of links used to communicate from node to everybody (node<->ALL). + - Split-Duplex links: auxiliary method to create split-duplex links in + the platform, easing its utilisation. It automatically creates both UP + and DOWN links (similarly as done in XML). + - Please refer to the documentation and the examples included: + e.g. examples/cpp/clusters-multicpu/ and examples/platforms/*.cpp. + - New plugin: Producer-Consumer with monitor. Just requires to include the + include/simgrid/plugins/ProducerConsumer.hpp header to be used. See the + associated example (examples/cpp/plugin-prodcons). + +S4U: + - New: s4u::Comm::wait_all_for() (like s4u::Comm::wait_all, but with a timeout), + s4u::Io::wait_any(), s4u::Io::wait_any_for(). + - Methods test_all/test_any/wait_all/wait_any in s4u now take their vector + parameter by reference, instead of a pointer. + - Fixed a bug where Activity::wait_for() killed the activity on timeout. + Explicitly cancel the activity to get back to previous behavior. + - New: Link::set_concurrency_limit() to limit the amount of concurrent flows. + +SMPI: + - The default SMPI compiler flags are no more taken from the environment. + They can be explicitly set through cmake parameters SMPI_C_FLAGS, + SMPI_CXX_FLAGS, or SMPI_Fortran_FLAGS. + - New options: + --cfg=smpi/finalization-barrier: which can be used to add + a barrier inside MPI_Finalize. This can help for some codes which cleanup + data attached to a process, but still used in other SMPI processes. + --cfg=smpi/errors-are-fatal: True by default, behaves like if MPI_ERRORS_RETURN + is active when set to false, to keep going after a small error + --cfg=smpi/pedantic: True by default. Do not report some harmless MPI errors + which may or may not be problematic in the end. + - Sampling: + - fix behaviour, as maximum iteration count could be ignored + - add SMPI_SAMPLE_LOCAL_TAG and SMPI_SAMPLE_GLOBAL_TAG macros, to allow user to + use sampling when the same kernel is called with a different set of parameters + which have an impact on the timing. + - realloc is now intercepted, to be coherent, as malloc/calloc/free were already. + It should now work with smpi/auto-shared-malloc-thresh. + - Improve error handling and reporting in multiple places + - Improve correctness checks on the MPI code.(MPI_Op and MPI_Datatype + validity checks, truncated messages are now an error, return errors + when explicitely deleted handles are reused, ...) + - RMA: multiple fixes and stability improvements. + - analysis (-analyze flag in smpirun): + - SMPI can now report buffer leaks as well as MPI handles leaks, + if code was compiled without SMPI_NO_OVERRIDE_MALLOC. + - if -trace-call-location is used when compiling, SMPI can report + origin of leaked handles/buffers + - group leaks by type/origin in output message if possible + - New implemented MPI calls: MPI_Comm_test_inter + +Models: + - Changed internal implementation of bandwidth factors in network models. + Models affected: CM02, LV08 (default), SMPI, IB. + Configuration affected: "network/bandwidth-factors" and "smpi/bw-factors". + Bandwidth factors are applied to communications to describe that users + cannot use 100% of the available bandwidth. For example, the default network model, + LV08, applies a factor of 0.97 to the bandwidth. In older versions, this + behavior was implemented by limiting the bandwidth available in the LMM + system for this flow. This may give the false impression that there is + bandwidth available for other flows due to its underutilization, especially + for the dynamic bandwidth factors used in SMPI models. + To avoid this, we have modified the implementation so that each flow uses the + maximum physical bandwidth according to the LMM system. + However, the actual throughput of the flow seen by the user is defined by + the physical bandwidth multiplied by the bandwidth factor. + This change impacts on the simulation results for all network models on + which we have bandwidth factors configured. + ***************************************** + *DO NOT MIX 3.28 RESULTS WITH OLDER ONES* + ***************************************** + This change may impact on the timing of your simulation results. + Take care when comparing simulations from different SimGrid's + versions. Sorry for the inconvenience. + - Dynamic network factors: users can configure a callback to define + the network factors dynamically. This API is available at + simgrid::kernel::resource::NetworkModelIntf. + - Users have access to complete information about the current communication + to decide which factor to apply. This includes: message size, source and + destination hosts, links and zones traversed. + - Dynamic factors for both latency and bandwidth. + - For more details, see the example in (examples/cpp/network-factors). + - Plugin host_energy: the "watt_off" and "watt_per_state" host properties, + deprecated since version 3.24, are no longer supported. Instead, use + "wattage_off" and "wattage_per_state". + +XBT: + - xbt_assert is not disabled anymore, even when built with enable_debug=off. + +Documentation: + - New tutorial: Model-checking and formal assessment + - New sections: "Demystifying the routing" and "C++ platforms" + - Update and improve the part on visualization in MPI and Algo tutorials. + - Improve the section on routing: how to define it, how it's used internally + - Fix many issues, broken links and missing references in doxygen and Sphinx + +LUA: + - Lua platform files are deprecated. Their support will be dropped after v3.31. + +Simix: + - Legacy functions deprecated in this release: SIMIX_get_clock(), SIMIX_run(). + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#47: Complete and fix tests from teshuite/s4u/activity-lifecycle + - FG#64: Configuring smpi/IB-penalty-factors + - FG#67: Running computation concurrently with MPI_Iallreduce + - FG#69: Tutorial misleading users of pre-v3.26 versions of SimGrid + - FG#71: Segmentation fault on invalid gw_src/gw_dst + - GH#322: Issue when an actor kills his host vm + +---------------------------------------------------------------------------- + +SimGrid (3.27) March 29. 2021 + +The Ragnar Release (anniversary of the siege of Paris by Vikings in 845) + +Main user-visible changes: + - All functions marked as deprecated for one year were removed. + - Removed all that was related to the "Storage" type in all APIs and bindings. + "Disks" have been introduced more than a year ago to replace them for the better. + It's time to finish this replacement. + - SMPI ABI breaks from previous version: MPI global handles (predefined datatypes, + ops, COMM_WORLD) are now defined differently, and can be used in initializers, + avoiding "initializer element is not constant" errors with some codes. Please be + careful to recompile your MPI codes if switching to this new SimGrid version. + - Network model ns-3 now requires version ns-3.28 or higher. + - Removed some option aliases that were deprecated since v3.23 (June 2019) + - 'verbose-exit' is now 'debug/verbose-exit' + - 'simix/breakpoint' is now 'debug/breakpoint' + - 'clean-atexit' is now 'debug/clean-atexit' + +S4U: + - New functions: Comm::sendto_{init,async} to initiate a communication + on between two (possibly remote) hosts. + - Define new template functions Mailbox::get_unique(), returning a std::unique_ptr. + - Functions Mailbox::get() and Mailbox::get_async() are now templated with the + type of the pointee. Untyped functions are deprecated. Use Mailbox::get() + or Mailbox::get_async() if you really want to play with void*. + - Unify the interface of Activity::on_{start/activity} + - New function: Comm::get_dst_data() + - New function: Comm::set_payload_size() to set the amount of bytes to simulate. + +Models: + - ptask should now work better with multicore hosts (FG#37). + - Internal cleanups and refactoring around activities, to prepare for SimDAG++. + - Internal cleanups and refactoring around models, to prepare for programmatic + platform creation. + - Modernize our tests of the performance models + +Documentation: + - User manual: + - New section: Configuring the logs from the command line. + - New section: Programming API of logs. + - New section: Getting realistic results + - New section: Examples of SMPI Usage + - New part: SimGrid's Internals, with a section on Design goals + - Move back to the classical Breathe tool chain, and fix many small glitches. + +SMPI: + - Handle leak checking: SMPI will now display a message if MPI handles are still + present at the end of a simulation. Option smpi/list-leaks:n will display the n + first leaked pointers and their types. + - smpi/display-allocs option : Display Memory allocations analysis, with hints + about the largest allocations. This only works for allocations with malloc/calloc + and not compiled with SMPI_NO_OVERRIDE_MALLOC + - "-analyze" option in smpirun will activate smpi/display-allocs and smpi/display-timing + - New implemented MPI calls: MPI_File_set_size, MPI_Aint_add, MPI_Aint_diff, + MPI_Type_get_extent_x, MPI_Type_get_true_extent_x, MPI_Status_set_elements_x + - Support for MPI C++ datatypes (MPI_CXX_BOOL, MPI_CXX_FLOAT_COMPLEX, + MPI_CXX_DOUBLE_COMPLEX, MPI_CXX_LONG_DOUBLE_COMPLEX) + - Replay now opens and parses the trace file only once for all ranks when possible. + - Misc fixes: + - better handling of MPI_PROC_NULL + - MPI_Group_incl was broken in some cases + - PAPI tracing now fixed + - MPI_Errhandlers are properly handled for MPI_COMM_WORLD + - SMPI will now fail properly when receive size < sent size, instead of + just setting MPI_ERR_TRUNCATE and keep going. + - Proxy Apps ( https://framagit.org/simgrid/SMPI-proxy-apps ): + - Remove useless build patches and split build/run patches on the page + - Build in parallel when possible + - Test on yum-based systems + - New nightly tested Proxy-apps: + - OpenMPI mpi-test-suite + - Coral MCB, LSMS, Qbox, CLOMP + - ECP/Mantevo Cloverleaf3D, Ember + - Codevault bhtree_mpi + +Model checker: + - Only initialize the Dwarf parser when needed, saving 10 seconds in most cases. + - Start to overhaul the implementation to modernize it (still ongoing). + +XBT: + - Drop simgrid::xbt::demangle(). Please use boost::core::demangle() instead. + - Drop xbt_dynar_cursor_rm(), xbt_dynar_search_or_negative(), + xbt_str_split_quoted(), xbt_str_split_quoted_in_place(). + - Deprecate xbt_dynar_to_array(). + - When an exception occurs in kernel mode, display both the actor and kernel stacks. + +C binding and interface: + - Rename (lowercase) functions sg_actor_get_PID, sg_actor_get_PPID, sg_actor_by_PID. + - Constness requirement has been lowered for parameter argv of sg_actor_start() + and sg_actor_create() It's now (char* const*). The previous version is still + available as sg_actor_start_() and sg_actor_create_(). + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GH: issues on GitHub) + - FG#37: Parallel tasks are limited to 1 core per host + - FG#62: Running "smpirun -replay" on large networks + - FG!46: Fix a few potential memory leaks in SMPI colls + - GH#343: Wrong consumed energy when an exec timeouts + +---------------------------------------------------------------------------- + +SimGrid (3.26) December 15. 2020 + +The Release release (the French lockdown was eased today). + +Important user-visible changes: + - SimGrid now requires a compiler with C++14 support. + Sibling projects should upgrade their FindSimgrid.cmake + - Surf precision default value is now 1e-9, instead of 1e-5. This was changed as + several users had difficulties to understand issues when using high bandwidth or + small latency events. The new value was already the default for SMPI and + should not cause too much performance hit. It can change some simulation + timings if simulation was skipping a lot of small events (which was actually + an issue per see). The old value can be restored with --cfg=surf/precision:1e-5. + - On some network models (L07, CM02, LV08), an implicit loopback link is created + for local communication. Default bandwidth was 500MBps and latency was 15µs, + which was rather bad. Default values have been changed to 10GBps and 0 latency. + Options network/loopback-lat and network/loopback-bw have been added to adapt + these values if needed. Network and routing models already providing separate + loopbacks (clusters and such) are not impacted by this change. + - SMPI now adds 16 bytes of header to each message sent, in order to simulate + metadata that are added to each MPI message by MPI libraries (tag, + src/dest, and communicator information). This may change simulation times + for very small message sizes, but should be more realistic. + +New features: + - New routing 'WiFi' that makes it easier to specify a wifi network zone. + - New plugins: wifi_energy and link_load. See the associated examples. + - New examples: wifi networks, and communication suspend. + +General: + - LTO is now enabled for Intel/clang compilers. + - LTO behavior on GCC can be parameterized using LTO_EXTRA_FLAG in cmake. + Setting it to "auto" will use all available cores, while setting it to n will + use n cores to speedup link step (usage: cmake -DLTO_EXTRA_FLAG=4). + - Remove obsolete runtime option 'contexts/parallel-threshold'. + - Runtime option 'tracing/msg/process' renamed to 'tracing/actor'. The old name + has been kept for compatibility. + - Finally remove obsolete snake_case() aliases for runtime options. + - Further improve the documentation. + +S4U: + - the on_exit() of each actor is also executed when the simulation deadlocks. + - New functions: s4u::Activity:: suspend(), resume() and is_suspended() + An example is provided for s4u::Comm but it should work with Exec and Io. + +SMPI: + - Update proxy apps coverage of new ECP apps: 60+ apps now tested nightly on + https://framagit.org/simgrid/SMPI-proxy-apps + - MPI/IO: supports names starting with "./" + - Fortran: many MPI/IO bindings, support for MPI_UB/LB, C MPI_INT/FLOAT/DOUBLE + datatypes. + - smpicc/cxx/ff/f90 now will actually perform definition checks at link time. When + building shared libraries, this may cause issues, so environment variable + SMPI_NO_UNDEFINED_CHECK can be added to disable this. + - most temporary files should now be created in /tmp dir (or equivalent). + If this one does not allow execution of code (noexec flag), this may cause issues. + Please use another tmp directory (using TMPDIR or equivalent system variable) + in this case. + +Model-Checker: + - A large refactoring is ongoing, but this should be transparent for users so far. + +XBT: + - Drop xbt_str_split(). + +C binding and interface: + - The return type of the sg_actor_on_exit() callbacks is now 'void' + (int value was previously ignored) + - Many C functions were renamed to follow a common naming style + (sg_object_get_something and sg_object_set_something). As usual, the deprecated + names are kept until version 3.30. + - Many MSG tests were converted to the new S4U's interface in C, that + was extended for that. + +Fixed bugs (FG#.. -> FramaGit bugs; FG!.. -> FG merge requests) + (FG: issues on Framagit; GF: issues on GForge; GH: issues on GitHub) + - FG#41: Add sg_actor_create C interface - FG#43: xbt::random needs some care - FG#48: The Impossible Did Happen (yet again) + - FG#50: Suspending an actor executed at the current timestamp fails + - FG#52: Zero-seconds timeout : "That's in the past already" + - FG#53: Crash while using ns-3 network model + - FG#54: How to suspend a comm? + - FG!22: Stochastic Profiles - FG!24: Documentation and fix for xbt/random + - FG!35: Add a modeling hint for parallel links in doc + - FG!36: [xbt/random] Read/Write the state of the RNG + - FG!42: Link cumulated load plugin + - FG!44: Test condition variable wait_until+notify_all + - GF#18137: Allow different stack sizes? + - GH#128: Parallelization of simulation with --cfg=contexts/nthreads - GH#139: Allow pthread creation in SMPI + - GH#336: Packet-level simulation using SMPI? + - GH#345: Error in the LMM while migrating a VM + - GH#346: [SMPI] error while loading shared libraries: libsimgrid.so + - GH#352: pip install / python setup.py install fail to find pybind11 + - GH!337: Fix link_energy plugin for wifi platforms + - GH!339: Add Mailbox set_receiver method to python binding + - GH!344: Cast hugepages macros parameters to int64 ---------------------------------------------------------------------------- @@ -15,76 +753,74 @@ SimGrid (3.25) February 2. 2020 (aka 02 02 2020) The Palindrom Day Release. Important user-visible changes: -- Improve the Python usability (stability and documentation). - - A nasty synchronization bug was ironed out, see also below. - - Python's doc was organized as a separate tree, now integrated with C++. - - C bindings of S4U were not part of the doc. - - The C++ doc was also improved as methods are now split by theme. -- Further deprecate MSG: you now have to pass -Denable_msg=ON to cmake. - - This is mandatory to use the Java bindings. - - OFF by default; The plan is to completely remove MSG by 2020Q4 or 2021Q1. -- SimDAG++: Automatic dependencies on S4U activities (experimental) - - Some features are already implemented but not all of them - - Cannot block an activity until it's scheduled on a resource - - No heterogeneous wait_any() that would mix Exec/Comm/Io activities. - - See examples/s4u/{io,exec,comm}-dependent for what's already there. - - + - Improve the Python usability (stability and documentation). + - A nasty synchronization bug was ironed out, see also below. + - Python's doc was organized as a separate tree, now integrated with C++. + - C bindings of S4U were not part of the doc. + - The C++ doc was also improved as methods are now split by theme. + - Further deprecate MSG: you now have to pass -Denable_msg=ON to cmake. + - This is mandatory to use the Java bindings. + - OFF by default; The plan is to completely remove MSG by 2020Q4 or 2021Q1. + - SimDAG++: Automatic dependencies on S4U activities (experimental) + - Some features are already implemented but not all of them + - Cannot block an activity until it's scheduled on a resource + - No heterogeneous wait_any() that would mix Exec/Comm/Io activities. + - See examples/s4u/{io,exec,comm}-dependent for what's already there. General: -- Upgrade documented Java dependency to Java 8 (earlier versions don't work). -- Drop support for Viva/Triva (old visualization tools), hence removed - TRACE_get_node_types() and TRACE_get_edge_types() functions. + - Upgrade documented Java dependency to Java 8 (earlier versions don't work). + - Drop support for Viva/Triva (old visualization tools), hence removed + TRACE_get_node_types() and TRACE_get_edge_types() functions. Python: -- Require pybind11 version 2.4. -- Greatly improve locking during context switches with Python. - - Declaring Python callbacks to this_actor::on_exit is now working. - - No more tests are failing currently, even on win32. + - Require pybind11 version 2.4. + - Greatly improve locking during context switches with Python. + - Declaring Python callbacks to this_actor::on_exit is now working. + - No more tests are failing currently, even on win32. S4U: -- Actor: Merge signals on_migration_start/end into on_host_change -- Actor: Rename migrate() into set_host() -- Disk: Allow users to get the read and write nominal bandwidth values -- Exec: Implement wait_for(timeout) -- Io: Implement wait_for(timeout) + - Actor: Merge signals on_migration_start/end into on_host_change + - Actor: Rename migrate() into set_host() + - Disk: Allow users to get the read and write nominal bandwidth values + - Exec: Implement wait_for(timeout) + - Io: Implement wait_for(timeout) XML: -- Parse errors now raise a simgrid::ParseError that you may want to catch. + - Parse errors now raise a simgrid::ParseError that you may want to catch. Kernel: -- In simgrid::kernel::resource::Model, the methods next_occuring_event*() have - been renamed to fix a spelling error. As usual, the previous definitions are - kept with a deprecation warning. However, to avoid runtime errors, it is now - forbidden to override these deprecated methods in a derived class. Please use - the new names immediately if you need to override them. + - In simgrid::kernel::resource::Model, the methods next_occuring_event*() have + been renamed to fix a spelling error. As usual, the previous definitions are + kept with a deprecation warning. However, to avoid runtime errors, it is now + forbidden to override these deprecated methods in a derived class. Please use + the new names immediately if you need to override them. XBT: -- Remove unused parameter 'free_ctn' for xbt_dict_set() and xbt_dict_set_ext(). -- Drop unused functions for dicts and dynars: xbt_dict_cursor_get_elm, - xbt_dict_get, xbt_dict_get_elm, xbt_dict_get_ext, xbt_dict_get_key, - xbt_dict_remove, xbt_dict_reset, xbt_dynar_compare, xbt_dynar_dump, - xbt_dynar_foreach_ptr, xbt_dynar_free_data, xbt_dynar_free_voidp, - xbt_dynar_init, xbt_dynar_insert_at_as, xbt_dynar_merge, - xbt_dynar_remove_n_at, xbt_dynar_replace, xbt_dynar_search, xbt_dynar_set. -- New module: random, providing classical random numbers generators. + - Remove unused parameter 'free_ctn' for xbt_dict_set() and xbt_dict_set_ext(). + - Drop unused functions for dicts and dynars: xbt_dict_cursor_get_elm, + xbt_dict_get, xbt_dict_get_elm, xbt_dict_get_ext, xbt_dict_get_key, + xbt_dict_remove, xbt_dict_reset, xbt_dynar_compare, xbt_dynar_dump, + xbt_dynar_foreach_ptr, xbt_dynar_free_data, xbt_dynar_free_voidp, + xbt_dynar_init, xbt_dynar_insert_at_as, xbt_dynar_merge, + xbt_dynar_remove_n_at, xbt_dynar_replace, xbt_dynar_search, xbt_dynar_set. + - New module: random, providing classical random numbers generators. SMPI: -- New option : "smpi/auto-shared-malloc-thresh" which sets a value for - allocations' size above which they are considered "shared" by default (as if - they were performed through SMPI_SHARED_MALLOC macros). - Default = 0 = disabled feature. - Note : malloc, calloc and free are now overriden by smpicc/cxx by default. - This can cause some troubles if codes are already overriding these. If this - is the case, defining SMPI_NO_OVERRIDE_MALLOC in the compilation flags can - help, but will make this feature unusable. -- Input check is now performed much more consistently and outputs precise - warnings for each error found -- New supported calls : MPI_File_set_view, MPI_File_get_view -- MPI I/O now only supports disks, as storage is discontinued. -- MPI I/O allows opening files without specifying absolute path -- Note : SMPI configuration options are now initialized by smpi_init_options() - call, which should be called if SMPI is not being used through smpirun. + - New option : "smpi/auto-shared-malloc-thresh" which sets a value for + allocations' size above which they are considered "shared" by default (as if + they were performed through SMPI_SHARED_MALLOC macros). + Default = 0 = disabled feature. + Note : malloc, calloc and free are now overridden by smpicc/cxx by default. + This can cause some troubles if codes are already overriding these. If this + is the case, defining SMPI_NO_OVERRIDE_MALLOC in the compilation flags can + help, but will make this feature unusable. + - Input check is now performed much more consistently and outputs precise + warnings for each error found + - New supported calls : MPI_File_set_view, MPI_File_get_view + - MPI I/O now only supports disks, as storage is discontinued. + - MPI I/O allows opening files without specifying absolute path + - Note : SMPI configuration options are now initialized by smpi_init_options() + call, which should be called if SMPI is not being used through smpirun. Fixed bugs (FG#.. -> framagit bugs; FG!.. -> framagit merge requests): - FG#9: Python bindings crashing @@ -395,7 +1131,7 @@ Documentation: (documenting all internals this way is overkill + doxygen scales badly) - New tutorial on S4U (highly inspired from the old MSG one, but with a git repository to fork as a starting point, and a docker image) - - Started but not finished a SMPI tutorial inspired from the JLPC'14 one. + - Started but not finished an SMPI tutorial inspired from the JLPC'14 one. - The Developper manual is still to be converted (not compiled until then) - Some parts are still missing in this conversion (such as the platform and deployment sections) while others were blindly converted @@ -531,7 +1267,7 @@ Plugins: - Rename Energy plugin into host_energy - Rename Load plugin into host_load -simix: +Simix: - Add parameter --cfg=simix/breakpoint to raise a SIGTRAP at given time. - kill simix::onDeadlock() that was somewhat dupplicating s4u::on_deadlock() - Improve performance when handling timeouts of simix synchros. @@ -823,7 +1559,7 @@ SimGrid (3.16) Released June 22. 2017. * This is (mainly) a backward compatible change: v4 are valid v4.1 files - can be used as a synonym for the now deprecated - can be used as a synonym for the now deprecated - - an be used as a synonym for the now deprecated + - can be used as a synonym for the now deprecated - can be used as a synonym for the now deprecated - state_file and avail_file periodicity is now easier to express (check the documentation) the old behavior should still work. @@ -1759,7 +2495,7 @@ SimGrid (3.8) stable; urgency=low TRACE: * Two new tracing options for adding comments to trace file so you can track your experiments (see --help-tracing for details). - * New option to generate a impoverished trace file (--cfg=tracing/basic:1) + * New option to generate an impoverished trace file (--cfg=tracing/basic:1) * Adding the SimGrid version that generated the trace file as a comment. * Instrumenting other MSG functions (MSG_task_isend_with_matching and MSG_task_dsend) * Fix to avoid key clashes on Paje links @@ -3710,7 +4446,7 @@ SimGrid (2.90) unstable; urgency=low MSG but a few things really had to disappear. The main differences with the previous version are: 1) no more m_links_t and the corresponding functions. Platforms are - directly read from a XML description and cannot be hard-coded + directly read from an XML description and cannot be hard-coded anymore. The same format is used for application deployment description. The new format is described in the documentation. Have a look in tools/platform_generation. There is a tiny script