Logo AND Algorithmique Numérique Distribuée

Public GIT Repository
simgrid.git
2 years agoAdjust timing of SMPI tests
Bruno Donassolo [Mon, 31 May 2021 12:52:56 +0000 (14:52 +0200)]
Adjust timing of SMPI tests

For sure SMPI is the most impacted by the bandwidth factors changes.
It seems especially impacted when collective comms are involved.

The old version used 2 factors for SMPI comms:
1) network/bandwidth-factor was used to reduce the link capacity (e.g.
0.97*C for LV08)
2) smpi/bw-factor was used for each communication, limitating the flow
capacity.

Now, the code is simplified, each communication has only 1 bw-factor
that is applied after it's done, at the update remaining phase.
In most cases, only the bw-factor are applied now but after the comm is
done, not before

2 years agoFix link-load test
Bruno Donassolo [Mon, 31 May 2021 08:51:47 +0000 (10:51 +0200)]
Fix link-load test

In the old version, our links capacity were 0.97*C, now it's just C.
So, more bytes can be transmitted through the links

2 years agoFix timing of Vivaldi/two_peers.xml tests
Bruno Donassolo [Mon, 31 May 2021 08:41:40 +0000 (10:41 +0200)]
Fix timing of Vivaldi/two_peers.xml tests

In this platform file, the communication is bounded by the big latency
between nodes (as consequence of the distance between vivaldi
coordinates).

Therefore, we now apply the bandwidth factor (0.97) on top of this
time, delaying a little the communications.

2 years agoFix timing of Wi-Fi tests.
Bruno Donassolo [Mon, 31 May 2021 08:18:41 +0000 (10:18 +0200)]
Fix timing of Wi-Fi tests.

Their timing were calculated considering that no factor were applied in
Wi-Fi communications.
This isn't the case anymore, since by default, we would apply the 0.97
factor from LV08 to these communications.
Setting CM02 as base network model since it doesn't apply any bandwidth
factor.

2 years agoAdjust dynamic network-factors test.
Bruno Donassolo [Fri, 28 May 2021 10:08:26 +0000 (12:08 +0200)]
Adjust dynamic network-factors test.

Improve test, including no crosstraffic config.

2 years agoNew implementation for bandwidth factors
Bruno Donassolo [Thu, 27 May 2021 14:57:57 +0000 (16:57 +0200)]
New implementation for bandwidth factors

Bandwidth factors are now implemented at the Action level, reducing the
speed that an action advances (e.g. the number of bytes transmitted).

In the past, the factor took place at the maxmin system, limiting the
amount of resources a communication could use.

For example, a bw factor of 0.97 (default for LV08 network model) was
reflected by reducing the link capacity to 0.97*C. So, a 100MBs link had
97MBs capacity in the maxmin system.

Now, a communication alone using this link may use the 100MBs, but after
one second, it'll transmit only 97MB of data.

NOTE: This change may impact the timing of your experiments.

2 years agoMissing include.
Arnaud Giersch [Tue, 1 Jun 2021 13:51:01 +0000 (15:51 +0200)]
Missing include.

2 years agoCoding style: no global "using namespace".
Arnaud Giersch [Tue, 1 Jun 2021 13:30:44 +0000 (15:30 +0200)]
Coding style: no global "using namespace".

2 years agoPrefer std algorithms.
Arnaud Giersch [Tue, 1 Jun 2021 13:26:10 +0000 (15:26 +0200)]
Prefer std algorithms.

2 years agoInvert tests to reduce depth of nesting.
Arnaud Giersch [Tue, 1 Jun 2021 12:53:37 +0000 (14:53 +0200)]
Invert tests to reduce depth of nesting.

2 years agoUseless tests for emptyness.
Arnaud Giersch [Tue, 1 Jun 2021 12:50:07 +0000 (14:50 +0200)]
Useless tests for emptyness.

2 years agoAdd attribute(printf) to xbt::string_printf.
Arnaud Giersch [Tue, 1 Jun 2021 10:59:34 +0000 (12:59 +0200)]
Add attribute(printf) to xbt::string_printf.

2 years agoPointer-to-const for Sonar.
Arnaud Giersch [Tue, 1 Jun 2021 07:38:57 +0000 (09:38 +0200)]
Pointer-to-const for Sonar.

2 years agoSet shared variable *before* mutex unlock.
Arnaud Giersch [Mon, 31 May 2021 21:20:24 +0000 (23:20 +0200)]
Set shared variable *before* mutex unlock.

2 years agoDie on unwanted function calls.
Arnaud Giersch [Mon, 31 May 2021 21:01:42 +0000 (23:01 +0200)]
Die on unwanted function calls.

2 years agoParameter 'assert' is a bit field.
Arnaud Giersch [Mon, 31 May 2021 20:53:55 +0000 (22:53 +0200)]
Parameter 'assert' is a bit field.

2 years agoFixes in UTs
Bruno Donassolo [Mon, 31 May 2021 17:22:28 +0000 (19:22 +0200)]
Fixes in UTs

2 years agoHandle case where different groups are given to MPI_Win_start and MPI_Win_post on...
Arnaud Giersch [Mon, 31 May 2021 13:40:40 +0000 (15:40 +0200)]
Handle case where different groups are given to MPI_Win_start and MPI_Win_post on a seame process.

2 years agoHandle duplicated datatypes within predefined MPI_Op.
Arnaud Giersch [Mon, 31 May 2021 13:14:10 +0000 (15:14 +0200)]
Handle duplicated datatypes within predefined MPI_Op.

2 years agoDocker: try to get apt update to run despite the cache
Martin Quinson [Mon, 31 May 2021 10:33:06 +0000 (12:33 +0200)]
Docker: try to get apt update to run despite the cache

2 years agoA few more 'const'.
Arnaud Giersch [Mon, 31 May 2021 09:32:23 +0000 (11:32 +0200)]
A few more 'const'.

2 years agoReplace redundant type with 'auto'.
Arnaud Giersch [Mon, 31 May 2021 09:28:24 +0000 (11:28 +0200)]
Replace redundant type with 'auto'.

2 years agoRemove useless temporary shadowing outer variable.
Arnaud Giersch [Mon, 31 May 2021 09:24:54 +0000 (11:24 +0200)]
Remove useless temporary shadowing outer variable.

2 years agoAdd some 'const' qualifiers.
Arnaud Giersch [Mon, 31 May 2021 08:22:02 +0000 (10:22 +0200)]
Add some 'const' qualifiers.

This started with NetPoint::get_englobing_zone() and propagated quickly...

2 years agoUnused exception parameter 'e'.
Arnaud Giersch [Mon, 31 May 2021 07:44:22 +0000 (09:44 +0200)]
Unused exception parameter 'e'.

2 years agoExplicitly delete the copy constructor and copy assignment operator (enforce rule...
Arnaud Giersch [Mon, 31 May 2021 07:40:15 +0000 (09:40 +0200)]
Explicitly delete the copy constructor and copy assignment operator (enforce rule-of-five).

2 years agoNo valgrind leak check for issue71.
Arnaud Giersch [Sat, 29 May 2021 12:25:53 +0000 (14:25 +0200)]
No valgrind leak check for issue71.

2 years agoxbt_replay: rethrow exception instead of xbt_die
Millian Poquet [Fri, 28 May 2021 21:03:01 +0000 (23:03 +0200)]
xbt_replay: rethrow exception instead of xbt_die

2 years agoRemove sg_bandwidth_factor from disks
Bruno Donassolo [Fri, 28 May 2021 19:16:55 +0000 (21:16 +0200)]
Remove sg_bandwidth_factor from disks

No need for bw_factor here

2 years agoremove a space to force update
Lucas M. Schnorr [Fri, 28 May 2021 19:10:50 +0000 (16:10 -0300)]
remove a space to force update

2 years agoIssue#71: add check in add_route for gw_src/gw_dst
Bruno Donassolo [Fri, 28 May 2021 17:49:45 +0000 (19:49 +0200)]
Issue#71: add check in add_route for gw_src/gw_dst

When adding a NetZoneRoute, check if gw_src and gw_dst belongs to the
respective netzones.

We need to recursively search for the netpoint inside the netzone since
the netpoint can be from one of its children.

2 years agoFix refcount for Datatype_contents.
Arnaud Giersch [Fri, 28 May 2021 15:16:30 +0000 (17:16 +0200)]
Fix refcount for Datatype_contents.

Error seen with Petsc test: vec_is_sf_tutorials-ex3_basic_dupped.

2 years agoAllow null ranks for MPI_Group_incl when n == 0.
Arnaud Giersch [Fri, 28 May 2021 09:52:30 +0000 (11:52 +0200)]
Allow null ranks for MPI_Group_incl when n == 0.

2 years agoupdate draw_gantt.R script to work with pajengr
Lucas M. Schnorr [Fri, 28 May 2021 14:48:12 +0000 (11:48 -0300)]
update draw_gantt.R script to work with pajengr

todo
- need to test with a recent version of S4U trace

2 years agoupdate pajengr installation procedure in S4U tutorial
Lucas M. Schnorr [Fri, 28 May 2021 14:41:29 +0000 (11:41 -0300)]
update pajengr installation procedure in S4U tutorial

2 years agoreplace r-cran-dplyr by r-cran-devtools
Lucas M. Schnorr [Fri, 28 May 2021 14:36:59 +0000 (11:36 -0300)]
replace r-cran-dplyr by r-cran-devtools

2 years agoupdate tuto-smpi Dockerfile to contain r-cran-tidyverse (replacing included R packages)
Lucas M. Schnorr [Fri, 28 May 2021 14:35:35 +0000 (11:35 -0300)]
update tuto-smpi Dockerfile to contain r-cran-tidyverse (replacing included R packages)

2 years agoadd additional necessary packages for pajengr installation
Lucas M. Schnorr [Fri, 28 May 2021 14:26:40 +0000 (11:26 -0300)]
add additional necessary packages for pajengr installation

2 years agoupdate pajengr R code and figure
Lucas M. Schnorr [Fri, 28 May 2021 13:56:14 +0000 (10:56 -0300)]
update pajengr R code and figure

details:
- note the addition of library(tidyverse)

2 years agouse new option in random test.
Augustin Degomme [Thu, 27 May 2021 14:55:48 +0000 (16:55 +0200)]
use new option in random test.

2 years agodocument new option
Augustin Degomme [Thu, 27 May 2021 14:47:48 +0000 (16:47 +0200)]
document new option

2 years agoAdd flag to provide an optional barrier in MPI_Finalize.
Augustin Degomme [Thu, 27 May 2021 13:40:44 +0000 (15:40 +0200)]
Add flag to provide an optional barrier in MPI_Finalize.
This is meant to help for codes which can aggressively cleanup memory at finalization, while other processes still have to use it.
For example when attributes are attached to a local communicator, in SMPI this communicator may be cleaned up by another process in the end, and the attribute has to still be valid at this point.
This was an issue with PETSC, for example.

2 years agoUse std::string for xbt_parse_units.
Arnaud Giersch [Thu, 27 May 2021 09:04:25 +0000 (11:04 +0200)]
Use std::string for xbt_parse_units.

Combine parameters 'entity_kind' and 'name'.
Also rename surf_parse_* to xbt_parse_*.

2 years agoDeprecate SIMIX_get_clock().
Arnaud Giersch [Thu, 27 May 2021 08:35:46 +0000 (10:35 +0200)]
Deprecate SIMIX_get_clock().

Use simgrid_get_clock() or Engine::get_clock().

2 years agoKeyval should always exist.
Arnaud Giersch [Thu, 27 May 2021 07:52:59 +0000 (09:52 +0200)]
Keyval should always exist.

Also please sonar by reducing depth for nested blocks.

2 years agoleaks analysis: when -analyze is given to smpirun, set value for list-leaks to 50.
Augustin Degomme [Thu, 27 May 2021 09:13:29 +0000 (11:13 +0200)]
leaks analysis: when -analyze is given to smpirun, set value for list-leaks to 50.

2 years agoleaks analysis : activate if list-leaks is asked manually (without smpi/display-alloc...
Augustin Degomme [Thu, 27 May 2021 09:12:17 +0000 (11:12 +0200)]
leaks analysis : activate if list-leaks is asked manually (without smpi/display-allocs or -analyze)

2 years agomake simix_global->maestro_ private
SUTER Frederic [Wed, 26 May 2021 14:39:49 +0000 (16:39 +0200)]
make simix_global->maestro_ private

2 years agoCheck types matching in MPI communications.
Augustin Degomme [Wed, 26 May 2021 14:49:56 +0000 (16:49 +0200)]
Check types matching in MPI communications.
should work with basic ones and duplicated ones.
todo: derived types matching, which is more tricky.

2 years agomark these as derived datatypes, and don't divide by 0 if possible
Augustin Degomme [Wed, 26 May 2021 14:48:09 +0000 (16:48 +0200)]
mark these as derived datatypes, and don't divide by 0 if possible

2 years agokeep track of which datatype was duplicated, to be able to match with it later
Augustin Degomme [Wed, 26 May 2021 14:47:22 +0000 (16:47 +0200)]
keep track of which datatype was duplicated, to be able to match with it later

2 years agonothing old here, just the type ..
Augustin Degomme [Wed, 26 May 2021 09:44:45 +0000 (11:44 +0200)]
nothing old here, just the type ..

2 years agoprepare transition of context_factory: make it private
SUTER Frederic [Wed, 26 May 2021 10:37:29 +0000 (12:37 +0200)]
prepare transition of context_factory: make it private

2 years agoMinor simplification in CM02::communicate
Bruno Donassolo [Wed, 26 May 2021 10:17:33 +0000 (12:17 +0200)]
Minor simplification in CM02::communicate

Don't need a function in NetworkModel for an "if" used only in CM02

2 years agoFix potential mem leak seen by scan-build.
Arnaud Giersch [Wed, 26 May 2021 07:30:23 +0000 (09:30 +0200)]
Fix potential mem leak seen by scan-build.

Note that ret should not normally be different than MPI_SUCCESS in the other case.

2 years agoDon't create mailbox before engine is turned on.
Arnaud Giersch [Tue, 25 May 2021 15:46:43 +0000 (17:46 +0200)]
Don't create mailbox before engine is turned on.

2 years agoLookup only once on mailbox creation.
Arnaud Giersch [Tue, 25 May 2021 14:58:42 +0000 (16:58 +0200)]
Lookup only once on mailbox creation.

2 years agoMisc sonar smells.
Arnaud Giersch [Tue, 25 May 2021 14:21:20 +0000 (16:21 +0200)]
Misc sonar smells.

2 years agovector::back() is good enough + assert for scan_build.
Arnaud Giersch [Tue, 25 May 2021 14:17:20 +0000 (16:17 +0200)]
vector::back() is good enough + assert for scan_build.

2 years ago[cppcheck] Don't let fields uninitialized.
Arnaud Giersch [Thu, 6 May 2021 18:21:59 +0000 (20:21 +0200)]
[cppcheck] Don't let fields uninitialized.

2 years ago[sonar] Extract assignments from expressions.
Arnaud Giersch [Thu, 6 May 2021 13:49:55 +0000 (15:49 +0200)]
[sonar] Extract assignments from expressions.

2 years agoPlug remaining leaks in mpich3-test/f77/attr.
Arnaud Giersch [Sun, 23 May 2021 22:20:38 +0000 (00:20 +0200)]
Plug remaining leaks in mpich3-test/f77/attr.

2 years agoPlug memleaks with Fortran bindings.
Arnaud Giersch [Sun, 23 May 2021 20:40:39 +0000 (22:40 +0200)]
Plug memleaks with Fortran bindings.

2 years agoReview allocation of MPI attributes with Fortran bindings.
Arnaud Giersch [Sun, 23 May 2021 20:26:58 +0000 (22:26 +0200)]
Review allocation of MPI attributes with Fortran bindings.

2 years agoFix refcounting of MPI keyvals when keyval_free is called twice.
Arnaud Giersch [Tue, 25 May 2021 12:33:56 +0000 (14:33 +0200)]
Fix refcounting of MPI keyvals when keyval_free is called twice.

2 years agoSimplify a bit Keyval::attr_delete.
Arnaud Giersch [Tue, 25 May 2021 12:23:59 +0000 (14:23 +0200)]
Simplify a bit Keyval::attr_delete.

2 years agoDon't increase refcount when existing elem is overwritten.
Arnaud Giersch [Sat, 22 May 2021 20:15:05 +0000 (22:15 +0200)]
Don't increase refcount when existing elem is overwritten.

2 years agoSave an indirection with smpi keyvals.
Arnaud Giersch [Sat, 22 May 2021 19:25:00 +0000 (21:25 +0200)]
Save an indirection with smpi keyvals.

2 years agorevalidate tesh
SUTER Frederic [Tue, 25 May 2021 14:50:11 +0000 (16:50 +0200)]
revalidate tesh

2 years agoremove cruft [ci-skip]
SUTER Frederic [Tue, 25 May 2021 14:28:56 +0000 (16:28 +0200)]
remove cruft [ci-skip]

2 years agoChange way Mailboxes are create, stored, and destroyed
SUTER Frederic [Tue, 25 May 2021 14:13:40 +0000 (16:13 +0200)]
Change way Mailboxes are create, stored, and destroyed

- keep the global map in EngineImpl (not as a static global in
  MailboxImpl.cpp)
- Delete mailboxes in ~EngineImpl() and get rid of SIMIX_mailbox_exit
- replace MailboxImpl::by_name_or_null and
  MailboxImpl::by_name_or_create by Engine::mailbox_by_name_or_create
  + better match with what is done for hosts, links, and actors
  + Mailbox::by_name cause two searchs in the map. One to check if
    name already points to a mailbox (by_name_or_null) and if not
    search again before creating a new mailbox. As there is no
    Mailbox::by_name_or_null, just keep the latter.
- Revalidate a bunch of tests (message ordering mostly)

2 years agodelete Timer when shutting down EngineImpl
SUTER Frederic [Tue, 25 May 2021 13:17:58 +0000 (15:17 +0200)]
delete Timer when shutting down EngineImpl

2 years agoContinue to migrate contents of simix_global to EngineImpl
SUTER Frederic [Tue, 25 May 2021 12:03:10 +0000 (14:03 +0200)]
Continue to migrate contents of simix_global to EngineImpl

thx @agiersch for unblocking this commit with the usual magic!

2 years agoPrefer using emplace() to insert() for smpi keyvals/attributes.
Arnaud Giersch [Sat, 22 May 2021 11:33:17 +0000 (13:33 +0200)]
Prefer using emplace() to insert() for smpi keyvals/attributes.

2 years agoMake smpi::Keyval::attribute() return a reference.
Arnaud Giersch [Sat, 22 May 2021 11:20:02 +0000 (13:20 +0200)]
Make smpi::Keyval::attribute() return a reference.

2 years agoNo need to test for attributes()->empty().
Arnaud Giersch [Sun, 23 May 2021 12:41:53 +0000 (14:41 +0200)]
No need to test for attributes()->empty().

2 years agoFix parameter names.
Arnaud Giersch [Sun, 23 May 2021 20:40:22 +0000 (22:40 +0200)]
Fix parameter names.

2 years agoCoding style: ! -> 'not'.
Arnaud Giersch [Sat, 22 May 2021 16:22:43 +0000 (18:22 +0200)]
Coding style: ! -> 'not'.

2 years agoUpdate filter line for recent versions of Asan.
Arnaud Giersch [Sun, 23 May 2021 22:39:33 +0000 (00:39 +0200)]
Update filter line for recent versions of Asan.

2 years agoDefine variables outside of header file.
Arnaud Giersch [Thu, 20 May 2021 08:37:48 +0000 (10:37 +0200)]
Define variables outside of header file.

2 years ago[sonar] Unused parameter.
Arnaud Giersch [Thu, 20 May 2021 08:28:36 +0000 (10:28 +0200)]
[sonar] Unused parameter.

2 years ago[sonar] Const/explicit methods.
Arnaud Giersch [Thu, 20 May 2021 08:23:58 +0000 (10:23 +0200)]
[sonar] Const/explicit methods.

2 years agoPlease sonar
Bruno Donassolo [Tue, 25 May 2021 08:35:34 +0000 (10:35 +0200)]
Please sonar

2 years agoAdjust test timing
Bruno Donassolo [Thu, 20 May 2021 15:13:59 +0000 (17:13 +0200)]
Adjust test timing

Minor changement in rounding up.
No real changement in timing (manually verified).

2 years agoSingle get_bandwidth_constraint function.
Bruno Donassolo [Thu, 20 May 2021 15:07:52 +0000 (17:07 +0200)]
Single get_bandwidth_constraint function.

Unify behavior of get_bandwidth_constraint function.

Note: Possible side-effect in SMPI communications with user defined
rate. Do not scale the user rate with bandwidth factor.

No impact on tests

2 years agoRefactor of NetworkCm02Model::communicate
Bruno Donassolo [Wed, 19 May 2021 18:55:39 +0000 (20:55 +0200)]
Refactor of NetworkCm02Model::communicate

Divide and conquer.
Reorganize code.
Cosmetics.

2 years agoDynamic factors: implement and test
Bruno Donassolo [Mon, 17 May 2021 17:26:18 +0000 (19:26 +0200)]
Dynamic factors: implement and test

Fully implements the dynamic factors through the NetworkModelIntf API.

Added an example showing its usage.

Added UT and doc.

2 years agoNetworkModelIntf: Interface to dynamic factors
Bruno Donassolo [Fri, 26 Mar 2021 17:27:38 +0000 (18:27 +0100)]
NetworkModelIntf: Interface to dynamic factors

Adding API to set user callbacks for bandwidth and network factors.
Valid for CM02 models and its childrens (SMPI, IB, etc).

Not fully working yet.

2 years agoGet netzones traversed by a route
Bruno Donassolo [Thu, 13 May 2021 16:41:26 +0000 (18:41 +0200)]
Get netzones traversed by a route

Preparing field to implement dynamic bandwidth and latency factors

2 years agomigrate actors_to_run and actors_that_ran to EngineImpl
SUTER Frederic [Fri, 21 May 2021 13:38:56 +0000 (15:38 +0200)]
migrate actors_to_run and actors_that_ran to EngineImpl

2 years agoSMPI should not eat up all exceptions in wait/waitany simcalls silently
Augustin Degomme [Fri, 21 May 2021 10:37:39 +0000 (12:37 +0200)]
SMPI should not eat up all exceptions in wait/waitany simcalls silently
It only deals with cancel ones

2 years agotypo --
Augustin Degomme [Fri, 21 May 2021 10:36:15 +0000 (12:36 +0200)]
typo --

2 years agoprocess -> actor
SUTER Frederic [Fri, 21 May 2021 07:46:15 +0000 (09:46 +0200)]
process -> actor

2 years agofix changelog [ci-skip]
Martin Quinson [Fri, 21 May 2021 08:12:08 +0000 (10:12 +0200)]
fix changelog [ci-skip]

2 years agoAdd a word to the tutorial for older versions of SimGrid
Martin Quinson [Fri, 21 May 2021 08:03:19 +0000 (10:03 +0200)]
Add a word to the tutorial for older versions of SimGrid

Closes https://framagit.org/simgrid/simgrid/-/issues/69

2 years ago [sonar] const
SUTER Frederic [Thu, 20 May 2021 07:30:18 +0000 (09:30 +0200)]
 [sonar] const

2 years ago[sonar] Don't mix public and private data members
SUTER Frederic [Thu, 20 May 2021 07:21:37 +0000 (09:21 +0200)]
[sonar] Don't mix public and private data members

2 years agointroduce a new plugin: stock implementation of a Producer-Consumer with a monitor
SUTER Frederic [Wed, 19 May 2021 16:27:18 +0000 (18:27 +0200)]
introduce a new plugin: stock implementation of a Producer-Consumer with a monitor

2 years agoTypos.
Arnaud Giersch [Wed, 19 May 2021 10:52:00 +0000 (12:52 +0200)]
Typos.

2 years agomigrate daemons from simix::Global to kernel::EngineImpl
SUTER Frederic [Mon, 17 May 2021 18:05:49 +0000 (20:05 +0200)]
migrate daemons from simix::Global to kernel::EngineImpl

go from a public vector to a private set (order doesn't matter) in the
process.