Logo AND Algorithmique Numérique Distribuée

Public GIT Repository
simgrid.git
3 years agogeneralize previous change to other MPI predefined globals.
Augustin Degomme [Mon, 8 Mar 2021 08:10:07 +0000 (09:10 +0100)]
generalize previous change to other MPI predefined globals.
Except MPI_COMM_WORLD because of its specificities (has to be a variable)

3 years agosonar is right here
Augustin Degomme [Mon, 8 Mar 2021 07:25:12 +0000 (08:25 +0100)]
sonar is right here

3 years agochange the way MPI datatypes are declared.
Augustin Degomme [Mon, 8 Mar 2021 00:05:00 +0000 (01:05 +0100)]
change the way MPI datatypes are declared.
Our previous solution did not let everything seen as const, and it was preventing some codes to compile (initializer element is not constant).
This allows to compile  https://github.com/open-mpi/mpi-test-suite.git without changint the code (will be added in proxy apps) in tst_types.c
todo : other types (op and co)

3 years agodebug message --
Augustin Degomme [Sun, 7 Mar 2021 19:38:24 +0000 (20:38 +0100)]
debug message --

3 years agoUse flags to handle cancellation instead of a tri-state int
Augustin Degomme [Sun, 7 Mar 2021 17:36:40 +0000 (18:36 +0100)]
Use flags to handle cancellation instead of a tri-state int

3 years agoChange handling of MPI_PROC_NULL as a src or a dest.
Augustin Degomme [Sun, 7 Mar 2021 17:10:58 +0000 (18:10 +0100)]
Change handling of MPI_PROC_NULL as a src or a dest.
We used to just skip the requests, but it was not possible to set status->MPI_SOURCE later in this case.

3 years agoThat was a stupid optimization.
Augustin Degomme [Sun, 7 Mar 2021 13:03:21 +0000 (14:03 +0100)]
That was a stupid optimization.
Processes can be reordered, so count is not a guarantee here at all.

3 years agoRegenerate.
Arnaud Giersch [Sun, 7 Mar 2021 12:58:45 +0000 (13:58 +0100)]
Regenerate.

3 years agoKill superfluous macros.
Arnaud Giersch [Sat, 6 Mar 2021 21:19:02 +0000 (22:19 +0100)]
Kill superfluous macros.

3 years agoZero-initialize struct to avoid valgrind warnings.
Arnaud Giersch [Sat, 6 Mar 2021 17:06:08 +0000 (18:06 +0100)]
Zero-initialize struct to avoid valgrind warnings.

3 years agoInline trivial function.
Arnaud Giersch [Sat, 6 Mar 2021 13:02:02 +0000 (14:02 +0100)]
Inline trivial function.

3 years agoUpdate .mailmap.
Arnaud Giersch [Fri, 5 Mar 2021 13:20:05 +0000 (14:20 +0100)]
Update .mailmap.

3 years agoAt most one resource allocation per statement (sonar).
Arnaud Giersch [Fri, 5 Mar 2021 10:33:18 +0000 (11:33 +0100)]
At most one resource allocation per statement (sonar).

3 years agoMC: untangle dependencies
Martin Quinson [Sat, 6 Mar 2021 22:14:23 +0000 (23:14 +0100)]
MC: untangle dependencies

3 years agoMC: more symmetry between WAITANY and TESTANY in dependency
Martin Quinson [Sat, 6 Mar 2021 22:03:37 +0000 (23:03 +0100)]
MC: more symmetry between WAITANY and TESTANY in dependency

No idea why TESTANY was doing specific cases on the first iteration,
nor even why it worked this way. But it was preventing the next commit
from occurring.

3 years agoMC: untangle the dependency code a tiny bit
Martin Quinson [Sat, 6 Mar 2021 20:57:56 +0000 (21:57 +0100)]
MC: untangle the dependency code a tiny bit

3 years agoMC: inline a function, kill a file
Martin Quinson [Sat, 6 Mar 2021 13:27:04 +0000 (14:27 +0100)]
MC: inline a function, kill a file

3 years agoAdd multiple unimplemented calls in smpi.h
Augustin Degomme [Sat, 6 Mar 2021 14:10:14 +0000 (15:10 +0100)]
Add multiple unimplemented calls in smpi.h
List gathered from mpi4py configuration tool

3 years agoImplement MPI_Aint_add and MPI_Aint_diff
Augustin Degomme [Sat, 6 Mar 2021 13:37:14 +0000 (14:37 +0100)]
Implement MPI_Aint_add and MPI_Aint_diff

3 years ago"Implement" MPI_Type_get_extent_x, MPI_Type_get_true_extent_x, and MPI_Status_set_ele...
Augustin Degomme [Sat, 6 Mar 2021 13:36:26 +0000 (14:36 +0100)]
"Implement" MPI_Type_get_extent_x, MPI_Type_get_true_extent_x, and MPI_Status_set_elements_x

3 years agodoc: add an hint on platform calibration, and various fixes
Martin Quinson [Fri, 5 Mar 2021 23:30:55 +0000 (00:30 +0100)]
doc: add an hint on platform calibration, and various fixes

3 years agosimgrid-mc also needs debugging symbols, so let's hint about it
Martin Quinson [Fri, 5 Mar 2021 23:00:30 +0000 (00:00 +0100)]
simgrid-mc also needs debugging symbols, so let's hint about it

3 years agodoc: fix a borken link
Martin Quinson [Fri, 5 Mar 2021 22:43:41 +0000 (23:43 +0100)]
doc: fix a borken link

3 years agoAdd a section presenting an overview of SimGrid design goals
Martin Quinson [Fri, 5 Mar 2021 22:03:36 +0000 (23:03 +0100)]
Add a section presenting an overview of SimGrid design goals

3 years agoclang ... override ... again
Frederic Suter [Fri, 5 Mar 2021 15:24:53 +0000 (16:24 +0100)]
clang ... override ... again

3 years agoRemove constraint from LinkImpl ctor
Frederic Suter [Fri, 5 Mar 2021 13:56:43 +0000 (14:56 +0100)]
Remove constraint from LinkImpl ctor

3 years agoadd boolean state to resources and protect set_core_count
Frederic Suter [Fri, 5 Mar 2021 11:31:01 +0000 (12:31 +0100)]
add boolean state to resources and protect set_core_count

3 years agofix pip builds
Martin Quinson [Fri, 5 Mar 2021 00:18:21 +0000 (01:18 +0100)]
fix pip builds

3 years agomc: rename Inspector to Observer
Martin Quinson [Thu, 4 Mar 2021 23:30:47 +0000 (00:30 +0100)]
mc: rename Inspector to Observer

3 years agoSimplify mc::api a bit
Martin Quinson [Thu, 4 Mar 2021 23:15:40 +0000 (00:15 +0100)]
Simplify mc::api a bit

No need to expose a call to test whether there is a deadlock and a
call to react to deadlocks. Have the first call react by itself to
this situation.

3 years agocosmetics
Martin Quinson [Thu, 4 Mar 2021 23:53:24 +0000 (00:53 +0100)]
cosmetics

3 years agoTry to use more common PAPI events for testing.
Arnaud Giersch [Thu, 4 Mar 2021 15:31:31 +0000 (16:31 +0100)]
Try to use more common PAPI events for testing.

Also fix the grep RE to not match the command line at the beginning of the file.

3 years agoDon't crash on empty PAPI event set.
Arnaud Giersch [Thu, 4 Mar 2021 15:20:26 +0000 (16:20 +0100)]
Don't crash on empty PAPI event set.

3 years agoCosmetics
Arnaud Giersch [Thu, 4 Mar 2021 15:14:16 +0000 (16:14 +0100)]
Cosmetics

3 years agouse CRTP dor resources
Frederic Suter [Wed, 3 Mar 2021 21:44:33 +0000 (22:44 +0100)]
use CRTP dor resources

3 years agoremove model from Link constructors
Frederic Suter [Wed, 3 Mar 2021 20:18:52 +0000 (21:18 +0100)]
remove model from Link constructors

3 years agoAdd a test for PAPI
Augustin Degomme [Wed, 3 Mar 2021 19:25:17 +0000 (20:25 +0100)]
Add a test for PAPI
next step : try to activate it on jenkins (for coverage, obviously)

3 years agoPAPI tracing was broken with latest instr changes.
Augustin Degomme [Wed, 3 Mar 2021 19:22:09 +0000 (20:22 +0100)]
PAPI tracing was broken with latest instr changes.
Expected because it's not tested.

3 years agoModernize simcall mutex_unlock.
Arnaud Giersch [Wed, 3 Mar 2021 14:40:35 +0000 (15:40 +0100)]
Modernize simcall mutex_unlock.

3 years agoMake SimcallInspector::dot_label and to_string pure virtual again.
Arnaud Giersch [Wed, 3 Mar 2021 16:46:30 +0000 (17:46 +0100)]
Make SimcallInspector::dot_label and to_string pure virtual again.

It was wronly removed in commit 99b0a6c6788a22a446be3519233dd524a44f1ffd.

3 years agoFix mem leak.
Arnaud Giersch [Wed, 3 Mar 2021 14:23:41 +0000 (15:23 +0100)]
Fix mem leak.

3 years agoPointer to const.
Arnaud Giersch [Wed, 3 Mar 2021 09:51:29 +0000 (10:51 +0100)]
Pointer to const.

3 years agostd::runtime error takes a const&.
Arnaud Giersch [Wed, 3 Mar 2021 09:47:45 +0000 (10:47 +0100)]
std::runtime error takes a const&.

3 years agoThese tests were missing mpi_finalize call (hence some leaks).
Augustin Degomme [Wed, 3 Mar 2021 11:55:21 +0000 (12:55 +0100)]
These tests were missing mpi_finalize call (hence some leaks).

3 years agochange message a bit
Augustin Degomme [Wed, 3 Mar 2021 11:54:48 +0000 (12:54 +0100)]
change message a bit

3 years agompi_comm_self was cleaned after mpi finalization during actor deletion, which was...
Augustin Degomme [Wed, 3 Mar 2021 11:54:33 +0000 (12:54 +0100)]
mpi_comm_self was cleaned after mpi finalization during actor deletion, which was annoying (after leaks check).
Have it done at MPI_Finalize level.

3 years agoAdd simple example using smpi_execute_* calls inside MPI code to simulate computation.
Augustin Degomme [Tue, 2 Mar 2021 22:36:56 +0000 (23:36 +0100)]
Add simple example using smpi_execute_* calls inside MPI code to simulate computation.
This could double as a basic MPI example, as we are missing one, I think.

3 years agoadd errhandlers support for mpi_comm_world
Augustin Degomme [Tue, 2 Mar 2021 15:00:07 +0000 (16:00 +0100)]
add errhandlers support for mpi_comm_world
As this communicator is actually shared, every process was overriding the errhandler, leading to using the same for every process, and leaking the others.

3 years agoadd leak checking for info and errhandlers
Augustin Degomme [Tue, 2 Mar 2021 14:58:38 +0000 (15:58 +0100)]
add leak checking for info and errhandlers

3 years agoRename mc_base.h -> .hpp (it's C++ only anyway).
Arnaud Giersch [Tue, 2 Mar 2021 19:54:07 +0000 (20:54 +0100)]
Rename mc_base.h -> .hpp (it's C++ only anyway).

3 years agoConst for methods.
Arnaud Giersch [Tue, 2 Mar 2021 15:24:04 +0000 (16:24 +0100)]
Const for methods.

3 years agoCosmetic rename.
Arnaud Giersch [Tue, 2 Mar 2021 15:20:54 +0000 (16:20 +0100)]
Cosmetic rename.

get_actor_string -> get_actor_dot_label
get_actor_string2 -> get_actor_string

3 years agoSimplify namespaces
Frederic Suter [Tue, 2 Mar 2021 00:30:45 +0000 (01:30 +0100)]
Simplify namespaces

3 years agocleanups
Frederic Suter [Mon, 1 Mar 2021 23:54:36 +0000 (00:54 +0100)]
cleanups

3 years agocosmetics
Frederic Suter [Mon, 1 Mar 2021 23:25:28 +0000 (00:25 +0100)]
cosmetics

3 years agoadd core_count setter
Frederic Suter [Mon, 1 Mar 2021 22:49:48 +0000 (23:49 +0100)]
add core_count setter

3 years agoAuto + std::make_unique.
Arnaud Giersch [Tue, 2 Mar 2021 15:13:45 +0000 (16:13 +0100)]
Auto + std::make_unique.

3 years agoUncomment.
Arnaud Giersch [Tue, 2 Mar 2021 14:57:09 +0000 (15:57 +0100)]
Uncomment.

3 years agoReduce code duplication.
Arnaud Giersch [Tue, 2 Mar 2021 14:26:06 +0000 (15:26 +0100)]
Reduce code duplication.

3 years agoProtect against buffer overflow.
Arnaud Giersch [Tue, 2 Mar 2021 13:58:47 +0000 (14:58 +0100)]
Protect against buffer overflow.

3 years agoMake Transition::textual a std::string.
Arnaud Giersch [Tue, 2 Mar 2021 13:45:21 +0000 (14:45 +0100)]
Make Transition::textual a std::string.

3 years agoRestore Rvalue references for xbt::Exception.
Arnaud Giersch [Fri, 26 Feb 2021 20:01:53 +0000 (21:01 +0100)]
Restore Rvalue references for xbt::Exception.

This was lost in commit 1361e008419329de33e4c92a6ad96621d40a9c40
and I don't remember any reason for that.

Exceptions are always built with temporaries, an d moving them should avoid
unnecessary copies.

3 years agoUncomment debug message.
Arnaud Giersch [Tue, 2 Mar 2021 10:52:21 +0000 (11:52 +0100)]
Uncomment debug message.

3 years agoMake field private (fixme--).
Arnaud Giersch [Tue, 2 Mar 2021 10:30:59 +0000 (11:30 +0100)]
Make field private (fixme--).

3 years agoUse std::copy.
Arnaud Giersch [Tue, 2 Mar 2021 10:09:32 +0000 (11:09 +0100)]
Use std::copy.

3 years agoType bool for boolean values.
Arnaud Giersch [Tue, 2 Mar 2021 09:41:28 +0000 (10:41 +0100)]
Type bool for boolean values.

3 years agoRemove some useless nested code blocks.
Arnaud Giersch [Mon, 1 Mar 2021 22:36:32 +0000 (23:36 +0100)]
Remove some useless nested code blocks.

3 years agoUsing C++ strings.
Arnaud Giersch [Mon, 1 Mar 2021 20:37:02 +0000 (21:37 +0100)]
Using C++ strings.

3 years agoUse xbt::string for MC hostnames.
Arnaud Giersch [Mon, 1 Mar 2021 16:21:04 +0000 (17:21 +0100)]
Use xbt::string for MC hostnames.

3 years agoKill useless function.
Arnaud Giersch [Mon, 1 Mar 2021 14:37:49 +0000 (15:37 +0100)]
Kill useless function.

3 years agoUse defined type alias.
Arnaud Giersch [Mon, 1 Mar 2021 14:31:09 +0000 (15:31 +0100)]
Use defined type alias.

3 years agoFix code generation (broken call to inspector_->is_pending), and reindent.
Arnaud Giersch [Mon, 1 Mar 2021 12:07:29 +0000 (13:07 +0100)]
Fix code generation (broken call to inspector_->is_pending), and reindent.

3 years agofix ns3
Frederic Suter [Mon, 1 Mar 2021 14:19:44 +0000 (15:19 +0100)]
fix ns3

3 years agoConsider Link latency as an optional parameter
Frederic Suter [Mon, 1 Mar 2021 13:14:55 +0000 (14:14 +0100)]
Consider Link latency as an optional parameter

3 years agomore fluent version
Frederic Suter [Mon, 1 Mar 2021 09:34:58 +0000 (10:34 +0100)]
more fluent version

3 years agoRollback a little in programmatic description of disks
Frederic Suter [Mon, 1 Mar 2021 09:24:24 +0000 (10:24 +0100)]
Rollback a little in programmatic description of disks
- add Required fields (w.r.t. the DTD) to the create_disk method
- leave the setters to enable changes of mind or dynamic stuff between
create and seal (TODO: add a state and some asserts)
- Resources always have a name so add it in the ctor

3 years agoFix usage string.
Arnaud Giersch [Mon, 1 Mar 2021 10:23:03 +0000 (11:23 +0100)]
Fix usage string.

3 years agoDeprecated simcall_mc_random.
Arnaud Giersch [Mon, 1 Mar 2021 09:47:19 +0000 (10:47 +0100)]
Deprecated simcall_mc_random.

Maybe this can be dropped immediately?.

3 years agoMake global variables 'const'.
Arnaud Giersch [Fri, 26 Feb 2021 14:23:07 +0000 (15:23 +0100)]
Make global variables 'const'.

3 years agoA few less global variables.
Arnaud Giersch [Fri, 26 Feb 2021 13:55:09 +0000 (14:55 +0100)]
A few less global variables.

3 years agoDon't require MC dependencies when building w/o MC
Martin Quinson [Mon, 1 Mar 2021 00:59:12 +0000 (01:59 +0100)]
Don't require MC dependencies when building w/o MC

3 years agoFix non-MC builds
Martin Quinson [Mon, 1 Mar 2021 00:46:38 +0000 (01:46 +0100)]
Fix non-MC builds

3 years agoFix make distcheck :-(
Martin Quinson [Mon, 1 Mar 2021 00:31:41 +0000 (01:31 +0100)]
Fix make distcheck :-(

3 years agoMissing file
Martin Quinson [Mon, 1 Mar 2021 00:30:18 +0000 (01:30 +0100)]
Missing file

3 years agoremove a workaround for a bug removed in previous commit
Martin Quinson [Mon, 1 Mar 2021 00:24:41 +0000 (01:24 +0100)]
remove a workaround for a bug removed in previous commit

Inspector::to_string() was sometimes called for simcalls w/o inspectors.

3 years agoSet the textual representation of a transition before executing it
Martin Quinson [Sun, 28 Feb 2021 23:38:00 +0000 (00:38 +0100)]
Set the textual representation of a transition before executing it

This way it will correctly be set if the App aborts or fails during that execution.

Also, don't use the execution path that is reserved to simcalls with
observers `mc_model_checker->simcall_to_string()` in any case.
Prefer the path that works with old simcalls too `Api::request_to_string()`

3 years agoSplit inspector::is_pending() in two logical parts
Martin Quinson [Sun, 28 Feb 2021 23:24:04 +0000 (00:24 +0100)]
Split inspector::is_pending() in two logical parts

- get_max_consider() returns how many times this simcall should be considered.
- prepare() does what needs to be done for the simcall to be fired in user space

This removes the RPC from MC to App to get this information:

 - The value of get_max_consider() is copied in the simcall structure
   when the AppSide prepares to wait for new instructions from the MC.
 - prepare() was already called by handle_simcall() on AppSide, to
   obey trace replays.

Less RPC is faster and less eror prone.

3 years agoNot sure why to_string is sometimes called when there is no inspector, but survive...
Martin Quinson [Sun, 28 Feb 2021 23:19:41 +0000 (00:19 +0100)]
Not sure why to_string is sometimes called when there is no inspector, but survive this bug

3 years agoRename ActorState::consider() and State::add_interleaved_set() as {ActorState,State...
Martin Quinson [Sun, 28 Feb 2021 20:32:27 +0000 (21:32 +0100)]
Rename ActorState::consider() and State::add_interleaved_set() as {ActorState,State}::mark_todo()

3 years agoFix build on clang
Martin Quinson [Sun, 28 Feb 2021 22:04:56 +0000 (23:04 +0100)]
Fix build on clang

3 years agoobjectification of MC simcall achieved -- many tests still failing
Martin Quinson [Sun, 28 Feb 2021 09:40:40 +0000 (10:40 +0100)]
objectification of MC simcall achieved -- many tests still failing

* MC_RANDOM is now a regular simcall, made observable by its SimcallInspector

  - remove it from simcall.in and remove all relevant elements

  - Other simcalls that are visible from MC (ie, that are generated by
    simcall.py and are handled in MC with big switch cases) are not
    modified at all, even if the patch reindent them since this execution
    path is now in a new conditionnal.

* SimcallInspectors are used everywhere where a specific handling was made for MC_RANDOM

  - In MC_state_choose_request_for_process, we get the next value to
    be picked by MC_random (or the next comm to be picked by waitany)
    depending on the amount of times we consider this simcall (ie, on the
    amount of decision forks we did on that transition while exploring the
    state space.
      This operation was not named before, it's now called is_pending()
    because it returns whether we should do another decision fork from
    that transition. Maybe is_branching() would be a better name.
      Or consider() since its parameter is times_considered, and there
    is one call location where its return value is ignored (when we
    replay all transitions on the path leading to the state we want to
    restore). But there is already a mc::ActorState::consider() where
    the Checker express interest in a given simcall.

  - The two other generic methods of SimcallInspectors are to_string()
    to get a textual representation of the transtion (eg to display the
    backtrace leading to the property violation), and dot_label() to
    get a label that can be used in dot(1). This is useful to get a
    visual representation of the state space.

  - There is a visible() method too, but I guess it's useless: Only
    visible simcall will be given an inspector. Or it will be more
    complex, because differing model checking algorithms will be
    interested in differing sets of simcalls (so visible() should be
    Checker-dependend). Leave it as is for now.

  - RandomSimcall is also implementing a get_value() method that
    allows its implementation. At the end of the day, the connexion
    between the actor side (out of the kernel) and the model-checker side
    is made with this code:

    auto observer = new simgrid::mc::RandomSimcall(SIMIX_process_self(), min, max);
    return simgrid::kernel::actor::simcall([observer] { return observer->get_value(); }, observer);

    The RandomSimcall object is really making the link between the two worlds.

  - When we will convert the other simcalls, something will be needed
    to handle transition dependency. Since MC_RANDOM is never dependent
    with anything, this is left unhandled in SimcallInspector for now.
    One problem at a time.

* SimcallInspectors live in the AppSide while these information are
  needed in the ModelChecker side. So the network protocol was
  extended to allow the Modelchecker to get this information thru RPC.

  This is not satisfactory because it probably slows things down, even
  if it's hard to say for sure because many tests just deadlock with it.
  When the AppSide is dead, you'd better not ask for a RPC to collect
  information about the simcalls that lead to the issue.

  I extended the State::Transition object (that is used to both replay
  a trace leading to a state I want to restore and to display the
  backtrace that lead to a violation) to include the textual
  representation of the transitions, to not do a RPC to the dead
  application on violation.

  But when the application abort(), I seem to have a stack corruption
  on the app side or something. And debugging the verified application
  is made very difficult by the Dwarf inspector that prevents the
  verification from starting if the subprocess is not compiled in one
  segment only (valgrind violates this rule). Plus valgrind has a
  subprocess itself, probably defeating the checkpoint/restore
  mechanism. Maybe I should try the Sanitizers.

* A better approach (if we fail to fix the curreny design) could be to
  pre-compute on AppSide all information that can be extracted from
  the inspectors, and copy it over to the CheckerSide once per
  scheduling round. That would speed things up, and ensure that there
  is no RPC after the death of the application. That should be done in
  place of MC_process_refresh_simix_actor_dynar() that copies some
  (less useful) information over.

3 years agoMerge branch 'master' of framagit.org:simgrid/simgrid
Martin Quinson [Sun, 28 Feb 2021 09:35:55 +0000 (10:35 +0100)]
Merge branch 'master' of framagit.org:simgrid/simgrid

3 years agoExtend xbt_stringify_args up to 25 arguments
Martin Quinson [Sat, 27 Feb 2021 22:18:48 +0000 (23:18 +0100)]
Extend xbt_stringify_args up to 25 arguments

3 years agoWelcome to modernity
Martin Quinson [Sat, 27 Feb 2021 22:17:23 +0000 (23:17 +0100)]
Welcome to modernity

3 years agoextend example with another disk
Frederic Suter [Fri, 26 Feb 2021 15:00:18 +0000 (16:00 +0100)]
extend example with another disk

3 years agoallow for programmatic creation of a disk
Frederic Suter [Fri, 26 Feb 2021 14:59:58 +0000 (15:59 +0100)]
allow for programmatic creation of a disk

3 years agotypo
Martin Quinson [Fri, 26 Feb 2021 09:46:34 +0000 (10:46 +0100)]
typo

3 years agoNeed at least ns-3.28.
Arnaud Giersch [Fri, 26 Feb 2021 09:16:01 +0000 (10:16 +0100)]
Need at least ns-3.28.

3 years agoDeclare function 'static' (and fix picky builders).
Arnaud Giersch [Fri, 26 Feb 2021 08:09:59 +0000 (09:09 +0100)]
Declare function 'static' (and fix picky builders).

3 years agoUpdate ASan suppressions for clang.
Arnaud Giersch [Thu, 25 Feb 2021 20:07:58 +0000 (21:07 +0100)]
Update ASan suppressions for clang.