Logo AND Algorithmique Numérique Distribuée

Public GIT Repository
simgrid.git
3 months agoFix tesh for the new mc-sendsend test
Martin Quinson [Sun, 18 Aug 2019 08:12:39 +0000 (10:12 +0200)]
Fix tesh for the new mc-sendsend test

- Test for all factories but thread (that is borken with MC)
- Don't run this test when Java, no matter what. Not sure how I came
  to such a stupid idea :)

3 months agofix make dist and python dist
Martin Quinson [Sun, 18 Aug 2019 00:34:08 +0000 (02:34 +0200)]
fix make dist and python dist

3 months agoadd a test of MC detecting blocking send/send patterns
Martin Quinson [Sat, 4 Nov 2017 17:51:02 +0000 (18:51 +0100)]
add a test of MC detecting blocking send/send patterns

3 months agoNew MC option: smpi/buffering, to control MPI buffering
Martin Quinson [Sat, 17 Aug 2019 23:33:14 +0000 (01:33 +0200)]
New MC option: smpi/buffering, to control MPI buffering

3 months agosmpirun: add a -quiet argument, allowing failing tests in tesh
Martin Quinson [Sat, 17 Aug 2019 23:19:27 +0000 (01:19 +0200)]
smpirun: add a -quiet argument, allowing failing tests in tesh

3 months agomore informative message when setting inexistant config items
Martin Quinson [Sat, 17 Aug 2019 22:42:48 +0000 (00:42 +0200)]
more informative message when setting inexistant config items

3 months agodocs: fix borken links
Martin Quinson [Sat, 17 Aug 2019 22:10:43 +0000 (00:10 +0200)]
docs: fix borken links

3 months agoPMPI_Cart_create: check that each dim is positive
Martin Quinson [Sat, 17 Aug 2019 21:07:21 +0000 (23:07 +0200)]
PMPI_Cart_create: check that each dim is positive

3 months agoHave SMPI fail on MPI_ERR_* in MC mode
Martin Quinson [Sat, 17 Aug 2019 20:24:44 +0000 (22:24 +0200)]
Have SMPI fail on MPI_ERR_* in MC mode

The standard says that upon error, implementations should call the
current MPI error handler, which is MPI_ERRORS_ARE_FATAL by default
but could be changed to MPI_ERRORS_RETURN on need.

Since we don't implement MPI_Comm_set_errhandler() to switch between
modes, the simulation mode of SMPI is only issuing a warning on
errors (which is similar to MPI_ERRORS_RETURN).

This commit adds a MC_assert() stating that every MPI call succeed.
This will lead to a property failure (visible only in MC mode) when a
MPI_ERR_* is issued by the implementation (which is somehow similar to
MPI_ERRORS_ARE_FATAL).

3 months agofix non-MC builds
Martin Quinson [Sat, 17 Aug 2019 15:53:57 +0000 (17:53 +0200)]
fix non-MC builds

3 months agouse assignment to non-trivial class rather than artificial trivialization and memset
Martin Quinson [Sat, 17 Aug 2019 14:57:04 +0000 (16:57 +0200)]
use assignment to non-trivial class rather than artificial trivialization and memset

plus some other small cleanups

3 months agoRestore triviality of s_smx_simcall to please GCC
Martin Quinson [Thu, 15 Aug 2019 16:28:24 +0000 (18:28 +0200)]
Restore triviality of s_smx_simcall to please GCC

Field initialization make this struct non-trivial, making GCC to panic
when we memset it. Even if we initialize some fields to nullptr and
memset it to 0.

3 months agofix java build
Martin Quinson [Thu, 15 Aug 2019 14:02:22 +0000 (16:02 +0200)]
fix java build

3 months agosmall comments improvements around a complex code
Martin Quinson [Thu, 15 Aug 2019 13:37:24 +0000 (15:37 +0200)]
small comments improvements around a complex code

3 months agonow, kernel::actor::simcall_blocking can return a value
Martin Quinson [Wed, 14 Aug 2019 00:03:56 +0000 (02:03 +0200)]
now, kernel::actor::simcall_blocking can return a value

3 months agoLet simcall.py produce valid code after recent renamings
Martin Quinson [Tue, 13 Aug 2019 23:50:00 +0000 (01:50 +0200)]
Let simcall.py produce valid code after recent renamings

3 months agocode simplification + replace a FIXME with an assert
Martin Quinson [Tue, 13 Aug 2019 23:26:51 +0000 (01:26 +0200)]
code simplification + replace a FIXME with an assert

3 months agosmall simplifications around simcalls
Martin Quinson [Tue, 13 Aug 2019 22:49:31 +0000 (00:49 +0200)]
small simplifications around simcalls

- Use ActivityImpl::register_simcall() where possible
- Uniformity in ActivityImpl::post() methods
- Rename processes into actors

3 months agosmall logic simplification
Martin Quinson [Tue, 13 Aug 2019 21:42:10 +0000 (23:42 +0200)]
small logic simplification

That function is a bit long, but that's not a reason to test twice
whether we should kill the issuer because it runs on a dead host.

3 months agoIntroduce a class mc::SimcallInspector, that allows MC to learn about the ongoing...
Martin Quinson [Tue, 13 Aug 2019 20:54:53 +0000 (22:54 +0200)]
Introduce a class mc::SimcallInspector, that allows MC to learn about the ongoing simcalls

Not quite used yet.

3 months agoMC: rename processes to actors
Martin Quinson [Tue, 13 Aug 2019 10:02:53 +0000 (12:02 +0200)]
MC: rename processes to actors

3 months agoobey our coding standards, and snake_case some parts of MC
Martin Quinson [Tue, 13 Aug 2019 09:47:52 +0000 (11:47 +0200)]
obey our coding standards, and snake_case some parts of MC

3 months agostart to make generic simcalls observable from the MC
Martin Quinson [Mon, 12 Aug 2019 10:21:51 +0000 (12:21 +0200)]
start to make generic simcalls observable from the MC

WIP, done in the easy parts for now.

Not finished in MC_state_get_request_for_process() yet. That's too bad
because this function is the core of the use of simcalls by the MC,
where the next simcall is picked.

Not done in the independence computing part either.

3 months agotransparent cleanups around simcalls mechanism
Martin Quinson [Mon, 12 Aug 2019 10:04:18 +0000 (12:04 +0200)]
transparent cleanups around simcalls mechanism

3 months agomove the simcall template to the kernel::actor namespace
Martin Quinson [Mon, 12 Aug 2019 09:37:10 +0000 (11:37 +0200)]
move the simcall template to the kernel::actor namespace

3 months agoprotect accesses to cnst->cnst_light_
Frederic Suter [Tue, 13 Aug 2019 08:07:45 +0000 (10:07 +0200)]
protect accesses to cnst->cnst_light_

3 months agoPort simcall_process_suspend to the modernity
Martin Quinson [Sat, 10 Aug 2019 22:51:16 +0000 (00:51 +0200)]
Port simcall_process_suspend to the modernity

Also simplify its logic by moving the s4u parts within the kernel

3 months agogetters don't have to be simcalls
Martin Quinson [Sat, 10 Aug 2019 20:41:56 +0000 (22:41 +0200)]
getters don't have to be simcalls

3 months agoconvert simcall_process_sleep to modernity
Martin Quinson [Sat, 10 Aug 2019 13:41:08 +0000 (15:41 +0200)]
convert simcall_process_sleep to modernity

3 months agoreduce the use of simcall_process_sleep() -> this_actor::sleep_for()
Martin Quinson [Sat, 10 Aug 2019 08:35:55 +0000 (10:35 +0200)]
reduce the use of simcall_process_sleep() -> this_actor::sleep_for()

3 months agoport a blocking simcall to the modernity
Martin Quinson [Fri, 9 Aug 2019 23:12:06 +0000 (01:12 +0200)]
port a blocking simcall to the modernity

3 months agosmall simplification
Martin Quinson [Fri, 9 Aug 2019 22:31:53 +0000 (00:31 +0200)]
small simplification

3 months agointroduce a simcall_blocking(), and improve the comments
Martin Quinson [Fri, 9 Aug 2019 22:24:53 +0000 (00:24 +0200)]
introduce a simcall_blocking(), and improve the comments

3 months agoFix GCC+MC builds
Martin Quinson [Fri, 9 Aug 2019 11:38:21 +0000 (13:38 +0200)]
Fix GCC+MC builds

3 months agoconvert SIMIX_simcall_answer() into ActorImpl::simcall_answer()
Martin Quinson [Fri, 9 Aug 2019 10:24:48 +0000 (12:24 +0200)]
convert SIMIX_simcall_answer() into ActorImpl::simcall_answer()

3 months agochange SIMIX_simcall_handle() into ActorImpl::simcall_handle()
Martin Quinson [Thu, 8 Aug 2019 21:20:44 +0000 (23:20 +0200)]
change SIMIX_simcall_handle() into ActorImpl::simcall_handle()

3 months agoanother assert to make one segfault more explicit
Martin Quinson [Thu, 8 Aug 2019 20:41:36 +0000 (22:41 +0200)]
another assert to make one segfault more explicit

3 months agonamespacify a global function of SIMIX
Martin Quinson [Thu, 8 Aug 2019 16:32:13 +0000 (18:32 +0200)]
namespacify a global function of SIMIX

3 months agoshy attempt at simplifying the simcall mechanism
Martin Quinson [Thu, 8 Aug 2019 16:08:43 +0000 (18:08 +0200)]
shy attempt at simplifying the simcall mechanism

4 months agoforcefully kill exiting actors even if their host is not off
Martin Quinson [Fri, 2 Aug 2019 18:12:24 +0000 (20:12 +0200)]
forcefully kill exiting actors even if their host is not off

Without this, this_actor::exit() deadlocks on Thread factory because
the actor does not exit soon enough to release maestro that is joining
its thread.

4 months agoimprove debug msg
Martin Quinson [Fri, 2 Aug 2019 18:11:41 +0000 (20:11 +0200)]
improve debug msg

4 months agorun Actor::on_destruction even if the actor was killed before starting, but not on...
Martin Quinson [Fri, 2 Aug 2019 16:55:49 +0000 (18:55 +0200)]
run Actor::on_destruction even if the actor was killed before starting, but not on maestro

It seems that the actors are sometimes killed after the simix_global
destruction... Maybe when the user code keeps a reference on them.

4 months agoNew signal: Actor::on_termination (when its code terminates)
Martin Quinson [Fri, 2 Aug 2019 15:50:49 +0000 (17:50 +0200)]
New signal: Actor::on_termination (when its code terminates)

4 months agoactivity-lifecycle: make tests shorter to ensure that they are really done when we...
Martin Quinson [Fri, 2 Aug 2019 15:49:14 +0000 (17:49 +0200)]
activity-lifecycle: make tests shorter to ensure that they are really done when we test them so

4 months agoActor: make the refcount observable, and improve debug messages
Martin Quinson [Thu, 1 Aug 2019 20:49:34 +0000 (22:49 +0200)]
Actor: make the refcount observable, and improve debug messages

4 months agosmpicxx: obey the VERBOSE environment variable and display what we do
Martin Quinson [Thu, 1 Aug 2019 20:47:23 +0000 (22:47 +0200)]
smpicxx: obey the VERBOSE environment variable and display what we do

4 months agoActor::by_pid: also search through the dead actors
Martin Quinson [Thu, 1 Aug 2019 20:44:06 +0000 (22:44 +0200)]
Actor::by_pid: also search through the dead actors

Finally, my trick works with it. MPI ranks are not garbage collected
as soon as they end because the MPI instance keeps a reference on
them. With this, by_pid() works properly, even on dead but not
collected actors.

But this only works until the trash is emptied, obviously. This seem
to be enough for the test I wanted to get running, so I will not fix
that point tonight.

4 months agoActorImpl: postpone the on_destroy signal to the destructor
Martin Quinson [Thu, 1 Aug 2019 20:41:59 +0000 (22:41 +0200)]
ActorImpl: postpone the on_destroy signal to the destructor

This is to prevent that the SMPI extension gets destroyed too early.
The problem is also that this extension is not really an extension but
manually handled, but I'll not fix that tonight.

4 months agoRestore the -no-privatize to the ampi example
Martin Quinson [Thu, 1 Aug 2019 20:40:14 +0000 (22:40 +0200)]
Restore the -no-privatize to the ampi example

4 months agofix ampi example by not cleaning the tracing submodule before the actor end
Martin Quinson [Thu, 1 Aug 2019 15:55:38 +0000 (17:55 +0200)]
fix ampi example by not cleaning the tracing submodule before the actor end

4 months agoadd a dependency of this test to mpi, as required on jenkins
Martin Quinson [Thu, 1 Aug 2019 12:57:31 +0000 (14:57 +0200)]
add a dependency of this test to mpi, as required on jenkins

4 months agoSMPI: redesign the end of actors/ranks' lifetime
Martin Quinson [Thu, 1 Aug 2019 07:34:53 +0000 (09:34 +0200)]
SMPI: redesign the end of actors/ranks' lifetime

The problem is that we don't use enough of refcounting in SMPI, so we
should not let any rank finish before the others, because it may be
involved in a communication or something.

Previously, there were a barrier at the end of the user code, so that
every ranks finishes exactly at the same time.

Now, the MPI instance keeps a reference on every actor it contains,
and the actor terminates with no delay after its code. The terminating
actors unregister from their MPI instance, but they are still
referenced until the last actor unregisters from the MPI instance.
Once the MPI instance is empty, it unregisters all the actors,
allowing their collection by the refcounting.

This commit changes the ending time of many ranks in many examples, as
expected. The ranks now terminate as soon as they are done, they are
not waiting the others anymore.

It introduces a segfault in ampi that I fail to understand. It seems
that a container is used after being collected in this example, but I
fail to see the reason so far.

4 months agosort out the functions on MPI init/fini
Martin Quinson [Thu, 1 Aug 2019 05:11:31 +0000 (07:11 +0200)]
sort out the functions on MPI init/fini

4 months agosmall simplification of MPI initialization
Martin Quinson [Thu, 1 Aug 2019 04:45:53 +0000 (06:45 +0200)]
small simplification of MPI initialization

4 months agoMPI: we don't mess with argc/argv anymore nowadays
Martin Quinson [Thu, 1 Aug 2019 03:53:31 +0000 (05:53 +0200)]
MPI: we don't mess with argc/argv anymore nowadays

Previously, the rank and instance were added in argv, mandating a
specific handling of MPI_Init parameters. But now, they are passed as
properties, and the argv is left unmodified. So there is no need to
deal specifically with the MPI_Init parameters.

4 months agosmpi_deployment: obey our coding rules
Martin Quinson [Thu, 1 Aug 2019 03:25:31 +0000 (05:25 +0200)]
smpi_deployment: obey our coding rules

4 months agoI'd like to turn process_data into a regular extension of Actor
Martin Quinson [Wed, 31 Jul 2019 16:55:08 +0000 (18:55 +0200)]
I'd like to turn process_data into a regular extension of Actor

4 months agoSMPI: prefer xbt_assert to 'if () xbt_die'
Martin Quinson [Wed, 31 Jul 2019 16:46:37 +0000 (18:46 +0200)]
SMPI: prefer xbt_assert to 'if () xbt_die'

4 months agoMPI init: inline a function and various small cleanups
Martin Quinson [Wed, 31 Jul 2019 16:35:02 +0000 (18:35 +0200)]
MPI init: inline a function and various small cleanups

4 months agosmpi::ActorExt: this constructor parameter is not used
Martin Quinson [Wed, 31 Jul 2019 16:12:10 +0000 (18:12 +0200)]
smpi::ActorExt: this constructor parameter is not used

4 months agoMPI init: inline a function
Martin Quinson [Wed, 31 Jul 2019 15:58:33 +0000 (17:58 +0200)]
MPI init: inline a function

4 months agoSMPI init: stringify and rename a variable
Martin Quinson [Wed, 31 Jul 2019 09:25:17 +0000 (11:25 +0200)]
SMPI init: stringify and rename a variable

4 months agokill smpi_process_count(), use smpi_get_universe_size() instead
Martin Quinson [Sun, 28 Jul 2019 23:55:25 +0000 (01:55 +0200)]
kill smpi_process_count(), use smpi_get_universe_size() instead

process_count was probably the original name while universe_size was
added to implement MPI_Attr_get(MPI_UNIVERSE_SIZE). Merge both to sort
things out.

While I'm at it, move all of it to smpi_deployment to reduce the
amount of globals made visible to more than one module.

4 months agocmake: build smpi internals before the collectives
Martin Quinson [Sun, 28 Jul 2019 23:42:33 +0000 (01:42 +0200)]
cmake: build smpi internals before the collectives

My changes break more often the internals than the collectives (that
are rarely modified)

4 months agoMC: kill a useless function
Martin Quinson [Sun, 28 Jul 2019 23:31:06 +0000 (01:31 +0200)]
MC: kill a useless function

4 months agosmpi: some useless cleanups while I read this code
Martin Quinson [Sun, 28 Jul 2019 22:38:00 +0000 (00:38 +0200)]
smpi: some useless cleanups while I read this code

4 months agoavoid mixing travis builders
Augustin Degomme [Sat, 27 Jul 2019 09:04:36 +0000 (11:04 +0200)]
avoid mixing travis builders

4 months agodocs: fix some typos reported by Gene
Martin Quinson [Fri, 26 Jul 2019 16:20:34 +0000 (18:20 +0200)]
docs: fix some typos reported by Gene

4 months agotypo...
Augustin Degomme [Fri, 26 Jul 2019 21:27:50 +0000 (23:27 +0200)]
typo...

4 months agomerge before pushing, next time
Augustin Degomme [Fri, 26 Jul 2019 21:26:08 +0000 (23:26 +0200)]
merge before pushing, next time

4 months agoreally build on mojave
Augustin Degomme [Fri, 26 Jul 2019 21:21:27 +0000 (23:21 +0200)]
really build on mojave

4 months agotry to fix broken detection of boost_stacktrace on mojave. Hope it does not break...
Augustin Degomme [Fri, 26 Jul 2019 20:57:25 +0000 (22:57 +0200)]
try to fix broken detection of boost_stacktrace on mojave. Hope it does not break cold cmakes

4 months agoadd color for travis and appveyor builds
Augustin Degomme [Fri, 26 Jul 2019 18:10:40 +0000 (20:10 +0200)]
add color for travis and appveyor builds

4 months agotypo
Augustin Degomme [Fri, 26 Jul 2019 17:42:25 +0000 (19:42 +0200)]
typo

4 months agoadd some comments on tests we don't pass
Augustin Degomme [Fri, 26 Jul 2019 14:28:37 +0000 (16:28 +0200)]
add some comments on tests we don't pass

4 months agoPartially revert 2b4dcee56 and activate a test.
Augustin Degomme [Fri, 26 Jul 2019 13:15:25 +0000 (15:15 +0200)]
Partially revert 2b4dcee56 and activate a test.
mpi_type_create_x actually handle MPI_Aint, only mpi_type_x need conversion

4 months agoactivate a fortran test
Augustin Degomme [Fri, 26 Jul 2019 12:34:49 +0000 (14:34 +0200)]
activate a fortran test

4 months agostandard says : "The names attached to opaque objects do not propagate when the objec...
Augustin Degomme [Fri, 26 Jul 2019 12:27:14 +0000 (14:27 +0200)]
standard says : "The names attached to opaque objects do not propagate when the object is duplicated or copied by MPI routines. "

4 months agoappveyor's back
Augustin Degomme [Fri, 26 Jul 2019 08:59:27 +0000 (10:59 +0200)]
appveyor's back

4 months agoMake SG_BARRIER_SERIAL_THREAD visible from C files too
Martin Quinson [Fri, 26 Jul 2019 13:47:34 +0000 (15:47 +0200)]
Make SG_BARRIER_SERIAL_THREAD visible from C files too

Fix https://framagit.org/simgrid/simgrid/issues/34 again, hopefully
properly this time.

4 months agosafety: give some basic logging at verbose level already
Martin Quinson [Wed, 24 Jul 2019 23:42:50 +0000 (01:42 +0200)]
safety: give some basic logging at verbose level already

4 months agoMove 'unit-tests' binary from 'all' target to 'tests'
Martin Quinson [Wed, 24 Jul 2019 23:27:12 +0000 (01:27 +0200)]
Move 'unit-tests' binary from 'all' target to 'tests'

4 months agosmpi: cut some includes
Martin Quinson [Wed, 24 Jul 2019 23:24:40 +0000 (01:24 +0200)]
smpi: cut some includes

4 months agoChange a bit F2C handling
Augustin Degomme [Fri, 26 Jul 2019 08:09:57 +0000 (10:09 +0200)]
Change a bit F2C handling
Store the f2c id in the objects, instead of stupidly browsing the entire map to get it each time
In some cases we had issues with old pointers not being removed from the map, and being reused for other objects, which could cause mismatches.

4 months agodon't referenece missing symbol
Augustin Degomme [Wed, 24 Jul 2019 23:46:48 +0000 (01:46 +0200)]
don't referenece missing symbol

4 months agotake care of some C/Fortran discrepancies
Augustin Degomme [Wed, 24 Jul 2019 23:13:33 +0000 (01:13 +0200)]
take care of some C/Fortran discrepancies

4 months agoactivate some C/Fortran tests
Augustin Degomme [Wed, 24 Jul 2019 21:38:33 +0000 (23:38 +0200)]
activate some C/Fortran tests

4 months agoconvert between Fortran and C MPI_*_NULL handles
Augustin Degomme [Wed, 24 Jul 2019 21:38:03 +0000 (23:38 +0200)]
convert between Fortran and C MPI_*_NULL handles

4 months agodon't test on a cancelled comm
Augustin Degomme [Wed, 24 Jul 2019 21:37:10 +0000 (23:37 +0200)]
don't test on a cancelled comm

4 months agoinline a function
Martin Quinson [Wed, 24 Jul 2019 22:03:53 +0000 (00:03 +0200)]
inline a function

4 months agodoc/mc: improve the replay documentation
Martin Quinson [Wed, 24 Jul 2019 22:02:36 +0000 (00:02 +0200)]
doc/mc: improve the replay documentation

4 months agocosmetic to make the code easier to read
Martin Quinson [Wed, 24 Jul 2019 21:50:55 +0000 (23:50 +0200)]
cosmetic to make the code easier to read

4 months agoDon't whine that smpi/host-speed was not set in MC mode
Martin Quinson [Wed, 24 Jul 2019 21:33:31 +0000 (23:33 +0200)]
Don't whine that smpi/host-speed was not set in MC mode

4 months agoexample s4u-platform-properties: test thehost->get_englobing_zone and diplay its...
Martin Quinson [Wed, 24 Jul 2019 13:19:37 +0000 (15:19 +0200)]
example s4u-platform-properties: test thehost->get_englobing_zone and diplay its properties

4 months agoHost::get_englobing_zone() returns the englobing netzone
Martin Quinson [Wed, 24 Jul 2019 13:18:44 +0000 (15:18 +0200)]
Host::get_englobing_zone() returns the englobing netzone

4 months agodon't swallow the zone properties while parsing the XML
Martin Quinson [Wed, 24 Jul 2019 13:17:33 +0000 (15:17 +0200)]
don't swallow the zone properties while parsing the XML

4 months agotravis: we now use travis
Martin Quinson [Wed, 24 Jul 2019 09:05:09 +0000 (11:05 +0200)]
travis: we now use travis

4 months agotravis: manually use apt-get since the apt addon is borken on bionic
Martin Quinson [Wed, 24 Jul 2019 08:29:55 +0000 (10:29 +0200)]
travis: manually use apt-get since the apt addon is borken on bionic

https://travis-ci.community/t/apt-addon-doesnt-run-in-bionic/4061/2

4 months agotravis: cleanups and simplify our use of the apt addon
Martin Quinson [Wed, 24 Jul 2019 08:04:57 +0000 (10:04 +0200)]
travis: cleanups and simplify our use of the apt addon

- Remove some extraneous parameters to the apt addon since it does not
  seem to start since I switched to bionic. The doc says that some
  features were removed in that addon, but I'm not sure of whether it
  should affect us given our config. Let's see if it's better this way.
- Kill dead code about lua. 5.3 is part of Debian since a very long time
- Reindent the build rules for linux and osx (no change)

4 months agofix the link of smpireplaymain on alpha and ia64
Martin Quinson [Wed, 24 Jul 2019 07:10:35 +0000 (09:10 +0200)]
fix the link of smpireplaymain on alpha and ia64