Logo AND Algorithmique Numérique Distribuée

Public GIT Repository
simgrid.git
2 years agoenable remote copy and move for disks
Frederic Suter [Wed, 11 Sep 2019 13:16:53 +0000 (15:16 +0200)]
enable remote copy and move for disks
* need to have a local disk on both sides. it amounts to reading on
one side, sending the read data and writing it on the other side.
* seemless remote accesses for read and write are no longer available.
* simplify the s4u-file-remote test (remote operations with storage is
still tested in teshsuite by an old MSG test)

2 years agoMerge branch 'master' into disk
Frederic Suter [Tue, 10 Sep 2019 12:14:28 +0000 (14:14 +0200)]
Merge branch 'master' into disk

2 years agopartially handle disks in file system plugin
Frederic Suter [Tue, 10 Sep 2019 12:08:53 +0000 (14:08 +0200)]
partially handle disks in file system plugin
* open/read/write/move/close/unlink
* remote read/write operations not supported yet
* update the s4u-io-file-system example
* To be used with this plugin, a <disk> has to declare some <props>
   * size (with unit) [mandatory]
   * mount            [mandatory]
   * content          [optional]
* TODO
   * write some documentation
   * add some checks/asserts if user tries to
      * use plugin with no <prop> declared
      * mix disk and storage

2 years agoMerge branch 'master' into 'master'
Martin Quinson [Tue, 10 Sep 2019 11:40:32 +0000 (13:40 +0200)]
Merge branch 'master' into 'master'

Fix the NS3 bindings to detect when several flows are simultaneously finishing

See merge request simgrid/simgrid!16

2 years agoDisks do not need this
Frederic Suter [Tue, 10 Sep 2019 10:42:57 +0000 (12:42 +0200)]
Disks do not need this

2 years agoneed to delay signal after properties are set
Frederic Suter [Tue, 10 Sep 2019 09:12:16 +0000 (11:12 +0200)]
need to delay signal after properties are set

2 years agoTo stop NS3 in a consistant state at its next occuring event time, make sure it has...
bcamus [Tue, 10 Sep 2019 08:39:59 +0000 (10:39 +0200)]
To stop NS3 in a consistant state at its next occuring event time, make sure it has processed all the flows that are finishing at this time.

2 years agoextend platorm with disks
Frederic Suter [Mon, 9 Sep 2019 12:09:50 +0000 (14:09 +0200)]
extend platorm with disks
* have two disks on bob
* share same disk name between bob and alice
* revalidate tesh files after modifications

2 years agosimplify disk management
Frederic Suter [Mon, 9 Sep 2019 11:53:37 +0000 (13:53 +0200)]
simplify disk management
* disks are related to a given host, there is no need to maintain a
global map.
* No more map means no by_name methods
* This allows for same name for disks on different hosts

2 years agocosmetics in the wanna be MC's simcall observer
Martin Quinson [Mon, 9 Sep 2019 00:18:06 +0000 (02:18 +0200)]
cosmetics in the wanna be MC's simcall observer

2 years agoimprove the doc of the MC reduction technics
Martin Quinson [Sun, 8 Sep 2019 23:55:31 +0000 (01:55 +0200)]
improve the doc of the MC reduction technics

2 years agoMerge branch 'master' into disk
Frederic Suter [Fri, 6 Sep 2019 15:22:14 +0000 (17:22 +0200)]
Merge branch 'master' into disk

2 years agoWifi: Clean obsolete code
Loic Guegan [Fri, 6 Sep 2019 10:45:37 +0000 (12:45 +0200)]
Wifi: Clean obsolete code

2 years agoWifi: Add crosstraffic on WIFI cells
Loic Guegan [Fri, 6 Sep 2019 09:44:30 +0000 (11:44 +0200)]
Wifi: Add crosstraffic on WIFI cells

2 years agoWifi: Add wifi tesh sources
Loic Guegan [Fri, 6 Sep 2019 09:36:12 +0000 (11:36 +0200)]
Wifi: Add wifi tesh sources

2 years agoMerge branch 'master' of https://framagit.org/simgrid/simgrid
Frederic Suter [Thu, 5 Sep 2019 11:23:43 +0000 (13:23 +0200)]
Merge branch 'master' of https://framagit.org/simgrid/simgrid

2 years agono more need to pass issuer to suspend
Frederic Suter [Thu, 5 Sep 2019 11:23:13 +0000 (13:23 +0200)]
no more need to pass issuer to suspend

2 years agoWifi: Ajust model on internal AP communication
Loic Guegan [Thu, 5 Sep 2019 09:09:42 +0000 (11:09 +0200)]
Wifi: Ajust model on internal AP communication

2 years agoWifi: Allow crosstraffic simulations
Loic Guegan [Thu, 5 Sep 2019 08:51:34 +0000 (10:51 +0200)]
Wifi: Allow crosstraffic simulations

2 years agoMerge branch 'master' into disk
Frederic Suter [Wed, 4 Sep 2019 14:19:55 +0000 (16:19 +0200)]
Merge branch 'master' into disk

2 years agomodify this example to use disks
Frederic Suter [Wed, 4 Sep 2019 13:39:40 +0000 (15:39 +0200)]
modify this example to use disks

2 years agoplug leak
Frederic Suter [Wed, 4 Sep 2019 06:56:46 +0000 (08:56 +0200)]
plug leak

2 years agoclang-tidy the ns3 model
Martin Quinson [Tue, 3 Sep 2019 21:42:54 +0000 (23:42 +0200)]
clang-tidy the ns3 model

2 years agoworking implementation of storage on disks
Frederic Suter [Tue, 3 Sep 2019 12:15:53 +0000 (14:15 +0200)]
working implementation of storage on disks
+ add a s4u::Disk
+ modify a lot of things to make it working
+ add a small example to test read/write

2 years agoconnect disks to their host
Frederic Suter [Tue, 3 Sep 2019 07:55:04 +0000 (09:55 +0200)]
connect disks to their host

2 years agoload disk model
Frederic Suter [Tue, 3 Sep 2019 07:12:15 +0000 (09:12 +0200)]
load disk model

2 years agodisk not mount
Frederic Suter [Tue, 3 Sep 2019 07:10:31 +0000 (09:10 +0200)]
disk not mount

2 years agofix distcheck
Frederic Suter [Mon, 2 Sep 2019 15:26:31 +0000 (17:26 +0200)]
fix distcheck

2 years agofix typo
Frederic Suter [Mon, 2 Sep 2019 15:20:12 +0000 (17:20 +0200)]
fix typo

2 years agoimplement sg_platf_new_disk
Frederic Suter [Mon, 2 Sep 2019 14:49:41 +0000 (16:49 +0200)]
implement sg_platf_new_disk

2 years agoadd a platform file using the new disk tag
Frederic Suter [Mon, 2 Sep 2019 14:37:08 +0000 (16:37 +0200)]
add a platform file using the new disk tag

2 years agostart implementing what is behind the new <disk> tag
Frederic Suter [Mon, 2 Sep 2019 14:12:40 +0000 (16:12 +0200)]
start implementing what is behind the new <disk> tag
* Not tested at all
* Mostly duplicates the current Storage stuff minus what we want to
get rid off
* Several FIXME where the link with S4U should be done

2 years agoExit with a special exit code on failure in the last stage.
Arnaud Giersch [Mon, 2 Sep 2019 10:08:58 +0000 (12:08 +0200)]
Exit with a special exit code on failure in the last stage.

This can be used in jenkins to mark the build "unstable" instead of "failed".

2 years agodoc: specify that you need to build the tests before ctest
Martin Quinson [Sun, 1 Sep 2019 00:54:50 +0000 (02:54 +0200)]
doc: specify that you need to build the tests before ctest

2 years agocmake: accept enable-model-checking as an alias to enable_model-checking
Martin Quinson [Sun, 1 Sep 2019 00:50:52 +0000 (02:50 +0200)]
cmake: accept enable-model-checking as an alias to enable_model-checking

This should be generalized to the many potential typos in the option
name, but I don't feel like scripting in cmake tonight.

2 years agoanother batch of small improvements advised by clang-tidy
Martin Quinson [Sun, 1 Sep 2019 00:33:24 +0000 (02:33 +0200)]
another batch of small improvements advised by clang-tidy

2 years agodoc: specify that ltl2ba is not part of SimGrid
Martin Quinson [Sun, 1 Sep 2019 00:12:09 +0000 (02:12 +0200)]
doc: specify that ltl2ba is not part of SimGrid

2 years agofurther very small steps to modernize our code. thanks clang-tidy
Martin Quinson [Thu, 29 Aug 2019 00:01:45 +0000 (02:01 +0200)]
further very small steps to modernize our code. thanks clang-tidy

2 years agomodernize our C++
Martin Quinson [Tue, 27 Aug 2019 07:28:06 +0000 (09:28 +0200)]
modernize our C++

- Use auto when using new to not repeat the type
- Pass parameter by value + std::move instead of pass by reference

This was advized by clang advanced warnings

2 years agoDon't name the unused parameter of this callback
Martin Quinson [Tue, 27 Aug 2019 07:22:19 +0000 (09:22 +0200)]
Don't name the unused parameter of this callback

2 years agofix a bug in a deprecated function
Martin Quinson [Tue, 27 Aug 2019 07:21:22 +0000 (09:21 +0200)]
fix a bug in a deprecated function

Found by a static analyzer because the function parameter was unused

2 years agoSpelling fixes & a few cases of polishing the English (#329)
Gene Cooperman [Sat, 31 Aug 2019 23:56:03 +0000 (19:56 -0400)]
Spelling fixes & a few cases of polishing the English (#329)

Some of the more specialized fixes were:
 - 'DEAD-LOCK' -> 'DEADLOCK'  (appears when MC fnids a deadlock.
 - 'allows to' -> 'allows one to' (and so on for allow/allows/permit/permits)
   'model-checker' -> 'model checker'
     ('the model-checker software', but 'the model checker')

2 years agoFactor common definition, and avoid empty component in LD_LIBRARY_PATH.
Arnaud Giersch [Wed, 28 Aug 2019 13:30:22 +0000 (15:30 +0200)]
Factor common definition, and avoid empty component in LD_LIBRARY_PATH.

2 years agoDrop spurious colon.
Arnaud Giersch [Wed, 28 Aug 2019 12:52:21 +0000 (14:52 +0200)]
Drop spurious colon.

2 years agodon't forget to set include path in this case.
Augustin Degomme [Wed, 28 Aug 2019 13:00:13 +0000 (15:00 +0200)]
don't forget to set include path in this case.

2 years agodon't throw away existing LD_LIBRARY_PATH in our tests.
Augustin Degomme [Wed, 28 Aug 2019 10:16:40 +0000 (12:16 +0200)]
don't throw away existing LD_LIBRARY_PATH in our tests.

2 years agoactually failsafe has to be applied in all cases.
Augustin Degomme [Wed, 28 Aug 2019 09:57:30 +0000 (11:57 +0200)]
actually failsafe has to be applied in all cases.
Windows on travis has a 1.58 boost, so it was failing the first attempt, but a recent cmake so it was avoiding the second.

2 years agofix mc build with gcc10
Augustin Degomme [Tue, 27 Aug 2019 23:09:24 +0000 (01:09 +0200)]
fix mc build with gcc10

2 years agolet's be gentle with old cmakes which can't handle missing optional boost libs
Augustin Degomme [Tue, 27 Aug 2019 23:09:12 +0000 (01:09 +0200)]
let's be gentle with old cmakes which can't handle missing optional boost libs

2 years agoAttempt to ease detection of boost libraries.
Augustin Degomme [Tue, 27 Aug 2019 21:27:45 +0000 (23:27 +0200)]
Attempt to ease detection of boost libraries.
Will probably break things everywhere.

2 years agoAdd some includes to please GCC 10
Augustin Degomme [Tue, 27 Aug 2019 10:29:49 +0000 (12:29 +0200)]
Add some includes to please GCC 10

2 years agoWorkaround build error seen with clang 10 on freebsd.
Arnaud Giersch [Tue, 27 Aug 2019 12:12:59 +0000 (14:12 +0200)]
Workaround build error seen with clang 10 on freebsd.

In file included from [...]/src/bindings/python/simgrid_python.cpp:11:
In file included from /usr/local/include/pybind11/functional.h:12:
/usr/local/include/pybind11/pybind11.h:1078:9: error: expression result unused [-Werror,-Wunused-value]
        PYBIND11_EXPAND_SIDE_EFFECTS(add_base<options>(record));
[...]

2 years agoUse C++-style cast.
Arnaud Giersch [Tue, 27 Aug 2019 09:57:16 +0000 (11:57 +0200)]
Use C++-style cast.

2 years agoFix build with MC.
Arnaud Giersch [Tue, 27 Aug 2019 09:38:03 +0000 (11:38 +0200)]
Fix build with MC.

2 years agoFiles were regenerated.
Arnaud Giersch [Tue, 27 Aug 2019 08:51:40 +0000 (10:51 +0200)]
Files were regenerated.

2 years agoFix: undefined reference to `typeinfo for simgrid::mc::SimcallInspector'
Arnaud Giersch [Tue, 27 Aug 2019 08:46:28 +0000 (10:46 +0200)]
Fix: undefined reference to `typeinfo for simgrid::mc::SimcallInspector'

2 years agoDisable test stack-overflow with tsan too (false positive).
Arnaud Giersch [Sun, 28 Jul 2019 14:30:38 +0000 (16:30 +0200)]
Disable test stack-overflow with tsan too (false positive).

2 years ago[cppcheck] Reduce scope for variable.
Arnaud Giersch [Wed, 24 Jul 2019 16:04:51 +0000 (18:04 +0200)]
[cppcheck] Reduce scope for variable.

2 years agoWIFI: Now implemented
Loic Guegan [Tue, 27 Aug 2019 08:08:47 +0000 (10:08 +0200)]
WIFI: Now implemented

2 years agofix pip packaging
Martin Quinson [Sun, 25 Aug 2019 20:44:39 +0000 (22:44 +0200)]
fix pip packaging

2 years agotesh kill timeouted processes with KILL also
Martin Quinson [Sun, 25 Aug 2019 19:41:40 +0000 (21:41 +0200)]
tesh kill timeouted processes with KILL also

It seems that the sanitizers processes are surviving their timeouts on
the server, so this may help fixing the build robots.

On the way, I switched to save the PID of the forked process instead
of its PGID. This helps when I try to see if the process is still
alive, later on: I can always do PID->PGID later (as I do now) while
it seems impossible to do PGID->PID to see whether the process is
still alive.

Fun fact: I think that PID=PGID on POSIX systems, but let's play safe :)

2 years agoDPOR: improve debug messages
Martin Quinson [Mon, 19 Aug 2019 14:50:52 +0000 (16:50 +0200)]
DPOR: improve debug messages

2 years agoadd test proposed in #39
Augustin Degomme [Thu, 22 Aug 2019 23:44:10 +0000 (01:44 +0200)]
add test proposed in #39
For now in teshsuite, might be moved to examples if someone asks

2 years agoshould work better with the hostfile in the dist.
Augustin Degomme [Thu, 22 Aug 2019 13:40:40 +0000 (15:40 +0200)]
should work better with the hostfile in the dist.

2 years agoOne rma test actually needs exactly 2 processes per node.
Augustin Degomme [Thu, 22 Aug 2019 12:49:48 +0000 (14:49 +0200)]
One rma test actually needs exactly 2 processes per node.
Let's indulge it and add a hostfile for mpich tests, with various ppn setups.

2 years agotypos
Augustin Degomme [Thu, 22 Aug 2019 08:46:20 +0000 (10:46 +0200)]
typos

2 years agoattach errhandlers to some forgotten calls
Augustin Degomme [Mon, 19 Aug 2019 09:49:56 +0000 (11:49 +0200)]
attach errhandlers to some forgotten calls

2 years agoErrors occurring during calls to routines that create MPI windows (e.g., MPI_WIN_CREA...
Augustin Degomme [Mon, 19 Aug 2019 09:49:40 +0000 (11:49 +0200)]
Errors occurring during calls to routines that create MPI windows (e.g., MPI_WIN_CREATE (...,comm,...)) cause the error handler currently associated with comm to be invoked.

2 years agoMerge branch 'trace_smpi_execute_flops' into 'master'
Augustin Degomme [Tue, 20 Aug 2019 19:26:20 +0000 (21:26 +0200)]
Merge branch 'trace_smpi_execute_flops' into 'master'

execute_flops now logs compute

See merge request simgrid/simgrid!15

2 years agofix bad private function location
Faure Adrien [Tue, 20 Aug 2019 19:21:32 +0000 (21:21 +0200)]
fix bad private function location
and fix bad reference to spmi_execute_benched

2 years agoexecute_flops now logs compute
Faure Adrien [Tue, 20 Aug 2019 18:08:28 +0000 (20:08 +0200)]
execute_flops now logs compute
I created a internal function that execute flops without tracing for
specific needs

2 years agoAllgatherv : don't output MPI_ERR_BUFFER if recvbuf is null if we don't receive any...
Augustin Degomme [Mon, 19 Aug 2019 09:33:25 +0000 (11:33 +0200)]
Allgatherv : don't output MPI_ERR_BUFFER if recvbuf is null if we don't receive any data.
FIXME : other collectives can have the same constraint relaxed

2 years agoFor File, we can change the default error handler by specifying MPI_FILE_NULL as...
Augustin Degomme [Mon, 19 Aug 2019 08:50:38 +0000 (10:50 +0200)]
For File, we can change the default error handler by specifying MPI_FILE_NULL as the fh argument

2 years agoUnlike errors on communicators and windows, the default behavior for files is to...
Augustin Degomme [Mon, 19 Aug 2019 08:09:22 +0000 (10:09 +0200)]
Unlike errors on communicators and windows, the default behavior for files is to have MPI_ERRORS_RETURN. ( End of advice to users.)
https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report/node223.htm#Node223
because why not.

2 years agoThese tests really need zero buffering
Martin Quinson [Sun, 18 Aug 2019 21:42:30 +0000 (23:42 +0200)]
These tests really need zero buffering

2 years agoswitch to infty buffering for now as some tests are broken in zero-buffering mode
Martin Quinson [Sun, 18 Aug 2019 21:29:10 +0000 (23:29 +0200)]
switch to infty buffering for now as some tests are broken in zero-buffering mode

2 years agoupdate changelog
Augustin Degomme [Sun, 18 Aug 2019 21:37:52 +0000 (23:37 +0200)]
update changelog

2 years agoThis particular RMA test is filled with stupid calls... We send errors for most of...
Augustin Degomme [Sun, 18 Aug 2019 21:36:50 +0000 (23:36 +0200)]
This particular RMA test is filled with stupid calls... We send errors for most of them.
So let's put that to rest by setting MPI_ERRORS_RETURN for this one.

2 years agofix request and coll failing tests
Augustin Degomme [Sun, 18 Aug 2019 21:35:24 +0000 (23:35 +0200)]
fix request and coll failing tests

2 years agotopo was a bit too eager to return errors.
Augustin Degomme [Sun, 18 Aug 2019 21:34:56 +0000 (23:34 +0200)]
topo was a bit too eager to return errors.

2 years agoDon't return an error when a key is not found in an MPI_Info.
Augustin Degomme [Sun, 18 Aug 2019 21:33:50 +0000 (23:33 +0200)]
Don't return an error when a key is not found in an MPI_Info.

2 years agoour coll-* tests include bad calls for coverage, they need MPI_ERRORS_RETURN
Augustin Degomme [Sun, 18 Aug 2019 21:32:48 +0000 (23:32 +0200)]
our coll-* tests include bad calls for coverage, they need MPI_ERRORS_RETURN

2 years agoset default error handler to MPI_ERRORS_ARE_FATAL, as in a proper MPI implementation
Augustin Degomme [Sun, 18 Aug 2019 21:31:58 +0000 (23:31 +0200)]
set default error handler to MPI_ERRORS_ARE_FATAL, as in a proper MPI implementation

2 years agodisalign tags for collectives from their nonblocking counterparts, to correctly deadl...
Augustin Degomme [Sun, 18 Aug 2019 19:02:50 +0000 (21:02 +0200)]
disalign tags for collectives from their nonblocking counterparts, to correctly deadlock when both are entangled
as some blocking ones used internally are actually implement with nonblocking+wait, use the right blocking tag in this case.

2 years agoadd tests for errhandlers
Augustin Degomme [Sun, 18 Aug 2019 18:40:32 +0000 (20:40 +0200)]
add tests for errhandlers
Most of them actually need mpi_add_error features, which are not really implemented yet

2 years agoAdd support for MPI Errhandlers in Comm, File, Win.
Augustin Degomme [Sun, 18 Aug 2019 18:39:15 +0000 (20:39 +0200)]
Add support for MPI Errhandlers in Comm, File, Win.
Default remains MPI_ERRORS_RETURN for now, pending test fixes

2 years agoApply the default settings of 'smpi/buffering' too
Martin Quinson [Sun, 18 Aug 2019 15:01:40 +0000 (17:01 +0200)]
Apply the default settings of 'smpi/buffering' too

Previously, we did obey to that option when given, but the default
value was ignored. This is because the handling was done in the value
verification callback, that is not used for the default value.

2 years agomore informative error message when checking params of MPI_Iallgather
Martin Quinson [Sun, 18 Aug 2019 14:21:56 +0000 (16:21 +0200)]
more informative error message when checking params of MPI_Iallgather

2 years agoappveyor: don't build python as it was not tested anyway, and don't build anymore
Martin Quinson [Sun, 18 Aug 2019 14:21:41 +0000 (16:21 +0200)]
appveyor: don't build python as it was not tested anyway, and don't build anymore

2 years agoMPI_Scatterv: sendcounts and displs params can be NULL on non-root ranks
Martin Quinson [Sun, 18 Aug 2019 10:34:26 +0000 (12:34 +0200)]
MPI_Scatterv: sendcounts and displs params can be NULL on non-root ranks

2 years agomore informative error messages on parameter checks of MPI_Iscatterv
Martin Quinson [Sun, 18 Aug 2019 10:21:35 +0000 (12:21 +0200)]
more informative error messages on parameter checks of MPI_Iscatterv

This should be done for each and every collective, but not today...

2 years agoFix tesh for the new mc-sendsend test
Martin Quinson [Sun, 18 Aug 2019 08:12:39 +0000 (10:12 +0200)]
Fix tesh for the new mc-sendsend test

- Test for all factories but thread (that is borken with MC)
- Don't run this test when Java, no matter what. Not sure how I came
  to such a stupid idea :)

2 years agofix make dist and python dist
Martin Quinson [Sun, 18 Aug 2019 00:34:08 +0000 (02:34 +0200)]
fix make dist and python dist

2 years agoadd a test of MC detecting blocking send/send patterns
Martin Quinson [Sat, 4 Nov 2017 17:51:02 +0000 (18:51 +0100)]
add a test of MC detecting blocking send/send patterns

2 years agoNew MC option: smpi/buffering, to control MPI buffering
Martin Quinson [Sat, 17 Aug 2019 23:33:14 +0000 (01:33 +0200)]
New MC option: smpi/buffering, to control MPI buffering

2 years agosmpirun: add a -quiet argument, allowing failing tests in tesh
Martin Quinson [Sat, 17 Aug 2019 23:19:27 +0000 (01:19 +0200)]
smpirun: add a -quiet argument, allowing failing tests in tesh

2 years agomore informative message when setting inexistant config items
Martin Quinson [Sat, 17 Aug 2019 22:42:48 +0000 (00:42 +0200)]
more informative message when setting inexistant config items

2 years agodocs: fix borken links
Martin Quinson [Sat, 17 Aug 2019 22:10:43 +0000 (00:10 +0200)]
docs: fix borken links

2 years agoPMPI_Cart_create: check that each dim is positive
Martin Quinson [Sat, 17 Aug 2019 21:07:21 +0000 (23:07 +0200)]
PMPI_Cart_create: check that each dim is positive

2 years agoHave SMPI fail on MPI_ERR_* in MC mode
Martin Quinson [Sat, 17 Aug 2019 20:24:44 +0000 (22:24 +0200)]
Have SMPI fail on MPI_ERR_* in MC mode

The standard says that upon error, implementations should call the
current MPI error handler, which is MPI_ERRORS_ARE_FATAL by default
but could be changed to MPI_ERRORS_RETURN on need.

Since we don't implement MPI_Comm_set_errhandler() to switch between
modes, the simulation mode of SMPI is only issuing a warning on
errors (which is similar to MPI_ERRORS_RETURN).

This commit adds a MC_assert() stating that every MPI call succeed.
This will lead to a property failure (visible only in MC mode) when a
MPI_ERR_* is issued by the implementation (which is somehow similar to
MPI_ERRORS_ARE_FATAL).