2 @defgroup SMPI_API SMPI: Simulate real MPI applications
3 @brief Programming environment for the simulation of MPI applications
5 This programming environment enables the study of MPI application by
6 emulating them on top of the SimGrid simulator. This is particularly
7 interesting to study existing MPI applications within the comfort of
8 the simulator. The motivation for this work is detailed in the
9 reference article (available at http://hal.inria.fr/inria-00527150).
11 Our goal is to enable the study of *unmodified* MPI applications,
12 although we are not quite there yet (see @ref SMPI_what). In
13 addition, you can modify your code to speed up your studies or
14 otherwise increase their scalability (see @ref SMPI_adapting).
16 \section SMPI_who Who should use SMPI (and who shouldn't)
18 SMPI is now considered as stable and you can use it in production. You
19 may probably want to read the scientific publications that detail the
20 models used and their limits, but this should not be absolutely
21 necessary. If you already fluently write and use MPI applications,
22 SMPI should sound very familiar to you. Use smpicc instead of mpicc,
23 and smpirun instead of mpirun (see below for more details).
25 Of course, if you don't know what MPI is, the documentation of SMPI
26 will seem a bit terse to you. You should pick up a good MPI tutorial
27 on the Internet (or a course in your favorite university) and come
28 back to SMPI once you know a bit more about MPI. Alternatively, you
29 may want to turn to the other SimGrid interfaces such as the
30 \ref MSG_API environment, or the \ref SD_API one.
32 \section SMPI_what What can run within SMPI?
34 You can run unmodified MPI applications (both C and Fortran) within
35 SMPI, provided that you only use MPI calls that we implemented in
36 MPI. Global variables should be handled correctly on Linux systems.
38 \subsection SMPI_what_coverage MPI coverage of SMPI
40 Our coverage of the interface is very decent, but still incomplete;
41 Given the size of the MPI standard, it may well be that we never
42 implement absolutely all existing primitives. One sided communications
43 and I/O primitives are not targeted for now. Our current state is
44 still very decent: we pass most of the MPICH coverage tests.
46 The full list of not yet implemented functions is documented in the
47 file <tt>include/smpi/smpi.h</tt> of the archive, between two lines
48 containing the <tt>FIXME</tt> marker. If you really need a missing
49 feature, please get in touch with us: we can guide you though the
50 SimGrid code to help you implementing it, and we'd glad to integrate
51 it in the main project afterward if you contribute them back.
53 \subsection SMPI_what_globals Global variables
55 Concerning the globals, the problem comes from the fact that usually,
56 MPI processes run as real UNIX processes while they are all folded
57 into threads of a unique system process in SMPI. Global variables are
58 usually private to each MPI process while they become shared between
59 the processes in SMPI. This point is rather problematic, and currently
60 forces to modify your application to privatize the global variables.
62 We tried several techniques to work this around. We used to have a
63 script that privatized automatically the globals through static
64 analysis of the source code, but it was not robust enough to be used
65 in production. This issue, as well as several potential solutions, is
66 discussed in this article: "Automatic Handling of Global Variables for
67 Multi-threaded MPI Programs",
68 available at http://charm.cs.illinois.edu/newPapers/11-23/paper.pdf
69 (note that this article does not deal with SMPI but with a concurrent
70 solution called AMPI that suffers of the same issue).
72 A method using dynamic switching of the .data and .bss segments of an
73 ELF executable has been introduced in SimGrid 3.11. By using the <tt>smpi/
74 privatize_global_variableles</tt> option to yes, SMPI will duplicate
75 the segments containing the global variables and when needed, will map
76 the right one in memory. This needs ELF executables and mmap on the system
77 (Linux and recent BSDs should be compatible). %As no copy is involved,
78 performance should not be altered (but memory occupation will be higher).
80 This solution actually works really good for a good number of MPI
81 applications. Its main limitation is that if the application loads dynamic
82 libraries, their global variables won't be privatized. This can be avoided
83 by linking statically with these libraries (but NOT with libsimgrid, as we
84 need SimGrid's own global varibles).
87 \section SMPI_compiling Compiling your code
89 This is very simply done with the <tt>smpicc</tt> script. If you
90 already compiled any MPI code before, you already know how to use it.
91 If not, you should try to get your MPI code running on top of MPI
92 before giving SMPI a spin. Actually, that's very simple even if it's
93 the first time you use MPI code: just use smpicc as a compiler (in
94 replacement of gcc or your usual compiler), and you're set.
97 Some configure scripts ("./configure") will test the compiler by
98 compiling some simple example scripts. To make this work, you
99 must set the <b>SMPI_PRETEND_CC</b> environment variable to
100 some arbitrary value, for instance "export SMPI_PRETEND_CC=1"
102 \section SMPI_executing Executing your code on top of the simulator
104 This is done though the <tt>smpirun</tt> script as follows.
105 <tt>my_hostfile.txt</tt> is a classical MPI hostfile (that is, this
106 file lists the machines on which the processes must be dispatched, one
107 per line) <tt>my_platform.xml</tt> is a classical SimGrid platform
108 file. Of course, the hosts of the hostfile must exist in the provided
109 platform. <tt>./program</tt> is the MPI program that you want to
110 simulate (must be compiled by <tt>smpicc</tt>) while <tt>-arg</tt> is
111 a command-line parameter passed to this program.
114 smpirun -hostfile my_hostfile.txt -platform my_platform.xml ./program -arg
117 smpirun accepts other parameters, such as <tt>-np</tt> if you don't
118 want to use all the hosts defined in the hostfile, <tt>-map</tt> to
119 display on which host each rank gets mapped of <tt>-trace</tt> to
120 activate the tracing during the simulation. You can get the full list
126 \section SMPI_adapting Adapting your MPI code to the use of SMPI
128 As detailed in the reference article (available at
129 http://hal.inria.fr/inria-00527150), you may want to adapt your code
130 to improve the simulation performance. But these tricks may seriously
131 hinder the result quality (or even prevent the app to run) if used
132 wrongly. We assume that if you want to simulate an HPC application,
133 you know what you are doing. Don't prove us wrong!
135 \section SMPI_adapting_size Reducing your memory footprint
137 If you get short on memory (the whole app is executed on a single node when
138 simulated), you should have a look at the SMPI_SHARED_MALLOC and
139 SMPI_SHARED_FREE macros. It allows to share memory areas between processes: The
140 purpose of these macro is that the same line malloc on each process will point
141 to the exact same memory area. So if you have a malloc of 2M and you have 16
142 processes, this macro will change your memory consumption from 2M*16 to 2M
143 only. Only one block for all processes.
145 If your program is ok with a block containing garbage value because all
146 processes write and read to the same place without any kind of coordination,
147 then this macro can dramatically shrink your memory consumption. For example,
148 that will be very beneficial to a matrix multiplication code, as all blocks will
149 be stored on the same area. Of course, the resulting computations will useless,
150 but you can still study the application behavior this way.
152 Naturally, this won't work if your code is data-dependent. For example, a Jacobi
153 iterative computation depends on the result computed by the code to detect
154 convergence conditions, so turning them into garbage by sharing the same memory
155 area between processes does not seem very wise. You cannot use the
156 SMPI_SHARED_MALLOC macro in this case, sorry.
158 This feature is demoed by the example file
159 <tt>examples/smpi/NAS/DT-folding/dt.c</tt>
161 \section SMPI_adapting_speed Toward faster simulations
163 If your application is too slow, try using SMPI_SAMPLE_LOCAL,
164 SMPI_SAMPLE_GLOBAL and friends to indicate which computation loops can
165 be sampled. Some of the loop iterations will be executed to measure
166 their duration, and this duration will be used for the subsequent
167 iterations. These samples are done per processor with
168 SMPI_SAMPLE_LOCAL, and shared between all processors with
169 SMPI_SAMPLE_GLOBAL. Of course, none of this will work if the execution
170 time of your loop iteration are not stable.
172 This feature is demoed by the example file
173 <tt>examples/smpi/NAS/EP-sampling/ep.c</tt>
176 \section SMPI_collective_algorithms Simulating collective operations
178 MPI collective operations can be implemented very differently from one library
179 to another. Actually, all existing libraries implement several algorithms
180 for each collective operation, and by default select at runtime which one
181 should be used for the current operation, depending on the sizes sent, the number
182 of nodes, the communicator, or the communication library being used. These
183 decisions are based on empirical results and theoretical complexity estimation,
184 but they can sometimes be suboptimal. Manual selection is possible in these cases,
185 to allow the user to tune the library and use the better collective if the
186 default one is not good enough.
188 SMPI tries to apply the same logic, regrouping algorithms from OpenMPI, MPICH
189 libraries, StarMPI (<a href="http://star-mpi.sourceforge.net/">STAR-MPI</a>), and MVAPICH2 libraries.
190 This collection of more than 115 algorithms allows a simple and effective
191 comparison of their behavior and performance, making SMPI a tool of choice for the
192 development of such algorithms.
194 \subsection Tracing_internals Tracing of internal communications
196 For each collective, default tracing only outputs global data.
197 Internal communication operations are not traced to avoid outputting too much data
198 to the trace. To debug and compare algorithm, this can be changed with the item
199 \b tracing/smpi/internals , which has 0 for default value.
200 Here are examples of two alltoall collective algorithms runs on 16 nodes,
201 the first one with a ring algorithm, the second with a pairwise one :
204 <a href="smpi_simgrid_alltoall_ring_16.png" border=0><img src="smpi_simgrid_alltoall_ring_16.png" width="30%" border=0 align="center"></a>
205 <a href="smpi_simgrid_alltoall_pair_16.png" border=0><img src="smpi_simgrid_alltoall_pair_16.png" width="30%" border=0 align="center"></a>
209 \subsection Selectors
211 The default selection logic implemented by default in OpenMPI (version 1.7)
212 and MPICH (version 3.0.4) has been replicated and can be used by setting the
213 \b smpi/coll_selector item to either ompi or mpich. A selector based on the selection logic of MVAPICH2 (version 1.9) tuned on the Stampede cluster as also been implemented, as well as a preliminary version of an Intel MPI selector (version 4.1.3, also tuned for the Stampede cluster). Due the closed source nature of Intel MPI, some of the algorithms described in the documentation are not available, and are replaced by mvapich ones.
215 Values for option \b smpi/coll_selector are :
222 The code and details for each
223 selector can be found in the <tt>src/smpi/colls/smpi_(openmpi/mpich/mvapich2/impi)_selector.c</tt> file.
224 As this is still in development, we do not insure that all algorithms are correctly
225 replicated and that they will behave exactly as the real ones. If you notice a difference,
226 please contact <a href="http://lists.gforge.inria.fr/mailman/listinfo/simgrid-devel">SimGrid developers mailing list</a>
228 The default selector uses the legacy algorithms used in versions of SimGrid
229 previous to the 3.10. they should not be used to perform performance study and
230 may be removed in the future, a different selector being used by default.
232 \subsection algos Available algorithms
234 For each one of the listed algorithms, several versions are available,
235 either coming from STAR-MPI, MPICH or OpenMPI implementations. Details can be
236 found in the code or in <a href="http://www.cs.arizona.edu/~dkl/research/papers/ics06.pdf">STAR-MPI</a> for STAR-MPI algorithms.
238 Each collective can be selected using the corresponding configuration item. For example, to use the pairwise alltoall algorithm, one should add \b --cfg=smpi/alltoall:pair to the line. This will override the selector (for this algorithm only) if provided, allowing better flexibility.
240 Warning: Some collective may require specific conditions to be executed correctly (for instance having a communicator with a power of two number of nodes only), which are currently not enforced by Simgrid. Some crashes can be expected while trying these algorithms with unusual sizes/parameters
242 \subsubsection MPI_Alltoall
244 Most of these are best described in <a href="http://www.cs.arizona.edu/~dkl/research/papers/ics06.pdf">STAR-MPI</a>
246 - default : naive one, by default
247 - ompi : use openmpi selector for the alltoall operations
248 - mpich : use mpich selector for the alltoall operations
249 - mvapich2 : use mvapich2 selector for the alltoall operations
250 - impi : use intel mpi selector for the alltoall operations
251 - automatic (experimental) : use an automatic self-benchmarking algorithm
252 - 2dmesh : organizes the nodes as a two dimensional mesh, and perform allgather
254 - 3dmesh : adds a third dimension to the previous algorithm
255 - rdb : recursive doubling : extends the mesh to a nth dimension, each one
257 - pair : pairwise exchange, only works for power of 2 procs, size-1 steps,
258 each process sends and receives from the same process at each step
259 - pair_light_barrier : same, with small barriers between steps to avoid contention
260 - pair_mpi_barrier : same, with MPI_Barrier used
261 - pair_one_barrier : only one barrier at the beginning
262 - ring : size-1 steps, at each step a process send to process (n+i)%size, and receives from (n-i)%size
263 - ring_light_barrier : same, with small barriers between some phases to avoid contention
264 - ring_mpi_barrier : same, with MPI_Barrier used
265 - ring_one_barrier : only one barrier at the beginning
266 - basic_linear : posts all receives and all sends,
267 starts the communications, and waits for all communication to finish
268 - mvapich2_scatter_dest : isend/irecv with scattered destinations, posting only a few messages at the same time
270 \subsubsection MPI_Alltoallv
272 - default : naive one, by default
273 - ompi : use openmpi selector for the alltoallv operations
274 - mpich : use mpich selector for the alltoallv operations
275 - mvapich2 : use mvapich2 selector for the alltoallv operations
276 - impi : use intel mpi selector for the alltoallv operations
277 - automatic (experimental) : use an automatic self-benchmarking algorithm
278 - bruck : same as alltoall
279 - pair : same as alltoall
280 - pair_light_barrier : same as alltoall
281 - pair_mpi_barrier : same as alltoall
282 - pair_one_barrier : same as alltoall
283 - ring : same as alltoall
284 - ring_light_barrier : same as alltoall
285 - ring_mpi_barrier : same as alltoall
286 - ring_one_barrier : same as alltoall
287 - ompi_basic_linear : same as alltoall
290 \subsubsection MPI_Gather
292 - default : naive one, by default
293 - ompi : use openmpi selector for the gather operations
294 - mpich : use mpich selector for the gather operations
295 - mvapich2 : use mvapich2 selector for the gather operations
296 - impi : use intel mpi selector for the gather operations
297 - automatic (experimental) : use an automatic self-benchmarking algorithm
298 which will iterate over all implemented versions and output the best
299 - ompi_basic_linear : basic linear algorithm from openmpi, each process sends to the root
300 - ompi_binomial : binomial tree algorithm
301 - ompi_linear_sync : same as basic linear, but with a synchronization at the
302 beginning and message cut into two segments.
303 - mvapich2_two_level : SMP-aware version from MVAPICH. Gather first intra-node (defaults to mpich's gather), and then exchange with only one process/node. Use mvapich2 selector to change these to tuned algorithms for Stampede cluster.
305 \subsubsection MPI_Barrier
306 - default : naive one, by default
307 - ompi : use openmpi selector for the barrier operations
308 - mpich : use mpich selector for the barrier operations
309 - mvapich2 : use mvapich2 selector for the barrier operations
310 - impi : use intel mpi selector for the barrier operations
311 - automatic (experimental) : use an automatic self-benchmarking algorithm
312 - ompi_basic_linear : all processes send to root
313 - ompi_two_procs : special case for two processes
314 - ompi_bruck : nsteps = sqrt(size), at each step, exchange data with rank-2^k and rank+2^k
315 - ompi_recursivedoubling : recursive doubling algorithm
316 - ompi_tree : recursive doubling type algorithm, with tree structure
317 - ompi_doublering : double ring algorithm
318 - mvapich2_pair : pairwise algorithm
321 \subsubsection MPI_Scatter
322 - default : naive one, by default
323 - ompi : use openmpi selector for the scatter operations
324 - mpich : use mpich selector for the scatter operations
325 - mvapich2 : use mvapich2 selector for the scatter operations
326 - impi : use intel mpi selector for the scatter operations
327 - automatic (experimental) : use an automatic self-benchmarking algorithm
328 - ompi_basic_linear : basic linear scatter
329 - ompi_binomial : binomial tree scatter
330 - mvapich2_two_level_direct : SMP aware algorithm, with an intra-node stage (default set to mpich selector), and then a basic linear inter node stage. Use mvapich2 selector to change these to tuned algorithms for Stampede cluster.
331 - mvapich2_two_level_binomial : SMP aware algorithm, with an intra-node stage (default set to mpich selector), and then a binomial phase. Use mvapich2 selector to change these to tuned algorithms for Stampede cluster.
335 \subsubsection MPI_Reduce
336 - default : naive one, by default
337 - ompi : use openmpi selector for the reduce operations
338 - mpich : use mpich selector for the reduce operations
339 - mvapich2 : use mvapich2 selector for the reduce operations
340 - impi : use intel mpi selector for the reduce operations
341 - automatic (experimental) : use an automatic self-benchmarking algorithm
342 - arrival_pattern_aware : root exchanges with the first process to arrive
343 - binomial : uses a binomial tree
344 - flat_tree : uses a flat tree
345 - NTSL : Non-topology-specific pipelined linear-bcast function
346 0->1, 1->2 ,2->3, ....., ->last node : in a pipeline fashion, with segments
348 - scatter_gather : scatter then gather
349 - ompi_chain : openmpi reduce algorithms are built on the same basis, but the
350 topology is generated differently for each flavor
351 chain = chain with spacing of size/2, and segment size of 64KB
352 - ompi_pipeline : same with pipeline (chain with spacing of 1), segment size
353 depends on the communicator size and the message size
354 - ompi_binary : same with binary tree, segment size of 32KB
355 - ompi_in_order_binary : same with binary tree, enforcing order on the
357 - ompi_binomial : same with binomial algo (redundant with default binomial
359 - ompi_basic_linear : basic algorithm, each process sends to root
360 - mvapich2_knomial : k-nomial algorithm. Default factor is 4 (mvapich2 selector adapts it through tuning)
361 - mvapich2_two_level : SMP-aware reduce, with default set to mpich both for intra and inter communicators. Use mvapich2 selector to change these to tuned algorithms for Stampede cluster.
362 - rab : <a href="https://fs.hlrs.de/projects/par/mpi//myreduce.html">Rabenseifner</a>'s reduce algorithm
364 \subsubsection MPI_Allreduce
365 - default : naive one, by default
366 - ompi : use openmpi selector for the allreduce operations
367 - mpich : use mpich selector for the allreduce operations
368 - mvapich2 : use mvapich2 selector for the allreduce operations
369 - impi : use intel mpi selector for the allreduce operations
370 - automatic (experimental) : use an automatic self-benchmarking algorithm
371 - lr : logical ring reduce-scatter then logical ring allgather
372 - rab1 : variations of the <a href="https://fs.hlrs.de/projects/par/mpi//myreduce.html">Rabenseifner</a> algorithm : reduce_scatter then allgather
373 - rab2 : variations of the <a href="https://fs.hlrs.de/projects/par/mpi//myreduce.html">Rabenseifner</a> algorithm : alltoall then allgather
374 - rab_rsag : variation of the <a href="https://fs.hlrs.de/projects/par/mpi//myreduce.html">Rabenseifner</a> algorithm : recursive doubling
375 reduce_scatter then recursive doubling allgather
376 - rdb : recursive doubling
377 - smp_binomial : binomial tree with smp : binomial intra
378 SMP reduce, inter reduce, inter broadcast then intra broadcast
379 - smp_binomial_pipeline : same with segment size = 4096 bytes
380 - smp_rdb : intra : binomial allreduce, inter : Recursive
381 doubling allreduce, intra : binomial broadcast
382 - smp_rsag : intra : binomial allreduce, inter : reduce-scatter,
383 inter:allgather, intra : binomial broadcast
384 - smp_rsag_lr : intra : binomial allreduce, inter : logical ring
385 reduce-scatter, logical ring inter:allgather, intra : binomial broadcast
386 - smp_rsag_rab : intra : binomial allreduce, inter : rab
387 reduce-scatter, rab inter:allgather, intra : binomial broadcast
388 - redbcast : reduce then broadcast, using default or tuned algorithms if specified
389 - ompi_ring_segmented : ring algorithm used by OpenMPI
390 - mvapich2_rs : rdb for small messages, reduce-scatter then allgather else
391 - mvapich2_two_level : SMP-aware algorithm, with mpich as intra algoritm, and rdb as inter (Change this behavior by using mvapich2 selector to use tuned values)
392 - rab : default <a href="https://fs.hlrs.de/projects/par/mpi//myreduce.html">Rabenseifner</a> implementation
394 \subsubsection MPI_Reduce_scatter
395 - default : naive one, by default
396 - ompi : use openmpi selector for the reduce_scatter operations
397 - mpich : use mpich selector for the reduce_scatter operations
398 - mvapich2 : use mvapich2 selector for the reduce_scatter operations
399 - impi : use intel mpi selector for the reduce_scatter operations
400 - automatic (experimental) : use an automatic self-benchmarking algorithm
401 - ompi_basic_recursivehalving : recursive halving version from OpenMPI
402 - ompi_ring : ring version from OpenMPI
403 - mpich_pair : pairwise exchange version from MPICH
404 - mpich_rdb : recursive doubling version from MPICH
405 - mpich_noncomm : only works for power of 2 procs, recursive doubling for noncommutative ops
408 \subsubsection MPI_Allgather
410 - default : naive one, by default
411 - ompi : use openmpi selector for the allgather operations
412 - mpich : use mpich selector for the allgather operations
413 - mvapich2 : use mvapich2 selector for the allgather operations
414 - impi : use intel mpi selector for the allgather operations
415 - automatic (experimental) : use an automatic self-benchmarking algorithm
416 - 2dmesh : see alltoall
417 - 3dmesh : see alltoall
418 - bruck : Described by Bruck et.al. in <a href="http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=642949">
419 Efficient algorithms for all-to-all communications in multiport message-passing systems</a>
420 - GB : Gather - Broadcast (uses tuned version if specified)
421 - loosely_lr : Logical Ring with grouping by core (hardcoded, default
423 - NTSLR : Non Topology Specific Logical Ring
424 - NTSLR_NB : Non Topology Specific Logical Ring, Non Blocking operations
425 - pair : see alltoall
427 - rhv : only power of 2 number of processes
428 - ring : see alltoall
429 - SMP_NTS : gather to root of each SMP, then every root of each SMP node
430 post INTER-SMP Sendrecv, then do INTRA-SMP Bcast for each receiving message,
431 using logical ring algorithm (hardcoded, default processes/SMP: 8)
432 - smp_simple : gather to root of each SMP, then every root of each SMP node
433 post INTER-SMP Sendrecv, then do INTRA-SMP Bcast for each receiving message,
434 using simple algorithm (hardcoded, default processes/SMP: 8)
435 - spreading_simple : from node i, order of communications is i -> i + 1, i ->
436 i + 2, ..., i -> (i + p -1) % P
437 - ompi_neighborexchange : Neighbor Exchange algorithm for allgather.
438 Described by Chen et.al. in <a href="http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=1592302">Performance Evaluation of Allgather Algorithms on Terascale Linux Cluster with Fast Ethernet</a>
439 - mvapich2_smp : SMP aware algorithm, performing intra-node gather, inter-node allgather with one process/node, and bcast intra-node
442 \subsubsection MPI_Allgatherv
443 - default : naive one, by default
444 - ompi : use openmpi selector for the allgatherv operations
445 - mpich : use mpich selector for the allgatherv operations
446 - mvapich2 : use mvapich2 selector for the allgatherv operations
447 - impi : use intel mpi selector for the allgatherv operations
448 - automatic (experimental) : use an automatic self-benchmarking algorithm
449 - GB : Gatherv - Broadcast (uses tuned version if specified, but only for
450 Bcast, gatherv is not tuned)
451 - pair : see alltoall
452 - ring : see alltoall
453 - ompi_neighborexchange : see allgather
454 - ompi_bruck : see allgather
455 - mpich_rdb : recursive doubling algorithm from MPICH
456 - mpich_ring : ring algorithm from MPICh - performs differently from the
459 \subsubsection MPI_Bcast
460 - default : naive one, by default
461 - ompi : use openmpi selector for the bcast operations
462 - mpich : use mpich selector for the bcast operations
463 - mvapich2 : use mvapich2 selector for the bcast operations
464 - impi : use intel mpi selector for the bcast operations
465 - automatic (experimental) : use an automatic self-benchmarking algorithm
466 - arrival_pattern_aware : root exchanges with the first process to arrive
467 - arrival_pattern_aware_wait : same with slight variation
468 - binomial_tree : binomial tree exchange
469 - flattree : flat tree exchange
470 - flattree_pipeline : flat tree exchange, message split into 8192 bytes pieces
471 - NTSB : Non-topology-specific pipelined binary tree with 8192 bytes pieces
472 - NTSL : Non-topology-specific pipelined linear with 8192 bytes pieces
473 - NTSL_Isend : Non-topology-specific pipelined linear with 8192 bytes pieces, asynchronous communications
474 - scatter_LR_allgather : scatter followed by logical ring allgather
475 - scatter_rdb_allgather : scatter followed by recursive doubling allgather
476 - arrival_scatter : arrival pattern aware scatter-allgather
477 - SMP_binary : binary tree algorithm with 8 cores/SMP
478 - SMP_binomial : binomial tree algorithm with 8 cores/SMP
479 - SMP_linear : linear algorithm with 8 cores/SMP
480 - ompi_split_bintree : binary tree algorithm from OpenMPI, with message split in 8192 bytes pieces
481 - ompi_pipeline : pipeline algorithm from OpenMPI, with message split in 128KB pieces
482 - mvapich2_inter_node : Inter node default mvapich worker
483 - mvapich2_intra_node : Intra node default mvapich worker
484 - mvapich2_knomial_intra_node : k-nomial intra node default mvapich worker. default factor is 4.
486 \subsection auto Automatic evaluation
488 (Warning : This is experimental and may be removed or crash easily)
490 An automatic version is available for each collective (or even as a selector). This specific
491 version will loop over all other implemented algorithm for this particular collective, and apply
492 them while benchmarking the time taken for each process. It will then output the quickest for
493 each process, and the global quickest. This is still unstable, and a few algorithms which need
494 specific number of nodes may crash.
497 \subsection add Add an algorithm
499 To add a new algorithm, one should check in the src/smpi/colls folder how other algorithms
500 are coded. Using plain MPI code inside Simgrid can't be done, so algorithms have to be
501 changed to use smpi version of the calls instead (MPI_Send will become smpi_mpi_send). Some functions may have different signatures than their MPI counterpart, please check the other algorithms or contact us using <a href="http://lists.gforge.inria.fr/mailman/listinfo/simgrid-devel">SimGrid developers mailing list</a>.
503 Example: adding a "pair" version of the Alltoall collective.
505 - Implement it in a file called alltoall-pair.c in the src/smpi/colls folder. This file should include colls_private.h.
507 - The name of the new algorithm function should be smpi_coll_tuned_alltoall_pair, with the same signature as MPI_Alltoall.
509 - Once the adaptation to SMPI code is done, add a reference to the file ("src/smpi/colls/alltoall-pair.c") in the SMPI_SRC part of the DefinePackages.cmake file inside buildtools/cmake, to allow the file to be built and distributed.
511 - To register the new version of the algorithm, simply add a line to the corresponding macro in src/smpi/colls/cools.h ( add a "COLL_APPLY(action, COLL_ALLTOALL_SIG, pair)" to the COLL_ALLTOALLS macro ). The algorithm should now be compiled and be selected when using --cfg=smpi/alltoall:pair at runtime.
513 - To add a test for the algorithm inside Simgrid's test suite, juste add the new algorithm name in the ALLTOALL_COLL list found inside teshsuite/smpi/CMakeLists.txt . When running ctest, a test for the new algorithm should be generated and executed. If it does not pass, please check your code or contact us.
515 - Feel free to push this new algorithm to the SMPI repository using Git.
