From: Takishipp Date: Wed, 16 Aug 2017 08:58:48 +0000 (+0200) Subject: Merge branch 'master' of github.com:simgrid/simgrid into s_type_cleanup X-Git-Tag: v3_17~203^2^2~10 X-Git-Url: http://info.iut-bm.univ-fcomte.fr/pub/gitweb/simgrid.git/commitdiff_plain/b87fc0829538ec066fd077e6c30ee6270b8abd78?hp=a2ff7cc7c5bf86435c85fcc21ca6edc6c20cde3b Merge branch 'master' of github.com:simgrid/simgrid into s_type_cleanup --- diff --git a/.gitignore b/.gitignore index 2b2b42be43..5c80711e9d 100644 --- a/.gitignore +++ b/.gitignore @@ -102,6 +102,7 @@ doc/simgrid.tag doc/doxygen/logcategories.doc doc/simgrid_modules.map doc/javadoc +doc/example_lists/ ### Specific of project .settings/ .csettings/ diff --git a/ChangeLog b/ChangeLog index 9f70fdbcc1..251ba08e59 100644 --- a/ChangeLog +++ b/ChangeLog @@ -4,6 +4,13 @@ SimGrid (3.17) UNRELEASED (release target: September 22 2017) - Comm.detach(): start and forget about asynchronous emission - this_actor::send(mailbox) is now mailbox->put() + SURF + - Unused option network/sender-gap is removed. + + MSG + - Deprecate MSG_task_isend_with_matching(): This unused feature + really complicates our internals. Will be removed in v3.20. + SimGrid (3.16) Released June 22. 2017. The Blooming Spring Release: developments are budding. @@ -52,7 +59,7 @@ SimGrid (3.16) Released June 22. 2017. SMPI - New algorithm to privatize globals: dlopen, with dynamic loading tricks - New option: smpi/keep-temps to not cleanup temp files - - New option : smpi/shared-malloc-blocksize . Relevant only when global shared + - New option : smpi/shared-malloc-blocksize . Relevant only when global shared mallocs mode is used, allows to change the size of the fake file used (default 1MB), to potentially limit the number of mappings for large runs. - Support for sparse privatized malloc with SMPI_PARTIAL_SHARED_MALLOC() @@ -93,7 +100,7 @@ SimGrid (3.15) Released March 22. 2017 MSG - New: MSG_process_yield(). Stop and yield to other processes. - - New: MSG_process_daemon(). Daemon processes are automatically killed + - New: MSG_process_daemon(). Daemon processes are automatically killed when the last non-daemon process terminates - New: MSG_process_ref/unref(). Fiddle with the process refcounting. - Renamed MSG_energy_plugin_init() -> MSG_host_energy_plugin_init() @@ -105,13 +112,13 @@ SimGrid (3.15) Released March 22. 2017 - Kill the obscure NativeException. Nobody want to survive the issues it denotes, so use JniException that is a RuntimeException (not to be caught explicitly). - - Partial bug fix in initialization. SimGrid flags on command line were - consumed at C level but stayed in the original Java String[] args. + - Partial bug fix in initialization. SimGrid flags on command line were + consumed at C level but stayed in the original Java String[] args. This could mess users' args[i] if SG flags were not put at the end of the command line. - The SimGrid flags are now removed from the Java arguments. However, + The SimGrid flags are now removed from the Java arguments. However, the number of arguments REMAINS UNCHANGED. It is then UNSAFE to test - if args.length is greater than the number of YOUR OWN ARGUMENTS. + if args.length is greater than the number of YOUR OWN ARGUMENTS. It might be if you have --log or --cfg flags in the command line. - Fix numerous memleaks all around the place. In particular, around VMs. @@ -133,21 +140,21 @@ SimGrid (3.15) Released March 22. 2017 SimDag - Backwards Compatibility breaks - - The SD_route_get_list and SD_route_get_size functions have been removed. - They are replaced by the unique + - The SD_route_get_list and SD_route_get_size functions have been removed. + They are replaced by the unique void sg_host_route(sg_host_t src, sg_host_t dst, xbt_dynar_t links) - The route from src to dst is built in the links dynar whose size can + The route from src to dst is built in the links dynar whose size can be obtained with xbt_dynar_length. - The SD_route_bandwidth and SD_route_latency functions have been replaced by - sg_host_route_bandwidth and sg_host_route_latency. + sg_host_route_bandwidth and sg_host_route_latency. Macros ensure the backwards compatibility, but you should fix your code SMPI - - Major C++ rewrite ongoing (SMPI used to be C compiled in C++). + - Major C++ rewrite ongoing (SMPI used to be C compiled in C++). This can break codes using SMPI internals (from private.h instead of the public smpi.h). - - Bump our claim of support from MPI 1.1 to MPI 2.2. + - Bump our claim of support from MPI 1.1 to MPI 2.2. We don't support 100% of it, but it should be enough. Contact us if not. - - MPI_Win_lock/unlock, MPI_Get_accumulate support added (as for all RMA, implementation is + - MPI_Win_lock/unlock, MPI_Get_accumulate support added (as for all RMA, implementation is naive and probably inaccurate) - New algorithm for SMPI_SHARED_MALLOC: global, where all blocks are mapped onto a unique small file using some system magic. @@ -194,13 +201,13 @@ SimGrid (3.14) Released December 24. 2016 SMPI * Privatization now works on FreeBSD - + * Privatization is now activated by default in smpirun You can turn it off with -no-privatize if something goes wrong. * Call-location tracing for SMPI. - You can add the exact location (filename / linenumber) of an MPI call to - your trace files and slow down or speed up the simulation between two + You can add the exact location (filename / linenumber) of an MPI call to + your trace files and slow down or speed up the simulation between two consecutive calls by using an adjustment file (see the documentation). * Fixed computation of timings for MPI_Send, MPI_Recv & possibly also others @@ -212,8 +219,8 @@ SimGrid (3.14) Released December 24. 2016 * smpi/cpu-threshold:-1 should become smpi/simulate-computation:no smpi/running-power is renamed to smpi/host-speed - * smpi/grow-injected-times option to enable or disable multiplication of the - timings injected in MPI_Iprobe, or MPI_Test. Enabled by default, which can + * smpi/grow-injected-times option to enable or disable multiplication of the + timings injected in MPI_Iprobe, or MPI_Test. Enabled by default, which can make simulation less precise (but also much faster). * smpirun script should be (much) faster for large deployments. @@ -243,11 +250,11 @@ SimGrid (3.14) Released December 24. 2016 - Ignored parameter of vm_create: core_nb, disk_path and disk_size. - Unimplemented save/restore methods * MSG_as_router_get_property_value() was redundent with - MSG_environment_as_get_property_value(). + MSG_environment_as_get_property_value(). - Removed MSG_as_router_*propert*() functions - Added MSG_environment_as_set_property_value() for consistency - * xbt heterogeneous dictionnaries (created with xbt_dict_new()). - Well, they are still there for now, but deprecated with a warning. + * xbt heterogeneous dictionnaries (created with xbt_dict_new()). + Well, they are still there for now, but deprecated with a warning. Please switch to xbt_dict_new_homogeneous() before this is removed for real. * Task affinity. Its intended behavior (that was very badly tested @@ -255,8 +262,8 @@ SimGrid (3.14) Released December 24. 2016 would have hoped here. * xbt_os_sem_get_value: unused internally, deprecated on OS X El Capitan * Option network/coordinates is now useless and should be dropped. - - Storage: + + Storage: * Added option storage/max_file_descriptors to allow more than 1024 files opened SURF: @@ -265,29 +272,29 @@ SimGrid (3.14) Released December 24. 2016 XML: * Add Exa- and Peta- units such as EiB, EB, Eib, Eb for size, and - EiBps, EBps, Eibps, Ebps for bandwidth. + EiBps, EBps, Eibps, Ebps for bandwidth. They may become useful to some lucky ones. - + Java: * New functions: msg.Comm.waitAll() and msg.Comm.waitAny() * ex/app_tokenring: new example, very similar to the MSG Token Ring * ex/async_waitAll: new example, on asynchronous communications MSG: - * Memory usage should be decreased for simulations with a large number + * Memory usage should be decreased for simulations with a large number of processes. This also helps for SMPI. SimGrid (3.13) stable; urgency=low The Half Release, a.k.a. the Zealous Easter Trim. - + - We removed half of the lines, that were mostly experimental cruft. v3.12 lasted 286000 lines of code, v3.13 is only 142000 lines (not counting blanks and comments -- according to openhub.net) - The internals are now compiled in C++ (and will soon be clean C++) - - We removed 75 klines of XML, 12 klines of Java, 5 klines of cmake, + - We removed 75 klines of XML, 12 klines of Java, 5 klines of cmake, 59 klines of C, etc. We added only 29 klines of C++ in replacement. - + * Backwards Compatibility breaks - Removed Lua simulation bindings (switch to C or Java for that). Lua can still be used to describe platforms @@ -299,11 +306,11 @@ SimGrid (3.13) stable; urgency=low way, with filter function. - MSG_task_listen_from_host - MSG_mailbox_get_count_host_waiting_tasks - - MSG_mailbox_put_with_timeout was removed. + - MSG_mailbox_put_with_timeout was removed. Please use MSG_task_send_with_timeout instead. - - In SimDag - - the SD_application_reinit function was removed. It has been a noop for a while. - - The ACCESS_MODE of SD_workstation has been removed. This feature was not really usable and should soon be + - In SimDag + - the SD_application_reinit function was removed. It has been a noop for a while. + - The ACCESS_MODE of SD_workstation has been removed. This feature was not really usable and should soon be replaced by a more flexible mechanism. - The following functions thus do not exist anymore - SD_workstation_get_access_mode @@ -313,7 +320,7 @@ SimGrid (3.13) stable; urgency=low - SD_route_get_communication_time => SG_route_get_latency() + amount / SD_route_get_bandwidth() - SD_workstation_get_computation_time => amount / sg_host_speed() - In Java - - VM.setBound(int load) is now VM.setBound(double bound) to meet the MSG semantics. + - VM.setBound(int load) is now VM.setBound(double bound) to meet the MSG semantics. Use VM.getSpeed()*load/100 for the legacy behavior. - In CMake - option enable_tracing was removed. It was not doing anything for a while. @@ -352,7 +359,7 @@ SimGrid (3.13) stable; urgency=low And the same in bits: 'Tibps', 'Gibps', 'Mibps', 'Kibps', 'Tbps', 'Gbps', 'Mbps', 'kbps', 'bps' - latency. Default: 's' second. Also defined: 'w' week, 'd' day, 'h' hour, 'm' minute, 'ms' millisecond, 'us' microsecond, 'ns' nanosecond, 'ps' picosecond - + * bin/simgrid_update_xml can upgrade your files automatically (won't convert unit-less values) tools/sg_xml_unit_converter.py may help (but it's just a warning and will probably ever be). @@ -363,14 +370,14 @@ SimGrid (3.13) stable; urgency=low MSG * The examples were completely reorganized (in C and Java), for your browsing pleasure. - * Kill all deprecated functions (the ones you had when declaring MSG_DEPRECATED). + * Kill all deprecated functions (the ones you had when declaring MSG_DEPRECATED). They were deprecated since a few years, and probably did not even compile anymore. SimDag - * The API has been profoundly modified to directly use the core objects instead of redefining its own. + * The API has been profoundly modified to directly use the core objects instead of redefining its own. SD_Workstation_t and SD_link_t are now sg_host_t and sg_link_t respectively. - Some functions have also been renamed for consistency. Backward compatibility is maintained, but users are - encouraged to update their codes. A list of the modified functions can be found at the end of + Some functions have also been renamed for consistency. Backward compatibility is maintained, but users are + encouraged to update their codes. A list of the modified functions can be found at the end of include/simgrid/simdag.h Simix @@ -406,7 +413,7 @@ SimGrid (3.13) stable; urgency=low * refactoring and cleanup of the code; * ongoing process to cleanly separate the model-checking algorithms from the code model-checking support. - + -- Wed Apr 27 21:00:53 CEST 2016 Da SimGrid team SimGrid (3.12) stable; urgency=low diff --git a/doc/Doxyfile.in b/doc/Doxyfile.in index 2ec75ddba2..500cbe435e 100644 --- a/doc/Doxyfile.in +++ b/doc/Doxyfile.in @@ -689,6 +689,7 @@ INPUT = doxygen/index.doc \ @CMAKE_HOME_DIRECTORY@/include/simgrid \ @CMAKE_HOME_DIRECTORY@/include/simgrid/s4u \ @CMAKE_HOME_DIRECTORY@/src/include/surf \ + @CMAKE_HOME_DIRECTORY@/src/include/xbt \ @CMAKE_HOME_DIRECTORY@/src/msg/ \ @CMAKE_HOME_DIRECTORY@/src/kernel/ \ @CMAKE_HOME_DIRECTORY@/src/kernel/activity/ \ diff --git a/doc/doxygen/options.doc b/doc/doxygen/options.doc index 2ec954e0ed..98f91ccf2b 100644 --- a/doc/doxygen/options.doc +++ b/doc/doxygen/options.doc @@ -112,7 +112,6 @@ int main(int argc, char *argv[]) { - \c network/maxmin-selective-update: \ref options_model_optim - \c network/model: \ref options_model_select - \c network/optim: \ref options_model_optim -- \c network/sender_gap: \ref options_model_network_sendergap - \c network/TCP-gamma: \ref options_model_network_gamma - \c network/weight-S: \ref options_model_network_coefs @@ -373,15 +372,6 @@ can be set to 0 (disable this feature) or 1 (enable it). Note that with the default host model this option is activated by default. -\subsubsection options_model_network_sendergap Simulating sender gap - -(this configuration item is experimental and may change or disapear) - -It is possible to specify a timing gap between consecutive emission on -the same network card through the \b network/sender-gap item. This -is still under investigation as of writting, and the default value is -to wait 10 microseconds (1e-5 seconds) between emissions. - \subsubsection options_model_network_asyncsend Simulating asyncronous send (this configuration item is experimental and may change or disapear) diff --git a/examples/java/app/bittorrent/Common.java b/examples/java/app/bittorrent/Common.java index 0eb1eaa0e1..b7f7718606 100644 --- a/examples/java/app/bittorrent/Common.java +++ b/examples/java/app/bittorrent/Common.java @@ -1,4 +1,4 @@ -/* Copyright (c) 2006-2014, 2016. The SimGrid Team. +/* Copyright (c) 2006-2014, 2016-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -18,7 +18,7 @@ class Common { public static final int MESSAGE_SIZE = 1; /* Information message size */ public static final int MAXIMUM_PEERS = 50; /* Max number of peers sent by the tracker to clients */ public static final int TRACKER_QUERY_INTERVAL = 1000; /* Interval of time where the peer should send a request to the tracker */ - public static final double TRACKER_COMM_SIZE = 0.01; /* Communication size for a task to the tracker */ + public static final double TRACKER_COMM_SIZE = 1; /* Communication size for a task to the tracker */ public static final int GET_PEERS_TIMEOUT = 10000; /* Timeout for the get peers data */ public static final int TIMEOUT_MESSAGE = 10; public static final int TRACKER_RECEIVE_TIMEOUT = 10; diff --git a/examples/msg/CMakeLists.txt b/examples/msg/CMakeLists.txt index ca06ab37ba..fc994e39e4 100644 --- a/examples/msg/CMakeLists.txt +++ b/examples/msg/CMakeLists.txt @@ -30,13 +30,6 @@ if(SIMGRID_HAVE_NS3) set_target_properties(network-ns3 PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/network-ns3) endif() -add_executable (bittorrent app-bittorrent/bittorrent.c app-bittorrent/messages.c app-bittorrent/peer.c app-bittorrent/tracker.c app-bittorrent/connection.c) -target_link_libraries(bittorrent simgrid) -set_target_properties(bittorrent PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/app-bittorrent) -foreach (file bittorrent connection messages peer tracker) - set(examples_src ${examples_src} ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/${file}.c ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/${file}.h) -endforeach() - add_executable (chainsend app-chainsend/chainsend.c app-chainsend/iterator.c app-chainsend/common.c app-chainsend/messages.c app-chainsend/broadcaster.c app-chainsend/peer.c) target_link_libraries(chainsend simgrid) set_target_properties(chainsend PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/app-chainsend) @@ -51,21 +44,19 @@ foreach (file answer dht-kademlia node routing_table task) set(examples_src ${examples_src} ${CMAKE_CURRENT_SOURCE_DIR}/dht-kademlia/${file}.c ${CMAKE_CURRENT_SOURCE_DIR}/dht-kademlia/${file}.h) endforeach() -foreach (file app-bittorrent app-chainsend app-masterworker app-pingpong async-wait +foreach (file app-chainsend app-masterworker app-pingpong async-wait async-waitall async-waitany dht-kademlia dht-pastry io-remote platform-properties process-yield task-priority) set(xml_files ${xml_files} ${CMAKE_CURRENT_SOURCE_DIR}/${file}/${file}_d.xml) endforeach() set(txt_files ${txt_files} ${CMAKE_CURRENT_SOURCE_DIR}/README.doc PARENT_SCOPE) -set(bin_files ${bin_files} ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/generate.py - ${CMAKE_CURRENT_SOURCE_DIR}/dht-kademlia/generate.py +set(bin_files ${bin_files} ${CMAKE_CURRENT_SOURCE_DIR}/dht-kademlia/generate.py ${CMAKE_CURRENT_SOURCE_DIR}/dht-pastry/generate.py PARENT_SCOPE) set(examples_src ${examples_src} ${CMAKE_CURRENT_SOURCE_DIR}/app-chainsend/chainsend.c ${CMAKE_CURRENT_SOURCE_DIR}/dht-kademlia/common.h ${CMAKE_CURRENT_SOURCE_DIR}/network-ns3/network-ns3.c PARENT_SCOPE) -set(tesh_files ${tesh_files} ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/app-bittorrent.tesh - ${CMAKE_CURRENT_SOURCE_DIR}/app-chainsend/app-chainsend.tesh +set(tesh_files ${tesh_files} ${CMAKE_CURRENT_SOURCE_DIR}/app-chainsend/app-chainsend.tesh ${CMAKE_CURRENT_SOURCE_DIR}/dht-kademlia/dht-kademlia.tesh ${CMAKE_CURRENT_SOURCE_DIR}/app-masterworker/app-masterworker-multicore.tesh ${CMAKE_CURRENT_SOURCE_DIR}/app-masterworker/app-masterworker-vivaldi.tesh @@ -89,7 +80,7 @@ set(xml_files ${xml_files} ${CMAKE_CURRENT_SOURCE_DIR}/app-masterworker/a ${CMAKE_CURRENT_SOURCE_DIR}/process-startkilltime/start_d.xml ${CMAKE_CURRENT_SOURCE_DIR}/process-startkilltime/start_kill_d.xml PARENT_SCOPE) -foreach(x app-bittorrent app-chainsend app-masterworker app-pingpong app-token-ring +foreach(x app-chainsend app-masterworker app-pingpong app-token-ring async-wait async-waitall async-waitany cloud-capping cloud-masterworker cloud-migration cloud-simple cloud-two-tasks dht-pastry dht-kademlia platform-failures io-file io-remote io-storage task-priority process-create process-daemon process-join process-kill process-migration process-startkilltime process-suspend process-yield @@ -105,7 +96,6 @@ foreach (x categories route-user-variables link-user-variables masterworker plat ADD_TESH(msg-trace-${x} --setenv bindir=${CMAKE_BINARY_DIR}/examples/msg/trace-${x} --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/platforms --cd ${CMAKE_HOME_DIRECTORY}/examples/msg/trace-${x} trace-${x}.tesh) endforeach() -ADD_TESH_FACTORIES(msg-app-bittorrent-parallel "thread;ucontext;raw" --cfg contexts/nthreads:4 ${CONTEXTS_SYNCHRO} --setenv bindir=${CMAKE_BINARY_DIR}/examples/msg/app-bittorrent --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/platforms --cd ${CMAKE_HOME_DIRECTORY}/examples/msg/app-bittorrent app-bittorrent.tesh) ADD_TESH_FACTORIES(msg-dht-kademlia-parallel "thread;ucontext;raw" --cfg contexts/nthreads:4 ${CONTEXTS_SYNCHRO} --setenv bindir=${CMAKE_BINARY_DIR}/examples/msg/dht-kademlia --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/platforms --cd ${CMAKE_HOME_DIRECTORY}/examples/msg/dht-kademlia dht-kademlia.tesh) ADD_TESH_FACTORIES(msg-energy-pstate-ptask "thread;ucontext;raw;boost" --cfg host/model:ptask_L07 --log xbt_cfg.threshold:critical --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/msg --cd ${CMAKE_BINARY_DIR}/examples/msg ${CMAKE_HOME_DIRECTORY}/examples/msg/energy-pstate/energy-pstate.tesh) ADD_TESH_FACTORIES(msg-energy-consumption-ptask "thread;ucontext;raw;boost" --cfg host/model:ptask_L07 --log xbt_cfg.threshold:critical --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/msg --cd ${CMAKE_BINARY_DIR}/examples/msg ${CMAKE_HOME_DIRECTORY}/examples/msg/energy-consumption/energy-consumption.tesh) diff --git a/examples/msg/app-bittorrent/messages.c b/examples/msg/app-bittorrent/messages.c deleted file mode 100644 index d4c7c2b971..0000000000 --- a/examples/msg/app-bittorrent/messages.c +++ /dev/null @@ -1,118 +0,0 @@ -/* Copyright (c) 2012-2014. The SimGrid Team. - * All rights reserved. */ - -/* This program is free software; you can redistribute it and/or modify it - * under the terms of the license (GNU LGPL) which comes with this package. */ - -#include "messages.h" -#include "bittorrent.h" - -XBT_LOG_NEW_DEFAULT_CATEGORY(msg_messages, "Messages specific for the message factory"); - -#define BITS_TO_BYTES(x) ((x / 8) + (x % 8) ? 1 : 0) - -/** @brief Build a new empty message - * @param type type of the message - * @param issuer_host_name hostname of the issuer, for debugging purposes - * @param mailbox mailbox where the peer should answer - * @param peer_id id of the issuer - * @param size message size in bytes - */ -msg_task_t task_message_new(e_message_type type, const char *issuer_host_name, const char *mailbox, int peer_id, - int size) -{ - message_t message = xbt_new(s_message_t, 1); - message->issuer_host_name = issuer_host_name; - message->peer_id = peer_id; - message->mailbox = mailbox; - message->type = type; - msg_task_t task = MSG_task_create(NULL, 0, size, message); - XBT_DEBUG("type: %d size: %d", (int) type, size); - return task; -} - -/** Builds a message containing an index. */ -msg_task_t task_message_index_new(e_message_type type, const char *issuer_host_name, const char *mailbox, int peer_id, - int index, int varsize) -{ - msg_task_t task = task_message_new(type, issuer_host_name, mailbox, peer_id, task_message_size(type) + varsize); - message_t message = MSG_task_get_data(task); - message->index = index; - return task; -} - -msg_task_t task_message_bitfield_new(const char *issuer_host_name, const char *mailbox, int peer_id, - unsigned int bitfield, int bitfield_size) -{ - msg_task_t task = task_message_new(MESSAGE_BITFIELD, issuer_host_name, mailbox, peer_id, - task_message_size(MESSAGE_BITFIELD) + BITS_TO_BYTES(bitfield_size)); - message_t message = MSG_task_get_data(task); - message->bitfield = bitfield; - return task; -} - -msg_task_t task_message_request_new(const char *issuer_host_name, const char *mailbox, int peer_id, int index, - int block_index, int block_length) -{ - msg_task_t task = task_message_index_new(MESSAGE_REQUEST, issuer_host_name, mailbox, peer_id, index, 0); - message_t message = MSG_task_get_data(task); - message->block_index = block_index; - message->block_length = block_length; - return task; -} - -msg_task_t task_message_piece_new(const char *issuer_host_name, const char *mailbox, int peer_id, int index, - int block_index, int block_length, int block_size) -{ - msg_task_t task = task_message_index_new(MESSAGE_PIECE, issuer_host_name, mailbox, peer_id, index, - block_length * block_size); - message_t message = MSG_task_get_data(task); - message->block_index = block_index; - message->block_length = block_length; - return task; -} - -void task_message_free(void *task) -{ - message_t message = MSG_task_get_data(task); - xbt_free(message); - MSG_task_destroy(task); -} - -int task_message_size(e_message_type type) -{ - int size = 0; - switch (type) { - case MESSAGE_HANDSHAKE: - size = MESSAGE_HANDSHAKE_SIZE; - break; - case MESSAGE_CHOKE: - size = MESSAGE_CHOKE_SIZE; - break; - case MESSAGE_UNCHOKE: - size = MESSAGE_UNCHOKE_SIZE; - break; - case MESSAGE_INTERESTED: - case MESSAGE_NOTINTERESTED: - size = MESSAGE_INTERESTED_SIZE; - break; - case MESSAGE_HAVE: - size = MESSAGE_HAVE_SIZE; - break; - case MESSAGE_BITFIELD: - size = MESSAGE_BITFIELD_SIZE; - break; - case MESSAGE_REQUEST: - size = MESSAGE_REQUEST_SIZE; - break; - case MESSAGE_PIECE: - size = MESSAGE_PIECE_SIZE; - break; - case MESSAGE_CANCEL: - size = MESSAGE_CANCEL_SIZE; - break; - default: - THROW_IMPOSSIBLE; - } - return size; -} diff --git a/examples/msg/app-bittorrent/tracker.h b/examples/msg/app-bittorrent/tracker.h deleted file mode 100644 index 40e7d9da65..0000000000 --- a/examples/msg/app-bittorrent/tracker.h +++ /dev/null @@ -1,45 +0,0 @@ -/* Copyright (c) 2012-2014. The SimGrid Team. - * All rights reserved. */ - -/* This program is free software; you can redistribute it and/or modify it - * under the terms of the license (GNU LGPL) which comes with this package. */ - -#ifndef BITTORRENT_TRACKER_H_ -#define BITTORRENT_TRACKER_H_ -#include -#include "bittorrent.h" -/** - * Tracker main function - */ -int tracker(int argc, char *argv[]); -/** - * Task types exchanged between a node and the tracker - */ -typedef enum { - TRACKER_TASK_QUERY, - TRACKER_TASK_ANSWER -} e_tracker_task_type_t; -/** - * Tasks exchanged between a tracker and peers. - */ -typedef struct s_tracker_task_data { - e_tracker_task_type_t type; //type of the task - const char *mailbox; //mailbox where the tracker should answer - const char *issuer_host_name; //hostname, for debug purposes - //Query data - int peer_id; //peer id - int uploaded; //how much the peer has already uploaded - int downloaded; //how much the peer has downloaded - int left; //how much the peer has left - //Answer data - int interval; //how often the peer should contact the tracker (unused for now) - xbt_dynar_t peers; //the peer list the peer has asked for. -} s_tracker_task_data_t; -typedef s_tracker_task_data_t *tracker_task_data_t; - -tracker_task_data_t tracker_task_data_new(const char *issuer_host_name, const char *mailbox, int peer_id, - int uploaded, int downloaded, int left); -void tracker_task_data_free(tracker_task_data_t task); - -int is_in_list(xbt_dynar_t peers, int id); -#endif /* BITTORRENT_TRACKER_H */ diff --git a/examples/msg/platform-properties/platform-properties.c b/examples/msg/platform-properties/platform-properties.c index a7b4d5963e..515849849a 100644 --- a/examples/msg/platform-properties/platform-properties.c +++ b/examples/msg/platform-properties/platform-properties.c @@ -33,7 +33,7 @@ static void test_host(const char*hostname) XBT_INFO(" Property: %s old value: %s", exist, value); XBT_INFO("== Trying to modify a host property"); - MSG_host_set_property_value(thehost, exist, xbt_strdup("250")); + MSG_host_set_property_value(thehost, exist, (char*)"250"); /* Test if we have changed the value */ value = MSG_host_get_property_value(thehost, exist); @@ -42,7 +42,9 @@ static void test_host(const char*hostname) XBT_INFO(" Property: %s old value: %s", exist, value); /* Restore the value for the next test */ - MSG_host_set_property_value(thehost, exist, xbt_strdup("180")); + MSG_host_set_property_value(thehost, exist, (char*)"180"); + + xbt_dict_free(&props); } static int alice(int argc, char *argv[]) { /* Dump what we have on the current host */ @@ -87,6 +89,7 @@ static int bob(int argc, char *argv[]) value = MSG_process_get_property_value(MSG_process_self(), noexist); xbt_assert(!value, "The property is defined (it shouldnt)"); + xbt_dict_free(&props); return 0; } diff --git a/examples/msg/trace-categories/trace-categories.tesh b/examples/msg/trace-categories/trace-categories.tesh index b58419e376..f2836b61a3 100644 --- a/examples/msg/trace-categories/trace-categories.tesh +++ b/examples/msg/trace-categories/trace-categories.tesh @@ -12,23 +12,23 @@ $ $SG_TEST_EXENV ${bindir:=.}/trace-categories$EXEEXT --cfg=tracing:yes --cfg=tr > [0.000000] [xbt_cfg/INFO] Configuration change: Set 'viva/uncategorized' to 'categories.uncat.plist' $ $SG_TEST_EXENV cat categories.cat.plist -> node = ("HOST","LINK"); -> edge = ("0-LINK4-LINK4","0-HOST1-LINK4","0-LINK4-HOST1"); +> node = ("LINK","HOST"); +> edge = ("0-LINK4-LINK4","0-LINK4-HOST1","0-HOST1-LINK4"); > > host = { > type = "square"; > size = "power"; -> values = ("pdata","pcompute","pfinalize","prequest"); +> values = ("pcompute","pdata","pfinalize","prequest"); > }; > link = { > type = "rhombus"; > size = "bandwidth"; -> values = ("bdata","bcompute","bfinalize","brequest"); +> values = ("bcompute","bdata","bfinalize","brequest"); > }; $ $SG_TEST_EXENV cat categories.uncat.plist -> node = ("HOST","LINK"); -> edge = ("0-LINK4-LINK4","0-HOST1-LINK4","0-LINK4-HOST1"); +> node = ("LINK","HOST"); +> edge = ("0-LINK4-LINK4","0-LINK4-HOST1","0-HOST1-LINK4"); > > host = { > type = "square"; diff --git a/examples/msg/trace-host-user-variables/trace-host-user-variables.tesh b/examples/msg/trace-host-user-variables/trace-host-user-variables.tesh index d45d68c70d..431b774ee5 100644 --- a/examples/msg/trace-host-user-variables/trace-host-user-variables.tesh +++ b/examples/msg/trace-host-user-variables/trace-host-user-variables.tesh @@ -9,12 +9,12 @@ $ $SG_TEST_EXENV ${bindir:=.}/trace-host-user-variables$EXEEXT --cfg=tracing:yes > [0.004078] [msg_test/INFO] HDD_utilization > [0.004078] [msg_test/INFO] Declared link variables: > [0.004078] [msg_test/INFO] Node types in the trace: -> [0.004078] [msg_test/INFO] LINK > [0.004078] [msg_test/INFO] HOST +> [0.004078] [msg_test/INFO] LINK > [0.004078] [msg_test/INFO] Node types in the trace: -> [0.004078] [msg_test/INFO] 0-LINK3-LINK3 > [0.004078] [msg_test/INFO] 0-HOST1-LINK3 > [0.004078] [msg_test/INFO] 0-LINK3-HOST1 +> [0.004078] [msg_test/INFO] 0-LINK3-LINK3 $ rm -f viva_graph.plist diff --git a/examples/msg/trace-masterworker/trace-masterworker.tesh b/examples/msg/trace-masterworker/trace-masterworker.tesh index 7690bfe874..d17cce3d64 100644 --- a/examples/msg/trace-masterworker/trace-masterworker.tesh +++ b/examples/msg/trace-masterworker/trace-masterworker.tesh @@ -10,10 +10,10 @@ $ $SG_TEST_EXENV ${bindir:=.}/trace-masterworker$EXEEXT --cfg=tracing:yes --cfg= > [0.000000] [xbt_cfg/INFO] Configuration change: Set 'viva/categorized' to 'trace-masterworker.cat.plist' > [0.000000] [xbt_cfg/INFO] Configuration change: Set 'viva/uncategorized' to 'trace-masterworker.uncat.plist' > [4.214821] [msg_trace_masterworker/INFO] Declared tracing categories: -> [4.214821] [msg_trace_masterworker/INFO] request +> [4.214821] [msg_trace_masterworker/INFO] compute > [4.214821] [msg_trace_masterworker/INFO] finalize > [4.214821] [msg_trace_masterworker/INFO] report -> [4.214821] [msg_trace_masterworker/INFO] compute +> [4.214821] [msg_trace_masterworker/INFO] request > [4.214821] [msg_trace_masterworker/INFO] Declared marks: > [4.214821] [msg_trace_masterworker/INFO] msmark @@ -27,10 +27,10 @@ $ $SG_TEST_EXENV ${bindir:=.}/trace-masterworker$EXEEXT ${srcdir:=.}/config_trac > [0.000000] [xbt_cfg/INFO] Configuration change: Set 'viva/categorized' to 'trace-masterworker.cat.plist' > [0.000000] [xbt_cfg/INFO] Configuration change: Set 'viva/uncategorized' to 'trace-masterworker.uncat.plist' > [4.214821] [msg_trace_masterworker/INFO] Declared tracing categories: -> [4.214821] [msg_trace_masterworker/INFO] request +> [4.214821] [msg_trace_masterworker/INFO] compute > [4.214821] [msg_trace_masterworker/INFO] finalize > [4.214821] [msg_trace_masterworker/INFO] report -> [4.214821] [msg_trace_masterworker/INFO] compute +> [4.214821] [msg_trace_masterworker/INFO] request > [4.214821] [msg_trace_masterworker/INFO] Declared marks: > [4.214821] [msg_trace_masterworker/INFO] msmark @@ -48,17 +48,17 @@ $ $SG_TEST_EXENV ${bindir:=.}/trace-masterworker$EXEEXT --cfg=tracing:yes --cfg= > [0.000000] [xbt_cfg/INFO] Configuration change: Set 'viva/categorized' to 'trace-masterworker.cat.plist' > [0.000000] [xbt_cfg/INFO] Configuration change: Set 'viva/uncategorized' to 'trace-masterworker.uncat.plist' > [4.214821] [msg_trace_masterworker/INFO] Declared tracing categories: -> [4.214821] [msg_trace_masterworker/INFO] request +> [4.214821] [msg_trace_masterworker/INFO] compute > [4.214821] [msg_trace_masterworker/INFO] finalize > [4.214821] [msg_trace_masterworker/INFO] report -> [4.214821] [msg_trace_masterworker/INFO] compute +> [4.214821] [msg_trace_masterworker/INFO] request > [4.214821] [msg_trace_masterworker/INFO] Declared marks: > [4.214821] [msg_trace_masterworker/INFO] msmark $ $SG_TEST_EXENV cat trace-masterworker.uncat.plist -> node = ("HOST","LINK"); +> node = ("LINK","HOST"); > edge = ("0-LINK13-LINK13","0-LINK13-HOST1","0-HOST1-LINK13"); -> +> > host = { > type = "square"; > size = "power"; @@ -71,18 +71,18 @@ $ $SG_TEST_EXENV cat trace-masterworker.uncat.plist > }; $ $SG_TEST_EXENV cat trace-masterworker.cat.plist -> node = ("HOST","LINK"); +> node = ("LINK","HOST"); > edge = ("0-LINK13-LINK13","0-LINK13-HOST1","0-HOST1-LINK13"); -> +> > host = { > type = "square"; > size = "power"; -> values = ("pcompute","preport","pfinalize","prequest"); +> values = ("pcompute","pfinalize","preport","prequest"); > }; > link = { > type = "rhombus"; > size = "bandwidth"; -> values = ("bcompute","breport","bfinalize","brequest"); +> values = ("bcompute","bfinalize","breport","brequest"); > }; $ rm -rf trace-masterworker.trace trace-masterworker.cat.plist trace-masterworker.uncat.plist diff --git a/examples/s4u/CMakeLists.txt b/examples/s4u/CMakeLists.txt index 72a43da51f..6064c0caa5 100644 --- a/examples/s4u/CMakeLists.txt +++ b/examples/s4u/CMakeLists.txt @@ -17,12 +17,23 @@ foreach (file s4u_dht-chord node) endforeach() set(examples_src ${examples_src} ${CMAKE_CURRENT_SOURCE_DIR}/dht-chord/s4u_dht-chord.hpp) +add_executable (s4u_bittorrent app-bittorrent/s4u_bittorrent.cpp app-bittorrent/s4u_peer.cpp + app-bittorrent/s4u_tracker.cpp) +target_link_libraries(s4u_bittorrent simgrid) +set_target_properties(s4u_bittorrent PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/app-bittorrent) +foreach (file s4u_bittorrent s4u_peer s4u_tracker) + set(examples_src ${examples_src} ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/${file}.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/${file}.hpp) +endforeach() + set(examples_src ${examples_src} PARENT_SCOPE) -set(tesh_files ${tesh_files} ${CMAKE_CURRENT_SOURCE_DIR}/dht-chord/s4u_dht-chord.tesh PARENT_SCOPE) +set(tesh_files ${tesh_files} ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/s4u_app-bittorrent.tesh + ${CMAKE_CURRENT_SOURCE_DIR}/dht-chord/s4u_dht-chord.tesh PARENT_SCOPE) set(xml_files ${xml_files} ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/s4u_actions-comm_split_d.xml ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/s4u_actions-comm_d.xml ${CMAKE_CURRENT_SOURCE_DIR}/actions-storage/s4u_actions-storage_d.xml ${CMAKE_CURRENT_SOURCE_DIR}/actor-create/s4u_actor-create_d.xml + ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/s4u_app-bittorrent_d.xml ${CMAKE_CURRENT_SOURCE_DIR}/app-masterworker/s4u_app-masterworker_d.xml ${CMAKE_CURRENT_SOURCE_DIR}/dht-chord/s4u_dht-chord_d.xml PARENT_SCOPE) set(txt_files ${txt_files} ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/s4u_actions-comm_split_p0.txt @@ -32,6 +43,6 @@ set(txt_files ${txt_files} ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/s4u_a ${CMAKE_CURRENT_SOURCE_DIR}/README.doc PARENT_SCOPE) foreach(example actions-comm actions-storage actor-create actor-daemon actor-kill actor-migration actor-suspend - app-masterworker app-pingpong app-token-ring dht-chord plugin-hostload io mutex ) + app-bittorrent app-masterworker app-pingpong app-token-ring dht-chord plugin-hostload io mutex ) ADD_TESH_FACTORIES(s4u-${example} "thread;ucontext;raw;boost" --setenv bindir=${CMAKE_CURRENT_BINARY_DIR}/${example} --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/platforms --cd ${CMAKE_HOME_DIRECTORY}/examples/s4u/${example} s4u_${example}.tesh) endforeach() diff --git a/examples/s4u/actions-storage/s4u_actions-storage.cpp b/examples/s4u/actions-storage/s4u_actions-storage.cpp index 85e71e13f6..85ffb9d975 100644 --- a/examples/s4u/actions-storage/s4u_actions-storage.cpp +++ b/examples/s4u/actions-storage/s4u_actions-storage.cpp @@ -29,7 +29,7 @@ static void log_action(const char* const* action, double date) } } -static simgrid::s4u::File* get_file_descriptor(const char* file_name) +static simgrid::s4u::File* get_file_descriptor(std::string file_name) { std::string full_name = simgrid::s4u::this_actor::getName() + ":" + file_name; @@ -60,11 +60,11 @@ public: /* My actions */ static void open(const char* const* action) { - const char* file_name = action[2]; + std::string file_name = action[2]; double clock = simgrid::s4u::Engine::getClock(); std::string full_name = simgrid::s4u::this_actor::getName() + ":" + file_name; - ACT_DEBUG("Entering Open: %s (filename: %s)", NAME, file_name); + ACT_DEBUG("Entering Open: %s (filename: %s)", NAME, file_name.c_str()); simgrid::s4u::File* file = new simgrid::s4u::File(file_name, NULL); opened_files.insert({full_name, file}); @@ -74,7 +74,7 @@ public: static void read(const char* const* action) { - const char* file_name = action[2]; + std::string file_name = action[2]; sg_size_t size = std::stoul(action[3]); double clock = simgrid::s4u::Engine::getClock(); @@ -88,12 +88,12 @@ public: static void close(const char* const* action) { - const char* file_name = action[2]; + std::string file_name = action[2]; double clock = simgrid::s4u::Engine::getClock(); simgrid::s4u::File* file = get_file_descriptor(file_name); - ACT_DEBUG("Entering Close: %s (filename: %s)", NAME, file_name); + ACT_DEBUG("Entering Close: %s (filename: %s)", NAME, file_name.c_str()); delete file; log_action(action, simgrid::s4u::Engine::getClock() - clock); diff --git a/examples/s4u/app-bittorrent/s4u_app-bittorrent.tesh b/examples/s4u/app-bittorrent/s4u_app-bittorrent.tesh new file mode 100644 index 0000000000..d309e01b18 --- /dev/null +++ b/examples/s4u/app-bittorrent/s4u_app-bittorrent.tesh @@ -0,0 +1,23 @@ +#! ./tesh + +p Testing the Bittorrent implementation with MSG + +! timeout 10 +! output sort 19 +$ $SG_TEST_EXENV ${bindir:=.}/s4u_bittorrent ${srcdir:=.}/cluster.xml ${srcdir:=.}/../s4u/app-bittorrent/s4u_app-bittorrent_d.xml "--log=root.fmt:[%12.6r]%e(%i:%P@%h)%e%m%n" +> [ 0.000000] (1:tracker@node-0.acme.org) Tracker launched. +> [ 0.000000] (2:peer@node-1.acme.org) Hi, I'm joining the network with id 2 +> [ 0.000000] (3:peer@node-2.acme.org) Hi, I'm joining the network with id 3 +> [ 0.000000] (4:peer@node-3.acme.org) Hi, I'm joining the network with id 4 +> [ 0.000000] (5:peer@node-4.acme.org) Hi, I'm joining the network with id 5 +> [ 0.000000] (6:peer@node-5.acme.org) Hi, I'm joining the network with id 6 +> [ 0.000000] (7:peer@node-6.acme.org) Hi, I'm joining the network with id 7 +> [ 0.000000] (8:peer@node-7.acme.org) Hi, I'm joining the network with id 8 +> [ 3000.000000] (1:tracker@node-0.acme.org) Tracker is leaving +> [ 5000.007806] (2:peer@node-1.acme.org) Here is my current status: 1111111111 +> [ 5000.007806] (3:peer@node-2.acme.org) Here is my current status: 1111111111 +> [ 5000.007806] (4:peer@node-3.acme.org) Here is my current status: 1111111111 +> [ 5000.007806] (5:peer@node-4.acme.org) Here is my current status: 1111111111 +> [ 5000.007806] (6:peer@node-5.acme.org) Here is my current status: 1111111111 +> [ 5000.007806] (7:peer@node-6.acme.org) Here is my current status: 1111111111 +> [ 5000.007806] (8:peer@node-7.acme.org) Here is my current status: 1111111111 diff --git a/examples/msg/app-bittorrent/app-bittorrent_d.xml b/examples/s4u/app-bittorrent/s4u_app-bittorrent_d.xml similarity index 100% rename from examples/msg/app-bittorrent/app-bittorrent_d.xml rename to examples/s4u/app-bittorrent/s4u_app-bittorrent_d.xml diff --git a/examples/s4u/app-bittorrent/s4u_bittorrent.cpp b/examples/s4u/app-bittorrent/s4u_bittorrent.cpp new file mode 100644 index 0000000000..cbc093b293 --- /dev/null +++ b/examples/s4u/app-bittorrent/s4u_bittorrent.cpp @@ -0,0 +1,37 @@ +/* Copyright (c) 2012-2017. The SimGrid Team. + * All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#include "s4u_bittorrent.hpp" +#include "s4u_peer.hpp" +#include "s4u_tracker.hpp" + +simgrid::xbt::Extension HostBittorrent::EXTENSION_ID; + +int main(int argc, char* argv[]) +{ + simgrid::s4u::Engine* e = new simgrid::s4u::Engine(&argc, argv); + + /* Check the arguments */ + xbt_assert(argc > 2, "Usage: %s platform_file deployment_file", argv[0]); + + e->loadPlatform(argv[1]); + + HostBittorrent::EXTENSION_ID = simgrid::s4u::Host::extension_create(); + + std::vector list; + simgrid::s4u::Engine::getInstance()->getHostList(&list); + for (auto host : list) + host->extension_set(new HostBittorrent(host)); + + e->registerFunction("tracker"); + e->registerFunction("peer"); + e->loadDeployment(argv[2]); + + e->run(); + + delete e; + return 0; +} diff --git a/examples/s4u/app-bittorrent/s4u_bittorrent.hpp b/examples/s4u/app-bittorrent/s4u_bittorrent.hpp new file mode 100644 index 0000000000..a3b296a2af --- /dev/null +++ b/examples/s4u/app-bittorrent/s4u_bittorrent.hpp @@ -0,0 +1,104 @@ +/* Copyright (c) 2012-2014, 2016-2017. The SimGrid Team. + * All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#ifndef BITTORRENT_BITTORRENT_HPP_ +#define BITTORRENT_BITTORRENT_HPP_ + +#include +#include + +#define MAILBOX_SIZE 40 +#define TRACKER_MAILBOX "tracker_mailbox" +/** Max number of peers sent by the tracker to clients */ +#define MAXIMUM_PEERS 50 +/** Interval of time where the peer should send a request to the tracker */ +#define TRACKER_QUERY_INTERVAL 1000 +/** Communication size for a task to the tracker */ +#define TRACKER_COMM_SIZE 1 +#define GET_PEERS_TIMEOUT 10000 +#define TIMEOUT_MESSAGE 10 +#define TRACKER_RECEIVE_TIMEOUT 10 +/** Number of peers that can be unchocked at a given time */ +#define MAX_UNCHOKED_PEERS 4 +/** Interval between each update of the choked peers */ +#define UPDATE_CHOKED_INTERVAL 30 +/** Number of pieces the peer asks for simultaneously */ +#define MAX_PIECES 1 + +/** Message sizes + * Sizes based on report by A. Legout et al, Understanding BitTorrent: An Experimental Perspective + * http://hal.inria.fr/inria-00000156/en + */ +#define MESSAGE_HANDSHAKE_SIZE 68 +#define MESSAGE_CHOKE_SIZE 5 +#define MESSAGE_UNCHOKE_SIZE 5 +#define MESSAGE_INTERESTED_SIZE 5 +#define MESSAGE_NOTINTERESTED_SIZE 5 +#define MESSAGE_HAVE_SIZE 9 +#define MESSAGE_BITFIELD_SIZE 5 +#define MESSAGE_REQUEST_SIZE 17 +#define MESSAGE_PIECE_SIZE 13 +#define MESSAGE_CANCEL_SIZE 17 + +/** Types of messages exchanged between two peers. */ +typedef enum { + MESSAGE_HANDSHAKE, + MESSAGE_CHOKE, + MESSAGE_UNCHOKE, + MESSAGE_INTERESTED, + MESSAGE_NOTINTERESTED, + MESSAGE_HAVE, + MESSAGE_BITFIELD, + MESSAGE_REQUEST, + MESSAGE_PIECE, + MESSAGE_CANCEL +} e_message_type; + +class Message { +public: + e_message_type type; + int peer_id; + simgrid::s4u::MailboxPtr return_mailbox; + unsigned int bitfield = 0U; + int piece = 0; + int block_index = 0; + int block_length = 0; + Message(e_message_type type, int peer_id, simgrid::s4u::MailboxPtr return_mailbox) + : type(type), peer_id(peer_id), return_mailbox(return_mailbox){}; + Message(e_message_type type, int peer_id, unsigned int bitfield, simgrid::s4u::MailboxPtr return_mailbox) + : type(type), peer_id(peer_id), return_mailbox(return_mailbox), bitfield(bitfield){}; + Message(e_message_type type, int peer_id, simgrid::s4u::MailboxPtr return_mailbox, int piece, int block_index, + int block_length) + : type(type) + , peer_id(peer_id) + , return_mailbox(return_mailbox) + , piece(piece) + , block_index(block_index) + , block_length(block_length){}; + Message(e_message_type type, int peer_id, simgrid::s4u::MailboxPtr return_mailbox, int piece) + : type(type), peer_id(peer_id), return_mailbox(return_mailbox), piece(piece){}; + ~Message() = default; +}; + +class HostBittorrent { + RngStream stream_; + simgrid::s4u::Host* host = nullptr; + +public: + static simgrid::xbt::Extension EXTENSION_ID; + + explicit HostBittorrent(simgrid::s4u::Host* ptr) : host(ptr) + { + std::string descr = std::string("RngSream<") + host->getCname() + ">"; + stream_ = RngStream_CreateStream(descr.c_str()); + } + + ~HostBittorrent() { RngStream_DeleteStream(&stream_); }; + + RngStream getStream() { return stream_; }; +}; + +#endif /* BITTORRENT_BITTORRENT_HPP_ */ diff --git a/examples/s4u/app-bittorrent/s4u_peer.cpp b/examples/s4u/app-bittorrent/s4u_peer.cpp new file mode 100644 index 0000000000..0b03bdbac8 --- /dev/null +++ b/examples/s4u/app-bittorrent/s4u_peer.cpp @@ -0,0 +1,671 @@ +/* Copyright (c) 2012-2017. The SimGrid Team. All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#include +#include + +#include "s4u_peer.hpp" +#include "s4u_tracker.hpp" + +XBT_LOG_NEW_DEFAULT_CATEGORY(s4u_bt_peer, "Messages specific for the peers"); + +/* + * User parameters for transferred file data. For the test, the default values are : + * File size: 10 pieces * 5 blocks/piece * 16384 bytes/block = 819200 bytes + */ +#define FILE_PIECES 10UL +#define PIECES_BLOCKS 5UL +#define BLOCK_SIZE 16384 + +/** Number of blocks asked by each request */ +#define BLOCKS_REQUESTED 2 + +#define ENABLE_END_GAME_MODE 1 +#define SLEEP_DURATION 1 +#define BITS_TO_BYTES(x) (((x) / 8 + (x) % 8) ? 1 : 0) + +Peer::Peer(std::vector args) +{ + // Check arguments + xbt_assert(args.size() == 3 || args.size() == 4, "Wrong number of arguments"); + try { + id = std::stoi(args[1]); + mailbox_ = simgrid::s4u::Mailbox::byName(std::to_string(id)); + } catch (std::invalid_argument& ia) { + throw std::invalid_argument(std::string("Invalid ID:") + args[1].c_str()); + } + + try { + deadline = std::stod(args[2]); + } catch (std::invalid_argument& ia) { + throw std::invalid_argument(std::string("Invalid deadline:") + args[2].c_str()); + } + xbt_assert(deadline > 0, "Wrong deadline supplied"); + + stream = simgrid::s4u::this_actor::getHost()->extension()->getStream(); + + if (args.size() == 4 && args[3] == "1") { + bitfield_ = (1U << FILE_PIECES) - 1U; + bitfield_blocks = (1ULL << (FILE_PIECES * PIECES_BLOCKS)) - 1ULL; + } + pieces_count = new short[FILE_PIECES]{0}; + + XBT_INFO("Hi, I'm joining the network with id %d", id); +} + +Peer::~Peer() +{ + for (auto peer : connected_peers) + delete peer.second; + delete[] pieces_count; +} + +/** Peer main function */ +void Peer::operator()() +{ + // Getting peer data from the tracker. + if (getPeersFromTracker()) { + XBT_DEBUG("Got %zu peers from the tracker. Current status is: %s", connected_peers.size(), getStatus().c_str()); + begin_receive_time = simgrid::s4u::Engine::getClock(); + mailbox_->setReceiver(simgrid::s4u::Actor::self()); + if (hasFinished()) { + sendHandshakeToAllPeers(); + } else { + leech(); + } + seed(); + } else { + XBT_INFO("Couldn't contact the tracker."); + } + + XBT_INFO("Here is my current status: %s", getStatus().c_str()); +} + +bool Peer::getPeersFromTracker() +{ + simgrid::s4u::MailboxPtr tracker_mailbox = simgrid::s4u::Mailbox::byName(TRACKER_MAILBOX); + // Build the task to send to the tracker + TrackerQuery* peer_request = new TrackerQuery(id, mailbox_); + try { + XBT_DEBUG("Sending a peer request to the tracker."); + tracker_mailbox->put(peer_request, TRACKER_COMM_SIZE, GET_PEERS_TIMEOUT); + } catch (xbt_ex& e) { + if (e.category == timeout_error) { + XBT_DEBUG("Timeout expired when requesting peers to tracker"); + delete peer_request; + return false; + } + } + + try { + TrackerAnswer* answer = static_cast(mailbox_->get(GET_PEERS_TIMEOUT)); + // Add the peers the tracker gave us to our peer list. + for (auto peer_id : *answer->getPeers()) + if (id != peer_id) + connected_peers[peer_id] = new Connection(peer_id); + delete answer; + } catch (xbt_ex& e) { + if (e.category == timeout_error) { + XBT_DEBUG("Timeout expired when requesting peers to tracker"); + return false; + } + } + return true; +} + +void Peer::sendHandshakeToAllPeers() +{ + for (auto kv : connected_peers) { + Connection* remote_peer = kv.second; + Message* handshake = new Message(MESSAGE_HANDSHAKE, id, mailbox_); + remote_peer->mailbox_->put_init(handshake, MESSAGE_HANDSHAKE_SIZE)->detach(); + XBT_DEBUG("Sending a HANDSHAKE to %d", remote_peer->id); + } +} + +void Peer::sendMessage(simgrid::s4u::MailboxPtr mailbox, e_message_type type, uint64_t size) +{ + const char* type_names[6] = {"HANDSHAKE", "CHOKE", "UNCHOKE", "INTERESTED", "NOTINTERESTED", "CANCEL"}; + XBT_DEBUG("Sending %s to %s", type_names[type], mailbox->getName()); + mailbox->put_init(new Message(type, id, bitfield_, mailbox_), size)->detach(); +} + +void Peer::sendBitfield(simgrid::s4u::MailboxPtr mailbox) +{ + XBT_DEBUG("Sending a BITFIELD to %s", mailbox->getName()); + mailbox + ->put_init(new Message(MESSAGE_BITFIELD, id, bitfield_, mailbox_), + MESSAGE_BITFIELD_SIZE + BITS_TO_BYTES(FILE_PIECES)) + ->detach(); +} + +void Peer::sendPiece(simgrid::s4u::MailboxPtr mailbox, unsigned int piece, int block_index, int block_length) +{ + xbt_assert(not hasNotPiece(piece), "Tried to send a unavailable piece."); + XBT_DEBUG("Sending the PIECE %u (%d,%d) to %s", piece, block_index, block_length, mailbox->getName()); + mailbox->put_init(new Message(MESSAGE_PIECE, id, mailbox_, piece, block_index, block_length), BLOCK_SIZE)->detach(); +} + +void Peer::sendHaveToAllPeers(unsigned int piece) +{ + XBT_DEBUG("Sending HAVE message to all my peers"); + for (auto kv : connected_peers) { + Connection* remote_peer = kv.second; + remote_peer->mailbox_->put_init(new Message(MESSAGE_HAVE, id, mailbox_, piece), MESSAGE_HAVE_SIZE)->detach(); + } +} + +void Peer::sendRequestTo(Connection* remote_peer, unsigned int piece) +{ + remote_peer->current_piece = piece; + xbt_assert(remote_peer->hasPiece(piece)); + int block_index = getFirstMissingBlockFrom(piece); + if (block_index != -1) { + int block_length = MIN(BLOCKS_REQUESTED, PIECES_BLOCKS - block_index); + XBT_DEBUG("Sending a REQUEST to %s for piece %u (%d,%d)", remote_peer->mailbox_->getName(), piece, block_index, + block_length); + remote_peer->mailbox_ + ->put_init(new Message(MESSAGE_REQUEST, id, mailbox_, piece, block_index, block_length), MESSAGE_REQUEST_SIZE) + ->detach(); + } +} + +std::string Peer::getStatus() +{ + std::string res = std::string(""); + for (int i = FILE_PIECES - 1; i >= 0; i--) + res = std::string((bitfield_ & (1U << i)) ? "1" : "0") + res; + return res; +} + +bool Peer::hasFinished() +{ + return bitfield_ == (1U << FILE_PIECES) - 1U; +} + +/** Indicates if the remote peer has a piece not stored by the local peer */ +bool Peer::isInterestedBy(Connection* remote_peer) +{ + return remote_peer->bitfield & (bitfield_ ^ ((1 << FILE_PIECES) - 1)); +} + +bool Peer::isInterestedByFree(Connection* remote_peer) +{ + for (unsigned int i = 0; i < FILE_PIECES; i++) + if (hasNotPiece(i) && remote_peer->hasPiece(i) && isNotDownloadingPiece(i)) + return true; + return false; +} + +void Peer::updatePiecesCountFromBitfield(unsigned int bitfield) +{ + for (unsigned int i = 0; i < FILE_PIECES; i++) + if (bitfield & (1U << i)) + pieces_count[i]++; +} + +unsigned int Peer::countPieces(unsigned int bitfield) +{ + unsigned int count = 0U; + unsigned int n = bitfield; + while (n) { + count += n & 1U; + n >>= 1U; + } + return count; +} + +int Peer::nbInterestedPeers() +{ + int nb = 0; + for (auto kv : connected_peers) + if (kv.second->interested) + nb++; + return nb; +} + +void Peer::leech() +{ + double next_choked_update = simgrid::s4u::Engine::getClock() + UPDATE_CHOKED_INTERVAL; + XBT_DEBUG("Start downloading."); + + /* Send a "handshake" message to all the peers it got (since it couldn't have gotten more than 50 peers) */ + sendHandshakeToAllPeers(); + XBT_DEBUG("Starting main leech loop listening on mailbox: %s", mailbox_->getName()); + + void* data = nullptr; + while (simgrid::s4u::Engine::getClock() < deadline && countPieces(bitfield_) < FILE_PIECES) { + if (comm_received == nullptr) { + comm_received = mailbox_->get_async(&data); + } + if (comm_received->test()) { + message = static_cast(data); + handleMessage(); + delete message; + comm_received = nullptr; + } else { + // We don't execute the choke algorithm if we don't already have a piece + if (simgrid::s4u::Engine::getClock() >= next_choked_update && countPieces(bitfield_) > 0) { + updateChokedPeers(); + next_choked_update += UPDATE_CHOKED_INTERVAL; + } else { + simgrid::s4u::this_actor::sleep_for(SLEEP_DURATION); + } + } + } + if (hasFinished()) + XBT_DEBUG("%d becomes a seeder", id); +} + +void Peer::seed() +{ + double next_choked_update = simgrid::s4u::Engine::getClock() + UPDATE_CHOKED_INTERVAL; + XBT_DEBUG("Start seeding."); + // start the main seed loop + void* data = nullptr; + while (simgrid::s4u::Engine::getClock() < deadline) { + if (comm_received == nullptr) { + comm_received = mailbox_->get_async(&data); + } + if (comm_received->test()) { + message = static_cast(data); + handleMessage(); + delete message; + comm_received = nullptr; + } else { + if (simgrid::s4u::Engine::getClock() >= next_choked_update) { + updateChokedPeers(); + // TODO: Change the choked peer algorithm when seeding. + next_choked_update += UPDATE_CHOKED_INTERVAL; + } else { + simgrid::s4u::this_actor::sleep_for(SLEEP_DURATION); + } + } + } +} + +void Peer::updateActivePeersSet(Connection* remote_peer) +{ + if (remote_peer->interested && not remote_peer->choked_upload) + active_peers.insert(remote_peer); + else + active_peers.erase(remote_peer); +} + +void Peer::handleMessage() +{ + const char* type_names[10] = {"HANDSHAKE", "CHOKE", "UNCHOKE", "INTERESTED", "NOTINTERESTED", + "HAVE", "BITFIELD", "REQUEST", "PIECE", "CANCEL"}; + + XBT_DEBUG("Received a %s message from %s", type_names[message->type], message->return_mailbox->getName()); + + auto known_peer = connected_peers.find(message->peer_id); + Connection* remote_peer = (known_peer == connected_peers.end()) ? nullptr : known_peer->second; + xbt_assert(remote_peer != nullptr || message->type == MESSAGE_HANDSHAKE, + "The impossible did happened: A not-in-our-list peer sent us a message."); + + switch (message->type) { + case MESSAGE_HANDSHAKE: + // Check if the peer is in our connection list. + if (remote_peer == nullptr) { + XBT_DEBUG("This peer %d was unknown, answer to its handshake", message->peer_id); + connected_peers[message->peer_id] = new Connection(message->peer_id); + sendMessage(message->return_mailbox, MESSAGE_HANDSHAKE, MESSAGE_HANDSHAKE_SIZE); + } + // Send our bitfield to the peer + sendBitfield(message->return_mailbox); + break; + case MESSAGE_BITFIELD: + // Update the pieces list + updatePiecesCountFromBitfield(message->bitfield); + // Store the bitfield + remote_peer->bitfield = message->bitfield; + xbt_assert(not remote_peer->am_interested, "Should not be interested at first"); + if (isInterestedBy(remote_peer)) { + remote_peer->am_interested = true; + sendMessage(message->return_mailbox, MESSAGE_INTERESTED, MESSAGE_INTERESTED_SIZE); + } + break; + case MESSAGE_INTERESTED: + // Update the interested state of the peer. + remote_peer->interested = true; + updateActivePeersSet(remote_peer); + break; + case MESSAGE_NOTINTERESTED: + remote_peer->interested = false; + updateActivePeersSet(remote_peer); + break; + case MESSAGE_UNCHOKE: + xbt_assert(remote_peer->choked_download); + remote_peer->choked_download = false; + // Send requests to the peer, since it has unchoked us + if (remote_peer->am_interested) + requestNewPieceTo(remote_peer); + break; + case MESSAGE_CHOKE: + xbt_assert(not remote_peer->choked_download); + remote_peer->choked_download = true; + if (remote_peer->current_piece != -1) + removeCurrentPiece(remote_peer, remote_peer->current_piece); + break; + case MESSAGE_HAVE: + XBT_DEBUG("\t for piece %d", message->piece); + xbt_assert((message->piece >= 0 && static_cast(message->piece) < FILE_PIECES), + "Wrong HAVE message received"); + remote_peer->bitfield = remote_peer->bitfield | (1U << static_cast(message->piece)); + pieces_count[message->piece]++; + // If the piece is in our pieces, we tell the peer that we are interested. + if (not remote_peer->am_interested && hasNotPiece(message->piece)) { + remote_peer->am_interested = true; + sendMessage(message->return_mailbox, MESSAGE_INTERESTED, MESSAGE_INTERESTED_SIZE); + if (not remote_peer->choked_download) + requestNewPieceTo(remote_peer); + } + break; + case MESSAGE_REQUEST: + xbt_assert(remote_peer->interested); + xbt_assert((message->piece >= 0 && static_cast(message->piece) < FILE_PIECES), + "Wrong HAVE message received"); + if (not remote_peer->choked_upload) { + XBT_DEBUG("\t for piece %d (%d,%d)", message->piece, message->block_index, + message->block_index + message->block_length); + if (not hasNotPiece(message->piece)) { + sendPiece(message->return_mailbox, message->piece, message->block_index, message->block_length); + } + } else { + XBT_DEBUG("\t for piece %d but he is choked.", message->peer_id); + } + break; + case MESSAGE_PIECE: + XBT_DEBUG(" \t for piece %d (%d,%d)", message->piece, message->block_index, + message->block_index + message->block_length); + xbt_assert(not remote_peer->choked_download); + xbt_assert(remote_peer->am_interested || ENABLE_END_GAME_MODE, + "Can't received a piece if I'm not interested without end-game mode!" + "piece (%d) bitfield (%u) remote bitfield (%u)", + message->piece, bitfield_, remote_peer->bitfield); + xbt_assert(not remote_peer->choked_download, "Can't received a piece if I'm choked !"); + xbt_assert((message->piece >= 0 && static_cast(message->piece) < FILE_PIECES), + "Wrong piece received"); + // TODO: Execute a computation. + if (hasNotPiece(static_cast(message->piece))) { + updateBitfieldBlocks(message->piece, message->block_index, message->block_length); + if (hasCompletedPiece(static_cast(message->piece))) { + // Removing the piece from our piece list + removeCurrentPiece(remote_peer, message->piece); + // Setting the fact that we have the piece + bitfield_ = bitfield_ | (1U << static_cast(message->piece)); + XBT_DEBUG("My status is now %s", getStatus().c_str()); + // Sending the information to all the peers we are connected to + sendHaveToAllPeers(message->piece); + // sending UNINTERESTED to peers that do not have what we want. + updateInterestedAfterReceive(); + } else { // piece not completed + sendRequestTo(remote_peer, message->piece); // ask for the next block + } + } else { + XBT_DEBUG("However, we already have it"); + xbt_assert(ENABLE_END_GAME_MODE, "Should not happen because we don't use end game mode !"); + requestNewPieceTo(remote_peer); + } + break; + case MESSAGE_CANCEL: + break; + default: + THROW_IMPOSSIBLE; + } + // Update the peer speed. + if (remote_peer) { + remote_peer->addSpeedValue(1.0 / (simgrid::s4u::Engine::getClock() - begin_receive_time)); + } + begin_receive_time = simgrid::s4u::Engine::getClock(); +} + +/** Selects the appropriate piece to download and requests it to the remote_peer */ +void Peer::requestNewPieceTo(Connection* remote_peer) +{ + int piece = selectPieceToDownload(remote_peer); + if (piece != -1) { + current_pieces |= (1U << (unsigned int)piece); + sendRequestTo(remote_peer, piece); + } +} + +void Peer::removeCurrentPiece(Connection* remote_peer, unsigned int current_piece) +{ + current_pieces &= ~(1U << current_piece); + remote_peer->current_piece = -1; +} + +/** @brief Return the piece to be downloaded + * There are two cases (as described in "Bittorrent Architecture Protocol", Ryan Toole : + * If a piece is partially downloaded, this piece will be selected prioritarily + * If the peer has strictly less than 4 pieces, he chooses a piece at random. + * If the peer has more than pieces, he downloads the pieces that are the less replicated (rarest policy). + * If all pieces have been downloaded or requested, we select a random requested piece (endgame mode). + * @param remote_peer: information about the connection + * @return the piece to download if possible. -1 otherwise + */ +int Peer::selectPieceToDownload(Connection* remote_peer) +{ + int piece = partiallyDownloadedPiece(remote_peer); + // strict priority policy + if (piece != -1) + return piece; + + // end game mode + if (countPieces(current_pieces) >= (FILE_PIECES - countPieces(bitfield_)) && isInterestedBy(remote_peer)) { +#if ENABLE_END_GAME_MODE == 0 + return -1; +#endif + int nb_interesting_pieces = 0; + // compute the number of interesting pieces + for (unsigned int i = 0; i < FILE_PIECES; i++) + if (hasNotPiece(i) && remote_peer->hasPiece(i)) + nb_interesting_pieces++; + + xbt_assert(nb_interesting_pieces != 0); + // get a random interesting piece + int random_piece_index = RngStream_RandInt(stream, 0, nb_interesting_pieces - 1); + int current_index = 0; + for (unsigned int i = 0; i < FILE_PIECES; i++) { + if (hasNotPiece(i) && remote_peer->hasPiece(i)) { + if (random_piece_index == current_index) { + piece = i; + break; + } + current_index++; + } + } + xbt_assert(piece != -1); + return piece; + } + // Random first policy + if (countPieces(bitfield_) < 4 && isInterestedByFree(remote_peer)) { + int nb_interesting_pieces = 0; + // compute the number of interesting pieces + for (unsigned int i = 0; i < FILE_PIECES; i++) + if (hasNotPiece(i) && remote_peer->hasPiece(i) && isNotDownloadingPiece(i)) + nb_interesting_pieces++; + xbt_assert(nb_interesting_pieces != 0); + // get a random interesting piece + int random_piece_index = RngStream_RandInt(stream, 0, nb_interesting_pieces - 1); + int current_index = 0; + for (unsigned int i = 0; i < FILE_PIECES; i++) { + if (hasNotPiece(i) && remote_peer->hasPiece(i) && isNotDownloadingPiece(i)) { + if (random_piece_index == current_index) { + piece = i; + break; + } + current_index++; + } + } + xbt_assert(piece != -1); + return piece; + } else { // Rarest first policy + short min = SHRT_MAX; + int nb_min_pieces = 0; + int current_index = 0; + // compute the smallest number of copies of available pieces + for (unsigned int i = 0; i < FILE_PIECES; i++) { + if (pieces_count[i] < min && hasNotPiece(i) && remote_peer->hasPiece(i) && isNotDownloadingPiece(i)) + min = pieces_count[i]; + } + + xbt_assert(min != SHRT_MAX || not isInterestedByFree(remote_peer)); + // compute the number of rarest pieces + for (unsigned int i = 0; i < FILE_PIECES; i++) + if (pieces_count[i] == min && hasNotPiece(i) && remote_peer->hasPiece(i) && isNotDownloadingPiece(i)) + nb_min_pieces++; + + xbt_assert(nb_min_pieces != 0 || not isInterestedByFree(remote_peer)); + // get a random rarest piece + int random_rarest_index = RngStream_RandInt(stream, 0, nb_min_pieces - 1); + for (unsigned int i = 0; i < FILE_PIECES; i++) + if (pieces_count[i] == min && hasNotPiece(i) && remote_peer->hasPiece(i) && isNotDownloadingPiece(i)) { + if (random_rarest_index == current_index) { + piece = i; + break; + } + current_index++; + } + + xbt_assert(piece != -1 || not isInterestedByFree(remote_peer)); + return piece; + } +} + +void Peer::updateChokedPeers() +{ + if (nbInterestedPeers() == 0) + return; + XBT_DEBUG("(%d) update_choked peers %zu active peers", id, active_peers.size()); + // update the current round + round_ = (round_ + 1) % 3; + Connection* chosen_peer = nullptr; + // select first active peer and remove it from the set + Connection* choked_peer = *(active_peers.begin()); + active_peers.erase(choked_peer); + + /**If we are currently seeding, we unchoke the peer which has been unchoked the last time.*/ + if (hasFinished()) { + Connection* remote_peer; + double unchoke_time = simgrid::s4u::Engine::getClock() + 1; + for (auto kv : connected_peers) { + remote_peer = kv.second; + if (remote_peer->last_unchoke < unchoke_time && remote_peer->interested && remote_peer->choked_upload) { + unchoke_time = remote_peer->last_unchoke; + chosen_peer = remote_peer; + } + } + } else { + // Random optimistic unchoking + if (round_ == 0) { + int j = 0; + do { + // We choose a random peer to unchoke. + std::unordered_map::iterator chosen_peer_it = connected_peers.begin(); + std::advance(chosen_peer_it, RngStream_RandInt(stream, 0, connected_peers.size() - 1)); + chosen_peer = chosen_peer_it->second; + if (chosen_peer == nullptr) + THROWF(unknown_error, 0, "A peer should have be selected at this point"); + else if (not chosen_peer->interested || not chosen_peer->choked_upload) + chosen_peer = nullptr; + else + XBT_DEBUG("Nothing to do, keep going"); + j++; + } while (chosen_peer == nullptr && j < MAXIMUM_PEERS); + } else { + // Use the "fastest download" policy. + double fastest_speed = 0.0; + for (auto kv : connected_peers) { + Connection* remote_peer = kv.second; + if (remote_peer->peer_speed > fastest_speed && remote_peer->choked_upload && remote_peer->interested) { + chosen_peer = remote_peer; + fastest_speed = remote_peer->peer_speed; + } + } + } + } + + if (chosen_peer != nullptr) + XBT_DEBUG("(%d) update_choked peers unchoked (%d) ; int (%d) ; choked (%d) ", id, chosen_peer->id, + chosen_peer->interested, chosen_peer->choked_upload); + + if (choked_peer != chosen_peer) { + if (choked_peer != nullptr) { + xbt_assert(not choked_peer->choked_upload, "Tries to choked a choked peer"); + choked_peer->choked_upload = true; + updateActivePeersSet(choked_peer); + XBT_DEBUG("(%d) Sending a CHOKE to %d", id, choked_peer->id); + sendMessage(choked_peer->mailbox_, MESSAGE_CHOKE, MESSAGE_CHOKE_SIZE); + } + if (chosen_peer != nullptr) { + xbt_assert((chosen_peer->choked_upload), "Tries to unchoked an unchoked peer"); + chosen_peer->choked_upload = false; + active_peers.insert(chosen_peer); + chosen_peer->last_unchoke = simgrid::s4u::Engine::getClock(); + XBT_DEBUG("(%d) Sending a UNCHOKE to %d", id, chosen_peer->id); + updateActivePeersSet(chosen_peer); + sendMessage(chosen_peer->mailbox_, MESSAGE_UNCHOKE, MESSAGE_UNCHOKE_SIZE); + } + } +} + +/** @brief Update "interested" state of peers: send "not interested" to peers that don't have any more pieces we want.*/ +void Peer::updateInterestedAfterReceive() +{ + for (auto kv : connected_peers) { + Connection* remote_peer = kv.second; + if (remote_peer->am_interested) { + bool interested = false; + // Check if the peer still has a piece we want. + for (unsigned int i = 0; i < FILE_PIECES; i++) + if (hasNotPiece(i) && remote_peer->hasPiece(i)) { + interested = true; + break; + } + + if (not interested) { // no more piece to download from connection + remote_peer->am_interested = false; + sendMessage(remote_peer->mailbox_, MESSAGE_NOTINTERESTED, MESSAGE_NOTINTERESTED_SIZE); + } + } + } +} + +void Peer::updateBitfieldBlocks(int piece, int block_index, int block_length) +{ + xbt_assert((piece >= 0 && static_cast(piece) <= FILE_PIECES), "Wrong piece."); + xbt_assert((block_index >= 0 && static_cast(block_index) <= PIECES_BLOCKS), "Wrong block : %d.", + block_index); + for (int i = block_index; i < (block_index + block_length); i++) + bitfield_blocks |= (1ULL << static_cast(piece * PIECES_BLOCKS + i)); +} + +bool Peer::hasCompletedPiece(unsigned int piece) +{ + for (unsigned int i = 0; i < PIECES_BLOCKS; i++) + if (not(bitfield_blocks & 1ULL << (piece * PIECES_BLOCKS + i))) + return false; + return true; +} + +int Peer::getFirstMissingBlockFrom(int piece) +{ + for (unsigned int i = 0; i < PIECES_BLOCKS; i++) + if (not(bitfield_blocks & 1ULL << (piece * PIECES_BLOCKS + i))) + return i; + return -1; +} + +/** Returns a piece that is partially downloaded and stored by the remote peer if any -1 otherwise. */ +int Peer::partiallyDownloadedPiece(Connection* remote_peer) +{ + for (unsigned int i = 0; i < FILE_PIECES; i++) + if (hasNotPiece(i) && remote_peer->hasPiece(i) && isNotDownloadingPiece(i) && getFirstMissingBlockFrom(i) > 0) + return i; + return -1; +} diff --git a/examples/s4u/app-bittorrent/s4u_peer.hpp b/examples/s4u/app-bittorrent/s4u_peer.hpp new file mode 100644 index 0000000000..bbbbcdd51f --- /dev/null +++ b/examples/s4u/app-bittorrent/s4u_peer.hpp @@ -0,0 +1,91 @@ +/* Copyright (c) 2012-2017. The SimGrid Team. + * All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#ifndef BITTORRENT_PEER_HPP +#define BITTORRENT_PEER_HPP +#include "s4u_bittorrent.hpp" +#include +#include + +class Connection { +public: + int id; // Peer id + simgrid::s4u::MailboxPtr mailbox_; + unsigned int bitfield = 0U; // Fields + // int messages_count; + double peer_speed = 0; + double last_unchoke = 0; + int current_piece = -1; + bool am_interested = false; // Indicates if we are interested in something the peer has + bool interested = false; // Indicates if the peer is interested in one of our pieces + bool choked_upload = true; // Indicates if the peer is choked for the current peer + bool choked_download = true; // Indicates if the peer has choked the current peer + + explicit Connection(int id) : id(id), mailbox_(simgrid::s4u::Mailbox::byName(std::to_string(id))){}; + ~Connection() = default; + void addSpeedValue(double speed) { peer_speed = peer_speed * 0.6 + speed * 0.4; } + bool hasPiece(unsigned int piece) { return bitfield & 1U << piece; } +}; + +class Peer { + int id; + double deadline; + RngStream stream; + simgrid::s4u::MailboxPtr mailbox_; + std::unordered_map connected_peers; + std::set active_peers; // active peers list + + unsigned int bitfield_ = 0; // list of pieces the peer has. + unsigned long long bitfield_blocks = 0; // list of blocks the peer has. + short* pieces_count = nullptr; // number of peers that have each piece. + unsigned int current_pieces = 0; // current pieces the peer is downloading + double begin_receive_time = 0; // time when the receiving communication has begun, useful for calculating host speed. + int round_ = 0; // current round for the chocking algorithm. + + simgrid::s4u::CommPtr comm_received = nullptr; // current comm + Message* message = nullptr; // current message being received +public: + explicit Peer(std::vector args); + Peer(const Peer&) = delete; + ~Peer(); + void operator()(); + + std::string getStatus(); + bool hasFinished(); + int nbInterestedPeers(); + bool isInterestedBy(Connection* remote_peer); + bool isInterestedByFree(Connection* remote_peer); + void updateActivePeersSet(Connection* remote_peer); + void updateInterestedAfterReceive(); + void updateChokedPeers(); + + bool hasNotPiece(unsigned int piece) { return not(bitfield_ & 1U << piece); } + bool hasCompletedPiece(unsigned int piece); + unsigned int countPieces(unsigned int bitfield); + /** Check that a piece is not currently being download by the peer. */ + bool isNotDownloadingPiece(unsigned int piece) { return not(current_pieces & 1U << piece); } + int partiallyDownloadedPiece(Connection* remote_peer); + void updatePiecesCountFromBitfield(unsigned int bitfield); + void removeCurrentPiece(Connection* remote_peer, unsigned int current_piece); + void updateBitfieldBlocks(int piece, int block_index, int block_length); + int getFirstMissingBlockFrom(int piece); + int selectPieceToDownload(Connection* remote_peer); + void requestNewPieceTo(Connection* remote_peer); + + bool getPeersFromTracker(); + void sendMessage(simgrid::s4u::MailboxPtr mailbox, e_message_type type, uint64_t size); + void sendBitfield(simgrid::s4u::MailboxPtr mailbox); + void sendPiece(simgrid::s4u::MailboxPtr mailbox, unsigned int piece, int block_index, int block_length); + void sendHandshakeToAllPeers(); + void sendHaveToAllPeers(unsigned int piece); + void sendRequestTo(Connection* remote_peer, unsigned int piece); + + void handleMessage(); + void leech(); + void seed(); +}; + +#endif /* BITTORRENT_PEER_HPP */ diff --git a/examples/s4u/app-bittorrent/s4u_tracker.cpp b/examples/s4u/app-bittorrent/s4u_tracker.cpp new file mode 100644 index 0000000000..7ddd5e095a --- /dev/null +++ b/examples/s4u/app-bittorrent/s4u_tracker.cpp @@ -0,0 +1,70 @@ +/* Copyright (c) 2012-2017. The SimGrid Team. + * All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#include "s4u_tracker.hpp" +#include + +XBT_LOG_NEW_DEFAULT_CATEGORY(s4u_bt_tracker, "Messages specific for the tracker"); + +Tracker::Tracker(std::vector args) +{ + // Checking arguments + xbt_assert(args.size() == 2, "Wrong number of arguments for the tracker."); + // Retrieving end time + try { + deadline = std::stod(args[1]); + } catch (std::invalid_argument& ia) { + throw std::invalid_argument(std::string("Invalid deadline:") + args[1].c_str()); + } + xbt_assert(deadline > 0, "Wrong deadline supplied"); + + stream = simgrid::s4u::this_actor::getHost()->extension()->getStream(); + + mailbox = simgrid::s4u::Mailbox::byName(TRACKER_MAILBOX); + + XBT_INFO("Tracker launched."); +} + +void Tracker::operator()() +{ + simgrid::s4u::CommPtr comm = nullptr; + while (simgrid::s4u::Engine::getClock() < deadline) { + void* received; + if (comm == nullptr) + comm = mailbox->get_async(&received); + if (comm->test()) { + // Retrieve the data sent by the peer. + TrackerQuery* tq = static_cast(received); + + // Add the peer to our peer list, if not already known. + if (known_peers.find(tq->getPeerId()) == known_peers.end()) { + known_peers.insert(tq->getPeerId()); + } + + // Sending back peers to the requesting peer + TrackerAnswer* ta = new TrackerAnswer(TRACKER_QUERY_INTERVAL); + std::set::iterator next_peer; + int nb_known_peers = known_peers.size(); + int max_tries = MIN(MAXIMUM_PEERS, nb_known_peers); + int tried = 0; + while (tried < max_tries) { + do { + next_peer = known_peers.begin(); + std::advance(next_peer, RngStream_RandInt(stream, 0, nb_known_peers - 1)); + } while (ta->getPeers()->find(*next_peer) != ta->getPeers()->end()); + ta->addPeer(*next_peer); + tried++; + } + tq->getReturnMailbox()->put_init(ta, TRACKER_COMM_SIZE)->detach(); + + delete tq; + comm = nullptr; + } else { + simgrid::s4u::this_actor::sleep_for(1); + } + } + XBT_INFO("Tracker is leaving"); +} diff --git a/examples/s4u/app-bittorrent/s4u_tracker.hpp b/examples/s4u/app-bittorrent/s4u_tracker.hpp new file mode 100644 index 0000000000..591233f9e3 --- /dev/null +++ b/examples/s4u/app-bittorrent/s4u_tracker.hpp @@ -0,0 +1,46 @@ +/* Copyright (c) 2012-2014, 2017. The SimGrid Team. + * All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#ifndef BITTORRENT_TRACKER_HPP_ +#define BITTORRENT_TRACKER_HPP_ + +#include "s4u_bittorrent.hpp" +#include + +class TrackerQuery { + int peer_id; // peer id + simgrid::s4u::MailboxPtr return_mailbox; +public: + explicit TrackerQuery(int peer_id, simgrid::s4u::MailboxPtr return_mailbox) + : peer_id(peer_id), return_mailbox(return_mailbox){}; + ~TrackerQuery() = default; + int getPeerId() { return peer_id; } + simgrid::s4u::MailboxPtr getReturnMailbox() { return return_mailbox; } +}; + +class TrackerAnswer { + XBT_ATTRIB_UNUSED int interval; // how often the peer should contact the tracker (unused for now) + std::set* peers; // the peer list the peer has asked for. +public: + explicit TrackerAnswer(int interval) : interval(interval) { peers = new std::set; } + TrackerAnswer(const TrackerAnswer&) = delete; + ~TrackerAnswer() { delete peers; }; + void addPeer(int peer) { peers->insert(peer); } + std::set* getPeers() { return peers; } +}; + +class Tracker { + double deadline; + RngStream stream; + simgrid::s4u::MailboxPtr mailbox; + std::set known_peers; + +public: + explicit Tracker(std::vector args); + void operator()(); +}; + +#endif /* BITTORRENT_TRACKER_HPP */ diff --git a/examples/s4u/app-masterworker/s4u_app-masterworker.cpp b/examples/s4u/app-masterworker/s4u_app-masterworker.cpp index 8e13fc5a76..04102b739c 100644 --- a/examples/s4u/app-masterworker/s4u_app-masterworker.cpp +++ b/examples/s4u/app-masterworker/s4u_app-masterworker.cpp @@ -40,17 +40,18 @@ public: XBT_INFO("Sending \"%s\" (of %ld) to mailbox \"%s\"", (std::string("Task_") + std::to_string(i)).c_str(), number_of_tasks, mailbox->getName()); - /* - Send the task to the @ref worker */ - char* payload = bprintf("%f", comp_size); - mailbox->put(payload, comm_size); + /* - Send the computation amount to the @ref worker */ + mailbox->put(static_cast(&comp_size), comm_size); } XBT_INFO("All tasks have been dispatched. Let's tell everybody the computation is over."); for (int i = 0; i < workers_count; i++) { /* - Eventually tell all the workers to stop by sending a "finalize" task */ mailbox = simgrid::s4u::Mailbox::byName(std::string("worker-") + std::to_string(i % workers_count)); - mailbox->put(xbt_strdup("finalize"), 0); + double finalize = -1; + mailbox->put(static_cast(&finalize), 0); } + simgrid::s4u::this_actor::sleep_for(.1); // Grace time to ensure everyone finishes. } }; @@ -70,19 +71,15 @@ public: void operator()() { while (1) { /* The worker waits in an infinite loop for tasks sent by the \ref master */ - char* res = static_cast(mailbox->get()); - xbt_assert(res != nullptr, "MSG_task_get failed"); - - if (strcmp(res, "finalize") == 0) { /* - Exit if 'finalize' is received */ - xbt_free(res); + double* comp_size = static_cast(mailbox->get()); + xbt_assert(comp_size != nullptr, "MSG_task_get failed"); + if (*comp_size < 0) { /* - Exit if 'finalize' is received */ + XBT_INFO("I'm done. See you!"); break; } /* - Otherwise, process the task */ - double comp_size = std::stod(res); - xbt_free(res); - simgrid::s4u::this_actor::execute(comp_size); + simgrid::s4u::this_actor::execute(*comp_size); } - XBT_INFO("I'm done. See you!"); } }; diff --git a/examples/s4u/app-masterworker/s4u_app-masterworker.tesh b/examples/s4u/app-masterworker/s4u_app-masterworker.tesh index f9b20df3fa..ff2092d8bc 100644 --- a/examples/s4u/app-masterworker/s4u_app-masterworker.tesh +++ b/examples/s4u/app-masterworker/s4u_app-masterworker.tesh @@ -31,6 +31,6 @@ $ $SG_TEST_EXENV ${bindir:=.}/s4u_app-masterworker$EXEEXT ${srcdir:=.}/small_pla > [ 4.057541] (worker@Jupiter) I'm done. See you! > [ 4.083249] (worker@Fafard) I'm done. See you! > [ 4.931805] (worker@Ginette) I'm done. See you! -> [ 5.094868] (maestro@) Simulation time 5.09487 > [ 5.094868] (worker@Bourassa) I'm done. See you! +> [ 5.194868] (maestro@) Simulation time 5.19487 diff --git a/examples/s4u/io/s4u_io.cpp b/examples/s4u/io/s4u_io.cpp index 378f271419..8c60b637d0 100644 --- a/examples/s4u/io/s4u_io.cpp +++ b/examples/s4u/io/s4u_io.cpp @@ -16,7 +16,7 @@ public: XBT_INFO("Storage info on %s:", simgrid::s4u::Host::current()->getCname()); for (const auto&kv : mounts) { - const char* mountpoint = kv.first.c_str(); + std::string mountpoint = kv.first; simgrid::s4u::Storage* storage = kv.second; // Retrieve disk's information @@ -24,8 +24,8 @@ public: sg_size_t used_size = storage->getSizeUsed(); sg_size_t size = storage->getSize(); - XBT_INFO(" %s (%s) Used: %llu; Free: %llu; Total: %llu.", storage->getName(), mountpoint, used_size, free_size, - size); + XBT_INFO(" %s (%s) Used: %llu; Free: %llu; Total: %llu.", storage->getName(), mountpoint.c_str(), used_size, + free_size, size); } } @@ -36,11 +36,11 @@ public: show_info(mounts); // Open an non-existing file to create it - const char* filename = "/home/tmp/data.txt"; + std::string filename = "/home/tmp/data.txt"; simgrid::s4u::File* file = new simgrid::s4u::File(filename, nullptr); sg_size_t write = file->write(200000); // Write 200,000 bytes - XBT_INFO("Create a %llu bytes file named '%s' on /sd1", write, filename); + XBT_INFO("Create a %llu bytes file named '%s' on /sd1", write, filename.c_str()); // check that sizes have changed show_info(mounts); @@ -49,22 +49,22 @@ public: const sg_size_t file_size = file->size(); file->seek(0); const sg_size_t read = file->read(file_size); - XBT_INFO("Read %llu bytes on %s", read, filename); + XBT_INFO("Read %llu bytes on %s", read, filename.c_str()); // Now write 100,000 bytes in tmp/data.txt write = file->write(100000); // Write 100,000 bytes - XBT_INFO("Write %llu bytes on %s", write, filename); + XBT_INFO("Write %llu bytes on %s", write, filename.c_str()); simgrid::s4u::Storage* storage = simgrid::s4u::Storage::byName("Disk4"); // Now rename file from ./tmp/data.txt to ./tmp/simgrid.readme - const char *newpath = "/home/tmp/simgrid.readme"; - XBT_INFO("Move '%s' to '%s'", file->getPath(), newpath); + std::string newpath = "/home/tmp/simgrid.readme"; + XBT_INFO("Move '%s' to '%s'", file->getPath(), newpath.c_str()); file->move(newpath); // Test attaching some user data to the file file->setUserdata(xbt_strdup("777")); - XBT_INFO("User data attached to the file: %s", (char*)file->getUserdata()); + XBT_INFO("User data attached to the file: %s", static_cast(file->getUserdata())); xbt_free(file->getUserdata()); // Close the file @@ -72,10 +72,10 @@ public: // Now attach some user data to disk1 XBT_INFO("Get/set data for storage element: %s", storage->getName()); - XBT_INFO(" Uninitialized storage data: '%s'", (char*)storage->getUserdata()); + XBT_INFO(" Uninitialized storage data: '%s'", static_cast(storage->getUserdata())); storage->setUserdata(xbt_strdup("Some user data")); - XBT_INFO(" Set and get data: '%s'", (char*)storage->getUserdata()); + XBT_INFO(" Set and get data: '%s'", static_cast(storage->getUserdata())); xbt_free(storage->getUserdata()); } diff --git a/examples/simdag/daxload/sd_daxload.tesh b/examples/simdag/daxload/sd_daxload.tesh index 34171340f4..8caeab782d 100644 --- a/examples/simdag/daxload/sd_daxload.tesh +++ b/examples/simdag/daxload/sd_daxload.tesh @@ -109,6 +109,8 @@ $ cat ${srcdir:=.}/daxload/smalldax.trace > [0.016600] node-1.acme.org compute 42000000000.000000 # 1@task1 > [0.016600] node-10.acme.org compute 42000000000.000000 # 2@task2 > [42.033200] node-11.acme.org compute 42000000000.000000 # 3@task1 +> [0.000000] node-0.acme.org send node-1.acme.org 1000000.000000 # root_i1_1@task1 +> [0.016600] node-1.acme.org recv node-0.acme.org 1000000.000000 # root_i1_1@task1 > [0.000000] node-0.acme.org send node-10.acme.org 1000000.000000 # root_i2_2@task2 > [0.016600] node-10.acme.org recv node-0.acme.org 1000000.000000 # root_i2_2@task2 > [42.016600] node-1.acme.org send node-11.acme.org 1000000.000000 # 1@task1_o1_3@task1 @@ -117,8 +119,6 @@ $ cat ${srcdir:=.}/daxload/smalldax.trace > [42.033200] node-11.acme.org recv node-10.acme.org 1000000.000000 # 2@task2_o2_3@task1 > [84.033200] node-11.acme.org send node-0.acme.org 4167312.000000 # 3@task1_o3_end > [84.067138] node-0.acme.org recv node-11.acme.org 4167312.000000 # 3@task1_o3_end -> [0.000000] node-0.acme.org send node-1.acme.org 1000000.000000 # root_i1_1@task1 -> [0.016600] node-1.acme.org recv node-0.acme.org 1000000.000000 # root_i1_1@task1 > [84.067138] node-0.acme.org compute 0.000000 # end $ cmake -E remove -f ${srcdir:=.}/dax.dot ${srcdir:=.}/daxload/smalldax.trace diff --git a/examples/simdag/properties/sd_properties.c b/examples/simdag/properties/sd_properties.c index 2e437de074..7e8d15926e 100644 --- a/examples/simdag/properties/sd_properties.c +++ b/examples/simdag/properties/sd_properties.c @@ -38,22 +38,24 @@ int main(int argc, char **argv) /* Print the properties of 'host1' */ - xbt_dict_foreach(props, cursor, key, data) { + xbt_dict_foreach (props, cursor, key, data) XBT_INFO("\tProperty: %s has value: %s", key, data); - } /* Try to get a property that does not exist */ value = sg_host_get_property_value(h1, noexist); XBT_INFO("\tProperty: %s has value: %s", noexist, value?value:"Undefined (NULL)"); + xbt_dict_free(&props); + /* Get the property list of 'host2' */ XBT_INFO("Property list for host %s", name2); props = sg_host_get_properties(h2); /* Print the properties of 'host2' */ - xbt_dict_foreach(props, cursor, key, data) { + xbt_dict_foreach (props, cursor, key, data) XBT_INFO("\tProperty: %s on host: %s", key, data); - } + + xbt_dict_free(&props); /* Modify an existing property test. First check it exists */ XBT_INFO("Modify an existing property"); diff --git a/include/simgrid/msg.h b/include/simgrid/msg.h index 1f9c363a7e..c6217599bb 100644 --- a/include/simgrid/msg.h +++ b/include/simgrid/msg.h @@ -1,5 +1,4 @@ -/* Copyright (c) 2004-2015. The SimGrid Team. - * All rights reserved. */ +/* Copyright (c) 2004-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ @@ -404,8 +403,10 @@ XBT_PUBLIC(msg_error_t) MSG_task_receive_bounded(msg_task_t * task, const char * XBT_PUBLIC(msg_comm_t) MSG_task_isend(msg_task_t task, const char *alias); XBT_PUBLIC(msg_comm_t) MSG_task_isend_bounded(msg_task_t task, const char *alias, double maxrate); XBT_PUBLIC(msg_comm_t) -MSG_task_isend_with_matching(msg_task_t task, const char* alias, int (*match_fun)(void*, void*, void*), - void* match_data); +XBT_ATTRIB_DEPRECATED_v320( + "This function will be removed from SimGrid v3.20. If you really need this function, please speak up quickly.") + MSG_task_isend_with_matching(msg_task_t task, const char* alias, int (*match_fun)(void*, void*, void*), + void* match_data); XBT_PUBLIC(void) MSG_task_dsend(msg_task_t task, const char *alias, void_f_pvoid_t cleanup); XBT_PUBLIC(void) MSG_task_dsend_bounded(msg_task_t task, const char *alias, void_f_pvoid_t cleanup, double maxrate); @@ -514,9 +515,6 @@ XBT_PUBLIC(void) MSG_vm_set_bound(msg_vm_t vm, double bound); /* Used only by the bindings -- unclean pimple, please ignore if you're not writing a binding */ XBT_PUBLIC(smx_context_t) MSG_process_get_smx_ctx(msg_process_t process); -/* Functions renamed in 3.14 */ -#define MSG_mailbox_get_head(m) MSG_mailbox_front(m) - SG_END_DECL() #ifdef __cplusplus diff --git a/include/simgrid/s4u/Engine.hpp b/include/simgrid/s4u/Engine.hpp index a42becaef6..e705bab613 100644 --- a/include/simgrid/s4u/Engine.hpp +++ b/include/simgrid/s4u/Engine.hpp @@ -75,7 +75,7 @@ public: simgrid::s4u::NetZone* getNetzoneByNameOrNull(const char* name); /** @brief Retrieve the netcard of the given name (or nullptr if not found) */ - simgrid::kernel::routing::NetPoint* getNetpointByNameOrNull(const char* name); + simgrid::kernel::routing::NetPoint* getNetpointByNameOrNull(std::string name); void getNetpointList(std::vector * list); void netpointRegister(simgrid::kernel::routing::NetPoint * card); void netpointUnregister(simgrid::kernel::routing::NetPoint * card); diff --git a/include/simgrid/s4u/File.hpp b/include/simgrid/s4u/File.hpp index 18d2926d66..cdc9b97a8c 100644 --- a/include/simgrid/s4u/File.hpp +++ b/include/simgrid/s4u/File.hpp @@ -25,12 +25,12 @@ namespace s4u { XBT_PUBLIC_CLASS File { public: - File(const char* fullpath, void* userdata); - File(const char* fullpath, sg_host_t host, void* userdata); + File(std::string fullpath, void* userdata); + File(std::string fullpath, sg_host_t host, void* userdata); ~File(); /** Retrieves the path to the file */ - const char* getPath() { return path_; } + const char* getPath() { return path_.c_str(); } /** Simulates a local read action. Returns the size of data actually read */ sg_size_t read(sg_size_t size); @@ -53,23 +53,21 @@ public: /** Retrieves the current file position */ sg_size_t tell(); - /** Rename a file - * - * WARNING: It is forbidden to move the file to another mount point */ - void move(const char* fullpath); + /** Rename a file. WARNING: It is forbidden to move the file to another mount point */ + void move(std::string fullpath); /** Remove a file from disk */ int unlink(); - const char* storage_type; - const char* storageId; + std::string storage_type; + std::string storageId; std::string mount_point; int desc_id = 0; private: surf_file_t pimpl_ = nullptr; - const char* path_ = nullptr; - void* userdata_ = nullptr; + std::string path_; + void* userdata_ = nullptr; }; } } // namespace simgrid::s4u diff --git a/include/simgrid/s4u/Host.hpp b/include/simgrid/s4u/Host.hpp index 3c8691e4eb..30c932e298 100644 --- a/include/simgrid/s4u/Host.hpp +++ b/include/simgrid/s4u/Host.hpp @@ -11,7 +11,6 @@ #include #include "xbt/Extendable.hpp" -#include "xbt/dict.h" #include "xbt/signal.hpp" #include "xbt/string.hpp" #include "xbt/swag.h" diff --git a/include/simgrid/s4u/NetZone.hpp b/include/simgrid/s4u/NetZone.hpp index fa5ca046e4..0f50b37606 100644 --- a/include/simgrid/s4u/NetZone.hpp +++ b/include/simgrid/s4u/NetZone.hpp @@ -38,13 +38,13 @@ XBT_PUBLIC_CLASS NetZone protected: friend simgrid::kernel::routing::NetZoneImpl; - explicit NetZone(NetZone * father, const char* name); + explicit NetZone(NetZone * father, std::string name); virtual ~NetZone(); public: /** @brief Seal your netzone once you're done adding content, and before routing stuff through it */ virtual void seal(); - char* getCname(); + const char* getCname(); NetZone* getFather(); std::vector* getChildren(); // Sub netzones @@ -80,7 +80,7 @@ private: std::unordered_map properties_; NetZone* father_ = nullptr; - char* name_ = nullptr; + std::string name_; bool sealed_ = false; // We cannot add more content when sealed diff --git a/include/simgrid/s4u/Storage.hpp b/include/simgrid/s4u/Storage.hpp index 38e30b7e49..efad7b826d 100644 --- a/include/simgrid/s4u/Storage.hpp +++ b/include/simgrid/s4u/Storage.hpp @@ -28,7 +28,7 @@ public: explicit Storage(surf::StorageImpl * pimpl) : pimpl_(pimpl) {} virtual ~Storage() = default; /** Retrieve a Storage by its name. It must exist in the platform file */ - static Storage* byName(const char* name); + static Storage* byName(std::string name); const char* getName(); const char* getType(); Host* getHost(); diff --git a/include/simgrid/s4u/VirtualMachine.hpp b/include/simgrid/s4u/VirtualMachine.hpp index b5dbf216d7..4453039929 100644 --- a/include/simgrid/s4u/VirtualMachine.hpp +++ b/include/simgrid/s4u/VirtualMachine.hpp @@ -45,6 +45,7 @@ private: virtual ~VirtualMachine(); public: + void start(); bool isMigrating(); void getParameters(vm_params_t params); diff --git a/include/simgrid/simix.h b/include/simgrid/simix.h index 44e263529b..de1ac3ad5f 100644 --- a/include/simgrid/simix.h +++ b/include/simgrid/simix.h @@ -72,9 +72,7 @@ extern int smx_context_guard_size_was_set; SG_BEGIN_DECL() -XBT_PUBLIC(xbt_dynar_t) SIMIX_process_get_runnable(); XBT_PUBLIC(smx_actor_t) SIMIX_process_from_PID(aid_t PID); -XBT_PUBLIC(xbt_dynar_t) SIMIX_processes_as_dynar(); /* parallelism */ XBT_PUBLIC(int) SIMIX_context_is_parallel(); @@ -201,7 +199,6 @@ XBT_PUBLIC(smx_actor_t) simcall_process_create(const char* name, xbt_main_func_t code, void* data, sg_host_t host, int argc, char** argv, std::map* properties); -XBT_PUBLIC(void) simcall_process_kill(smx_actor_t process); XBT_PUBLIC(void) simcall_process_killall(int reset_pid); XBT_PUBLIC(void) SIMIX_process_throw(smx_actor_t process, xbt_errcat_t cat, int value, const char *msg); diff --git a/include/simgrid/simix.hpp b/include/simgrid/simix.hpp index eb7e26b88a..f21fa3d539 100644 --- a/include/simgrid/simix.hpp +++ b/include/simgrid/simix.hpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2007-2010, 2012-2015. The SimGrid Team. +/* Copyright (c) 2007-2010, 2012-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -69,6 +69,7 @@ typename std::result_of::type kernelImmediate(F&& code) return result.get(); } +XBT_PUBLIC(const std::vector&) process_get_runnable(); XBT_PUBLIC(void) set_maestro(std::function code); XBT_PUBLIC(void) create_maestro(std::function code); diff --git a/include/xbt/algorithm.hpp b/include/xbt/algorithm.hpp new file mode 100644 index 0000000000..1a914527d5 --- /dev/null +++ b/include/xbt/algorithm.hpp @@ -0,0 +1,46 @@ +/* Copyright (c) 2015-2017. The SimGrid Team. All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#ifndef XBT_ALGORITHM_HPP +#define XBT_ALGORITHM_HPP + +namespace simgrid { +namespace xbt { + +/** @brief Sorts the elements of the sequence [first, last) according to their color assuming elements can have only + * three colors. Since there are only three colors, it is linear and much faster than a classical sort. See for + * example http://en.wikipedia.org/wiki/Dutch_national_flag_problem + * + * \param first forward iterators to the initial position of the sequence to partition. + * \param last forward iterators to the final position of the sequence to partition. + * \param color the color function that accepts an element in the range as argument, and returns a value of 0, 1, or 2. + * + * At the end of the call, elements with color 0 are at the beginning of the range, elements with color 2 are at the end + * and elements with color 1 are in the middle. + */ +template +void three_way_partition(ForwardIterator first, ForwardIterator last, ColorFunction color) +{ + ForwardIterator iter = first; + while (iter < last) { + int c = color(*iter); + if (c == 1) { + ++iter; + } else if (c == 0) { + if (iter != first) + std::swap(*iter, *first); + ++iter; + ++first; + } else { // c == 2 + --last; + if (iter != last) + std::swap(*iter, *last); + } + } +} +} +} + +#endif diff --git a/include/xbt/dynar.h b/include/xbt/dynar.h index 86751e859d..5484ccdc41 100644 --- a/include/xbt/dynar.h +++ b/include/xbt/dynar.h @@ -93,7 +93,6 @@ XBT_PUBLIC(signed int) xbt_dynar_search_or_negative(xbt_dynar_t const dynar, voi XBT_PUBLIC(int) xbt_dynar_member(xbt_dynar_t const dynar, void *elem); XBT_PUBLIC(void) xbt_dynar_sort(xbt_dynar_t const dynar, int_f_cpvoid_cpvoid_t compar_fn); XBT_PUBLIC(xbt_dynar_t) xbt_dynar_sort_strings(xbt_dynar_t dynar); -XBT_PUBLIC(void) xbt_dynar_three_way_partition(xbt_dynar_t const dynar, int_f_pvoid_t color); XBT_PUBLIC(int) xbt_dynar_compare(xbt_dynar_t d1, xbt_dynar_t d2, int(*compar)(const void *, const void *)); XBT_PUBLIC(void *) xbt_dynar_to_array (xbt_dynar_t dynar); diff --git a/include/xbt/dynar.hpp b/include/xbt/dynar.hpp index 77bb871fdf..d9f77b5c76 100644 --- a/include/xbt/dynar.hpp +++ b/include/xbt/dynar.hpp @@ -1,11 +1,11 @@ -/* Copyright (c) 2004-2007, 2009-2015. The SimGrid Team. +/* Copyright (c) 2004-2007, 2009-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ -#ifndef SIMGRId_XBT_DYNAR_HPP -#define SIMGRId_XBT_DYNAR_HPP +#ifndef SIMGRID_XBT_DYNAR_HPP +#define SIMGRID_XBT_DYNAR_HPP #include #include diff --git a/include/xbt/function_types.h b/include/xbt/function_types.h index 03aec81f82..f9883a51d7 100644 --- a/include/xbt/function_types.h +++ b/include/xbt/function_types.h @@ -1,7 +1,6 @@ /* function_type.h - classical types for pointer to function */ -/* Copyright (c) 2006-2007, 2009-2010, 2012-2015. The SimGrid Team. - * All rights reserved. */ +/* Copyright (c) 2006-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ @@ -18,8 +17,6 @@ typedef void *(*pvoid_f_void_t) (void); typedef void *(*pvoid_f_pvoid_t) (void *); typedef void (*void_f_void_t) (void); -typedef int (*int_f_pvoid_t) (void*); - typedef int (*int_f_pvoid_pvoid_t) (void *, void *); typedef int (*int_f_cpvoid_cpvoid_t) (const void *, const void *); diff --git a/include/xbt/parmap.h b/include/xbt/parmap.h index 905b12c16f..d12a2f2ac1 100644 --- a/include/xbt/parmap.h +++ b/include/xbt/parmap.h @@ -1,6 +1,6 @@ /* A thread pool. */ -/* Copyright (c) 2007, 2009-2014. The SimGrid Team. +/* Copyright (c) 2007, 2009-2014, 2016-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -9,28 +9,19 @@ #ifndef XBT_PARMAP_H #define XBT_PARMAP_H -#include "xbt/misc.h" /* SG_BEGIN_DECL */ -#include "xbt/function_types.h" -#include "xbt/dynar.h" - -SG_BEGIN_DECL() - /** \addtogroup XBT_parmap * \ingroup XBT_misc * \brief Parallel map. * - * A function is applied to all elements of a dynar in parallel with n worker threads. - * The worker threads are persistent until the destruction of the parmap. + * A function is applied to all elements of a std::vector in parallel with n worker threads. The worker threads are + * persistent until the destruction of the parmap. * - * If there are more than n elements in the dynar, the worker threads are allowed to fetch themselves remaining work - * with xbt_parmap_next() and execute it. + * If there are more than n elements in the vector, the worker threads are allowed to fetch themselves remaining work + * with method next() and execute it. * * \{ */ -/** \brief Parallel map data type (opaque type) */ -typedef struct s_xbt_parmap *xbt_parmap_t; - /** \brief Synchronization mode of the worker threads of a parmap. */ typedef enum { XBT_PARMAP_POSIX, /**< use POSIX synchronization primitives */ @@ -39,13 +30,6 @@ typedef enum { XBT_PARMAP_DEFAULT /**< futex if available, posix otherwise */ } e_xbt_parmap_mode_t; -XBT_PUBLIC(xbt_parmap_t) xbt_parmap_new(unsigned int num_workers, e_xbt_parmap_mode_t mode); -XBT_PUBLIC(void) xbt_parmap_destroy(xbt_parmap_t parmap); -XBT_PUBLIC(void) xbt_parmap_apply(xbt_parmap_t parmap, void_f_pvoid_t fun, xbt_dynar_t data); -XBT_PUBLIC(void*) xbt_parmap_next(xbt_parmap_t parmap); - /** \} */ -SG_END_DECL() - #endif diff --git a/src/bindings/java/JavaContext.cpp b/src/bindings/java/JavaContext.cpp index e39a7b9740..c796b18c0d 100644 --- a/src/bindings/java/JavaContext.cpp +++ b/src/bindings/java/JavaContext.cpp @@ -47,10 +47,7 @@ JavaContext* JavaContextFactory::create_context( void JavaContextFactory::run_all() { - xbt_dynar_t processes = SIMIX_process_get_runnable(); - smx_actor_t process; - unsigned int cursor; - xbt_dynar_foreach(processes, cursor, process) { + for (smx_actor_t process : simgrid::simix::process_get_runnable()) { static_cast(process->context)->resume(); } } diff --git a/src/bindings/java/jmsg_comm.cpp b/src/bindings/java/jmsg_comm.cpp index 8d52bfedf2..47f14d60a6 100644 --- a/src/bindings/java/jmsg_comm.cpp +++ b/src/bindings/java/jmsg_comm.cpp @@ -10,6 +10,8 @@ #include "jmsg.h" #include +#include + XBT_LOG_EXTERNAL_DEFAULT_CATEGORY(java); SG_BEGIN_DECL() @@ -79,7 +81,7 @@ JNIEXPORT jboolean JNICALL Java_org_simgrid_msg_Comm_test(JNIEnv *env, jobject j } if (not comm) { - jxbt_throw_null(env, bprintf("comm is null")); + jxbt_throw_null(env, "comm is null"); return JNI_FALSE; } @@ -99,7 +101,7 @@ JNIEXPORT jboolean JNICALL Java_org_simgrid_msg_Comm_test(JNIEnv *env, jobject j JNIEXPORT void JNICALL Java_org_simgrid_msg_Comm_waitCompletion(JNIEnv *env, jobject jcomm, jdouble timeout) { msg_comm_t comm = (msg_comm_t) (uintptr_t) env->GetLongField(jcomm, jcomm_field_Comm_bind); if (not comm) { - jxbt_throw_null(env, bprintf("comm is null")); + jxbt_throw_null(env, "comm is null"); return; } @@ -130,7 +132,7 @@ static msg_comm_t* jarray_to_commArray(JNIEnv *env, jobjectArray jcomms, /* OUT comms[i] = (msg_comm_t) (uintptr_t) env->GetLongField(jcomm, jcomm_field_Comm_bind); if (not comms[i]) { - jxbt_throw_null(env, bprintf("comm at rank %d is null", i)); + jxbt_throw_null(env, std::string("comm at rank ") + std::to_string(i) + " is null"); return nullptr; } diff --git a/src/bindings/java/jmsg_host.cpp b/src/bindings/java/jmsg_host.cpp index e2208b381b..28804e76e9 100644 --- a/src/bindings/java/jmsg_host.cpp +++ b/src/bindings/java/jmsg_host.cpp @@ -7,6 +7,7 @@ #include "simgrid/plugins/energy.h" #include "simgrid/s4u/Host.hpp" +#include "simgrid/s4u/Storage.hpp" #include "jmsg.h" #include "jmsg_host.h" @@ -55,7 +56,7 @@ JNIEXPORT jobject JNICALL Java_org_simgrid_msg_Host_getByName(JNIEnv * env, jcla /* get the C string from the java string */ if (jname == nullptr) { - jxbt_throw_null(env,bprintf("No host can have a null name")); + jxbt_throw_null(env, "No host can have a null name"); return nullptr; } const char *name = env->GetStringUTFChars(jname, 0); @@ -237,8 +238,8 @@ JNIEXPORT jobjectArray JNICALL Java_org_simgrid_msg_Host_getMountedStorage(JNIEn int index = 0; jobjectArray jtable; - xbt_dict_t dict = MSG_host_get_mounted_storage_list(host); - int count = xbt_dict_length(dict); + std::unordered_map mounted_storages = host->getMountedStorages(); + int count = mounted_storages.size(); jclass cls = env->FindClass("org/simgrid/msg/Storage"); jtable = env->NewObjectArray((jsize) count, cls, nullptr); @@ -248,17 +249,12 @@ JNIEXPORT jobjectArray JNICALL Java_org_simgrid_msg_Host_getMountedStorage(JNIEn return nullptr; } - xbt_dict_cursor_t cursor=nullptr; - const char* mount_name; - const char* storage_name; - - xbt_dict_foreach(dict,cursor,mount_name,storage_name) { - jname = env->NewStringUTF(storage_name); + for (auto elm : mounted_storages) { + jname = env->NewStringUTF(elm.second->getName()); jstorage = Java_org_simgrid_msg_Storage_getByName(env,cls,jname); env->SetObjectArrayElement(jtable, index, jstorage); index++; } - xbt_dict_free(&dict); return jtable; } diff --git a/src/bindings/java/jmsg_process.cpp b/src/bindings/java/jmsg_process.cpp index 184e259cf5..000d1d4c4a 100644 --- a/src/bindings/java/jmsg_process.cpp +++ b/src/bindings/java/jmsg_process.cpp @@ -101,7 +101,7 @@ JNIEXPORT jobject JNICALL Java_org_simgrid_msg_Process_fromPID(JNIEnv * env, jcl msg_process_t process = MSG_process_from_PID(pid); if (not process) { - jxbt_throw_process_not_found(env, bprintf("PID = %d",static_cast(pid))); + jxbt_throw_process_not_found(env, std::string("PID = ") + std::to_string(static_cast(pid))); return nullptr; } @@ -156,7 +156,7 @@ JNIEXPORT void JNICALL Java_org_simgrid_msg_Process_suspend(JNIEnv * env, jobjec /* try to suspend the process */ msg_error_t rv = MSG_process_suspend(process); - jxbt_check_res("MSG_process_suspend()", rv, MSG_OK, bprintf("unexpected error , please report this bug")); + jxbt_check_res("MSG_process_suspend()", rv, MSG_OK, "unexpected error , please report this bug"); } JNIEXPORT void JNICALL Java_org_simgrid_msg_Process_resume(JNIEnv * env, jobject jprocess) @@ -170,7 +170,7 @@ JNIEXPORT void JNICALL Java_org_simgrid_msg_Process_resume(JNIEnv * env, jobject /* try to resume the process */ msg_error_t res = MSG_process_resume(process); - jxbt_check_res("MSG_process_resume()", res, MSG_OK, bprintf("unexpected error , please report this bug")); + jxbt_check_res("MSG_process_resume()", res, MSG_OK, "unexpected error , please report this bug"); } JNIEXPORT void diff --git a/src/bindings/java/jmsg_storage.cpp b/src/bindings/java/jmsg_storage.cpp index 2d3a02931c..7b5003f6c1 100644 --- a/src/bindings/java/jmsg_storage.cpp +++ b/src/bindings/java/jmsg_storage.cpp @@ -56,7 +56,7 @@ JNIEXPORT jobject JNICALL Java_org_simgrid_msg_Storage_getByName(JNIEnv * env, j /* get the C string from the java string */ if (jname == nullptr) { - jxbt_throw_null(env,bprintf("No host can have a null name")); + jxbt_throw_null(env, "No host can have a null name"); return nullptr; } const char *name = env->GetStringUTFChars(jname, 0); diff --git a/src/bindings/java/jmsg_task.cpp b/src/bindings/java/jmsg_task.cpp index 47dfe47f93..eb5ca2ddb6 100644 --- a/src/bindings/java/jmsg_task.cpp +++ b/src/bindings/java/jmsg_task.cpp @@ -112,7 +112,7 @@ JNIEXPORT void JNICALL Java_org_simgrid_msg_Task_cancel(JNIEnv * env, jobject jt msg_error_t rv = MSG_task_cancel(ptask); - jxbt_check_res("MSG_task_cancel()", rv, MSG_OK, bprintf("unexpected error , please report this bug")); + jxbt_check_res("MSG_task_cancel()", rv, MSG_OK, "unexpected error , please report this bug"); } JNIEXPORT void JNICALL Java_org_simgrid_msg_Task_execute(JNIEnv * env, jobject jtask) diff --git a/src/bindings/java/jmsg_vm.cpp b/src/bindings/java/jmsg_vm.cpp index d7e9ea12b6..b570d72103 100644 --- a/src/bindings/java/jmsg_vm.cpp +++ b/src/bindings/java/jmsg_vm.cpp @@ -174,7 +174,7 @@ JNIEXPORT jobject JNICALL Java_org_simgrid_msg_VM_getVMByName(JNIEnv* env, jclas /* get the C string from the java string */ if (jname == nullptr) { - jxbt_throw_null(env, bprintf("No VM can have a null name")); + jxbt_throw_null(env, "No VM can have a null name"); return nullptr; } const char* name = env->GetStringUTFChars(jname, 0); diff --git a/src/bindings/lua/lua_platf.cpp b/src/bindings/lua/lua_platf.cpp index c0b812cad9..f0ca4380da 100644 --- a/src/bindings/lua/lua_platf.cpp +++ b/src/bindings/lua/lua_platf.cpp @@ -1,5 +1,4 @@ -/* Copyright (c) 2010, 2012-2017. The SimGrid Team. - * All rights reserved. */ +/* Copyright (c) 2010-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ @@ -20,7 +19,7 @@ extern "C" { #include "src/surf/surf_private.h" #include #include -#include +#include #include #include @@ -102,18 +101,16 @@ int console_add_backbone(lua_State *L) { } sg_platf_new_link(&link); - routing_cluster_add_backbone(simgrid::surf::LinkImpl::byName(link.id.c_str())); + routing_cluster_add_backbone(simgrid::surf::LinkImpl::byName(link.id)); return 0; } int console_add_host___link(lua_State *L) { - s_sg_platf_host_link_cbarg_t hostlink; - memset(&hostlink,0,sizeof(hostlink)); + HostLinkCreationArgs hostlink; int type; - lua_ensure(lua_istable(L, -1), - "Bad Arguments to create host_link in Lua. Should be a table with named arguments."); + lua_ensure(lua_istable(L, -1), "Bad Arguments to create host_link in Lua. Should be a table with named arguments."); lua_pushstring(L, "id"); type = lua_gettable(L, -2); @@ -135,7 +132,7 @@ int console_add_host___link(lua_State *L) { hostlink.link_down = lua_tostring(L, -1); lua_pop(L, 1); - XBT_DEBUG("Create a host_link for host %s", hostlink.id); + XBT_DEBUG("Create a host_link for host %s", hostlink.id.c_str()); sg_platf_new_hostlink(&hostlink); return 0; @@ -285,8 +282,7 @@ int console_add_link(lua_State *L) { * add Router to AS components */ int console_add_router(lua_State* L) { - lua_ensure(lua_istable(L, -1), - "Bad Arguments to create router, Should be a table with named arguments"); + lua_ensure(lua_istable(L, -1), "Bad Arguments to create router, Should be a table with named arguments"); lua_pushstring(L, "id"); int type = lua_gettable(L, -2); @@ -343,7 +339,7 @@ int console_add_route(lua_State *L) { // Several names separated by , \t\r\n for (auto name : names) { if (name.length() > 0) { - simgrid::surf::LinkImpl* link = simgrid::surf::LinkImpl::byName(name.c_str()); + simgrid::surf::LinkImpl* link = simgrid::surf::LinkImpl::byName(name); route.link_list->push_back(link); } } @@ -424,7 +420,7 @@ int console_add_ASroute(lua_State *L) { // Several names separated by , \t\r\n for (auto name : names) { if (name.length() > 0) { - simgrid::surf::LinkImpl* link = simgrid::surf::LinkImpl::byName(name.c_str()); + simgrid::surf::LinkImpl* link = simgrid::surf::LinkImpl::byName(name); ASroute.link_list->push_back(link); } } @@ -452,23 +448,19 @@ int console_add_ASroute(lua_State *L) { } int console_AS_open(lua_State *L) { - const char *id; - const char *mode; - int type; - XBT_DEBUG("Opening AS"); lua_ensure(lua_istable(L, 1), "Bad Arguments to AS_open, Should be a table with named arguments"); lua_pushstring(L, "id"); - type = lua_gettable(L, -2); + int type = lua_gettable(L, -2); lua_ensure(type == LUA_TSTRING, "Attribute 'id' must be specified for any AS and must be a string."); - id = lua_tostring(L, -1); + const char* id = lua_tostring(L, -1); lua_pop(L, 1); lua_pushstring(L, "mode"); lua_gettable(L, -2); - mode = lua_tostring(L, -1); + const char* mode = lua_tostring(L, -1); lua_pop(L, 1); int mode_int = A_surfxml_AS_routing_None; @@ -488,10 +480,10 @@ int console_AS_open(lua_State *L) { mode_int = A_surfxml_AS_routing_None; else xbt_die("Don't have the model name '%s'",mode); - s_sg_platf_AS_cbarg_t AS; + ZoneCreationArgs AS; AS.id = id; AS.routing = mode_int; - simgrid::s4u::NetZone* new_as = sg_platf_new_AS_begin(&AS); + simgrid::s4u::NetZone* new_as = sg_platf_new_Zone_begin(&AS); /* Build a Lua representation of the new AS on the stack */ lua_newtable(L); @@ -506,37 +498,33 @@ int console_AS_open(lua_State *L) { } int console_AS_seal(lua_State *L) { XBT_DEBUG("Sealing AS"); - sg_platf_new_AS_seal(); + sg_platf_new_Zone_seal(); return 0; } int console_host_set_property(lua_State *L) { - const char* name =""; - const char* prop_id = ""; - const char* prop_value = ""; lua_ensure(lua_istable(L, -1), "Bad Arguments to create link, Should be a table with named arguments"); // get Host id lua_pushstring(L, "host"); lua_gettable(L, -2); - name = lua_tostring(L, -1); + const char* name = lua_tostring(L, -1); lua_pop(L, 1); // get prop Name lua_pushstring(L, "prop"); lua_gettable(L, -2); - prop_id = lua_tostring(L, -1); + const char* prop_id = lua_tostring(L, -1); lua_pop(L, 1); //get args lua_pushstring(L,"value"); lua_gettable(L, -2); - prop_value = lua_tostring(L,-1); + const char* prop_value = lua_tostring(L, -1); lua_pop(L, 1); sg_host_t host = sg_host_by_name(name); lua_ensure(host, "no host '%s' found",name); - xbt_dict_t props = sg_host_get_properties(host); - xbt_dict_set(props,prop_id,xbt_strdup(prop_value),nullptr); + host->setProperty(prop_id, prop_value); return 0; } diff --git a/src/include/instr/instr_interface.h b/src/include/instr/instr_interface.h index 4e6ff0b414..a3262e054a 100644 --- a/src/include/instr/instr_interface.h +++ b/src/include/instr/instr_interface.h @@ -14,8 +14,6 @@ XBT_PUBLIC(int) TRACE_start (); XBT_PUBLIC(int) TRACE_end (); XBT_PUBLIC(void) TRACE_global_init(); XBT_PUBLIC(void) TRACE_help(int detailed); -XBT_PUBLIC(void) TRACE_surf_resource_utilization_alloc(); -XBT_PUBLIC(void) TRACE_surf_resource_utilization_release(); SG_END_DECL() diff --git a/src/include/surf/surf.h b/src/include/surf/surf.h index f1f3ce2eb2..532f1cff49 100644 --- a/src/include/surf/surf.h +++ b/src/include/surf/surf.h @@ -60,7 +60,6 @@ SG_BEGIN_DECL() /* user-visible parameters */ extern XBT_PRIVATE double sg_tcp_gamma; -extern XBT_PRIVATE double sg_sender_gap; extern XBT_PRIVATE double sg_latency_factor; extern XBT_PRIVATE double sg_bandwidth_factor; extern XBT_PRIVATE double sg_weight_S_parameter; @@ -434,8 +433,6 @@ const char *__surf_get_initial_path(); /********** Tracing **********/ /* from surf_instr.c */ void TRACE_surf_action(surf_action_t surf_action, const char *category); -void TRACE_surf_alloc(); -void TRACE_surf_release(); /* instr_routing.c */ void instr_routing_define_callbacks (); diff --git a/src/include/xbt/parmap.hpp b/src/include/xbt/parmap.hpp new file mode 100644 index 0000000000..24affcc830 --- /dev/null +++ b/src/include/xbt/parmap.hpp @@ -0,0 +1,451 @@ +/* A thread pool (C++ version). */ + +/* Copyright (c) 2004-2017 The SimGrid Team. + * All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#ifndef XBT_PARMAP_HPP +#define XBT_PARMAP_HPP + +#include "src/internal_config.h" // HAVE_FUTEX_H +#include "src/kernel/context/Context.hpp" +#include +#include +#include +#include +#include +#include +#include + +#if HAVE_FUTEX_H +#include +#include +#include +#endif + +XBT_LOG_EXTERNAL_CATEGORY(xbt_parmap); + +namespace simgrid { +namespace xbt { + +/** \addtogroup XBT_parmap + * \ingroup XBT_misc + * \brief Parallel map class + * \{ + */ +template class Parmap { +public: + Parmap(unsigned num_workers, e_xbt_parmap_mode_t mode); + Parmap(const Parmap&) = delete; + ~Parmap(); + void apply(void (*fun)(T), const std::vector& data); + boost::optional next(); + +private: + enum Flag { PARMAP_WORK, PARMAP_DESTROY }; + + /** + * \brief Thread data transmission structure + */ + class ThreadData { + public: + ThreadData(Parmap& parmap, int id) : parmap(parmap), worker_id(id) {} + Parmap& parmap; + int worker_id; + }; + + /** + * \brief Synchronization object (different specializations). + */ + class Synchro { + public: + explicit Synchro(Parmap& parmap) : parmap(parmap) {} + virtual ~Synchro() {} + /** + * \brief Wakes all workers and waits for them to finish the tasks. + * + * This function is called by the controller thread. + */ + virtual void master_signal() = 0; + /** + * \brief Starts the parmap: waits for all workers to be ready and returns. + * + * This function is called by the controller thread. + */ + virtual void master_wait() = 0; + /** + * \brief Ends the parmap: wakes the controller thread when all workers terminate. + * + * This function is called by all worker threads when they end (not including the controller). + */ + virtual void worker_signal() = 0; + /** + * \brief Waits for some work to process. + * + * This function is called by each worker thread (not including the controller) when it has no more work to do. + * + * \param round the expected round number + */ + virtual void worker_wait(unsigned) = 0; + + protected: + Parmap& parmap; + }; + + class PosixSynchro : public Synchro { + public: + explicit PosixSynchro(Parmap& parmap); + ~PosixSynchro(); + void master_signal(); + void master_wait(); + void worker_signal(); + void worker_wait(unsigned round); + + private: + xbt_os_cond_t ready_cond; + xbt_os_mutex_t ready_mutex; + xbt_os_cond_t done_cond; + xbt_os_mutex_t done_mutex; + }; + +#if HAVE_FUTEX_H + class FutexSynchro : public Synchro { + public: + explicit FutexSynchro(Parmap& parmap) : Synchro(parmap) {} + void master_signal(); + void master_wait(); + void worker_signal(); + void worker_wait(unsigned); + + private: + static void futex_wait(unsigned* uaddr, unsigned val); + static void futex_wake(unsigned* uaddr, unsigned val); + }; +#endif + + class BusyWaitSynchro : public Synchro { + public: + explicit BusyWaitSynchro(Parmap& parmap) : Synchro(parmap) {} + void master_signal(); + void master_wait(); + void worker_signal(); + void worker_wait(unsigned); + }; + + static void* worker_main(void* arg); + Synchro* new_synchro(e_xbt_parmap_mode_t mode); + void work(); + + Flag status; /**< is the parmap active or being destroyed? */ + unsigned work_round; /**< index of the current round */ + xbt_os_thread_t* workers; /**< worker thread handlers */ + unsigned num_workers; /**< total number of worker threads including the controller */ + Synchro* synchro; /**< synchronization object */ + + unsigned thread_counter = 0; /**< number of workers that have done the work */ + void (*fun)(const T) = nullptr; /**< function to run in parallel on each element of data */ + const std::vector* data = nullptr; /**< parameters to pass to fun in parallel */ + std::atomic index; /**< index of the next element of data to pick */ +}; + +/** + * \brief Creates a parallel map object + * \param num_workers number of worker threads to create + * \param mode how to synchronize the worker threads + */ +template Parmap::Parmap(unsigned num_workers, e_xbt_parmap_mode_t mode) +{ + XBT_CDEBUG(xbt_parmap, "Create new parmap (%u workers)", num_workers); + + /* Initialize the thread pool data structure */ + this->status = PARMAP_WORK; + this->work_round = 0; + this->workers = new xbt_os_thread_t[num_workers]; + this->num_workers = num_workers; + this->synchro = new_synchro(mode); + + /* Create the pool of worker threads */ + this->workers[0] = nullptr; +#if HAVE_PTHREAD_SETAFFINITY + int core_bind = 0; +#endif + for (unsigned i = 1; i < num_workers; i++) { + ThreadData* data = new ThreadData(*this, i); + this->workers[i] = xbt_os_thread_create(nullptr, worker_main, data, nullptr); +#if HAVE_PTHREAD_SETAFFINITY + xbt_os_thread_bind(this->workers[i], core_bind); + if (core_bind != xbt_os_get_numcores() - 1) + core_bind++; + else + core_bind = 0; +#endif + } +} + +/** + * \brief Destroys a parmap + */ +template Parmap::~Parmap() +{ + status = PARMAP_DESTROY; + synchro->master_signal(); + + for (unsigned i = 1; i < num_workers; i++) + xbt_os_thread_join(workers[i], nullptr); + + delete[] workers; + delete synchro; +} + +/** + * \brief Applies a list of tasks in parallel. + * \param fun the function to call in parallel + * \param data each element of this vector will be passed as an argument to fun + */ +template void Parmap::apply(void (*fun)(T), const std::vector& data) +{ + /* Assign resources to worker threads (we are maestro here)*/ + this->fun = fun; + this->data = &data; + this->index = 0; + this->synchro->master_signal(); // maestro runs futex_wait to wake all the minions (the working threads) + this->work(); // maestro works with its minions + this->synchro->master_wait(); // When there is no more work to do, then maestro waits for the last minion to stop + XBT_CDEBUG(xbt_parmap, "Job done"); // ... and proceeds +} + +/** + * \brief Returns a next task to process. + * + * Worker threads call this function to get more work. + * + * \return the next task to process, or throws a std::out_of_range exception if there is no more work + */ +template boost::optional Parmap::next() +{ + unsigned index = this->index++; + if (index < this->data->size()) + return (*this->data)[index]; + else + return boost::none; +} + +/** + * \brief Main work loop: applies fun to elements in turn. + */ +template void Parmap::work() +{ + unsigned index = this->index++; + unsigned length = this->data->size(); + while (index < length) { + this->fun((*this->data)[index]); + index = this->index++; + } +} + +/** + * Get a synchronization object for given mode. + * \param mode the synchronization mode + */ +template typename Parmap::Synchro* Parmap::new_synchro(e_xbt_parmap_mode_t mode) +{ + if (mode == XBT_PARMAP_DEFAULT) { +#if HAVE_FUTEX_H + mode = XBT_PARMAP_FUTEX; +#else + mode = XBT_PARMAP_POSIX; +#endif + } + Synchro* res; + switch (mode) { + case XBT_PARMAP_POSIX: + res = new PosixSynchro(*this); + break; + case XBT_PARMAP_FUTEX: +#if HAVE_FUTEX_H + res = new FutexSynchro(*this); +#else + xbt_die("Fute is not available on this OS."); +#endif + break; + case XBT_PARMAP_BUSY_WAIT: + res = new BusyWaitSynchro(*this); + break; + default: + THROW_IMPOSSIBLE; + } + return res; +} + +/** + * \brief Main function of a worker thread. + */ +template void* Parmap::worker_main(void* arg) +{ + ThreadData* data = static_cast(arg); + Parmap& parmap = data->parmap; + unsigned round = 0; + smx_context_t context = SIMIX_context_new(std::function(), nullptr, nullptr); + SIMIX_context_set_current(context); + + XBT_CDEBUG(xbt_parmap, "New worker thread created"); + + /* Worker's main loop */ + while (1) { + round++; + parmap.synchro->worker_wait(round); + if (parmap.status == PARMAP_DESTROY) + break; + + XBT_CDEBUG(xbt_parmap, "Worker %d got a job", data->worker_id); + parmap.work(); + parmap.synchro->worker_signal(); + XBT_CDEBUG(xbt_parmap, "Worker %d has finished", data->worker_id); + } + /* We are destroying the parmap */ + delete context; + delete data; + return nullptr; +} + +template Parmap::PosixSynchro::PosixSynchro(Parmap& parmap) : Synchro(parmap) +{ + ready_cond = xbt_os_cond_init(); + ready_mutex = xbt_os_mutex_init(); + done_cond = xbt_os_cond_init(); + done_mutex = xbt_os_mutex_init(); +} + +template Parmap::PosixSynchro::~PosixSynchro() +{ + xbt_os_cond_destroy(ready_cond); + xbt_os_mutex_destroy(ready_mutex); + xbt_os_cond_destroy(done_cond); + xbt_os_mutex_destroy(done_mutex); +} + +template void Parmap::PosixSynchro::master_signal() +{ + xbt_os_mutex_acquire(ready_mutex); + this->parmap.thread_counter = 1; + this->parmap.work_round++; + /* wake all workers */ + xbt_os_cond_broadcast(ready_cond); + xbt_os_mutex_release(ready_mutex); +} + +template void Parmap::PosixSynchro::master_wait() +{ + xbt_os_mutex_acquire(done_mutex); + if (this->parmap.thread_counter < this->parmap.num_workers) { + /* wait for all workers to be ready */ + xbt_os_cond_wait(done_cond, done_mutex); + } + xbt_os_mutex_release(done_mutex); +} + +template void Parmap::PosixSynchro::worker_signal() +{ + xbt_os_mutex_acquire(done_mutex); + this->parmap.thread_counter++; + if (this->parmap.thread_counter == this->parmap.num_workers) { + /* all workers have finished, wake the controller */ + xbt_os_cond_signal(done_cond); + } + xbt_os_mutex_release(done_mutex); +} + +template void Parmap::PosixSynchro::worker_wait(unsigned round) +{ + xbt_os_mutex_acquire(ready_mutex); + /* wait for more work */ + if (this->parmap.work_round != round) { + xbt_os_cond_wait(ready_cond, ready_mutex); + } + xbt_os_mutex_release(ready_mutex); +} + +#if HAVE_FUTEX_H +template inline void Parmap::FutexSynchro::futex_wait(unsigned* uaddr, unsigned val) +{ + XBT_CVERB(xbt_parmap, "Waiting on futex %p", uaddr); + syscall(SYS_futex, uaddr, FUTEX_WAIT_PRIVATE, val, nullptr, nullptr, 0); +} + +template inline void Parmap::FutexSynchro::futex_wake(unsigned* uaddr, unsigned val) +{ + XBT_CVERB(xbt_parmap, "Waking futex %p", uaddr); + syscall(SYS_futex, uaddr, FUTEX_WAKE_PRIVATE, val, nullptr, nullptr, 0); +} + +template void Parmap::FutexSynchro::master_signal() +{ + this->parmap.thread_counter = 1; + __sync_add_and_fetch(&this->parmap.work_round, 1); + /* wake all workers */ + futex_wake(&this->parmap.work_round, std::numeric_limits::max()); +} + +template void Parmap::FutexSynchro::master_wait() +{ + unsigned count = this->parmap.thread_counter; + while (count < this->parmap.num_workers) { + /* wait for all workers to be ready */ + futex_wait(&this->parmap.thread_counter, count); + count = this->parmap.thread_counter; + } +} + +template void Parmap::FutexSynchro::worker_signal() +{ + unsigned count = __sync_add_and_fetch(&this->parmap.thread_counter, 1); + if (count == this->parmap.num_workers) { + /* all workers have finished, wake the controller */ + futex_wake(&this->parmap.thread_counter, std::numeric_limits::max()); + } +} + +template void Parmap::FutexSynchro::worker_wait(unsigned round) +{ + unsigned work_round = this->parmap.work_round; + /* wait for more work */ + while (work_round != round) { + futex_wait(&this->parmap.work_round, work_round); + work_round = this->parmap.work_round; + } +} +#endif + +template void Parmap::BusyWaitSynchro::master_signal() +{ + this->parmap.thread_counter = 1; + __sync_add_and_fetch(&this->parmap.work_round, 1); +} + +template void Parmap::BusyWaitSynchro::master_wait() +{ + while (this->parmap.thread_counter < this->parmap.num_workers) { + xbt_os_thread_yield(); + } +} + +template void Parmap::BusyWaitSynchro::worker_signal() +{ + __sync_add_and_fetch(&this->parmap.thread_counter, 1); +} + +template void Parmap::BusyWaitSynchro::worker_wait(unsigned round) +{ + /* wait for more work */ + while (this->parmap.work_round != round) { + xbt_os_thread_yield(); + } +} + +/** \} */ +} +} + +#endif diff --git a/src/instr/instr_config.cpp b/src/instr/instr_config.cpp index 8455e27199..9a14d50624 100644 --- a/src/instr/instr_config.cpp +++ b/src/instr/instr_config.cpp @@ -1,11 +1,11 @@ -/* Copyright (c) 2010-2015. The SimGrid Team. - * All rights reserved. */ +/* Copyright (c) 2010-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ #include "src/instr/instr_private.h" #include "surf/surf.h" +#include #include XBT_LOG_NEW_CATEGORY(instr, "Logging the behavior of the tracing system (used for Visualization/Analysis of simulations)"); @@ -122,13 +122,6 @@ int TRACE_start() } trace_active = 1; XBT_DEBUG("Tracing is on"); - - /* other trace initialization */ - created_categories = xbt_dict_new_homogeneous(xbt_free_f); - declared_marks = xbt_dict_new_homogeneous(xbt_free_f); - user_host_variables = xbt_dict_new_homogeneous(xbt_free_f); - user_vm_variables = xbt_dict_new_homogeneous(xbt_free_f); - user_link_variables = xbt_dict_new_homogeneous(xbt_free_f); } return 0; } @@ -154,12 +147,6 @@ int TRACE_end() PJ_container_release(); rootType = nullptr; - xbt_dict_free(&user_link_variables); - xbt_dict_free(&user_host_variables); - xbt_dict_free(&user_vm_variables); - xbt_dict_free(&declared_marks); - xbt_dict_free(&created_categories); - /* close the trace files */ const char* format = xbt_cfg_get_string(OPT_TRACING_FORMAT); XBT_DEBUG("Tracing format %s\n", format); @@ -491,20 +478,19 @@ static void output_types (const char *name, xbt_dynar_t types, FILE *file) xbt_dynar_free (&types); } -static void output_categories (const char *name, xbt_dynar_t cats, FILE *file) +static void output_categories(const char* name, FILE* file) { - unsigned int i; + unsigned int i = created_categories.size(); fprintf (file, " values = ("); - for (i = xbt_dynar_length(cats); i > 0; i--) { - char *cat = *(static_cast(xbt_dynar_get_ptr(cats, i - 1))); - fprintf (file, "\"%s%s\"", name, cat); - if (i - 1 > 0){ + for (auto cat : created_categories) { + --i; + fprintf(file, "\"%s%s\"", name, cat.c_str()); + if (i > 0) { fprintf (file, ","); }else{ fprintf (file, ");\n"); } } - xbt_dynar_free (&cats); } static void uncat_configuration (FILE *file) @@ -541,13 +527,13 @@ static void cat_configuration (FILE *file) " host = {\n" " type = \"square\";\n" " size = \"power\";\n"); - output_categories ("p", TRACE_get_categories(), file); + output_categories("p", file); fprintf (file, " };\n" " link = {\n" " type = \"rhombus\";\n" " size = \"bandwidth\";\n"); - output_categories ("b", TRACE_get_categories(), file); + output_categories("b", file); fprintf (file, " };\n"); //close } @@ -574,7 +560,7 @@ static void generate_cat_configuration (const char *output, const char *name, in { if (output && strlen(output) > 0){ //check if we do have categories declared - if (xbt_dict_is_empty(created_categories)){ + if (created_categories.empty()) { XBT_INFO("No categories declared, ignoring generation of %s graph configuration", name); return; } diff --git a/src/instr/instr_interface.cpp b/src/instr/instr_interface.cpp index 726b68ac9c..24b187f70a 100644 --- a/src/instr/instr_interface.cpp +++ b/src/instr/instr_interface.cpp @@ -10,6 +10,7 @@ #include "src/surf/network_interface.hpp" #include "src/surf/surf_private.h" #include "surf/surf.h" +#include typedef enum { INSTR_US_DECLARE, @@ -20,26 +21,23 @@ typedef enum { XBT_LOG_NEW_DEFAULT_SUBCATEGORY (instr_api, instr, "API"); -xbt_dict_t created_categories = nullptr; -xbt_dict_t declared_marks = nullptr; -xbt_dict_t user_host_variables = nullptr; -xbt_dict_t user_vm_variables = nullptr; -xbt_dict_t user_link_variables = nullptr; -extern xbt_dict_t trivaNodeTypes; -extern xbt_dict_t trivaEdgeTypes; +std::set created_categories; +std::set declared_marks; +std::set user_host_variables; +std::set user_vm_variables; +std::set user_link_variables; +extern std::set trivaNodeTypes; +extern std::set trivaEdgeTypes; -static xbt_dynar_t instr_dict_to_dynar (xbt_dict_t filter) +static xbt_dynar_t instr_set_to_dynar(std::set* filter) { if (not TRACE_is_enabled() || not TRACE_needs_platform()) return nullptr; xbt_dynar_t ret = xbt_dynar_new (sizeof(char*), &xbt_free_ref); - xbt_dict_cursor_t cursor = nullptr; - char *name; - char *value; - xbt_dict_foreach(filter, cursor, name, value) { - xbt_dynar_push_as (ret, char*, xbt_strdup(name)); - } + for (auto name : *filter) + xbt_dynar_push_as(ret, char*, xbt_strdup(name.c_str())); + return ret; } @@ -88,10 +86,10 @@ void TRACE_category_with_color (const char *category, const char *color) return; //check if category is already created - if (xbt_dict_get_or_null(created_categories, category) != nullptr) + if (created_categories.find(category) != created_categories.end()) return; - - xbt_dict_set (created_categories, category, xbt_strdup("1"), nullptr); + else + created_categories.insert(category); //define final_color char final_color[INSTR_DEFAULT_STR_SIZE]; @@ -127,8 +125,7 @@ xbt_dynar_t TRACE_get_categories () { if (not TRACE_is_enabled() || not TRACE_categorized()) return nullptr; - - return instr_dict_to_dynar (created_categories); + return instr_set_to_dynar(&created_categories); } /** \ingroup TRACE_mark @@ -151,13 +148,13 @@ void TRACE_declare_mark(const char *mark_type) THROWF (tracing_error, 1, "mark_type is nullptr"); //check if mark_type is already declared - if (xbt_dict_get_or_null(declared_marks, mark_type) != nullptr) { + if (declared_marks.find(mark_type) != declared_marks.end()) { THROWF (tracing_error, 1, "mark_type with name (%s) is already declared", mark_type); } XBT_DEBUG("MARK,declare %s", mark_type); PJ_type_event_new(mark_type, PJ_type_get_root()); - xbt_dict_set (declared_marks, mark_type, xbt_strdup("1"), nullptr); + declared_marks.insert(mark_type); } /** \ingroup TRACE_mark @@ -263,25 +260,25 @@ xbt_dynar_t TRACE_get_marks () if (not TRACE_is_enabled()) return nullptr; - return instr_dict_to_dynar (declared_marks); + return instr_set_to_dynar(&declared_marks); } -static void instr_user_variable(double time, const char *resource, const char *variable, const char *father_type, - double value, InstrUserVariable what, const char *color, xbt_dict_t filter) +static void instr_user_variable(double time, const char* resource, const char* variable, const char* father_type, + double value, InstrUserVariable what, const char* color, std::set* filter) { /* safe switches. tracing has to be activated and if platform is not traced, we don't allow user variables */ if (not TRACE_is_enabled() || not TRACE_needs_platform()) return; //check if variable is already declared - char *created = (char*)xbt_dict_get_or_null(filter, variable); + auto created = filter->find(variable); if (what == INSTR_US_DECLARE){ - if (not created) { // not declared yet - xbt_dict_set (filter, variable, xbt_strdup("1"), nullptr); + if (created == filter->end()) { // not declared yet + filter->insert(variable); instr_new_user_variable_type (father_type, variable, color); } }else{ - if (created) { // declared, let's work + if (created != filter->end()) { // declared, let's work char valuestr[100]; snprintf(valuestr, 100, "%g", value); container_t container = PJ_container_get(resource); @@ -318,7 +315,7 @@ static void instr_user_srcdst_variable(double time, const char *src, const char std::vector route; simgrid::kernel::routing::NetZoneImpl::getGlobalRoute(src_elm, dst_elm, &route, nullptr); for (auto link : route) - instr_user_variable(time, link->cname(), variable, father_type, value, what, nullptr, user_link_variables); + instr_user_variable(time, link->cname(), variable, father_type, value, what, nullptr, &user_link_variables); } /** \ingroup TRACE_API @@ -364,7 +361,7 @@ int TRACE_platform_graph_export_graphviz (const char *filename) */ void TRACE_vm_variable_declare (const char *variable) { - instr_user_variable(0, nullptr, variable, "MSG_VM", 0, INSTR_US_DECLARE, nullptr, user_vm_variables); + instr_user_variable(0, nullptr, variable, "MSG_VM", 0, INSTR_US_DECLARE, nullptr, &user_vm_variables); } /** \ingroup TRACE_user_variables @@ -379,7 +376,7 @@ void TRACE_vm_variable_declare (const char *variable) */ void TRACE_vm_variable_declare_with_color (const char *variable, const char *color) { - instr_user_variable(0, nullptr, variable, "MSG_VM", 0, INSTR_US_DECLARE, color, user_vm_variables); + instr_user_variable(0, nullptr, variable, "MSG_VM", 0, INSTR_US_DECLARE, color, &user_vm_variables); } /** \ingroup TRACE_user_variables @@ -441,7 +438,7 @@ void TRACE_vm_variable_sub (const char *vm, const char *variable, double value) */ void TRACE_vm_variable_set_with_time (double time, const char *vm, const char *variable, double value) { - instr_user_variable(time, vm, variable, "MSG_VM", value, INSTR_US_SET, nullptr, user_vm_variables); + instr_user_variable(time, vm, variable, "MSG_VM", value, INSTR_US_SET, nullptr, &user_vm_variables); } /** \ingroup TRACE_user_variables @@ -461,7 +458,7 @@ void TRACE_vm_variable_set_with_time (double time, const char *vm, const char *v */ void TRACE_vm_variable_add_with_time (double time, const char *vm, const char *variable, double value) { - instr_user_variable(time, vm, variable, "MSG_VM", value, INSTR_US_ADD, nullptr, user_vm_variables); + instr_user_variable(time, vm, variable, "MSG_VM", value, INSTR_US_ADD, nullptr, &user_vm_variables); } /** \ingroup TRACE_user_variables @@ -481,7 +478,7 @@ void TRACE_vm_variable_add_with_time (double time, const char *vm, const char *v */ void TRACE_vm_variable_sub_with_time (double time, const char *vm, const char *variable, double value) { - instr_user_variable(time, vm, variable, "MSG_VM", value, INSTR_US_SUB, nullptr, user_vm_variables); + instr_user_variable(time, vm, variable, "MSG_VM", value, INSTR_US_SUB, nullptr, &user_vm_variables); } /* for host variables */ @@ -498,7 +495,7 @@ void TRACE_vm_variable_sub_with_time (double time, const char *vm, const char *v */ void TRACE_host_variable_declare (const char *variable) { - instr_user_variable(0, nullptr, variable, "HOST", 0, INSTR_US_DECLARE, nullptr, user_host_variables); + instr_user_variable(0, nullptr, variable, "HOST", 0, INSTR_US_DECLARE, nullptr, &user_host_variables); } /** \ingroup TRACE_user_variables @@ -513,7 +510,7 @@ void TRACE_host_variable_declare (const char *variable) */ void TRACE_host_variable_declare_with_color (const char *variable, const char *color) { - instr_user_variable(0, nullptr, variable, "HOST", 0, INSTR_US_DECLARE, color, user_host_variables); + instr_user_variable(0, nullptr, variable, "HOST", 0, INSTR_US_DECLARE, color, &user_host_variables); } /** \ingroup TRACE_user_variables @@ -575,7 +572,7 @@ void TRACE_host_variable_sub (const char *host, const char *variable, double val */ void TRACE_host_variable_set_with_time (double time, const char *host, const char *variable, double value) { - instr_user_variable(time, host, variable, "HOST", value, INSTR_US_SET, nullptr, user_host_variables); + instr_user_variable(time, host, variable, "HOST", value, INSTR_US_SET, nullptr, &user_host_variables); } /** \ingroup TRACE_user_variables @@ -595,7 +592,7 @@ void TRACE_host_variable_set_with_time (double time, const char *host, const cha */ void TRACE_host_variable_add_with_time (double time, const char *host, const char *variable, double value) { - instr_user_variable(time, host, variable, "HOST", value, INSTR_US_ADD, nullptr, user_host_variables); + instr_user_variable(time, host, variable, "HOST", value, INSTR_US_ADD, nullptr, &user_host_variables); } /** \ingroup TRACE_user_variables @@ -615,7 +612,7 @@ void TRACE_host_variable_add_with_time (double time, const char *host, const cha */ void TRACE_host_variable_sub_with_time (double time, const char *host, const char *variable, double value) { - instr_user_variable(time, host, variable, "HOST", value, INSTR_US_SUB, nullptr, user_host_variables); + instr_user_variable(time, host, variable, "HOST", value, INSTR_US_SUB, nullptr, &user_host_variables); } /** \ingroup TRACE_user_variables @@ -628,7 +625,7 @@ void TRACE_host_variable_sub_with_time (double time, const char *host, const cha */ xbt_dynar_t TRACE_get_host_variables () { - return instr_dict_to_dynar (user_host_variables); + return instr_set_to_dynar(&user_host_variables); } /* for link variables */ @@ -645,7 +642,7 @@ xbt_dynar_t TRACE_get_host_variables () */ void TRACE_link_variable_declare (const char *variable) { - instr_user_variable (0, nullptr, variable, "LINK", 0, INSTR_US_DECLARE, nullptr, user_link_variables); + instr_user_variable(0, nullptr, variable, "LINK", 0, INSTR_US_DECLARE, nullptr, &user_link_variables); } /** \ingroup TRACE_user_variables @@ -660,7 +657,7 @@ void TRACE_link_variable_declare (const char *variable) */ void TRACE_link_variable_declare_with_color (const char *variable, const char *color) { - instr_user_variable (0, nullptr, variable, "LINK", 0, INSTR_US_DECLARE, color, user_link_variables); + instr_user_variable(0, nullptr, variable, "LINK", 0, INSTR_US_DECLARE, color, &user_link_variables); } /** \ingroup TRACE_user_variables @@ -722,7 +719,7 @@ void TRACE_link_variable_sub (const char *link, const char *variable, double val */ void TRACE_link_variable_set_with_time (double time, const char *link, const char *variable, double value) { - instr_user_variable (time, link, variable, "LINK", value, INSTR_US_SET, nullptr, user_link_variables); + instr_user_variable(time, link, variable, "LINK", value, INSTR_US_SET, nullptr, &user_link_variables); } /** \ingroup TRACE_user_variables @@ -742,7 +739,7 @@ void TRACE_link_variable_set_with_time (double time, const char *link, const cha */ void TRACE_link_variable_add_with_time (double time, const char *link, const char *variable, double value) { - instr_user_variable (time, link, variable, "LINK", value, INSTR_US_ADD, nullptr, user_link_variables); + instr_user_variable(time, link, variable, "LINK", value, INSTR_US_ADD, nullptr, &user_link_variables); } /** \ingroup TRACE_user_variables @@ -762,7 +759,7 @@ void TRACE_link_variable_add_with_time (double time, const char *link, const cha */ void TRACE_link_variable_sub_with_time (double time, const char *link, const char *variable, double value) { - instr_user_variable (time, link, variable, "LINK", value, INSTR_US_SUB, nullptr, user_link_variables); + instr_user_variable(time, link, variable, "LINK", value, INSTR_US_SUB, nullptr, &user_link_variables); } /* for link variables, but with src and dst used for get_route */ @@ -899,7 +896,7 @@ void TRACE_link_srcdst_variable_sub_with_time (double time, const char *src, con */ xbt_dynar_t TRACE_get_link_variables () { - return instr_dict_to_dynar (user_link_variables); + return instr_set_to_dynar(&user_link_variables); } /** \ingroup TRACE_user_variables @@ -1001,7 +998,7 @@ void TRACE_host_pop_state (const char *host, const char *state) */ xbt_dynar_t TRACE_get_node_types () { - return instr_dict_to_dynar (trivaNodeTypes); + return instr_set_to_dynar(&trivaNodeTypes); } /** \ingroup TRACE_API @@ -1014,5 +1011,5 @@ xbt_dynar_t TRACE_get_node_types () */ xbt_dynar_t TRACE_get_edge_types () { - return instr_dict_to_dynar (trivaEdgeTypes); + return instr_set_to_dynar(&trivaEdgeTypes); } diff --git a/src/instr/instr_paje_containers.cpp b/src/instr/instr_paje_containers.cpp index 6f2331d22e..d65e46cba9 100644 --- a/src/instr/instr_paje_containers.cpp +++ b/src/instr/instr_paje_containers.cpp @@ -14,8 +14,8 @@ XBT_LOG_NEW_DEFAULT_SUBCATEGORY (instr_paje_containers, instr, "Paje tracing eve static container_t rootContainer = nullptr; /* the root container */ static xbt_dict_t allContainers = nullptr; /* all created containers indexed by name */ -xbt_dict_t trivaNodeTypes = nullptr; /* all host types defined */ -xbt_dict_t trivaEdgeTypes = nullptr; /* all link types defined */ +std::set trivaNodeTypes; /* all host types defined */ +std::set trivaEdgeTypes; /* all link types defined */ long long int instr_new_paje_id () { @@ -26,15 +26,11 @@ long long int instr_new_paje_id () void PJ_container_alloc () { allContainers = xbt_dict_new_homogeneous(nullptr); - trivaNodeTypes = xbt_dict_new_homogeneous(xbt_free_f); - trivaEdgeTypes = xbt_dict_new_homogeneous(xbt_free_f); } void PJ_container_release () { xbt_dict_free (&allContainers); - xbt_dict_free (&trivaNodeTypes); - xbt_dict_free (&trivaEdgeTypes); } void PJ_container_set_root (container_t root) @@ -151,7 +147,7 @@ container_t PJ_container_new (const char *name, e_container_types kind, containe //register NODE types for triva configuration if (newContainer->kind == INSTR_HOST || newContainer->kind == INSTR_LINK || newContainer->kind == INSTR_ROUTER) { - xbt_dict_set (trivaNodeTypes, newContainer->type->name, xbt_strdup("1"), nullptr); + trivaNodeTypes.insert(newContainer->type->name); } return newContainer; } diff --git a/src/instr/instr_private.h b/src/instr/instr_private.h index 9cf8031370..e8e38d4536 100644 --- a/src/instr/instr_private.h +++ b/src/instr/instr_private.h @@ -8,10 +8,11 @@ #include -#include "simgrid/instr.h" #include "instr/instr_interface.h" -#include "src/internal_config.h" +#include "simgrid/instr.h" #include "simgrid_config.h" +#include "src/internal_config.h" +#include SG_BEGIN_DECL() @@ -265,12 +266,11 @@ public: }; - -extern XBT_PRIVATE xbt_dict_t created_categories; -extern XBT_PRIVATE xbt_dict_t declared_marks; -extern XBT_PRIVATE xbt_dict_t user_host_variables; -extern XBT_PRIVATE xbt_dict_t user_vm_variables; -extern XBT_PRIVATE xbt_dict_t user_link_variables; +extern XBT_PRIVATE std::set created_categories; +extern XBT_PRIVATE std::set declared_marks; +extern XBT_PRIVATE std::set user_host_variables; +extern XBT_PRIVATE std::set user_vm_variables; +extern XBT_PRIVATE std::set user_link_variables; extern XBT_PRIVATE double TRACE_last_timestamp_to_dump; /* instr_paje_header.c */ @@ -325,8 +325,8 @@ XBT_PRIVATE void TRACE_surf_link_set_utilization(const char *resource,const char XBT_PUBLIC(void) TRACE_surf_resource_utilization_alloc(); /* instr_paje.c */ -extern XBT_PRIVATE xbt_dict_t trivaNodeTypes; -extern XBT_PRIVATE xbt_dict_t trivaEdgeTypes; +extern XBT_PRIVATE std::set trivaNodeTypes; +extern XBT_PRIVATE std::set trivaEdgeTypes; XBT_PRIVATE long long int instr_new_paje_id (); XBT_PRIVATE void PJ_container_alloc (); XBT_PRIVATE void PJ_container_release (); diff --git a/src/instr/instr_resource_utilization.cpp b/src/instr/instr_resource_utilization.cpp index 3135882228..7606f22e5c 100644 --- a/src/instr/instr_resource_utilization.cpp +++ b/src/instr/instr_resource_utilization.cpp @@ -1,15 +1,17 @@ -/* Copyright (c) 2010-2015. The SimGrid Team. +/* Copyright (c) 2010-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ #include "src/instr/instr_private.h" +#include +#include XBT_LOG_NEW_DEFAULT_SUBCATEGORY (instr_resource, instr, "tracing (un)-categorized resource utilization"); //to check if variables were previously set to 0, otherwise paje won't simulate them -static xbt_dict_t platform_variables; +static std::unordered_map platform_variables; //used by all methods static void __TRACE_surf_check_variable_set_to_zero(double now, const char *variable, const char *resource) @@ -21,18 +23,15 @@ static void __TRACE_surf_check_variable_set_to_zero(double now, const char *vari */ // create a key considering the resource and variable - int n = strlen(variable)+strlen(resource)+1; - char *key = (char*)xbt_malloc(n*sizeof(char)); - snprintf (key, n, "%s%s", resource, variable); + std::string key = std::string(resource) + variable; // check if key exists: if it doesn't, set the variable to zero and mark this in the dict - if (not xbt_dict_get_or_null(platform_variables, key)) { + if (platform_variables.find(key) == platform_variables.end()) { container_t container = PJ_container_get (resource); type_t type = PJ_type_get (variable, container->type); new SetVariableEvent (now, container, type, 0); - xbt_dict_set(platform_variables, key, (char*)"", nullptr); + platform_variables[key] = std::string(""); } - xbt_free(key); } static void instr_event (double now, double delta, type_t variable, container_t resource, double value) @@ -100,13 +99,3 @@ void TRACE_surf_host_set_utilization(const char *resource, const char *category, instr_event (now, delta, type, container, value); } } - -void TRACE_surf_resource_utilization_alloc() -{ - platform_variables = xbt_dict_new_homogeneous(nullptr); -} - -void TRACE_surf_resource_utilization_release() -{ - xbt_dict_free(&platform_variables); -} diff --git a/src/instr/jedule/jedule_platform.cpp b/src/instr/jedule/jedule_platform.cpp index 5109fd7dc5..babb894623 100644 --- a/src/instr/jedule/jedule_platform.cpp +++ b/src/instr/jedule/jedule_platform.cpp @@ -33,7 +33,6 @@ Container::~Container() void Container::addChild(jed_container_t child) { - xbt_assert(this != nullptr); xbt_assert(child != nullptr); this->children.push_back(child); child->parent = this; @@ -76,8 +75,6 @@ void Container::createHierarchy(sg_netzone_t from_as) std::vector Container::getHierarchy() { - xbt_assert( this!= nullptr ); - if(this->parent != nullptr ) { if (not this->parent->children.empty()) { @@ -147,7 +144,6 @@ void Container::printResources(FILE * jed_file) void Container::print(FILE* jed_file) { - xbt_assert( this != nullptr ); fprintf(jed_file, " \n", this->name.c_str()); if (not this->children.empty()) { for (auto child: this->children) { diff --git a/src/kernel/activity/MailboxImpl.cpp b/src/kernel/activity/MailboxImpl.cpp index 3f8cedc888..936b249718 100644 --- a/src/kernel/activity/MailboxImpl.cpp +++ b/src/kernel/activity/MailboxImpl.cpp @@ -9,13 +9,13 @@ XBT_LOG_NEW_DEFAULT_SUBCATEGORY(simix_mailbox, simix, "Mailbox implementation"); -static xbt_dict_t mailboxes = xbt_dict_new_homogeneous([](void* data) { - delete static_cast(data); -}); +static std::map* mailboxes = new std::map; void SIMIX_mailbox_exit() { - xbt_dict_free(&mailboxes); + for (auto elm : *mailboxes) + delete elm.second; + delete mailboxes; } /******************************************************************************/ @@ -28,20 +28,25 @@ namespace activity { /** @brief Returns the mailbox of that name, or nullptr */ MailboxImpl* MailboxImpl::byNameOrNull(const char* name) { - return static_cast(xbt_dict_get_or_null(mailboxes, name)); + auto mbox = mailboxes->find(name); + if (mbox != mailboxes->end()) + return mbox->second; + else + return nullptr; } /** @brief Returns the mailbox of that name, newly created on need */ MailboxImpl* MailboxImpl::byNameOrCreate(const char* name) { xbt_assert(name, "Mailboxes must have a name"); /* two processes may have pushed the same mbox_create simcall at the same time */ - smx_mailbox_t mbox = static_cast(xbt_dict_get_or_null(mailboxes, name)); - if (not mbox) { - mbox = new MailboxImpl(name); + auto m = mailboxes->find(name); + if (m == mailboxes->end()) { + smx_mailbox_t mbox = new MailboxImpl(name); XBT_DEBUG("Creating a mailbox at %p with name %s", mbox, name); - xbt_dict_set(mailboxes, mbox->name_, mbox, nullptr); - } - return mbox; + (*mailboxes)[mbox->name_] = mbox; + return mbox; + } else + return m->second; } /** @brief set the receiver of the mailbox to allow eager sends * \param actor The receiving dude diff --git a/src/kernel/context/Context.hpp b/src/kernel/context/Context.hpp index 97614a9d97..8102fb130e 100644 --- a/src/kernel/context/Context.hpp +++ b/src/kernel/context/Context.hpp @@ -147,10 +147,8 @@ SG_BEGIN_DECL() XBT_PRIVATE void SIMIX_context_mod_init(); XBT_PRIVATE void SIMIX_context_mod_exit(); -XBT_PRIVATE smx_context_t SIMIX_context_new( - std::function code, - void_pfn_smxprocess_t cleanup_func, - smx_actor_t simix_process); +XBT_PUBLIC(smx_context_t) +SIMIX_context_new(std::function code, void_pfn_smxprocess_t cleanup_func, smx_actor_t simix_process); #ifndef WIN32 XBT_PUBLIC_DATA(char sigsegv_stack[SIGSTKSZ]); @@ -174,7 +172,7 @@ XBT_PUBLIC(smx_context_t) SIMIX_context_self(); // public because it's used in s XBT_PRIVATE void *SIMIX_context_stack_new(); XBT_PRIVATE void SIMIX_context_stack_delete(void *stack); -XBT_PRIVATE void SIMIX_context_set_current(smx_context_t context); +XBT_PUBLIC(void) SIMIX_context_set_current(smx_context_t context); XBT_PRIVATE smx_context_t SIMIX_context_get_current(); XBT_PUBLIC(int) SIMIX_process_get_maxpid(); diff --git a/src/kernel/context/ContextBoost.cpp b/src/kernel/context/ContextBoost.cpp index a88920d898..3028445dd9 100644 --- a/src/kernel/context/ContextBoost.cpp +++ b/src/kernel/context/ContextBoost.cpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2015. The SimGrid Team. All rights reserved. */ +/* Copyright (c) 2015-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ @@ -49,12 +49,12 @@ public: // BoostContextFactory -bool BoostContext::parallel_ = false; -xbt_parmap_t BoostContext::parmap_ = nullptr; -uintptr_t BoostContext::threads_working_ = 0; +bool BoostContext::parallel_ = false; +simgrid::xbt::Parmap* BoostContext::parmap_ = nullptr; +uintptr_t BoostContext::threads_working_ = 0; xbt_os_thread_key_t BoostContext::worker_id_key_; -unsigned long BoostContext::process_index_ = 0; -BoostContext* BoostContext::maestro_context_ = nullptr; +unsigned long BoostContext::process_index_ = 0; +BoostContext* BoostContext::maestro_context_ = nullptr; std::vector BoostContext::workers_context_; BoostContextFactory::BoostContextFactory() @@ -64,7 +64,7 @@ BoostContextFactory::BoostContextFactory() if (BoostContext::parallel_) { #if HAVE_THREAD_CONTEXTS int nthreads = SIMIX_context_get_nthreads(); - BoostContext::parmap_ = xbt_parmap_new(nthreads, SIMIX_context_get_parallel_mode()); + BoostContext::parmap_ = new simgrid::xbt::Parmap(nthreads, SIMIX_context_get_parallel_mode()); BoostContext::workers_context_.clear(); BoostContext::workers_context_.resize(nthreads, nullptr); BoostContext::maestro_context_ = nullptr; @@ -79,7 +79,7 @@ BoostContextFactory::~BoostContextFactory() { #if HAVE_THREAD_CONTEXTS if (BoostContext::parmap_) { - xbt_parmap_destroy(BoostContext::parmap_); + delete BoostContext::parmap_; BoostContext::parmap_ = nullptr; } BoostContext::workers_context_.clear(); @@ -106,19 +106,18 @@ void BoostContextFactory::run_all() #if HAVE_THREAD_CONTEXTS if (BoostContext::parallel_) { BoostContext::threads_working_ = 0; - xbt_parmap_apply(BoostContext::parmap_, - [](void* arg) { - smx_actor_t process = static_cast(arg); - BoostContext* context = static_cast(process->context); - return context->resume(); - }, - simix_global->process_to_run); + BoostContext::parmap_->apply( + [](smx_actor_t process) { + BoostContext* context = static_cast(process->context); + return context->resume(); + }, + simix_global->process_to_run); } else #endif { - if (xbt_dynar_is_empty(simix_global->process_to_run)) + if (simix_global->process_to_run.empty()) return; - smx_actor_t first_process = xbt_dynar_get_as(simix_global->process_to_run, 0, smx_actor_t); + smx_actor_t first_process = simix_global->process_to_run.front(); BoostContext::process_index_ = 1; /* execute the first process */ static_cast(first_process->context)->resume(); @@ -193,11 +192,10 @@ void BoostSerialContext::suspend() unsigned long int i = process_index_; process_index_++; - if (i < xbt_dynar_length(simix_global->process_to_run)) { + if (i < simix_global->process_to_run.size()) { /* execute the next process */ XBT_DEBUG("Run next process"); - next_context = - static_cast(xbt_dynar_get_as(simix_global->process_to_run, i, smx_actor_t)->context); + next_context = static_cast(simix_global->process_to_run[i]->context); } else { /* all processes were run, return to maestro */ XBT_DEBUG("No more process to run"); @@ -223,12 +221,11 @@ void BoostSerialContext::stop() void BoostParallelContext::suspend() { - smx_actor_t next_work = static_cast(xbt_parmap_next(parmap_)); - BoostParallelContext* next_context = nullptr; - - if (next_work != nullptr) { + boost::optional next_work = parmap_->next(); + BoostParallelContext* next_context; + if (next_work) { XBT_DEBUG("Run next process"); - next_context = static_cast(next_work->context); + next_context = static_cast(next_work.get()->context); } else { XBT_DEBUG("No more processes to run"); uintptr_t worker_id = (uintptr_t)xbt_os_thread_get_specific(worker_id_key_); diff --git a/src/kernel/context/ContextBoost.hpp b/src/kernel/context/ContextBoost.hpp index 578fe10932..e8a34c2229 100644 --- a/src/kernel/context/ContextBoost.hpp +++ b/src/kernel/context/ContextBoost.hpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2015. The SimGrid Team. All rights reserved. */ +/* Copyright (c) 2015-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ @@ -9,7 +9,7 @@ #include #include -#include +#include #include @@ -27,7 +27,7 @@ class BoostContextFactory; class BoostContext : public Context { protected: // static static bool parallel_; - static xbt_parmap_t parmap_; + static simgrid::xbt::Parmap* parmap_; static std::vector workers_context_; static uintptr_t threads_working_; static xbt_os_thread_key_t worker_id_key_; diff --git a/src/kernel/context/ContextRaw.cpp b/src/kernel/context/ContextRaw.cpp index 057d46e8da..53726aa300 100644 --- a/src/kernel/context/ContextRaw.cpp +++ b/src/kernel/context/ContextRaw.cpp @@ -5,7 +5,7 @@ #include "src/internal_config.h" -#include "xbt/parmap.h" +#include "xbt/parmap.hpp" #include "src/simix/smx_private.h" #include "mc/mc.h" @@ -73,7 +73,7 @@ ContextFactory* raw_factory() // ***** Loads of static stuff #if HAVE_THREAD_CONTEXTS -static xbt_parmap_t raw_parmap; +static simgrid::xbt::Parmap* raw_parmap; static simgrid::kernel::context::RawContext** raw_workers_context; /* space to save the worker context in each thread */ static uintptr_t raw_threads_working; /* number of threads that have started their work */ static xbt_os_thread_key_t raw_worker_id_key; /* thread-specific storage for the thread id */ @@ -277,8 +277,7 @@ RawContextFactory::RawContextFactory() RawContextFactory::~RawContextFactory() { #if HAVE_THREAD_CONTEXTS - if (raw_parmap) - xbt_parmap_destroy(raw_parmap); + delete raw_parmap; xbt_free(raw_workers_context); #endif } @@ -337,11 +336,10 @@ void RawContextFactory::run_all() void RawContextFactory::run_all_serial() { - if (xbt_dynar_is_empty(simix_global->process_to_run)) + if (simix_global->process_to_run.empty()) return; - smx_actor_t first_process = - xbt_dynar_get_as(simix_global->process_to_run, 0, smx_actor_t); + smx_actor_t first_process = simix_global->process_to_run.front(); raw_process_index = 1; static_cast(first_process->context)->resume_serial(); } @@ -351,11 +349,9 @@ void RawContextFactory::run_all_parallel() #if HAVE_THREAD_CONTEXTS raw_threads_working = 0; if (raw_parmap == nullptr) - raw_parmap = xbt_parmap_new( - SIMIX_context_get_nthreads(), SIMIX_context_get_parallel_mode()); - xbt_parmap_apply(raw_parmap, - [](void* arg) { - smx_actor_t process = static_cast(arg); + raw_parmap = new simgrid::xbt::Parmap(SIMIX_context_get_nthreads(), SIMIX_context_get_parallel_mode()); + raw_parmap->apply( + [](smx_actor_t process) { RawContext* context = static_cast(process->context); context->resume_parallel(); }, @@ -379,10 +375,10 @@ void RawContext::suspend_serial() RawContext* next_context = nullptr; unsigned long int i = raw_process_index; raw_process_index++; - if (i < xbt_dynar_length(simix_global->process_to_run)) { + if (i < simix_global->process_to_run.size()) { /* execute the next process */ XBT_DEBUG("Run next process"); - next_context = static_cast(xbt_dynar_get_as(simix_global->process_to_run, i, smx_actor_t)->context); + next_context = static_cast(simix_global->process_to_run[i]->context); } else { /* all processes were run, return to maestro */ XBT_DEBUG("No more process to run"); @@ -396,13 +392,12 @@ void RawContext::suspend_parallel() { #if HAVE_THREAD_CONTEXTS /* determine the next context */ - smx_actor_t next_work = static_cast(xbt_parmap_next(raw_parmap)); - RawContext* next_context = nullptr; - - if (next_work != nullptr) { + boost::optional next_work = raw_parmap->next(); + RawContext* next_context; + if (next_work) { /* there is a next process to resume */ XBT_DEBUG("Run next process"); - next_context = static_cast(next_work->context); + next_context = static_cast(next_work.get()->context); } else { /* all processes were run, go to the barrier */ XBT_DEBUG("No more processes to run"); @@ -450,7 +445,7 @@ void RawContext::resume_parallel() /** @brief Resumes all processes ready to run. */ void RawContextFactory::run_all_adaptative() { - unsigned long nb_processes = xbt_dynar_length(simix_global->process_to_run); + unsigned long nb_processes = simix_global->process_to_run.size(); if (SIMIX_context_is_parallel() && static_cast(SIMIX_context_get_parallel_threshold()) < nb_processes) { raw_context_parallel = true; diff --git a/src/kernel/context/ContextThread.cpp b/src/kernel/context/ContextThread.cpp index 85ac35a62a..4ed2f29012 100644 --- a/src/kernel/context/ContextThread.cpp +++ b/src/kernel/context/ContextThread.cpp @@ -58,9 +58,7 @@ void ThreadContextFactory::run_all() { if (smx_ctx_thread_sem == nullptr) { // Serial execution - smx_actor_t process; - unsigned int cursor; - xbt_dynar_foreach(simix_global->process_to_run, cursor, process) { + for (smx_actor_t process : simix_global->process_to_run) { XBT_DEBUG("Handling %p",process); ThreadContext* context = static_cast(process->context); xbt_os_sem_release(context->begin_); @@ -68,12 +66,10 @@ void ThreadContextFactory::run_all() } } else { // Parallel execution - unsigned int index; - smx_actor_t process; - xbt_dynar_foreach(simix_global->process_to_run, index, process) + for (smx_actor_t process : simix_global->process_to_run) xbt_os_sem_release(static_cast(process->context)->begin_); - xbt_dynar_foreach(simix_global->process_to_run, index, process) - xbt_os_sem_acquire(static_cast(process->context)->end_); + for (smx_actor_t process : simix_global->process_to_run) + xbt_os_sem_acquire(static_cast(process->context)->end_); } } diff --git a/src/kernel/context/ContextUnix.cpp b/src/kernel/context/ContextUnix.cpp index ac2a4b5fbe..ec26e437ce 100644 --- a/src/kernel/context/ContextUnix.cpp +++ b/src/kernel/context/ContextUnix.cpp @@ -7,12 +7,11 @@ #include /* context relative declarations */ -#include "src/simix/ActorImpl.hpp" -#include "src/simix/smx_private.h" -#include "xbt/parmap.h" #include "mc/mc.h" #include "src/mc/mc_ignore.h" - +#include "src/simix/ActorImpl.hpp" +#include "src/simix/smx_private.h" +#include "xbt/parmap.hpp" /** Many integers are needed to store a pointer * @@ -55,7 +54,7 @@ namespace context { }}} #if HAVE_THREAD_CONTEXTS -static xbt_parmap_t sysv_parmap; +static simgrid::xbt::Parmap* sysv_parmap; static simgrid::kernel::context::ParallelUContext** sysv_workers_context; /* space to save the worker's context in each thread */ static uintptr_t sysv_threads_working; /* number of threads that have started their work */ static xbt_os_thread_key_t sysv_worker_id_key; /* thread-specific storage for the thread id */ @@ -146,8 +145,7 @@ UContextFactory::UContextFactory() : ContextFactory("UContextFactory") UContextFactory::~UContextFactory() { #if HAVE_THREAD_CONTEXTS - if (sysv_parmap) - xbt_parmap_destroy(sysv_parmap); + delete sysv_parmap; xbt_free(sysv_workers_context); #endif } @@ -169,25 +167,24 @@ void UContextFactory::run_all() // with simix_global->context_factory (which might not be initialized // when bootstrapping): if (sysv_parmap == nullptr) - sysv_parmap = xbt_parmap_new( - SIMIX_context_get_nthreads(), SIMIX_context_get_parallel_mode()); - - xbt_parmap_apply(sysv_parmap, - [](void* arg) { - smx_actor_t process = (smx_actor_t) arg; - ParallelUContext* context = static_cast(process->context); - context->resume(); - }, - simix_global->process_to_run); + sysv_parmap = + new simgrid::xbt::Parmap(SIMIX_context_get_nthreads(), SIMIX_context_get_parallel_mode()); + + sysv_parmap->apply( + [](smx_actor_t process) { + ParallelUContext* context = static_cast(process->context); + context->resume(); + }, + simix_global->process_to_run); #else xbt_die("You asked for a parallel execution, but you don't have any threads."); #endif } else { // Serial: - if (xbt_dynar_is_empty(simix_global->process_to_run)) + if (simix_global->process_to_run.empty()) return; - smx_actor_t first_process = xbt_dynar_get_as(simix_global->process_to_run, 0, smx_actor_t); + smx_actor_t first_process = simix_global->process_to_run.front(); sysv_process_index = 1; SerialUContext* context = static_cast(first_process->context); context->resume(); @@ -270,15 +267,14 @@ void SerialUContext::suspend() SerialUContext* next_context = nullptr; unsigned long int i = sysv_process_index++; - if (i < xbt_dynar_length(simix_global->process_to_run)) { + if (i < simix_global->process_to_run.size()) { /* execute the next process */ XBT_DEBUG("Run next process"); - next_context = (SerialUContext*) xbt_dynar_get_as( - simix_global->process_to_run,i, smx_actor_t)->context; + next_context = static_cast(simix_global->process_to_run[i]->context); } else { /* all processes were run, return to maestro */ XBT_DEBUG("No more process to run"); - next_context = (SerialUContext*) sysv_maestro_context; + next_context = static_cast(sysv_maestro_context); } SIMIX_context_set_current(next_context); swapcontext(&this->uc_, &next_context->uc_); @@ -289,7 +285,7 @@ void SerialUContext::suspend() void SerialUContext::resume() { SIMIX_context_set_current(this); - swapcontext(&((SerialUContext*)sysv_maestro_context)->uc_, &this->uc_); + swapcontext(&static_cast(sysv_maestro_context)->uc_, &this->uc_); } void ParallelUContext::stop() @@ -307,11 +303,11 @@ void ParallelUContext::resume() // Store the number of my containing body in os-thread-specific area : xbt_os_thread_set_specific(sysv_worker_id_key, (void*) worker_id); // Get my current soul: - ParallelUContext* worker_context = (ParallelUContext*) SIMIX_context_self(); + ParallelUContext* worker_context = static_cast(SIMIX_context_self()); // Write down that this soul is hosted in that body (for now) sysv_workers_context[worker_id] = worker_context; // Retrieve the system-level info that fuels this soul: - ucontext_t* worker_stack = &((ParallelUContext*) worker_context)->uc_; + ucontext_t* worker_stack = &worker_context->uc_; // Write in simix that I switched my soul SIMIX_context_set_current(this); // Actually do that using the relevant library call: @@ -345,15 +341,12 @@ void ParallelUContext::suspend() #if HAVE_THREAD_CONTEXTS /* determine the next context */ // Get the next soul to embody now: - smx_actor_t next_work = (smx_actor_t) xbt_parmap_next(sysv_parmap); - ParallelUContext* next_context = nullptr; - // Will contain the next soul to run, either simulated or initial minion's one - ucontext_t* next_stack; - - if (next_work != nullptr) { + boost::optional next_work = sysv_parmap->next(); + ParallelUContext* next_context; + if (next_work) { // There is a next soul to embody (ie, a next process to resume) XBT_DEBUG("Run next process"); - next_context = (ParallelUContext*) next_work->context; + next_context = static_cast(next_work.get()->context); } else { // All processes were run, go to the barrier XBT_DEBUG("No more processes to run"); @@ -362,11 +355,12 @@ void ParallelUContext::suspend() uintptr_t worker_id = (uintptr_t) xbt_os_thread_get_specific(sysv_worker_id_key); // Deduce the initial soul of that body - next_context = (ParallelUContext*) sysv_workers_context[worker_id]; + next_context = sysv_workers_context[worker_id]; // When given that soul, the body will wait for the next scheduling round } - next_stack = &next_context->uc_; + // Will contain the next soul to run, either simulated or initial minion's one + ucontext_t* next_stack = &next_context->uc_; SIMIX_context_set_current(next_context); // Get that next soul: diff --git a/src/kernel/routing/ClusterZone.cpp b/src/kernel/routing/ClusterZone.cpp index 52f8bb1d8c..296f3f8e05 100644 --- a/src/kernel/routing/ClusterZone.cpp +++ b/src/kernel/routing/ClusterZone.cpp @@ -16,7 +16,7 @@ XBT_LOG_NEW_DEFAULT_SUBCATEGORY(surf_route_cluster, surf, "Routing part of surf" namespace simgrid { namespace kernel { namespace routing { -ClusterZone::ClusterZone(NetZone* father, const char* name) : NetZoneImpl(father, name) +ClusterZone::ClusterZone(NetZone* father, std::string name) : NetZoneImpl(father, name) { } @@ -119,9 +119,9 @@ void ClusterZone::getGraph(xbt_graph_t graph, xbt_dict_t nodes, xbt_dict_t edges } } -void ClusterZone::create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id, int /*rank*/, int position) +void ClusterZone::create_links_for_node(ClusterCreationArgs* cluster, int id, int /*rank*/, int position) { - char* link_id = bprintf("%s_link_%d", cluster->id, id); + std::string link_id = cluster->id + "_link_" + std::to_string(id); LinkCreationArgs link; link.id = link_id; @@ -133,17 +133,12 @@ void ClusterZone::create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id surf::LinkImpl *linkUp; surf::LinkImpl *linkDown; if (link.policy == SURF_LINK_FULLDUPLEX) { - char* tmp_link = bprintf("%s_UP", link_id); - linkUp = surf::LinkImpl::byName(tmp_link); - xbt_free(tmp_link); - tmp_link = bprintf("%s_DOWN", link_id); - linkDown = surf::LinkImpl::byName(tmp_link); - xbt_free(tmp_link); + linkUp = surf::LinkImpl::byName(link_id + "_UP"); + linkDown = surf::LinkImpl::byName(link_id + "_DOWN"); } else { linkUp = surf::LinkImpl::byName(link_id); linkDown = linkUp; } - xbt_free(link_id); privateLinks_.insert({position, {linkUp, linkDown}}); } } diff --git a/src/kernel/routing/ClusterZone.hpp b/src/kernel/routing/ClusterZone.hpp index 7a347a86af..9fe165366d 100644 --- a/src/kernel/routing/ClusterZone.hpp +++ b/src/kernel/routing/ClusterZone.hpp @@ -67,13 +67,13 @@ namespace routing { class XBT_PRIVATE ClusterZone : public NetZoneImpl { public: - explicit ClusterZone(NetZone* father, const char* name); + explicit ClusterZone(NetZone* father, std::string name); void getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cbarg_t into, double* latency) override; void getGraph(xbt_graph_t graph, xbt_dict_t nodes, xbt_dict_t edges) override; - virtual void create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id, int rank, int position); - virtual void parse_specific_arguments(sg_platf_cluster_cbarg_t cluster) + virtual void create_links_for_node(ClusterCreationArgs* cluster, int id, int rank, int position); + virtual void parse_specific_arguments(ClusterCreationArgs* cluster) { /* this routing method does not require any specific argument */ } diff --git a/src/kernel/routing/DijkstraZone.cpp b/src/kernel/routing/DijkstraZone.cpp index b5c53135c5..a76b6178f4 100644 --- a/src/kernel/routing/DijkstraZone.cpp +++ b/src/kernel/routing/DijkstraZone.cpp @@ -307,7 +307,7 @@ DijkstraZone::~DijkstraZone() /* Creation routing model functions */ -DijkstraZone::DijkstraZone(NetZone* father, const char* name, bool cached) : RoutedZone(father, name) +DijkstraZone::DijkstraZone(NetZone* father, std::string name, bool cached) : RoutedZone(father, name) { if (cached) routeCache_ = xbt_dict_new_homogeneous(&route_cache_elem_free); diff --git a/src/kernel/routing/DijkstraZone.hpp b/src/kernel/routing/DijkstraZone.hpp index 2e089ede3c..29e8b90c79 100644 --- a/src/kernel/routing/DijkstraZone.hpp +++ b/src/kernel/routing/DijkstraZone.hpp @@ -43,7 +43,7 @@ namespace routing { */ class XBT_PRIVATE DijkstraZone : public RoutedZone { public: - DijkstraZone(NetZone* father, const char* name, bool cached); + DijkstraZone(NetZone* father, std::string name, bool cached); void seal() override; ~DijkstraZone() override; diff --git a/src/kernel/routing/DragonflyZone.cpp b/src/kernel/routing/DragonflyZone.cpp index 4975caf85c..2b76d0b7d1 100644 --- a/src/kernel/routing/DragonflyZone.cpp +++ b/src/kernel/routing/DragonflyZone.cpp @@ -17,7 +17,7 @@ namespace simgrid { namespace kernel { namespace routing { -DragonflyZone::DragonflyZone(NetZone* father, const char* name) : ClusterZone(father, name) +DragonflyZone::DragonflyZone(NetZone* father, std::string name) : ClusterZone(father, name) { } @@ -41,7 +41,7 @@ void DragonflyZone::rankId_to_coords(int rankId, unsigned int (*coords)[4]) (*coords)[3] = rankId % numNodesPerBlade_; } -void DragonflyZone::parse_specific_arguments(sg_platf_cluster_cbarg_t cluster) +void DragonflyZone::parse_specific_arguments(ClusterCreationArgs* cluster) { std::vector parameters; std::vector tmp; @@ -176,13 +176,13 @@ void DragonflyZone::createLink(std::string id, int numlinks, surf::LinkImpl** li std::string tmpID; if (this->cluster_->sharing_policy == SURF_LINK_FULLDUPLEX) { tmpID = linkTemplate.id + "_UP"; - link = surf::LinkImpl::byName(tmpID.c_str()); + link = surf::LinkImpl::byName(tmpID); *linkup = link; // check link? tmpID = linkTemplate.id + "_DOWN"; - link = surf::LinkImpl::byName(tmpID.c_str()); + link = surf::LinkImpl::byName(tmpID); *linkdown = link; // check link ? } else { - link = surf::LinkImpl::byName(linkTemplate.id.c_str()); + link = surf::LinkImpl::byName(linkTemplate.id); *linkup = link; *linkdown = link; } diff --git a/src/kernel/routing/DragonflyZone.hpp b/src/kernel/routing/DragonflyZone.hpp index b974803eaf..ceeeda348f 100644 --- a/src/kernel/routing/DragonflyZone.hpp +++ b/src/kernel/routing/DragonflyZone.hpp @@ -60,11 +60,11 @@ public: */ class XBT_PRIVATE DragonflyZone : public ClusterZone { public: - explicit DragonflyZone(NetZone* father, const char* name); + explicit DragonflyZone(NetZone* father, std::string name); ~DragonflyZone() override; // void create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id, int rank, int position) override; void getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cbarg_t into, double* latency) override; - void parse_specific_arguments(sg_platf_cluster_cbarg_t cluster) override; + void parse_specific_arguments(ClusterCreationArgs* cluster) override; void seal() override; void generateRouters(); void generateLinks(); @@ -72,7 +72,7 @@ public: private: void rankId_to_coords(int rankId, unsigned int (*coords)[4]); - sg_platf_cluster_cbarg_t cluster_; + ClusterCreationArgs* cluster_; unsigned int numNodesPerBlade_ = 0; unsigned int numBladesPerChassis_ = 0; unsigned int numChassisPerGroup_ = 0; diff --git a/src/kernel/routing/EmptyZone.cpp b/src/kernel/routing/EmptyZone.cpp index 732cab82c1..3ef273611c 100644 --- a/src/kernel/routing/EmptyZone.cpp +++ b/src/kernel/routing/EmptyZone.cpp @@ -15,7 +15,7 @@ namespace simgrid { namespace kernel { namespace routing { -EmptyZone::EmptyZone(NetZone* father, const char* name) : NetZoneImpl(father, name) +EmptyZone::EmptyZone(NetZone* father, std::string name) : NetZoneImpl(father, name) { } diff --git a/src/kernel/routing/EmptyZone.hpp b/src/kernel/routing/EmptyZone.hpp index a63d75f5b2..888d744e15 100644 --- a/src/kernel/routing/EmptyZone.hpp +++ b/src/kernel/routing/EmptyZone.hpp @@ -21,7 +21,7 @@ namespace routing { class XBT_PRIVATE EmptyZone : public NetZoneImpl { public: - explicit EmptyZone(NetZone* father, const char* name); + explicit EmptyZone(NetZone* father, std::string name); ~EmptyZone() override; void getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cbarg_t into, double* latency) override diff --git a/src/kernel/routing/FatTreeZone.cpp b/src/kernel/routing/FatTreeZone.cpp index 49f8b8bfce..c6485c8540 100644 --- a/src/kernel/routing/FatTreeZone.cpp +++ b/src/kernel/routing/FatTreeZone.cpp @@ -20,7 +20,7 @@ namespace simgrid { namespace kernel { namespace routing { -FatTreeZone::FatTreeZone(NetZone* father, const char* name) : ClusterZone(father, name) +FatTreeZone::FatTreeZone(NetZone* father, std::string name) : ClusterZone(father, name) { XBT_DEBUG("Creating a new fat tree."); } @@ -241,9 +241,9 @@ void FatTreeZone::generateSwitches() this->nodesByLevel_[0] *= this->lowerLevelNodesNumber_[i]; if (this->nodesByLevel_[0] != this->nodes_.size()) { - surf_parse_error("The number of provided nodes does not fit with the wanted topology." - " Please check your platform description (We need %u nodes, we got %zu)", - this->nodesByLevel_[0], this->nodes_.size()); + surf_parse_error(std::string("The number of provided nodes does not fit with the wanted topology.") + + " Please check your platform description (We need " + std::to_string(this->nodesByLevel_[0]) + + "nodes, we got " + std::to_string(this->nodes_.size())); return; } @@ -356,7 +356,7 @@ void FatTreeZone::addLink(FatTreeNode* parent, unsigned int parentPort, FatTreeN this->links_.push_back(newLink); } -void FatTreeZone::parse_specific_arguments(sg_platf_cluster_cbarg_t cluster) +void FatTreeZone::parse_specific_arguments(ClusterCreationArgs* cluster) { std::vector parameters; std::vector tmp; @@ -441,7 +441,7 @@ void FatTreeZone::generateDotFile(const std::string& filename) const file.close(); } -FatTreeNode::FatTreeNode(sg_platf_cluster_cbarg_t cluster, int id, int level, int position) +FatTreeNode::FatTreeNode(ClusterCreationArgs* cluster, int id, int level, int position) : id(id), level(level), position(position) { LinkCreationArgs linkTemplate; @@ -451,7 +451,7 @@ FatTreeNode::FatTreeNode(sg_platf_cluster_cbarg_t cluster, int id, int level, in linkTemplate.policy = SURF_LINK_SHARED; linkTemplate.id = "limiter_"+std::to_string(id); sg_platf_new_link(&linkTemplate); - this->limiterLink = surf::LinkImpl::byName(linkTemplate.id.c_str()); + this->limiterLink = surf::LinkImpl::byName(linkTemplate.id); } if (cluster->loopback_bw || cluster->loopback_lat) { linkTemplate.bandwidth = cluster->loopback_bw; @@ -459,11 +459,11 @@ FatTreeNode::FatTreeNode(sg_platf_cluster_cbarg_t cluster, int id, int level, in linkTemplate.policy = SURF_LINK_FATPIPE; linkTemplate.id = "loopback_"+ std::to_string(id); sg_platf_new_link(&linkTemplate); - this->loopback = surf::LinkImpl::byName(linkTemplate.id.c_str()); + this->loopback = surf::LinkImpl::byName(linkTemplate.id); } } -FatTreeLink::FatTreeLink(sg_platf_cluster_cbarg_t cluster, FatTreeNode* downNode, FatTreeNode* upNode) +FatTreeLink::FatTreeLink(ClusterCreationArgs* cluster, FatTreeNode* downNode, FatTreeNode* upNode) : upNode(upNode), downNode(downNode) { static int uniqueId = 0; @@ -477,11 +477,11 @@ FatTreeLink::FatTreeLink(sg_platf_cluster_cbarg_t cluster, FatTreeNode* downNode if (cluster->sharing_policy == SURF_LINK_FULLDUPLEX) { std::string tmpID = std::string(linkTemplate.id) + "_UP"; - this->upLink = surf::LinkImpl::byName(tmpID.c_str()); // check link? + this->upLink = surf::LinkImpl::byName(tmpID); // check link? tmpID = std::string(linkTemplate.id) + "_DOWN"; - this->downLink = surf::LinkImpl::byName(tmpID.c_str()); // check link ? + this->downLink = surf::LinkImpl::byName(tmpID); // check link ? } else { - this->upLink = surf::LinkImpl::byName(linkTemplate.id.c_str()); + this->upLink = surf::LinkImpl::byName(linkTemplate.id); this->downLink = this->upLink; } uniqueId++; diff --git a/src/kernel/routing/FatTreeZone.hpp b/src/kernel/routing/FatTreeZone.hpp index 40b439aa96..c08e69eff5 100644 --- a/src/kernel/routing/FatTreeZone.hpp +++ b/src/kernel/routing/FatTreeZone.hpp @@ -51,7 +51,7 @@ public: * instead of passing by an upper level switch. */ surf::LinkImpl* loopback; - FatTreeNode(sg_platf_cluster_cbarg_t cluster, int id, int level, int position); + FatTreeNode(ClusterCreationArgs* cluster, int id, int level, int position); }; /** \brief Link in a fat tree (@ref FatTreeZone). @@ -61,7 +61,7 @@ public: */ class FatTreeLink { public: - FatTreeLink(sg_platf_cluster_cbarg_t cluster, FatTreeNode* source, FatTreeNode* destination); + FatTreeLink(ClusterCreationArgs* cluster, FatTreeNode* source, FatTreeNode* destination); /** Link going up in the tree */ surf::LinkImpl* upLink; /** Link going down in the tree */ @@ -98,7 +98,7 @@ public: */ class XBT_PRIVATE FatTreeZone : public ClusterZone { public: - explicit FatTreeZone(NetZone* father, const char* name); + explicit FatTreeZone(NetZone* father, std::string name); ~FatTreeZone() override; void getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cbarg_t into, double* latency) override; @@ -113,7 +113,7 @@ public: * * It will also store the cluster for future use. */ - void parse_specific_arguments(sg_platf_cluster_cbarg_t cluster) override; + void parse_specific_arguments(ClusterCreationArgs* cluster) override; void addProcessingNode(int id); void generateDotFile(const std::string& filename = "fatTree.dot") const; @@ -129,7 +129,7 @@ private: std::vector links_; std::vector nodesByLevel_; - sg_platf_cluster_cbarg_t cluster_ = nullptr; + ClusterCreationArgs* cluster_ = nullptr; void addLink(FatTreeNode* parent, unsigned int parentPort, FatTreeNode* child, unsigned int childPort); int getLevelPosition(const unsigned int level); diff --git a/src/kernel/routing/FloydZone.cpp b/src/kernel/routing/FloydZone.cpp index 863c876583..329eff905f 100644 --- a/src/kernel/routing/FloydZone.cpp +++ b/src/kernel/routing/FloydZone.cpp @@ -21,7 +21,7 @@ namespace simgrid { namespace kernel { namespace routing { -FloydZone::FloydZone(NetZone* father, const char* name) : RoutedZone(father, name) +FloydZone::FloydZone(NetZone* father, std::string name) : RoutedZone(father, name) { predecessorTable_ = nullptr; costTable_ = nullptr; diff --git a/src/kernel/routing/FloydZone.hpp b/src/kernel/routing/FloydZone.hpp index f1cb156936..a612425422 100644 --- a/src/kernel/routing/FloydZone.hpp +++ b/src/kernel/routing/FloydZone.hpp @@ -23,7 +23,7 @@ namespace routing { */ class XBT_PRIVATE FloydZone : public RoutedZone { public: - explicit FloydZone(NetZone* father, const char* name); + explicit FloydZone(NetZone* father, std::string name); ~FloydZone() override; void getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cbarg_t into, double* latency) override; diff --git a/src/kernel/routing/FullZone.cpp b/src/kernel/routing/FullZone.cpp index 9bae53937e..dbb5a80023 100644 --- a/src/kernel/routing/FullZone.cpp +++ b/src/kernel/routing/FullZone.cpp @@ -14,7 +14,7 @@ XBT_LOG_NEW_DEFAULT_SUBCATEGORY(surf_route_full, surf, "Routing part of surf"); namespace simgrid { namespace kernel { namespace routing { -FullZone::FullZone(NetZone* father, const char* name) : RoutedZone(father, name) +FullZone::FullZone(NetZone* father, std::string name) : RoutedZone(father, name) { } diff --git a/src/kernel/routing/FullZone.hpp b/src/kernel/routing/FullZone.hpp index f13764ffa8..ab13bc5bb4 100644 --- a/src/kernel/routing/FullZone.hpp +++ b/src/kernel/routing/FullZone.hpp @@ -15,13 +15,12 @@ namespace routing { /** @ingroup ROUTING_API * @brief NetZone with an explicit routing provided by the user * - * The full communication matrix is provided at creation, so this model - * has the highest expressive power and the lowest computational requirements, - * but also the highest memory requirements (both in platform file and in memory). + * The full communication matrix is provided at creation, so this model has the highest expressive power and the lowest + * computational requirements, but also the highest memory requirements (both in platform file and in memory). */ class XBT_PRIVATE FullZone : public RoutedZone { public: - explicit FullZone(NetZone* father, const char* name); + explicit FullZone(NetZone* father, std::string name); void seal() override; ~FullZone() override; diff --git a/src/kernel/routing/NetZoneImpl.cpp b/src/kernel/routing/NetZoneImpl.cpp index 5f846b1017..e128d57a34 100644 --- a/src/kernel/routing/NetZoneImpl.cpp +++ b/src/kernel/routing/NetZoneImpl.cpp @@ -26,14 +26,15 @@ public: std::vector links; }; -NetZoneImpl::NetZoneImpl(NetZone* father, const char* name) : NetZone(father, name) +NetZoneImpl::NetZoneImpl(NetZone* father, std::string name) : NetZone(father, name) { - xbt_assert(nullptr == simgrid::s4u::Engine::getInstance()->getNetpointByNameOrNull(name), - "Refusing to create a second NetZone called '%s'.", name); + xbt_assert(nullptr == simgrid::s4u::Engine::getInstance()->getNetpointByNameOrNull(name.c_str()), + "Refusing to create a second NetZone called '%s'.", name.c_str()); netpoint_ = new NetPoint(name, NetPoint::Type::NetZone, static_cast(father)); - XBT_DEBUG("NetZone '%s' created with the id '%u'", name, netpoint_->id()); + XBT_DEBUG("NetZone '%s' created with the id '%u'", name.c_str(), netpoint_->id()); } + NetZoneImpl::~NetZoneImpl() { for (auto& kv : bypassRoutes_) @@ -260,20 +261,18 @@ bool NetZoneImpl::getBypassRoute(routing::NetPoint* src, routing::NetPoint* dst, for (int i = 0; i < max; i++) { if (i <= max_index_src && max <= max_index_dst) { key = {path_src.at(i)->netpoint_, path_dst.at(max)->netpoint_}; - try { - bypassedRoute = bypassRoutes_.at(key); + auto bpr = bypassRoutes_.find(key); + if (bpr != bypassRoutes_.end()) { + bypassedRoute = bpr->second; break; - } catch (std::out_of_range& unfound) { - // Do nothing } } if (max <= max_index_src && i <= max_index_dst) { key = {path_src.at(max)->netpoint_, path_dst.at(i)->netpoint_}; - try { - bypassedRoute = bypassRoutes_.at(key); + auto bpr = bypassRoutes_.find(key); + if (bpr != bypassRoutes_.end()) { + bypassedRoute = bpr->second; break; - } catch (std::out_of_range& unfound) { - // Do nothing } } } @@ -283,11 +282,10 @@ bool NetZoneImpl::getBypassRoute(routing::NetPoint* src, routing::NetPoint* dst, if (max <= max_index_src && max <= max_index_dst) { key = {path_src.at(max)->netpoint_, path_dst.at(max)->netpoint_}; - try { - bypassedRoute = bypassRoutes_.at(key); + auto bpr = bypassRoutes_.find(key); + if (bpr != bypassRoutes_.end()) { + bypassedRoute = bpr->second; break; - } catch (std::out_of_range& unfound) { - // Do nothing } } } diff --git a/src/kernel/routing/NetZoneImpl.hpp b/src/kernel/routing/NetZoneImpl.hpp index e4a297caf7..98296b511b 100644 --- a/src/kernel/routing/NetZoneImpl.hpp +++ b/src/kernel/routing/NetZoneImpl.hpp @@ -53,7 +53,7 @@ XBT_PUBLIC_CLASS NetZoneImpl : public s4u::NetZone friend simgrid::kernel::EngineImpl; // it destroys netRoot_ protected: - explicit NetZoneImpl(NetZone * father, const char* name); + explicit NetZoneImpl(NetZone * father, std::string name); virtual ~NetZoneImpl(); public: diff --git a/src/kernel/routing/RoutedZone.cpp b/src/kernel/routing/RoutedZone.cpp index 2647781a29..cd223f1594 100644 --- a/src/kernel/routing/RoutedZone.cpp +++ b/src/kernel/routing/RoutedZone.cpp @@ -65,7 +65,7 @@ namespace simgrid { namespace kernel { namespace routing { -RoutedZone::RoutedZone(NetZone* father, const char* name) : NetZoneImpl(father, name) +RoutedZone::RoutedZone(NetZone* father, std::string name) : NetZoneImpl(father, name) { } diff --git a/src/kernel/routing/RoutedZone.hpp b/src/kernel/routing/RoutedZone.hpp index 25be9b45e5..3904232c22 100644 --- a/src/kernel/routing/RoutedZone.hpp +++ b/src/kernel/routing/RoutedZone.hpp @@ -50,7 +50,7 @@ namespace routing { class XBT_PRIVATE RoutedZone : public NetZoneImpl { public: - explicit RoutedZone(NetZone* father, const char* name); + explicit RoutedZone(NetZone* father, std::string name); void getGraph(xbt_graph_t graph, xbt_dict_t nodes, xbt_dict_t edges) override; virtual sg_platf_route_cbarg_t newExtendedRoute(RoutingMode hierarchy, sg_platf_route_cbarg_t routearg, diff --git a/src/kernel/routing/TorusZone.cpp b/src/kernel/routing/TorusZone.cpp index 399ac2e4dc..945fa9eb8e 100644 --- a/src/kernel/routing/TorusZone.cpp +++ b/src/kernel/routing/TorusZone.cpp @@ -27,11 +27,11 @@ inline void rankId_to_coords(int rankId, std::vector dimensions, u namespace simgrid { namespace kernel { namespace routing { -TorusZone::TorusZone(NetZone* father, const char* name) : ClusterZone(father, name) +TorusZone::TorusZone(NetZone* father, std::string name) : ClusterZone(father, name) { } -void TorusZone::create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id, int rank, int position) +void TorusZone::create_links_for_node(ClusterCreationArgs* cluster, int id, int rank, int position) { /* Create all links that exist in the torus. Each rank creates @a dimensions-1 links */ int dim_product = 1; // Needed to calculate the next neighbor_id @@ -45,7 +45,8 @@ void TorusZone::create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id, ? rank - (current_dimension - 1) * dim_product : rank + dim_product; // name of neighbor is not right for non contiguous cluster radicals (as id != rank in this case) - char* link_id = bprintf("%s_link_from_%i_to_%i", cluster->id, id, neighbor_rank_id); + std::string link_id = + std::string(cluster->id) + "_link_from_" + std::to_string(id) + "_to_" + std::to_string(neighbor_rank_id); link.id = link_id; link.bandwidth = cluster->bw; link.latency = cluster->lat; @@ -54,12 +55,10 @@ void TorusZone::create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id, surf::LinkImpl* linkUp; surf::LinkImpl* linkDown; if (link.policy == SURF_LINK_FULLDUPLEX) { - char* tmp_link = bprintf("%s_UP", link_id); + std::string tmp_link = link_id + "_UP"; linkUp = surf::LinkImpl::byName(tmp_link); - free(tmp_link); - tmp_link = bprintf("%s_DOWN", link_id); + tmp_link = link_id + "_DOWN"; linkDown = surf::LinkImpl::byName(tmp_link); - free(tmp_link); } else { linkUp = surf::LinkImpl::byName(link_id); linkDown = linkUp; @@ -71,24 +70,22 @@ void TorusZone::create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id, */ privateLinks_.insert({position + j, {linkUp, linkDown}}); dim_product *= current_dimension; - xbt_free(link_id); } rank++; } -void TorusZone::parse_specific_arguments(sg_platf_cluster_cbarg_t cluster) +void TorusZone::parse_specific_arguments(ClusterCreationArgs* cluster) { std::vector dimensions; boost::split(dimensions, cluster->topo_parameters, boost::is_any_of(",")); if (not dimensions.empty()) { /* We are in a torus cluster - * Parse attribute dimensions="dim1,dim2,dim3,...,dimN" and safe it in a vector. + * Parse attribute dimensions="dim1,dim2,dim3,...,dimN" and save them into a vector. * Additionally, we need to know how many ranks we have in total */ - for (auto group : dimensions) { - dimensions_.push_back(surf_parse_get_int(group.c_str())); - } + for (auto group : dimensions) + dimensions_.push_back(surf_parse_get_int(group)); linkCountPerNode_ = dimensions_.size(); } @@ -119,21 +116,16 @@ void TorusZone::getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cbarg unsigned int current_node = src->id(); unsigned int next_node = 0; /* - * Arrays that hold the coordinates of the current node and - * the target; comparing the values at the i-th position of - * both arrays, we can easily assess whether we need to route - * into this dimension or not. + * Arrays that hold the coordinates of the current node andthe target; comparing the values at the i-th position of + * both arrays, we can easily assess whether we need to route into this dimension or not. */ unsigned int myCoords[4]; rankId_to_coords(src->id(), dimensions_, &myCoords); unsigned int targetCoords[4]; rankId_to_coords(dst->id(), dimensions_, &targetCoords); /* - * linkOffset describes the offset where the link - * we want to use is stored - * (+1 is added because each node has a link from itself to itself, - * which can only be the case if src->m_id == dst->m_id -- see above - * for this special case) + * linkOffset describes the offset where the link we want to use is stored(+1 is added because each node has a link + * from itself to itself, which can only be the case if src->m_id == dst->m_id -- see above for this special case) */ int nodeOffset = (dimensions_.size() + 1) * src->id(); diff --git a/src/kernel/routing/TorusZone.hpp b/src/kernel/routing/TorusZone.hpp index bf0c845941..56d4c7008c 100644 --- a/src/kernel/routing/TorusZone.hpp +++ b/src/kernel/routing/TorusZone.hpp @@ -20,10 +20,10 @@ namespace routing { class XBT_PRIVATE TorusZone : public ClusterZone { public: - explicit TorusZone(NetZone* father, const char* name); - void create_links_for_node(sg_platf_cluster_cbarg_t cluster, int id, int rank, int position) override; + explicit TorusZone(NetZone* father, std::string name); + void create_links_for_node(ClusterCreationArgs* cluster, int id, int rank, int position) override; void getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cbarg_t into, double* latency) override; - void parse_specific_arguments(sg_platf_cluster_cbarg_t cluster) override; + void parse_specific_arguments(ClusterCreationArgs* cluster) override; private: std::vector dimensions_; diff --git a/src/kernel/routing/VivaldiZone.cpp b/src/kernel/routing/VivaldiZone.cpp index 8ebb7fa996..99017222bb 100644 --- a/src/kernel/routing/VivaldiZone.cpp +++ b/src/kernel/routing/VivaldiZone.cpp @@ -57,7 +57,8 @@ static std::vector* getCoordsFromNetpoint(NetPoint* np) (np->isNetZone() ? "Netzone" : (np->isHost() ? "Host" : "Router")), np->cname(), np); return &coords->coords; } -VivaldiZone::VivaldiZone(NetZone* father, const char* name) : ClusterZone(father, name) + +VivaldiZone::VivaldiZone(NetZone* father, std::string name) : ClusterZone(father, name) { } @@ -86,25 +87,27 @@ void VivaldiZone::getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cba } /* Retrieve the private links */ - try { - std::pair info = privateLinks_.at(src->id()); + auto src_link = privateLinks_.find(src->id()); + if (src_link != privateLinks_.end()) { + std::pair info = src_link->second; if (info.first) { route->link_list->push_back(info.first); if (lat) *lat += info.first->latency(); } - } catch (std::out_of_range& unfound) { + } else { XBT_DEBUG("Source of private link (%u) doesn't exist", src->id()); } - try { - std::pair info = privateLinks_.at(dst->id()); + auto dst_link = privateLinks_.find(dst->id()); + if (dst_link != privateLinks_.end()) { + std::pair info = dst_link->second; if (info.second) { route->link_list->push_back(info.second); if (lat) *lat += info.second->latency(); } - } catch (std::out_of_range& unfound) { + } else { XBT_DEBUG("Destination of private link (%u) doesn't exist", dst->id()); } diff --git a/src/kernel/routing/VivaldiZone.hpp b/src/kernel/routing/VivaldiZone.hpp index 83fc6b24e1..64ea9bd5f2 100644 --- a/src/kernel/routing/VivaldiZone.hpp +++ b/src/kernel/routing/VivaldiZone.hpp @@ -45,7 +45,7 @@ namespace routing { class XBT_PRIVATE VivaldiZone : public ClusterZone { public: - explicit VivaldiZone(NetZone* father, const char* name); + explicit VivaldiZone(NetZone* father, std::string name); void setPeerLink(NetPoint* netpoint, double bw_in, double bw_out, std::string coord); void getLocalRoute(NetPoint* src, NetPoint* dst, sg_platf_route_cbarg_t into, double* latency) override; diff --git a/src/mc/LocationList.cpp b/src/mc/LocationList.cpp index 0ad937520a..8bb46615c4 100644 --- a/src/mc/LocationList.cpp +++ b/src/mc/LocationList.cpp @@ -1,5 +1,4 @@ -/* Copyright (c) 2004-2015. The SimGrid Team. - * All rights reserved. */ +/* Copyright (c) 2004-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ diff --git a/src/mc/LocationList.hpp b/src/mc/LocationList.hpp index 58b9fe24b8..5fa74b3071 100644 --- a/src/mc/LocationList.hpp +++ b/src/mc/LocationList.hpp @@ -1,5 +1,4 @@ -/* Copyright (c) 2004-2015. The SimGrid Team. - * All rights reserved. */ +/* Copyright (c) 2004-2015. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ @@ -63,7 +62,8 @@ typedef std::vector LocationList; class Location { private: void* memory_; - int register_id_; + int register_id_ = 0; + public: explicit Location(void* x) : memory_(x) {} explicit Location(int register_id) : memory_(nullptr), register_id_(register_id) {} diff --git a/src/mc/mc_base.cpp b/src/mc/mc_base.cpp index f3bc7d96e4..a12f15c1e7 100644 --- a/src/mc/mc_base.cpp +++ b/src/mc/mc_base.cpp @@ -35,13 +35,9 @@ void wait_for_requests() #if SIMGRID_HAVE_MC xbt_assert(mc_model_checker == nullptr, "This must be called from the client"); #endif - - smx_actor_t process; - unsigned int iter; - - while (not xbt_dynar_is_empty(simix_global->process_to_run)) { + while (not simix_global->process_to_run.empty()) { SIMIX_process_runall(); - xbt_dynar_foreach(simix_global->process_that_ran, iter, process) { + for (smx_actor_t process : simix_global->process_that_ran) { smx_simcall_t req = &process->simcall; if (req->call != SIMCALL_NONE && not simgrid::mc::request_is_visible(req)) SIMIX_simcall_handle(req, 0); diff --git a/src/msg/msg_global.cpp b/src/msg/msg_global.cpp index 4a0509d674..f2f926b0bc 100644 --- a/src/msg/msg_global.cpp +++ b/src/msg/msg_global.cpp @@ -113,7 +113,6 @@ static void MSG_exit() { if (msg_global==nullptr) return; - TRACE_surf_resource_utilization_release(); TRACE_end(); delete msg_global; msg_global = nullptr; diff --git a/src/msg/msg_io.cpp b/src/msg/msg_io.cpp index 776adfe90b..3b06983292 100644 --- a/src/msg/msg_io.cpp +++ b/src/msg/msg_io.cpp @@ -73,7 +73,8 @@ void MSG_file_dump (msg_file_t fd){ "\t\tStorage Id: '%s'\n" "\t\tStorage Type: '%s'\n" "\t\tFile Descriptor Id: %d", - fd->getPath(), fd->size(), fd->mount_point.c_str(), fd->storageId, fd->storage_type, fd->desc_id); + fd->getPath(), fd->size(), fd->mount_point.c_str(), fd->storageId.c_str(), fd->storage_type.c_str(), + fd->desc_id); } /** \ingroup msg_file diff --git a/src/msg/msg_process.cpp b/src/msg/msg_process.cpp index 18878b0407..d26778eccf 100644 --- a/src/msg/msg_process.cpp +++ b/src/msg/msg_process.cpp @@ -6,6 +6,7 @@ #include "msg_private.h" #include "simgrid/s4u/Host.hpp" #include "src/simix/ActorImpl.hpp" +#include "src/simix/smx_private.h" XBT_LOG_NEW_DEFAULT_SUBCATEGORY(msg_process, msg, "Logging specific to MSG (process)"); @@ -304,7 +305,12 @@ msg_process_t MSG_process_from_PID(int PID) /** @brief returns a list of all currently existing processes */ xbt_dynar_t MSG_processes_as_dynar() { - return SIMIX_processes_as_dynar(); + xbt_dynar_t res = xbt_dynar_new(sizeof(smx_actor_t), nullptr); + for (auto kv : simix_global->process_list) { + smx_actor_t actor = kv.second; + xbt_dynar_push(res, &actor); + } + return res; } /** @brief Return the current number MSG processes. */ diff --git a/src/msg/msg_vm.cpp b/src/msg/msg_vm.cpp index 99a3c5d15a..1b3829c447 100644 --- a/src/msg/msg_vm.cpp +++ b/src/msg/msg_vm.cpp @@ -184,34 +184,7 @@ void MSG_vm_destroy(msg_vm_t vm) */ void MSG_vm_start(msg_vm_t vm) { - simgrid::simix::kernelImmediate([vm]() { - simgrid::vm::VmHostExt::ensureVmExtInstalled(); - - simgrid::s4u::Host* pm = vm->pimpl_vm_->getPm(); - if (pm->extension() == nullptr) - pm->extension_set(new simgrid::vm::VmHostExt()); - - long pm_ramsize = pm->extension()->ramsize; - int pm_overcommit = pm->extension()->overcommit; - long vm_ramsize = vm->getRamsize(); - - if (pm_ramsize && not pm_overcommit) { /* Only verify that we don't overcommit on need */ - /* Retrieve the memory occupied by the VMs on that host. Yep, we have to traverse all VMs of all hosts for that */ - long total_ramsize_of_vms = 0; - for (simgrid::s4u::VirtualMachine* ws_vm : simgrid::vm::VirtualMachineImpl::allVms_) - if (pm == ws_vm->pimpl_vm_->getPm()) - total_ramsize_of_vms += ws_vm->pimpl_vm_->getRamsize(); - - if (vm_ramsize > pm_ramsize - total_ramsize_of_vms) { - XBT_WARN("cannnot start %s@%s due to memory shortage: vm_ramsize %ld, free %ld, pm_ramsize %ld (bytes).", - vm->getCname(), pm->getCname(), vm_ramsize, pm_ramsize - total_ramsize_of_vms, pm_ramsize); - THROWF(vm_error, 0, "Memory shortage on host '%s', VM '%s' cannot be started", pm->getCname(), vm->getCname()); - } - } - - vm->pimpl_vm_->setState(SURF_VM_STATE_RUNNING); - }); - + vm->start(); if (TRACE_msg_vm_is_enabled()) { container_t vm_container = PJ_container_get(vm->getCname()); type_t type = PJ_type_get("MSG_VM_STATE", vm_container->type); @@ -229,27 +202,27 @@ void MSG_vm_start(msg_vm_t vm) */ void MSG_vm_shutdown(msg_vm_t vm) { - smx_actor_t issuer=SIMIX_process_self(); + smx_actor_t issuer = SIMIX_process_self(); simgrid::simix::kernelImmediate([vm, issuer]() { vm->pimpl_vm_->shutdown(issuer); }); - // Make sure that the processes in the VM are killed in this scheduling round before processing - // (eg with the VM destroy) + // Make sure that processes in the VM are killed in this scheduling round before processing (eg with the VM destroy) MSG_process_sleep(0.); } -static inline char *get_mig_process_tx_name(msg_vm_t vm, msg_host_t src_pm, msg_host_t dst_pm) +static std::string get_mig_process_tx_name(msg_vm_t vm, msg_host_t src_pm, msg_host_t dst_pm) { - return bprintf("__pr_mig_tx:%s(%s-%s)", vm->getCname(), src_pm->getCname(), dst_pm->getCname()); + return std::string("__pr_mig_tx:") + vm->getCname() + "(" + src_pm->getCname() + "-" + dst_pm->getCname() + ")"; } -static inline char *get_mig_process_rx_name(msg_vm_t vm, msg_host_t src_pm, msg_host_t dst_pm) +static std::string get_mig_process_rx_name(msg_vm_t vm, msg_host_t src_pm, msg_host_t dst_pm) { - return bprintf("__pr_mig_rx:%s(%s-%s)", vm->getCname(), src_pm->getCname(), dst_pm->getCname()); + return std::string("__pr_mig_rx:") + vm->getCname() + "(" + src_pm->getCname() + "-" + dst_pm->getCname() + ")"; } -static inline char *get_mig_task_name(msg_vm_t vm, msg_host_t src_pm, msg_host_t dst_pm, int stage) +static std::string get_mig_task_name(msg_vm_t vm, msg_host_t src_pm, msg_host_t dst_pm, int stage) { - return bprintf("__task_mig_stage%d:%s(%s-%s)", stage, vm->getCname(), src_pm->getCname(), dst_pm->getCname()); + return std::string("__task_mig_stage") + std::to_string(stage) + ":" + vm->getCname() + "(" + src_pm->getCname() + + "-" + dst_pm->getCname() + ")"; } struct migration_session { @@ -273,7 +246,7 @@ static int migration_rx_fun(int argc, char *argv[]) bool received_finalize = false; - char *finalize_task_name = get_mig_task_name(ms->vm, ms->src_pm, ms->dst_pm, 3); + std::string finalize_task_name = get_mig_task_name(ms->vm, ms->src_pm, ms->dst_pm, 3); while (not received_finalize) { msg_task_t task = nullptr; int ret = MSG_task_recv(&task, ms->mbox); @@ -281,16 +254,14 @@ static int migration_rx_fun(int argc, char *argv[]) if (ret != MSG_OK) { // An error occurred, clean the code and return // The owner did not change, hence the task should be only destroyed on the other side - xbt_free(finalize_task_name); return 0; } - if (strcmp(task->name, finalize_task_name) == 0) + if (finalize_task_name == task->name) received_finalize = 1; MSG_task_destroy(task); } - xbt_free(finalize_task_name); // Here Stage 1, 2 and 3 have been performed. // Hence complete the migration @@ -343,8 +314,8 @@ static int migration_rx_fun(int argc, char *argv[]) } // Inform the SRC that the migration has been correctly performed - char *task_name = get_mig_task_name(ms->vm, ms->src_pm, ms->dst_pm, 4); - msg_task_t task = MSG_task_create(task_name, 0, 0, nullptr); + std::string task_name = get_mig_task_name(ms->vm, ms->src_pm, ms->dst_pm, 4); + msg_task_t task = MSG_task_create(task_name.c_str(), 0, 0, nullptr); msg_error_t ret = MSG_task_send(task, ms->mbox_ctl); // xbt_assert(ret == MSG_OK); if(ret == MSG_HOST_FAILURE){ @@ -356,7 +327,6 @@ static int migration_rx_fun(int argc, char *argv[]) // The SRC has crashed, this is not a problem has the VM has been correctly migrated on the DST node MSG_task_destroy(task); } - xbt_free(task_name); XBT_DEBUG("mig: rx_done"); return 0; @@ -479,8 +449,8 @@ static sg_size_t send_migration_data(msg_vm_t vm, msg_host_t src_pm, msg_host_t int stage, int stage2_round, double mig_speed, double timeout) { sg_size_t sent = 0; - char *task_name = get_mig_task_name(vm, src_pm, dst_pm, stage); - msg_task_t task = MSG_task_create(task_name, 0, static_cast(size), nullptr); + std::string task_name = get_mig_task_name(vm, src_pm, dst_pm, stage); + msg_task_t task = MSG_task_create(task_name.c_str(), 0, static_cast(size), nullptr); double clock_sta = MSG_get_clock(); @@ -490,8 +460,6 @@ static sg_size_t send_migration_data(msg_vm_t vm, msg_host_t src_pm, msg_host_t else ret = MSG_task_send(task, mbox); - xbt_free(task_name); - if (ret == MSG_OK) { sent = size; } else if (ret == MSG_TIMEOUT) { @@ -752,18 +720,12 @@ void MSG_vm_migrate(msg_vm_t vm, msg_host_t dst_pm) ms->mbox_ctl = bprintf("__mbox_mig_ctl:%s(%s-%s)", vm->getCname(), src_pm->getCname(), dst_pm->getCname()); ms->mbox = bprintf("__mbox_mig_src_dst:%s(%s-%s)", vm->getCname(), src_pm->getCname(), dst_pm->getCname()); - char *pr_rx_name = get_mig_process_rx_name(vm, src_pm, dst_pm); - char *pr_tx_name = get_mig_process_tx_name(vm, src_pm, dst_pm); + std::string pr_rx_name = get_mig_process_rx_name(vm, src_pm, dst_pm); + std::string pr_tx_name = get_mig_process_tx_name(vm, src_pm, dst_pm); - char** argv = xbt_new(char*, 2); - argv[0] = pr_rx_name; - argv[1] = nullptr; - MSG_process_create_with_arguments(pr_rx_name, migration_rx_fun, ms, dst_pm, 1, argv); + MSG_process_create(pr_rx_name.c_str(), migration_rx_fun, ms, dst_pm); - argv = xbt_new(char*, 2); - argv[0] = pr_tx_name; - argv[1] = nullptr; - MSG_process_create_with_arguments(pr_tx_name, migration_tx_fun, ms, src_pm, 1, argv); + MSG_process_create(pr_tx_name.c_str(), migration_tx_fun, ms, src_pm); /* wait until the migration have finished or on error has occurred */ XBT_DEBUG("wait for reception of the final ACK (i.e. migration has been correctly performed"); @@ -791,9 +753,7 @@ void MSG_vm_migrate(msg_vm_t vm, msg_host_t dst_pm) vm->getCname()); } - char* expected_task_name = get_mig_task_name(vm, src_pm, dst_pm, 4); - xbt_assert(strcmp(task->name, expected_task_name) == 0); - xbt_free(expected_task_name); + xbt_assert(get_mig_task_name(vm, src_pm, dst_pm, 4) == task->name); MSG_task_destroy(task); } diff --git a/src/plugins/vm/s4u_VirtualMachine.cpp b/src/plugins/vm/s4u_VirtualMachine.cpp index c874d48283..7c4f791459 100644 --- a/src/plugins/vm/s4u_VirtualMachine.cpp +++ b/src/plugins/vm/s4u_VirtualMachine.cpp @@ -5,6 +5,7 @@ #include "src/instr/instr_private.h" #include "src/plugins/vm/VirtualMachineImpl.hpp" +#include "src/plugins/vm/VmHostExt.hpp" #include "src/simix/smx_host_private.h" #include "src/surf/cpu_cas01.hpp" @@ -54,6 +55,38 @@ VirtualMachine::~VirtualMachine() pimpl_netpoint = nullptr; } +void VirtualMachine::start() +{ + simgrid::simix::kernelImmediate([this]() { + simgrid::vm::VmHostExt::ensureVmExtInstalled(); + + simgrid::s4u::Host* pm = this->pimpl_vm_->getPm(); + if (pm->extension() == nullptr) + pm->extension_set(new simgrid::vm::VmHostExt()); + + long pm_ramsize = pm->extension()->ramsize; + int pm_overcommit = pm->extension()->overcommit; + long vm_ramsize = this->getRamsize(); + + if (pm_ramsize && not pm_overcommit) { /* Only verify that we don't overcommit on need */ + /* Retrieve the memory occupied by the VMs on that host. Yep, we have to traverse all VMs of all hosts for that */ + long total_ramsize_of_vms = 0; + for (simgrid::s4u::VirtualMachine* ws_vm : simgrid::vm::VirtualMachineImpl::allVms_) + if (pm == ws_vm->pimpl_vm_->getPm()) + total_ramsize_of_vms += ws_vm->pimpl_vm_->getRamsize(); + + if (vm_ramsize > pm_ramsize - total_ramsize_of_vms) { + XBT_WARN("cannnot start %s@%s due to memory shortage: vm_ramsize %ld, free %ld, pm_ramsize %ld (bytes).", + this->getCname(), pm->getCname(), vm_ramsize, pm_ramsize - total_ramsize_of_vms, pm_ramsize); + THROWF(vm_error, 0, "Memory shortage on host '%s', VM '%s' cannot be started", pm->getCname(), + this->getCname()); + } + } + + this->pimpl_vm_->setState(SURF_VM_STATE_RUNNING); + }); +} + bool VirtualMachine::isMigrating() { return pimpl_vm_ && pimpl_vm_->isMigrating; @@ -80,7 +113,7 @@ void VirtualMachine::getParameters(vm_params_t params) /** @brief Sets the params of that VM/PM */ void VirtualMachine::setParameters(vm_params_t params) { - simgrid::simix::kernelImmediate([&]() { pimpl_vm_->setParams(params); }); + simgrid::simix::kernelImmediate([this, params] { pimpl_vm_->setParams(params); }); } } // namespace simgrid diff --git a/src/s4u/s4u_actor.cpp b/src/s4u/s4u_actor.cpp index 7dcef8fee5..f3f5ab06bf 100644 --- a/src/s4u/s4u_actor.cpp +++ b/src/s4u/s4u_actor.cpp @@ -134,7 +134,7 @@ void Actor::kill(aid_t pid) { smx_actor_t process = SIMIX_process_from_PID(pid); if(process != nullptr) { - simcall_process_kill(process); + simgrid::simix::kernelImmediate([process] { SIMIX_process_kill(process, process); }); } else { std::ostringstream oss; oss << "kill: ("<< pid <<") - No such process" << std::endl; @@ -147,7 +147,9 @@ smx_actor_t Actor::getImpl() { } void Actor::kill() { - simcall_process_kill(pimpl_); + smx_actor_t process = SIMIX_process_self(); + simgrid::simix::kernelImmediate( + [this, process] { SIMIX_process_kill(pimpl_, (pimpl_ == simix_global->maestro_process) ? pimpl_ : process); }); } // ***** Static functions ***** @@ -294,7 +296,8 @@ bool isSuspended() void kill() { - simcall_process_kill(SIMIX_process_self()); + smx_actor_t process = SIMIX_process_self(); + simgrid::simix::kernelImmediate([process] { SIMIX_process_kill(process, process); }); } void onExit(int_f_pvoid_pvoid_t fun, void* data) diff --git a/src/s4u/s4u_engine.cpp b/src/s4u/s4u_engine.cpp index e061713a9c..50d287e334 100644 --- a/src/s4u/s4u_engine.cpp +++ b/src/s4u/s4u_engine.cpp @@ -126,14 +126,12 @@ NetZone* Engine::getNetzoneByNameOrNull(const char* name) } /** @brief Retrieve the netpoint of the given name (or nullptr if not found) */ -simgrid::kernel::routing::NetPoint* Engine::getNetpointByNameOrNull(const char* name) +simgrid::kernel::routing::NetPoint* Engine::getNetpointByNameOrNull(std::string name) { - try { - return pimpl->netpoints_.at(name); - } catch (std::out_of_range& unfound) { - return nullptr; - } + auto netp = pimpl->netpoints_.find(name); + return netp == pimpl->netpoints_.end() ? nullptr : netp->second; } + /** @brief Fill the provided vector with all existing netpoints */ void Engine::getNetpointList(std::vector* list) { diff --git a/src/s4u/s4u_file.cpp b/src/s4u/s4u_file.cpp index 043f6d5d8d..35f59e74d7 100644 --- a/src/s4u/s4u_file.cpp +++ b/src/s4u/s4u_file.cpp @@ -17,19 +17,19 @@ XBT_LOG_NEW_DEFAULT_CATEGORY(s4u_file,"S4U files"); namespace simgrid { namespace s4u { -File::File(const char* fullpath, void* userdata) : File(fullpath, Host::current(), userdata){}; +File::File(std::string fullpath, void* userdata) : File(fullpath, Host::current(), userdata){}; -File::File(const char* fullpath, sg_host_t host, void* userdata) : path_(fullpath), userdata_(userdata) +File::File(std::string fullpath, sg_host_t host, void* userdata) : path_(fullpath), userdata_(userdata) { // this cannot fail because we get a xbt_die if the mountpoint does not exist Storage* st = nullptr; size_t longest_prefix_length = 0; std::string path; - XBT_DEBUG("Search for storage name for '%s' on '%s'", fullpath, host->getCname()); + XBT_DEBUG("Search for storage name for '%s' on '%s'", fullpath.c_str(), host->getCname()); for (auto mnt : host->getMountedStorages()) { XBT_DEBUG("See '%s'", mnt.first.c_str()); - mount_point = std::string(fullpath).substr(0, mnt.first.size()); + mount_point = fullpath.substr(0, mnt.first.length()); if (mount_point == mnt.first && mnt.first.length() > longest_prefix_length) { /* The current mount name is found in the full path and is bigger than the previous*/ @@ -38,10 +38,10 @@ File::File(const char* fullpath, sg_host_t host, void* userdata) : path_(fullpat } } if (longest_prefix_length > 0) { /* Mount point found, split fullpath into mount_name and path+filename*/ - mount_point = std::string(fullpath).substr(0, longest_prefix_length); - path = std::string(fullpath).substr(longest_prefix_length, strlen(fullpath)); + mount_point = fullpath.substr(0, longest_prefix_length); + path = fullpath.substr(longest_prefix_length, fullpath.length()); } else - xbt_die("Can't find mount point for '%s' on '%s'", fullpath, host->getCname()); + xbt_die("Can't find mount point for '%s' on '%s'", fullpath.c_str(), host->getCname()); pimpl_ = simgrid::simix::kernelImmediate([this, st, path] { return new simgrid::surf::FileImpl(st, path, mount_point); }); @@ -84,7 +84,7 @@ sg_size_t File::tell() return simgrid::simix::kernelImmediate([this] { return pimpl_->tell(); }); } -void File::move(const char* fullpath) +void File::move(std::string fullpath) { simgrid::simix::kernelImmediate([this, fullpath] { pimpl_->move(fullpath); }); } diff --git a/src/s4u/s4u_host.cpp b/src/s4u/s4u_host.cpp index cd083646d2..5439ce8887 100644 --- a/src/s4u/s4u_host.cpp +++ b/src/s4u/s4u_host.cpp @@ -87,11 +87,8 @@ Host* Host::by_name_or_null(const char* name) } Host* Host::by_name_or_null(std::string name) { - try { - return host_list.at(name); - } catch (std::out_of_range& unfound) { - return nullptr; - } + auto host = host_list.find(name); + return host == host_list.end() ? nullptr : host->second; } Host *Host::current(){ diff --git a/src/s4u/s4u_mailbox.cpp b/src/s4u/s4u_mailbox.cpp index b53d652f30..245bd5334b 100644 --- a/src/s4u/s4u_mailbox.cpp +++ b/src/s4u/s4u_mailbox.cpp @@ -53,9 +53,7 @@ smx_activity_t Mailbox::front() } void Mailbox::setReceiver(ActorPtr actor) { - simix::kernelImmediate([this, actor]() { - this->pimpl_->setReceiver(actor); - }); + simix::kernelImmediate([this, actor]() { this->pimpl_->setReceiver(actor); }); } /** @brief get the receiver (process associated to the mailbox) */ diff --git a/src/s4u/s4u_netzone.cpp b/src/s4u/s4u_netzone.cpp index 33dbf91c0a..1490c8f942 100644 --- a/src/s4u/s4u_netzone.cpp +++ b/src/s4u/s4u_netzone.cpp @@ -23,7 +23,7 @@ simgrid::xbt::signal NetZone::onCreation; simgrid::xbt::signal NetZone::onSeal; -NetZone::NetZone(NetZone* father, const char* name) : father_(father), name_(xbt_strdup(name)) +NetZone::NetZone(NetZone* father, std::string name) : father_(father), name_(name) { children_ = new std::vector(); } @@ -38,7 +38,6 @@ NetZone::~NetZone() for (auto nz : *children_) delete nz; delete children_; - xbt_free(name_); } std::unordered_map* NetZone::getProperties() @@ -64,9 +63,9 @@ std::vector* NetZone::getChildren() { return children_; } -char* NetZone::getCname() +const char* NetZone::getCname() { - return name_; + return name_.c_str(); } NetZone* NetZone::getFather() { @@ -90,7 +89,7 @@ int NetZone::addComponent(kernel::routing::NetPoint* elm) void NetZone::addRoute(sg_platf_route_cbarg_t /*route*/) { - xbt_die("NetZone '%s' does not accept new routes (wrong class).", name_); + xbt_die("NetZone '%s' does not accept new routes (wrong class).", name_.c_str()); } } }; // namespace simgrid::s4u diff --git a/src/s4u/s4u_storage.cpp b/src/s4u/s4u_storage.cpp index 38be6654ba..3890ce3c90 100644 --- a/src/s4u/s4u_storage.cpp +++ b/src/s4u/s4u_storage.cpp @@ -22,7 +22,7 @@ std::map* allStorages() return res; } -Storage* Storage::byName(const char* name) +Storage* Storage::byName(std::string name) { surf::StorageImpl* res = surf::StorageImpl::byName(name); if (res == nullptr) diff --git a/src/simdag/sd_daxloader.cpp b/src/simdag/sd_daxloader.cpp index 843e43e312..17ce94e88f 100644 --- a/src/simdag/sd_daxloader.cpp +++ b/src/simdag/sd_daxloader.cpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2009-2016. The SimGrid Team. +/* Copyright (c) 2009-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -10,7 +10,7 @@ #include "xbt/log.h" #include "xbt/misc.h" #include "xbt/str.h" -#include +#include XBT_LOG_NEW_DEFAULT_SUBCATEGORY(sd_daxparse, sd, "Parsing DAX files"); @@ -141,8 +141,8 @@ bool acyclic_graph_detail(xbt_dynar_t dag){ static YY_BUFFER_STATE input_buffer; static xbt_dynar_t result; -static std::unordered_map jobs; -static xbt_dict_t files; +static std::map jobs; +static std::map files; static SD_task_t current_job; static SD_task_t root_task; static SD_task_t end_task; @@ -158,9 +158,7 @@ static void dax_task_free(void *task) */ xbt_dynar_t SD_daxload(const char *filename) { - xbt_dict_cursor_t cursor; SD_task_t file; - char* name; FILE* in_file = fopen(filename, "r"); xbt_assert(in_file, "Unable to open \"%s\"\n", filename); input_buffer = dax__create_buffer(in_file, 10); @@ -168,7 +166,6 @@ xbt_dynar_t SD_daxload(const char *filename) dax_lineno = 1; result = xbt_dynar_new(sizeof(SD_task_t), dax_task_free); - files = xbt_dict_new_homogeneous(&dax_task_free); root_task = SD_task_create_comp_seq("root", nullptr, 0); /* by design the root task is always SCHEDULABLE */ SD_task_set_state(root_task, SD_SCHEDULABLE); @@ -189,7 +186,8 @@ xbt_dynar_t SD_daxload(const char *filename) * Files not consumed in the system are said to be consumed by end task (bottom of DAG). */ - xbt_dict_foreach(files, cursor, name, file) { + for (auto elm : files) { + file = elm.second; SD_task_t newfile; if (file->predecessors->empty()) { for (SD_task_t it : *file->successors) { @@ -225,7 +223,8 @@ xbt_dynar_t SD_daxload(const char *filename) xbt_dynar_push(result, &end_task); /* Free previous copy of the files */ - xbt_dict_free(&files); + for (auto elm : files) + SD_task_destroy(elm.second); unsigned int cpt; xbt_dynar_foreach(result, cpt, file) { if (SD_task_get_kind(file) == SD_TASK_COMM_E2E) { @@ -295,12 +294,14 @@ void STag_dax__uses() bool is_input = (A_dax__uses_link == A_dax__uses_link_input); XBT_DEBUG("See ",A_dax__uses_file,(is_input?"in":"out")); - SD_task_t file = static_cast(xbt_dict_get_or_null(files, A_dax__uses_file)); - if (file == nullptr) { + auto it = files.find(A_dax__uses_file); + SD_task_t file; + if (it == files.end()) { file = SD_task_create_comm_e2e(A_dax__uses_file, nullptr, size); sd_global->initial_tasks->erase(file); - xbt_dict_set(files, A_dax__uses_file, file, nullptr); + files[A_dax__uses_file] = file; } else { + file = it->second; if (file->amount < size || file->amount > size) { XBT_WARN("Ignore file %s size redefinition from %.0f to %.0f", A_dax__uses_file, SD_task_get_amount(file), size); } @@ -318,9 +319,10 @@ void STag_dax__uses() static SD_task_t current_child; void STag_dax__child() { - try { - current_child = jobs.at(A_dax__child_ref); - } catch (std::out_of_range& unfound) { + auto job = jobs.find(A_dax__child_ref); + if (job != jobs.end()) { + current_child = job->second; + } else { throw std::out_of_range(std::string("Parse error on line ") + std::to_string(dax_lineno) + ": Asked to add dependencies to the non-existent " + A_dax__child_ref + "task"); } @@ -333,11 +335,12 @@ void ETag_dax__child() void STag_dax__parent() { - try { - SD_task_t parent = jobs.at(A_dax__parent_ref); + auto job = jobs.find(A_dax__parent_ref); + if (job != jobs.end()) { + SD_task_t parent = job->second; SD_task_dependency_add(nullptr, nullptr, parent, current_child); XBT_DEBUG("Control-flow dependency from %s to %s", current_child->name, parent->name); - } catch (std::out_of_range& unfound) { + } else { throw std::out_of_range(std::string("Parse error on line ") + std::to_string(dax_lineno) + ": Asked to add a dependency from " + current_child->name + " to " + A_dax__parent_ref + ", but " + A_dax__parent_ref + " does not exist"); diff --git a/src/simdag/sd_dotloader.cpp b/src/simdag/sd_dotloader.cpp index b2008872e7..0960689062 100644 --- a/src/simdag/sd_dotloader.cpp +++ b/src/simdag/sd_dotloader.cpp @@ -103,9 +103,10 @@ xbt_dynar_t SD_dotload_generic(const char* filename, bool sequential, bool sched if ((performer != -1 && order != -1) && performer < static_cast(sg_host_count())) { /* required parameters are given and less performers than hosts are required */ XBT_DEBUG ("Task '%s' is scheduled on workstation '%d' in position '%d'", task->name, performer, order); - try { - computer = computers.at(char_performer); - } catch (std::out_of_range& unfound) { + auto comp = computers.find(char_performer); + if (comp != computers.end()) { + computer = comp->second; + } else { computer = new std::vector; computers.insert({char_performer, computer}); } diff --git a/src/simdag/simdag_private.hpp b/src/simdag/simdag_private.hpp index 58d88e7bd0..b5e52f0251 100644 --- a/src/simdag/simdag_private.hpp +++ b/src/simdag/simdag_private.hpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2006-2015. The SimGrid Team. +/* Copyright (c) 2006-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -21,6 +21,7 @@ namespace sd{ class Global { public: explicit Global(); + Global(const Global&) = delete; ~Global(); bool watch_point_reached; /* has a task just reached a watch point? */ std::set *initial_tasks; diff --git a/src/simgrid/sg_config.cpp b/src/simgrid/sg_config.cpp index 5bc246a2af..0da6263839 100644 --- a/src/simgrid/sg_config.cpp +++ b/src/simgrid/sg_config.cpp @@ -341,11 +341,6 @@ void sg_config_init(int *argc, char **argv) /* The parameters of network models */ - // real default for "network/sender-gap" is set in network_smpi.cpp: - sg_sender_gap = NAN; - simgrid::config::bindFlag(sg_sender_gap, {"network/sender-gap", "network/sender_gap"}, - "Minimum gap between two overlapping sends"); - sg_latency_factor = 13.01; // comes from the default LV08 network model simgrid::config::bindFlag(sg_latency_factor, {"network/latency-factor", "network/latency_factor"}, "Correction factor to apply to the provided latency (default value set by network model)"); diff --git a/src/simix/ActorImpl.cpp b/src/simix/ActorImpl.cpp index 6bd02caaa7..610678fe55 100644 --- a/src/simix/ActorImpl.cpp +++ b/src/simix/ActorImpl.cpp @@ -79,8 +79,9 @@ void SIMIX_process_cleanup(smx_actor_t process) xbt_os_mutex_acquire(simix_global->mutex); /* cancel non-blocking communications */ - smx_activity_t synchro = process->comms.front(); while (not process->comms.empty()) { + smx_activity_t synchro = process->comms.front(); + process->comms.pop_front(); simgrid::kernel::activity::CommImplPtr comm = boost::static_pointer_cast(synchro); @@ -104,8 +105,6 @@ void SIMIX_process_cleanup(smx_actor_t process) } else { xbt_die("Communication synchro %p is in my list but I'm not the sender nor the receiver", synchro.get()); } - process->comms.pop_front(); - synchro = process->comms.front(); comm->cancel(); } @@ -346,7 +345,7 @@ smx_actor_t SIMIX_process_create(const char* name, std::function code, v /* Now insert it in the global process list and in the process to run list */ simix_global->process_list[process->pid] = process; XBT_DEBUG("Inserting %s(%s) in the to_run list", process->cname(), host->getCname()); - xbt_dynar_push_as(simix_global->process_to_run, smx_actor_t, process); + simix_global->process_to_run.push_back(process); intrusive_ptr_add_ref(process); /* Tracing the process creation */ @@ -410,7 +409,7 @@ smx_actor_t SIMIX_process_attach(const char* name, void* data, const char* hostn /* Now insert it in the global process list and in the process to run list */ simix_global->process_list[process->pid] = process; XBT_DEBUG("Inserting %s(%s) in the to_run list", process->cname(), host->getCname()); - xbt_dynar_push_as(simix_global->process_to_run, smx_actor_t, process); + simix_global->process_to_run.push_back(process); /* Tracing the process creation */ TRACE_msg_process_create(process->cname(), process->pid, process->host); @@ -454,15 +453,10 @@ void SIMIX_process_runall() { SIMIX_context_runall(); - xbt_dynar_t tmp = simix_global->process_that_ran; - simix_global->process_that_ran = simix_global->process_to_run; - simix_global->process_to_run = tmp; - xbt_dynar_reset(simix_global->process_to_run); + simix_global->process_to_run.swap(simix_global->process_that_ran); + simix_global->process_to_run.clear(); } -void simcall_HANDLER_process_kill(smx_simcall_t simcall, smx_actor_t process) { - SIMIX_process_kill(process, simcall->issuer); -} /** * \brief Internal function to kill a SIMIX process. * @@ -525,9 +519,11 @@ void SIMIX_process_kill(smx_actor_t process, smx_actor_t issuer) { process->waiting_synchro = nullptr; } - if (not xbt_dynar_member(simix_global->process_to_run, &(process)) && process != issuer) { + if (std::find(begin(simix_global->process_to_run), end(simix_global->process_to_run), process) == + end(simix_global->process_to_run) && + process != issuer) { XBT_DEBUG("Inserting %s in the to_run list", process->name.c_str()); - xbt_dynar_push_as(simix_global->process_to_run, smx_actor_t, process); + simix_global->process_to_run.push_back(process); } } @@ -564,9 +560,11 @@ void SIMIX_process_throw(smx_actor_t process, xbt_errcat_t cat, int value, const boost::dynamic_pointer_cast(process->waiting_synchro); if (sleep != nullptr) { SIMIX_process_sleep_destroy(process->waiting_synchro); - if (not xbt_dynar_member(simix_global->process_to_run, &(process)) && process != SIMIX_process_self()) { + if (std::find(begin(simix_global->process_to_run), end(simix_global->process_to_run), process) == + end(simix_global->process_to_run) && + process != SIMIX_process_self()) { XBT_DEBUG("Inserting %s in the to_run list", process->name.c_str()); - xbt_dynar_push_as(simix_global->process_to_run, smx_actor_t, process); + simix_global->process_to_run.push_back(process); } } @@ -821,7 +819,7 @@ void SIMIX_process_exception_terminate(xbt_ex_t * e) } /** @brief Returns the list of processes to run. */ -xbt_dynar_t SIMIX_process_get_runnable() +const std::vector& simgrid::simix::process_get_runnable() { return simix_global->process_to_run; } @@ -829,21 +827,8 @@ xbt_dynar_t SIMIX_process_get_runnable() /** @brief Returns the process from PID. */ smx_actor_t SIMIX_process_from_PID(aid_t PID) { - try { - return simix_global->process_list.at(PID); - } catch (std::out_of_range& unfound) { - return nullptr; - } -} - -/** @brief returns a dynar containing all currently existing processes */ -xbt_dynar_t SIMIX_processes_as_dynar() { - xbt_dynar_t res = xbt_dynar_new(sizeof(smx_actor_t),nullptr); - for (auto kv : simix_global->process_list) { - smx_actor_t proc = kv.second; - xbt_dynar_push(res,&proc); - } - return res; + auto process = simix_global->process_list.find(PID); + return process == simix_global->process_list.end() ? nullptr : process->second; } void SIMIX_process_on_exit_runall(smx_actor_t process) { diff --git a/src/simix/libsmx.cpp b/src/simix/libsmx.cpp index 045f8519e5..d8a0a0f711 100644 --- a/src/simix/libsmx.cpp +++ b/src/simix/libsmx.cpp @@ -154,19 +154,6 @@ e_smx_state_t simcall_execution_wait(smx_activity_t execution) return (e_smx_state_t) simcall_BODY_execution_wait(execution); } -/** - * \ingroup simix_process_management - * \brief Kills a SIMIX process. - * - * This function simply kills a process. - * - * \param process poor victim - */ -void simcall_process_kill(smx_actor_t process) -{ - simcall_BODY_process_kill(process); -} - /** * \ingroup simix_process_management * \brief Kills all SIMIX processes. @@ -239,7 +226,7 @@ void simcall_process_set_kill_time(smx_actor_t process, double kill_time) if (kill_time <= SIMIX_get_clock() || simix_global->kill_process_function == nullptr) return; XBT_DEBUG("Set kill time %f for process %s@%s", kill_time, process->cname(), process->host->getCname()); - process->kill_timer = SIMIX_timer_set(kill_time, [=] { + process->kill_timer = SIMIX_timer_set(kill_time, [process] { simix_global->kill_process_function(process); process->kill_timer=nullptr; }); diff --git a/src/simix/popping.cpp b/src/simix/popping.cpp index 2cefef94e6..91a79b1091 100644 --- a/src/simix/popping.cpp +++ b/src/simix/popping.cpp @@ -24,12 +24,11 @@ void SIMIX_simcall_answer(smx_simcall_t simcall) XBT_DEBUG("Answer simcall %s (%d) issued by %s (%p)", SIMIX_simcall_name(simcall->call), (int)simcall->call, simcall->issuer->name.c_str(), simcall->issuer); simcall->issuer->simcall.call = SIMCALL_NONE; - xbt_assert(SIMIX_is_maestro(), "Ugh! This code path is reserved for maestro, but I'm '%s' on '%s'", - SIMIX_process_self()->cname(), sg_host_get_name(SIMIX_process_self()->host)); /* This check should be useless and slows everyone. Reactivate if you see something weird in process scheduling. */ - // if (xbt_dynar_member(simix_global->process_to_run, &(simcall->issuer))) + // if (std::find(begin(simix_global->process_to_run), end(simix_global->process_to_run), simcall->issuer) != + // end(simix_global->process_to_run)) // DIE_IMPOSSIBLE; - xbt_dynar_push_as(simix_global->process_to_run, smx_actor_t, simcall->issuer); + simix_global->process_to_run.push_back(simcall->issuer); } } diff --git a/src/simix/popping_accessors.h b/src/simix/popping_accessors.h index d16dc06c78..eb130f4bd7 100644 --- a/src/simix/popping_accessors.h +++ b/src/simix/popping_accessors.h @@ -15,19 +15,6 @@ */ #include "src/simix/popping_private.h" -static inline smx_actor_t simcall_process_kill__get__process(smx_simcall_t simcall) -{ - return simgrid::simix::unmarshal(simcall->args[0]); -} -static inline smx_actor_t simcall_process_kill__getraw__process(smx_simcall_t simcall) -{ - return simgrid::simix::unmarshal_raw(simcall->args[0]); -} -static inline void simcall_process_kill__set__process(smx_simcall_t simcall, smx_actor_t arg) -{ - simgrid::simix::marshal(simcall->args[0], arg); -} - static inline int simcall_process_killall__get__reset_pid(smx_simcall_t simcall) { return simgrid::simix::unmarshal(simcall->args[0]); @@ -1461,7 +1448,6 @@ static inline void simcall_run_blocking__set__code(smx_simcall_t simcall, std::f /* The prototype of all simcall handlers, automatically generated for you */ -XBT_PRIVATE void simcall_HANDLER_process_kill(smx_simcall_t simcall, smx_actor_t process); XBT_PRIVATE void simcall_HANDLER_process_killall(smx_simcall_t simcall, int reset_pid); XBT_PRIVATE void simcall_HANDLER_process_suspend(smx_simcall_t simcall, smx_actor_t process); XBT_PRIVATE void simcall_HANDLER_process_join(smx_simcall_t simcall, smx_actor_t process, double timeout); diff --git a/src/simix/popping_bodies.cpp b/src/simix/popping_bodies.cpp index c2c83cc915..3a1bd5edbb 100644 --- a/src/simix/popping_bodies.cpp +++ b/src/simix/popping_bodies.cpp @@ -36,12 +36,6 @@ inline static R simcall(e_smx_simcall_t call, T const&... t) return simgrid::simix::unmarshal(self->simcall.result); } -inline static void simcall_BODY_process_kill(smx_actor_t process) { - /* Go to that function to follow the code flow through the simcall barrier */ - if (0) simcall_HANDLER_process_kill(&SIMIX_process_self()->simcall, process); - return simcall(SIMCALL_PROCESS_KILL, process); - } - inline static void simcall_BODY_process_killall(int reset_pid) { /* Go to that function to follow the code flow through the simcall barrier */ if (0) simcall_HANDLER_process_killall(&SIMIX_process_self()->simcall, reset_pid); diff --git a/src/simix/popping_enum.h b/src/simix/popping_enum.h index 8c5c93730d..a2f3155909 100644 --- a/src/simix/popping_enum.h +++ b/src/simix/popping_enum.h @@ -19,7 +19,6 @@ */ typedef enum { SIMCALL_NONE, - SIMCALL_PROCESS_KILL, SIMCALL_PROCESS_KILLALL, SIMCALL_PROCESS_CLEANUP, SIMCALL_PROCESS_SUSPEND, diff --git a/src/simix/popping_generated.cpp b/src/simix/popping_generated.cpp index d388546ea4..e52c41f726 100644 --- a/src/simix/popping_generated.cpp +++ b/src/simix/popping_generated.cpp @@ -25,7 +25,6 @@ XBT_LOG_EXTERNAL_DEFAULT_CATEGORY(simix_popping); /** @brief Simcalls' names (generated from src/simix/simcalls.in) */ const char* simcall_names[] = { "SIMCALL_NONE", - "SIMCALL_PROCESS_KILL", "SIMCALL_PROCESS_KILLALL", "SIMCALL_PROCESS_CLEANUP", "SIMCALL_PROCESS_SUSPEND", @@ -76,11 +75,6 @@ void SIMIX_simcall_handle(smx_simcall_t simcall, int value) { if (simcall->issuer->context->iwannadie && simcall->call != SIMCALL_PROCESS_CLEANUP) return; switch (simcall->call) { -case SIMCALL_PROCESS_KILL: - simcall_HANDLER_process_kill(simcall, simgrid::simix::unmarshal(simcall->args[0])); - SIMIX_simcall_answer(simcall); - break; - case SIMCALL_PROCESS_KILLALL: simcall_HANDLER_process_killall(simcall, simgrid::simix::unmarshal(simcall->args[0])); SIMIX_simcall_answer(simcall); diff --git a/src/simix/simcalls.in b/src/simix/simcalls.in index 3a6cc0f0af..c13413f840 100644 --- a/src/simix/simcalls.in +++ b/src/simix/simcalls.in @@ -35,7 +35,6 @@ # Last but not the least, you should declare the new simix call in # ./include/simgrid/simix.h (otherwise you will get a warning at compile time) -void process_kill(smx_actor_t process); void process_killall(int reset_pid); void process_cleanup(smx_actor_t process) [[nohandler]]; void process_suspend(smx_actor_t process) [[block]]; diff --git a/src/simix/smx_global.cpp b/src/simix/smx_global.cpp index d00955b753..0e6973544e 100644 --- a/src/simix/smx_global.cpp +++ b/src/simix/smx_global.cpp @@ -10,6 +10,7 @@ #include /* Signal handling */ #include +#include #include #include "simgrid/s4u/Engine.hpp" @@ -204,8 +205,6 @@ void SIMIX_global_init(int *argc, char **argv) simix_global = std::unique_ptr(new simgrid::simix::Global()); simgrid::simix::ActorImpl proc; - simix_global->process_to_run = xbt_dynar_new(sizeof(smx_actor_t), nullptr); - simix_global->process_that_ran = xbt_dynar_new(sizeof(smx_actor_t), nullptr); simix_global->process_to_destroy = xbt_swag_new(xbt_swag_offset(proc, destroy_hookup)); simix_global->maestro_process = nullptr; simix_global->create_process_function = &SIMIX_process_create; @@ -277,7 +276,7 @@ void SIMIX_clean() smx_cleaned = 1; XBT_DEBUG("SIMIX_clean called. Simulation's over."); - if (not xbt_dynar_is_empty(simix_global->process_to_run) && SIMIX_get_clock() <= 0.0) { + if (not simix_global->process_to_run.empty() && SIMIX_get_clock() <= 0.0) { XBT_CRITICAL(" "); XBT_CRITICAL("The time is still 0, and you still have processes ready to run."); XBT_CRITICAL("It seems that you forgot to run the simulation that you setup."); @@ -292,8 +291,8 @@ void SIMIX_clean() xbt_heap_free(simix_timers); simix_timers = nullptr; /* Free the remaining data structures */ - xbt_dynar_free(&simix_global->process_to_run); - xbt_dynar_free(&simix_global->process_that_ran); + simix_global->process_to_run.clear(); + simix_global->process_that_ran.clear(); xbt_swag_free(simix_global->process_to_destroy); simix_global->process_list.clear(); simix_global->process_to_destroy = nullptr; @@ -334,19 +333,6 @@ double SIMIX_get_clock() } } -static int process_syscall_color(void *p) -{ - switch ((*(smx_actor_t *)p)->simcall.call) { - case SIMCALL_NONE: - case SIMCALL_PROCESS_KILL: - return 2; - // case SIMCALL_PROCESS_RESUME: - // return 1; - default: - return 0; - } -} - /** Wake up all processes waiting for a Surf action to finish */ static void SIMIX_wake_processes() { @@ -429,19 +415,16 @@ void SIMIX_run() double time = 0; do { - XBT_DEBUG("New Schedule Round; size(queue)=%lu", xbt_dynar_length(simix_global->process_to_run)); + XBT_DEBUG("New Schedule Round; size(queue)=%zu", simix_global->process_to_run.size()); SIMIX_execute_tasks(); - while (not xbt_dynar_is_empty(simix_global->process_to_run)) { - XBT_DEBUG("New Sub-Schedule Round; size(queue)=%lu", xbt_dynar_length(simix_global->process_to_run)); + while (not simix_global->process_to_run.empty()) { + XBT_DEBUG("New Sub-Schedule Round; size(queue)=%zu", simix_global->process_to_run.size()); /* Run all processes that are ready to run, possibly in parallel */ SIMIX_process_runall(); - /* Move all killer processes to the end of the list, because killing a process that have an ongoing simcall is a bad idea */ - xbt_dynar_three_way_partition(simix_global->process_that_ran, process_syscall_color); - /* answer sequentially and in a fixed arbitrary order all the simcalls that were issued during that sub-round */ /* WARNING, the order *must* be fixed or you'll jeopardize the simulation reproducibility (see RR-7653) */ @@ -497,9 +480,7 @@ void SIMIX_run() * That would thus be a pure waste of time. */ - unsigned int iter; - smx_actor_t process; - xbt_dynar_foreach(simix_global->process_that_ran, iter, process) { + for (smx_actor_t process : simix_global->process_that_ran) { if (process->simcall.call != SIMCALL_NONE) { SIMIX_simcall_handle(&process->simcall, 0); } @@ -548,13 +529,13 @@ void SIMIX_run() /* Clean processes to destroy */ SIMIX_process_empty_trash(); - XBT_DEBUG("### time %f, #processes %zu, #to_run %lu", time, simix_global->process_list.size(), - xbt_dynar_length(simix_global->process_to_run)); + XBT_DEBUG("### time %f, #processes %zu, #to_run %zu", time, simix_global->process_list.size(), + simix_global->process_to_run.size()); - if (xbt_dynar_is_empty(simix_global->process_to_run) && not simix_global->process_list.empty()) + if (simix_global->process_to_run.empty() && not simix_global->process_list.empty()) simgrid::simix::onDeadlock(); - } while (time > -1.0 || not xbt_dynar_is_empty(simix_global->process_to_run)); + } while (time > -1.0 || not simix_global->process_to_run.empty()); if (simix_global->process_list.size() != 0) { diff --git a/src/simix/smx_private.h b/src/simix/smx_private.h index a0c713eba7..f10bf4d126 100644 --- a/src/simix/smx_private.h +++ b/src/simix/smx_private.h @@ -11,6 +11,7 @@ #include #include +#include /********************************** Simix Global ******************************/ @@ -22,8 +23,8 @@ class Global { public: smx_context_factory_t context_factory = nullptr; - xbt_dynar_t process_to_run = nullptr; - xbt_dynar_t process_that_ran = nullptr; + std::vector process_to_run; + std::vector process_that_ran; std::map process_list; #if SIMGRID_HAVE_MC /* MCer cannot read the std::map above in the remote process, so we copy the info it needs in a dynar. diff --git a/src/smpi/bindings/smpi_pmpi.cpp b/src/smpi/bindings/smpi_pmpi.cpp index ff6a24857d..59561a4699 100644 --- a/src/smpi/bindings/smpi_pmpi.cpp +++ b/src/smpi/bindings/smpi_pmpi.cpp @@ -3,13 +3,14 @@ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ +#include "private.h" #include "simgrid/s4u/Engine.hpp" #include "simgrid/s4u/Host.hpp" -#include "private.h" #include "smpi_comm.hpp" #include "smpi_datatype_derived.hpp" #include "smpi_process.hpp" #include "smpi_status.hpp" +#include "src/simix/ActorImpl.hpp" XBT_LOG_NEW_DEFAULT_SUBCATEGORY(smpi_pmpi, smpi, "Logging specific to SMPI (pmpi)"); @@ -119,7 +120,8 @@ int PMPI_Abort(MPI_Comm comm, int errorcode) { smpi_bench_end(); // FIXME: should kill all processes in comm instead - simcall_process_kill(SIMIX_process_self()); + smx_actor_t process = SIMIX_process_self(); + simgrid::simix::kernelImmediate([process] { SIMIX_process_kill(process, process); }); return MPI_SUCCESS; } diff --git a/src/smpi/bindings/smpi_pmpi_request.cpp b/src/smpi/bindings/smpi_pmpi_request.cpp index 40cc76e08b..926b55c349 100644 --- a/src/smpi/bindings/smpi_pmpi_request.cpp +++ b/src/smpi/bindings/smpi_pmpi_request.cpp @@ -20,16 +20,16 @@ int PMPI_Send_init(void *buf, int count, MPI_Datatype datatype, int dst, int tag smpi_bench_end(); if (request == nullptr) { - retval = MPI_ERR_ARG; + retval = MPI_ERR_ARG; } else if (comm == MPI_COMM_NULL) { - retval = MPI_ERR_COMM; + retval = MPI_ERR_COMM; } else if (not datatype->is_valid()) { retval = MPI_ERR_TYPE; } else if (dst == MPI_PROC_NULL) { - retval = MPI_SUCCESS; + retval = MPI_SUCCESS; } else { - *request = simgrid::smpi::Request::send_init(buf, count, datatype, dst, tag, comm); - retval = MPI_SUCCESS; + *request = simgrid::smpi::Request::send_init(buf, count, datatype, dst, tag, comm); + retval = MPI_SUCCESS; } smpi_bench_begin(); if (retval != MPI_SUCCESS && request != nullptr) @@ -423,8 +423,8 @@ int PMPI_Ssend(void* buf, int count, MPI_Datatype datatype, int dst, int tag, MP return retval; } -int PMPI_Sendrecv(void *sendbuf, int sendcount, MPI_Datatype sendtype, int dst, int sendtag, void *recvbuf, - int recvcount, MPI_Datatype recvtype, int src, int recvtag, MPI_Comm comm, MPI_Status * status) +int PMPI_Sendrecv(void* sendbuf, int sendcount, MPI_Datatype sendtype, int dst, int sendtag, void* recvbuf, + int recvcount, MPI_Datatype recvtype, int src, int recvtag, MPI_Comm comm, MPI_Status* status) { int retval = 0; @@ -448,34 +448,34 @@ int PMPI_Sendrecv(void *sendbuf, int sendcount, MPI_Datatype sendtype, int dst, retval = MPI_ERR_TAG; } else { - int rank = comm != MPI_COMM_NULL ? smpi_process()->index() : -1; - int dst_traced = comm->group()->index(dst); - int src_traced = comm->group()->index(src); - instr_extra_data extra = xbt_new0(s_instr_extra_data_t,1); - extra->type = TRACING_SENDRECV; - extra->src = src_traced; - extra->dst = dst_traced; - int known=0; - extra->datatype1 = encode_datatype(sendtype, &known); - int dt_size_send = 1; - if(known==0) - dt_size_send = sendtype->size(); - extra->send_size = sendcount*dt_size_send; - extra->datatype2 = encode_datatype(recvtype, &known); - int dt_size_recv = 1; - if(known==0) - dt_size_recv = recvtype->size(); - extra->recv_size = recvcount*dt_size_recv; - - TRACE_smpi_ptp_in(rank, __FUNCTION__, extra); - TRACE_smpi_send(rank, rank, dst_traced, sendtag,sendcount*sendtype->size()); - - simgrid::smpi::Request::sendrecv(sendbuf, sendcount, sendtype, dst, sendtag, recvbuf, recvcount, recvtype, src, recvtag, comm, - status); - retval = MPI_SUCCESS; - - TRACE_smpi_ptp_out(rank, dst_traced, __FUNCTION__); - TRACE_smpi_recv(src_traced, rank, recvtag); + int rank = comm != MPI_COMM_NULL ? smpi_process()->index() : -1; + int dst_traced = comm->group()->index(dst); + int src_traced = comm->group()->index(src); + instr_extra_data extra = xbt_new0(s_instr_extra_data_t, 1); + extra->type = TRACING_SENDRECV; + extra->src = src_traced; + extra->dst = dst_traced; + int known = 0; + extra->datatype1 = encode_datatype(sendtype, &known); + int dt_size_send = 1; + if (known == 0) + dt_size_send = sendtype->size(); + extra->send_size = sendcount * dt_size_send; + extra->datatype2 = encode_datatype(recvtype, &known); + int dt_size_recv = 1; + if (known == 0) + dt_size_recv = recvtype->size(); + extra->recv_size = recvcount * dt_size_recv; + + TRACE_smpi_ptp_in(rank, __FUNCTION__, extra); + TRACE_smpi_send(rank, rank, dst_traced, sendtag, sendcount * sendtype->size()); + + simgrid::smpi::Request::sendrecv(sendbuf, sendcount, sendtype, dst, sendtag, recvbuf, recvcount, recvtype, src, + recvtag, comm, status); + retval = MPI_SUCCESS; + + TRACE_smpi_ptp_out(rank, dst_traced, __FUNCTION__); + TRACE_smpi_recv(src_traced, rank, recvtag); } smpi_bench_begin(); @@ -495,7 +495,7 @@ int PMPI_Sendrecv_replace(void* buf, int count, MPI_Datatype datatype, int dst, void* recvbuf = xbt_new0(char, size); retval = MPI_Sendrecv(buf, count, datatype, dst, sendtag, recvbuf, count, datatype, src, recvtag, comm, status); if(retval==MPI_SUCCESS){ - simgrid::smpi::Datatype::copy(recvbuf, count, datatype, buf, count, datatype); + simgrid::smpi::Datatype::copy(recvbuf, count, datatype, buf, count, datatype); } xbt_free(recvbuf); @@ -632,9 +632,7 @@ int PMPI_Wait(MPI_Request * request, MPI_Status * status) TRACE_smpi_ptp_out(rank, dst_traced, __FUNCTION__); if (is_wait_for_receive) { if(src_traced==MPI_ANY_SOURCE) - src_traced = (status!=MPI_STATUS_IGNORE) ? - comm->group()->rank(status->MPI_SOURCE) : - src_traced; + src_traced = (status != MPI_STATUS_IGNORE) ? comm->group()->rank(status->MPI_SOURCE) : src_traced; TRACE_smpi_recv(src_traced, dst_traced, tag_traced); } } @@ -682,9 +680,8 @@ int PMPI_Waitany(int count, MPI_Request requests[], int *index, MPI_Status * sta int is_wait_for_receive = savedvals[*index].recv; if (is_wait_for_receive) { if(savedvals[*index].src==MPI_ANY_SOURCE) - src_traced = (status != MPI_STATUSES_IGNORE) - ? savedvals[*index].comm->group()->rank(status->MPI_SOURCE) - : savedvals[*index].src; + src_traced = (status != MPI_STATUSES_IGNORE) ? savedvals[*index].comm->group()->rank(status->MPI_SOURCE) + : savedvals[*index].src; TRACE_smpi_recv(src_traced, dst_traced, savedvals[*index].tag); } TRACE_smpi_ptp_out(rank_traced, dst_traced, __FUNCTION__); @@ -727,14 +724,14 @@ int PMPI_Waitall(int count, MPI_Request requests[], MPI_Status status[]) for (int i = 0; i < count; i++) { if(savedvals[i].valid){ - //the src may not have been known at the beginning of the recv (MPI_ANY_SOURCE) + // the src may not have been known at the beginning of the recv (MPI_ANY_SOURCE) int src_traced = savedvals[i].src; int dst_traced = savedvals[i].dst; int is_wait_for_receive = savedvals[i].recv; if (is_wait_for_receive) { if(src_traced==MPI_ANY_SOURCE) - src_traced = (status!=MPI_STATUSES_IGNORE) ? - savedvals[i].comm->group()->rank(status[i].MPI_SOURCE) : savedvals[i].src; + src_traced = (status != MPI_STATUSES_IGNORE) ? savedvals[i].comm->group()->rank(status[i].MPI_SOURCE) + : savedvals[i].src; TRACE_smpi_recv(src_traced, dst_traced,savedvals[i].tag); } } diff --git a/src/smpi/include/smpi_keyvals.hpp b/src/smpi/include/smpi_keyvals.hpp index bc3e2af943..986540f4e9 100644 --- a/src/smpi/include/smpi_keyvals.hpp +++ b/src/smpi/include/smpi_keyvals.hpp @@ -109,11 +109,12 @@ template int Keyval::attr_get(int keyval, void* attr_value, int* fl *flag=0; return MPI_SUCCESS; } - try { - *static_cast(attr_value) = attributes()->at(keyval); + const auto& attribs = attributes(); + auto attr = attribs->find(keyval); + if (attr != attribs->end()) { + *static_cast(attr_value) = attr->second; *flag=1; - } - catch (const std::out_of_range& oor) { + } else { *flag=0; } return MPI_SUCCESS; @@ -140,12 +141,13 @@ template void Keyval::cleanup_attr(){ if (not attributes()->empty()) { int flag=0; for(auto it : attributes_){ - try{ - smpi_key_elem elem = T::keyvals_.at(it.first); + auto elm = T::keyvals_.find(it.first); + if (elm != T::keyvals_.end()) { + smpi_key_elem elem = elm->second; if(elem != nullptr){ call_deleter((T*)this, elem, it.first,it.second,&flag); } - }catch(const std::out_of_range& oor) { + } else { //already deleted, not a problem; flag=0; } diff --git a/src/smpi/internals/instr_smpi.cpp b/src/smpi/internals/instr_smpi.cpp index ec33a27bc8..9318b356fc 100644 --- a/src/smpi/internals/instr_smpi.cpp +++ b/src/smpi/internals/instr_smpi.cpp @@ -9,11 +9,12 @@ #include #include #include +#include #include XBT_LOG_NEW_DEFAULT_SUBCATEGORY(instr_smpi, instr, "Tracing SMPI"); -static xbt_dict_t keys; +static std::unordered_map*> keys; static const char *smpi_colors[] ={ "recv", "1 0 0", @@ -100,12 +101,14 @@ static char *TRACE_smpi_put_key(int src, int dst, int tag, char *key, int n, int //get the dynar for src#dst char aux[INSTR_DEFAULT_STR_SIZE]; snprintf(aux, INSTR_DEFAULT_STR_SIZE, "%d#%d#%d#%d", src, dst, tag, send); - xbt_dynar_t d = static_cast(xbt_dict_get_or_null(keys, aux)); + auto it = keys.find(aux); + std::deque* d; - if (d == nullptr) { - d = xbt_dynar_new(sizeof(char *), &xbt_free_ref); - xbt_dict_set(keys, aux, d, nullptr); - } + if (it == keys.end()) { + d = new std::deque; + keys[aux] = d; + } else + d = it->second; //generate the key static unsigned long long counter = 0; @@ -113,8 +116,7 @@ static char *TRACE_smpi_put_key(int src, int dst, int tag, char *key, int n, int snprintf(key, n, "%d_%d_%d_%llu", src, dst, tag, counter); //push it - char *a = static_cast (xbt_strdup(key)); - xbt_dynar_push_as(d, char *, a); + d->push_back(key); return key; } @@ -123,21 +125,18 @@ static char *TRACE_smpi_get_key(int src, int dst, int tag, char *key, int n, int { char aux[INSTR_DEFAULT_STR_SIZE]; snprintf(aux, INSTR_DEFAULT_STR_SIZE, "%d#%d#%d#%d", src, dst, tag, send==1?0:1); - xbt_dynar_t d = static_cast(xbt_dict_get_or_null(keys, aux)); - - // first posted - if(xbt_dynar_is_empty(d)){ - TRACE_smpi_put_key(src, dst, tag, key, n, send); - return key; + auto it = keys.find(aux); + if (it == keys.end()) { + // first posted + TRACE_smpi_put_key(src, dst, tag, key, n, send); + } else { + snprintf(key, n, "%s", it->second->front().c_str()); + it->second->pop_front(); } - - char *s = xbt_dynar_get_as (d, 0, char *); - snprintf (key, n, "%s", s); - xbt_dynar_remove_at (d, 0, nullptr); return key; } -static xbt_dict_t process_category; +static std::unordered_map process_category; static void cleanup_extra_data (instr_extra_data extra){ if(extra!=nullptr){ @@ -157,12 +156,8 @@ void TRACE_internal_smpi_set_category (const char *category) //declare category TRACE_category (category); - char processid[INSTR_DEFAULT_STR_SIZE]; - snprintf (processid, INSTR_DEFAULT_STR_SIZE, "%p", SIMIX_process_self()); - if (xbt_dict_get_or_null (process_category, processid)) - xbt_dict_remove (process_category, processid); if (category != nullptr) - xbt_dict_set (process_category, processid, xbt_strdup(category), nullptr); + process_category[SIMIX_process_self()] = category; } const char *TRACE_internal_smpi_get_category () @@ -170,21 +165,19 @@ const char *TRACE_internal_smpi_get_category () if (not TRACE_smpi_is_enabled()) return nullptr; - char processid[INSTR_DEFAULT_STR_SIZE]; - snprintf (processid, INSTR_DEFAULT_STR_SIZE, "%p", SIMIX_process_self()); - return static_cast(xbt_dict_get_or_null (process_category, processid)); + auto it = process_category.find(SIMIX_process_self()); + return (it == process_category.end()) ? nullptr : it->second.c_str(); } void TRACE_smpi_alloc() { - keys = xbt_dict_new_homogeneous(xbt_dynar_free_voidp); - process_category = xbt_dict_new_homogeneous(xbt_free_f); + // for symmetry } void TRACE_smpi_release() { - xbt_dict_free(&keys); - xbt_dict_free(&process_category); + for (auto elm : keys) + delete elm.second; } void TRACE_smpi_init(int rank) diff --git a/src/smpi/internals/smpi_bench.cpp b/src/smpi/internals/smpi_bench.cpp index 44fdedfcfa..8e17376e70 100644 --- a/src/smpi/internals/smpi_bench.cpp +++ b/src/smpi/internals/smpi_bench.cpp @@ -3,13 +3,14 @@ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ -#include "src/internal_config.h" #include "private.h" #include "private.hpp" #include "simgrid/modelchecker.h" -#include "src/mc/mc_replay.h" -#include "smpi_process.hpp" #include "smpi_comm.hpp" +#include "smpi_process.hpp" +#include "src/internal_config.h" +#include "src/mc/mc_replay.h" +#include #ifndef WIN32 #include @@ -22,9 +23,6 @@ XBT_LOG_NEW_DEFAULT_SUBCATEGORY(smpi_bench, smpi, "Logging specific to SMPI (benchmarking)"); - -xbt_dict_t samples = nullptr; /* Allocated on first use */ - double smpi_cpu_threshold = -1; double smpi_host_speed; @@ -32,11 +30,6 @@ shared_malloc_type smpi_cfg_shared_malloc = shmalloc_global; double smpi_total_benched_time = 0; smpi_privatization_region_t smpi_privatization_regions; -void smpi_bench_destroy() -{ - xbt_dict_free(&samples); -} - extern "C" XBT_PUBLIC(void) smpi_execute_flops_(double *flops); void smpi_execute_flops_(double *flops) { @@ -282,11 +275,14 @@ typedef struct { int benching; /* 1: we are benchmarking; 0: we have enough data, no bench anymore */ } local_data_t; -static char *sample_location(int global, const char *file, int line) { +std::unordered_map samples; /* Allocated on first use */ + +static std::string sample_location(int global, const char* file, int line) +{ if (global) { - return bprintf("%s:%d", file, line); + return std::string(file) + ":" + std::to_string(line); } else { - return bprintf("%s:%d:%d", file, line, smpi_process()->index()); + return std::string(file) + ":" + std::to_string(line) + ":" + std::to_string(smpi_process()->index()); } } @@ -305,19 +301,17 @@ static int sample_enough_benchs(local_data_t *data) { void smpi_sample_1(int global, const char *file, int line, int iters, double threshold) { - char *loc = sample_location(global, file, line); + std::string loc = sample_location(global, file, line); smpi_bench_end(); /* Take time from previous, unrelated computation into account */ smpi_process()->set_sampling(1); - if (samples==nullptr) - samples = xbt_dict_new_homogeneous(free); - - local_data_t *data = static_cast(xbt_dict_get_or_null(samples, loc)); - if (data==nullptr) { + auto ld = samples.find(loc); + local_data_t* data; + if (ld == samples.end()) { xbt_assert(threshold>0 || iters>0, "You should provide either a positive amount of iterations to bench, or a positive maximal stderr (or both)"); - data = static_cast( xbt_new(local_data_t, 1)); + data = static_cast(xbt_new(local_data_t, 1)); data->count = 0; data->sum = 0.0; data->sum_pow2 = 0.0; @@ -325,34 +319,34 @@ void smpi_sample_1(int global, const char *file, int line, int iters, double thr data->threshold = threshold; data->benching = 1; // If we have no data, we need at least one data->mean = 0; - xbt_dict_set(samples, loc, data, nullptr); - XBT_DEBUG("XXXXX First time ever on benched nest %s.",loc); + samples[loc] = data; + XBT_DEBUG("XXXXX First time ever on benched nest %s.", loc.c_str()); } else { + data = ld->second; if (data->iters != iters || data->threshold != threshold) { XBT_ERROR("Asked to bench block %s with different settings %d, %f is not %d, %f. " "How did you manage to give two numbers at the same line??", - loc, data->iters, data->threshold, iters, threshold); + loc.c_str(), data->iters, data->threshold, iters, threshold); THROW_IMPOSSIBLE; } // if we already have some data, check whether sample_2 should get one more bench or whether it should emulate // the computation instead data->benching = (sample_enough_benchs(data) == 0); - XBT_DEBUG("XXXX Re-entering the benched nest %s. %s", loc, + XBT_DEBUG("XXXX Re-entering the benched nest %s. %s", loc.c_str(), (data->benching ? "more benching needed" : "we have enough data, skip computes")); } - xbt_free(loc); } int smpi_sample_2(int global, const char *file, int line) { - char *loc = sample_location(global, file, line); + std::string loc = sample_location(global, file, line); int res; - xbt_assert(samples, "Y U NO use SMPI_SAMPLE_* macros? Stop messing directly with smpi_sample_* functions!"); - local_data_t *data = static_cast(xbt_dict_get(samples, loc)); - XBT_DEBUG("sample2 %s",loc); - xbt_free(loc); + xbt_assert(not samples.empty(), + "Y U NO use SMPI_SAMPLE_* macros? Stop messing directly with smpi_sample_* functions!"); + local_data_t* data = samples.at(loc); + XBT_DEBUG("sample2 %s", loc.c_str()); if (data->benching==1) { // we need to run a new bench @@ -375,12 +369,12 @@ int smpi_sample_2(int global, const char *file, int line) void smpi_sample_3(int global, const char *file, int line) { - char *loc = sample_location(global, file, line); + std::string loc = sample_location(global, file, line); - xbt_assert(samples, "Y U NO use SMPI_SAMPLE_* macros? Stop messing directly with smpi_sample_* functions!"); - local_data_t *data = static_cast(xbt_dict_get(samples, loc)); - XBT_DEBUG("sample3 %s",loc); - xbt_free(loc); + xbt_assert(not samples.empty(), + "Y U NO use SMPI_SAMPLE_* macros? Stop messing directly with smpi_sample_* functions!"); + local_data_t* data = samples.at(loc); + XBT_DEBUG("sample3 %s", loc.c_str()); if (data->benching==0) THROW_IMPOSSIBLE; @@ -408,30 +402,36 @@ void smpi_sample_3(int global, const char *file, int line) } extern "C" { /** These functions will be called from the user code **/ - smpi_trace_call_location_t* smpi_trace_get_call_location() { - return smpi_process()->call_location(); - } +smpi_trace_call_location_t* smpi_trace_get_call_location() +{ + return smpi_process()->call_location(); +} - void smpi_trace_set_call_location(const char* file, const int line) { - smpi_trace_call_location_t* loc = smpi_process()->call_location(); +void smpi_trace_set_call_location(const char* file, const int line) +{ + smpi_trace_call_location_t* loc = smpi_process()->call_location(); - loc->previous_filename = loc->filename; - loc->previous_linenumber = loc->linenumber; - loc->filename = file; - loc->linenumber = line; - } + loc->previous_filename = loc->filename; + loc->previous_linenumber = loc->linenumber; + loc->filename = file; + loc->linenumber = line; +} - /** - * Required for Fortran bindings - */ - void smpi_trace_set_call_location_(const char* file, int* line) { - smpi_trace_set_call_location(file, *line); - } +/** Required for Fortran bindings */ +void smpi_trace_set_call_location_(const char* file, int* line) +{ + smpi_trace_set_call_location(file, *line); +} - /** - * Required for Fortran if -fsecond-underscore is activated - */ - void smpi_trace_set_call_location__(const char* file, int* line) { - smpi_trace_set_call_location(file, *line); - } +/** Required for Fortran if -fsecond-underscore is activated */ +void smpi_trace_set_call_location__(const char* file, int* line) +{ + smpi_trace_set_call_location(file, *line); +} +} + +void smpi_bench_destroy() +{ + for (auto elm : samples) + xbt_free(elm.second); } diff --git a/src/smpi/internals/smpi_global.cpp b/src/smpi/internals/smpi_global.cpp index e864e819a1..f1226a27ea 100644 --- a/src/smpi/internals/smpi_global.cpp +++ b/src/smpi/internals/smpi_global.cpp @@ -127,20 +127,18 @@ static void print(std::vector> vec) { std::fprintf(stderr, "{"); for (auto elt : vec) { std::fprintf(stderr, "(0x%zx, 0x%zx),", elt.first, elt.second); - } - std::fprintf(stderr, "}\n"); + } + std::fprintf(stderr, "}\n"); } static void memcpy_private(void* dest, const void* src, std::vector>& private_blocks) { - for(auto block : private_blocks) { + for (auto block : private_blocks) memcpy((uint8_t*)dest+block.first, (uint8_t*)src+block.first, block.second-block.first); - } } static void check_blocks(std::vector> &private_blocks, size_t buff_size) { - for(auto block : private_blocks) { + for (auto block : private_blocks) xbt_assert(block.first <= block.second && block.second <= buff_size, "Oops, bug in shared malloc."); - } } void smpi_comm_copy_buffer_callback(smx_activity_t synchro, void *buff, size_t buff_size) @@ -205,8 +203,8 @@ void smpi_comm_copy_buffer_callback(smx_activity_t synchro, void *buff, size_t b //xbt_free(comm->comm.src_data);// inside SMPI the request is kept inside the user data and should be free comm->src_buff = nullptr; } - if(tmpbuff!=buff)xbt_free(tmpbuff); - + if (tmpbuff != buff) + xbt_free(tmpbuff); } void smpi_comm_null_copy_buffer_callback(smx_activity_t comm, void *buff, size_t buff_size) @@ -282,10 +280,8 @@ void smpi_global_init() std::string str = std::string(xbt_cfg_get_string("smpi/papi-events")); Tokenizer tokens(str, separator_units); - // Iterate over all the computational units. This could be - // processes, hosts, threads, ranks... You name it. I'm not exactly - // sure what we will support eventually, so I'll leave it at the - // general term "units". + // Iterate over all the computational units. This could be processes, hosts, threads, ranks... You name it. + // I'm not exactly sure what we will support eventually, so I'll leave it at the general term "units". for (auto& unit_it : tokens) { boost::char_separator separator_events(":"); Tokenizer event_tokens(unit_it, separator_events); @@ -306,7 +302,7 @@ void smpi_global_init() // Note that we need to remove the name of the unit // (that could also be the "default" value), which always comes first. // Hence, we start at ++(events.begin())! - for (Tokenizer::iterator events_it = ++(event_tokens.begin()); events_it != event_tokens.end(); events_it++) { + for (Tokenizer::iterator events_it = ++(event_tokens.begin()); events_it != event_tokens.end(); ++events_it) { int event_code = PAPI_NULL; char* event_name = const_cast((*events_it).c_str()); diff --git a/src/smpi/internals/smpi_memory.cpp b/src/smpi/internals/smpi_memory.cpp index 9a1147f73d..4fce186361 100644 --- a/src/smpi/internals/smpi_memory.cpp +++ b/src/smpi/internals/smpi_memory.cpp @@ -108,7 +108,8 @@ void smpi_really_switch_data_segment(int dest) int smpi_is_privatization_file(char* file) { - return strncmp("/dev/shm/my-buffer-", file, std::strlen("/dev/shm/my-buffer-")) == 0; + const std::string buffer_path {"/dev/shm/my-buffer-"}; + return buffer_path.compare(file) == 0; } void smpi_initialize_global_memory_segments() diff --git a/src/smpi/internals/smpi_shared.cpp b/src/smpi/internals/smpi_shared.cpp index a213733c8f..27bd066e70 100644 --- a/src/smpi/internals/smpi_shared.cpp +++ b/src/smpi/internals/smpi_shared.cpp @@ -7,7 +7,7 @@ * Associated data and metadata are used as follows: * * mmap #1 - * `allocs' dict ---- -. + * `allocs' map ---- -. * ---------- shared_data_t shared_metadata_t / | | | * .->| | ---> -------------------- <--. ----------------- | | | | * | ---------- | fd of | | | size of mmap | --| | | | @@ -15,7 +15,7 @@ * `----------------- | | | ----------------- ---- | * -------------------- | ^ | * | | | - * | | `allocs_metadata' dict | + * | | `allocs_metadata' map | * | | ---------------------- | * | `-- | |<-' * | .-- | |<-. @@ -38,8 +38,6 @@ #include "private.h" #include "private.hpp" -#include "xbt/dict.h" -#include "xbt/ex.hpp" #include #include @@ -122,7 +120,8 @@ typedef struct { } shared_metadata_t; std::map allocs_metadata; -xbt_dict_t calls = nullptr; /* Allocated on first use */ +std::map calls; + #ifndef WIN32 static int smpi_shared_malloc_bogusfile = -1; static int smpi_shared_malloc_bogusfile_huge_page = -1; @@ -135,7 +134,7 @@ void smpi_shared_destroy() { allocs.clear(); allocs_metadata.clear(); - xbt_dict_free(&calls); + calls.clear(); } static size_t shm_size(int fd) { @@ -487,46 +486,18 @@ void smpi_shared_free(void *ptr) int smpi_shared_known_call(const char* func, const char* input) { - char* loc = bprintf("%s:%s", func, input); - int known = 0; - - if (calls==nullptr) { - calls = xbt_dict_new_homogeneous(nullptr); - } - try { - xbt_dict_get(calls, loc); /* Succeed or throw */ - known = 1; - xbt_free(loc); - } - catch (xbt_ex& ex) { - xbt_free(loc); - if (ex.category != not_found_error) - throw; - } - catch(...) { - xbt_free(loc); - throw; - } - return known; + std::string loc = std::string(func) + ":" + input; + return calls.find(loc) != calls.end(); } void* smpi_shared_get_call(const char* func, const char* input) { - char* loc = bprintf("%s:%s", func, input); + std::string loc = std::string(func) + ":" + input; - if (calls == nullptr) - calls = xbt_dict_new_homogeneous(nullptr); - void* data = xbt_dict_get(calls, loc); - xbt_free(loc); - return data; + return calls.at(loc); } void* smpi_shared_set_call(const char* func, const char* input, void* data) { - char* loc = bprintf("%s:%s", func, input); - - if (calls == nullptr) - calls = xbt_dict_new_homogeneous(nullptr); - xbt_dict_set(calls, loc, data, nullptr); - xbt_free(loc); + std::string loc = std::string(func) + ":" + input; + calls[loc] = data; return data; } - diff --git a/src/smpi/internals/smpi_utils.cpp b/src/smpi/internals/smpi_utils.cpp index b1a9e096f9..7951ecf7fa 100644 --- a/src/smpi/internals/smpi_utils.cpp +++ b/src/smpi/internals/smpi_utils.cpp @@ -30,7 +30,7 @@ std::vector parse_factor(const char *smpi_coef_string) * E --> F * G --> H */ - for (Tokenizer::iterator token_iter = tokens.begin(); token_iter != tokens.end(); token_iter++) { + for (Tokenizer::iterator token_iter = tokens.begin(); token_iter != tokens.end(); ++token_iter) { XBT_DEBUG("token : %s", token_iter->c_str()); Tokenizer factor_values(*token_iter, factor_separator); s_smpi_factor_t fact; @@ -38,7 +38,7 @@ std::vector parse_factor(const char *smpi_coef_string) xbt_die("Malformed radical for smpi factor: '%s'", smpi_coef_string); } unsigned int iteration = 0; - for (Tokenizer::iterator factor_iter = factor_values.begin(); factor_iter != factor_values.end(); factor_iter++) { + for (Tokenizer::iterator factor_iter = factor_values.begin(); factor_iter != factor_values.end(); ++factor_iter) { iteration++; if (factor_iter == factor_values.begin()) { /* first element */ diff --git a/src/smpi/mpi/smpi_comm.cpp b/src/smpi/mpi/smpi_comm.cpp index 23a4bd6ad7..051be8cba8 100644 --- a/src/smpi/mpi/smpi_comm.cpp +++ b/src/smpi/mpi/smpi_comm.cpp @@ -452,11 +452,9 @@ MPI_Comm Comm::f2c(int id) { return MPI_COMM_WORLD; } else if(F2C::f2c_lookup() != nullptr && id >= 0) { char key[KEY_SIZE]; - try { - return static_cast(F2C::f2c_lookup()->at(get_key_id(key, id))); - } catch (std::out_of_range& unfound) { - return MPI_COMM_NULL; - } + const auto& lookup = F2C::f2c_lookup(); + auto comm = lookup->find(get_key_id(key, id)); + return comm == lookup->end() ? MPI_COMM_NULL : static_cast(comm->second); } else { return MPI_COMM_NULL; } diff --git a/src/smpi/mpi/smpi_f2c.cpp b/src/smpi/mpi/smpi_f2c.cpp index ffd795fdbb..532f3e497a 100644 --- a/src/smpi/mpi/smpi_f2c.cpp +++ b/src/smpi/mpi/smpi_f2c.cpp @@ -90,11 +90,8 @@ F2C* F2C::f2c(int id) if(id >= 0){ char key[KEY_SIZE]; - try { - return f2c_lookup_->at(get_key(key, id)); - } catch (std::out_of_range& unfound) { - return nullptr; - } + auto comm = f2c_lookup_->find(get_key(key, id)); + return comm == f2c_lookup_->end() ? nullptr : comm->second; }else return nullptr; } diff --git a/src/smpi/mpi/smpi_group.cpp b/src/smpi/mpi/smpi_group.cpp index 8cc21c33e7..232076857d 100644 --- a/src/smpi/mpi/smpi_group.cpp +++ b/src/smpi/mpi/smpi_group.cpp @@ -73,11 +73,8 @@ int Group::rank(int index) { if (this == MPI_GROUP_EMPTY) return MPI_UNDEFINED; - try { - return index_to_rank_map_.at(index); - } catch (std::out_of_range& unfound) { - return MPI_UNDEFINED; - } + auto rank = index_to_rank_map_.find(index); + return rank == index_to_rank_map_.end() ? MPI_UNDEFINED : rank->second; } void Group::ref() diff --git a/src/smpi/mpi/smpi_info.cpp b/src/smpi/mpi/smpi_info.cpp index ff3c8535d8..3698e30d40 100644 --- a/src/smpi/mpi/smpi_info.cpp +++ b/src/smpi/mpi/smpi_info.cpp @@ -32,15 +32,16 @@ void Info::set(char *key, char *value){ int Info::get(char *key, int valuelen, char *value, int *flag){ *flag=false; - try { - std::string tmpvalue = map_.at(key); + auto val = map_.find(key); + if (val != map_.end()) { + std::string tmpvalue = val->second; memset(value, 0, valuelen); memcpy(value, tmpvalue.c_str(), (tmpvalue.length() + 1 < static_cast(valuelen)) ? tmpvalue.length() + 1 : valuelen); *flag=true; return MPI_SUCCESS; - } catch (std::out_of_range& unfound) { + } else { return MPI_ERR_INFO_KEY; } } @@ -71,11 +72,12 @@ int Info::get_nthkey(int n, char *key){ int Info::get_valuelen(char *key, int *valuelen, int *flag){ *flag=false; - try { - *valuelen = map_.at(key).length(); + auto val = map_.find(key); + if (val != map_.end()) { + *valuelen = val->second.length(); *flag=true; return MPI_SUCCESS; - } catch (std::out_of_range& unfound) { + } else { return MPI_ERR_INFO_KEY; } } diff --git a/src/smpi/mpi/smpi_request.cpp b/src/smpi/mpi/smpi_request.cpp index 7007143bcc..0edf5281b6 100644 --- a/src/smpi/mpi/smpi_request.cpp +++ b/src/smpi/mpi/smpi_request.cpp @@ -221,11 +221,11 @@ MPI_Request Request::rma_recv_init(void *buf, int count, MPI_Datatype datatype, { MPI_Request request = nullptr; /* MC needs the comm to be set to nullptr during the call */ if(op==MPI_OP_NULL){ - request = new Request(buf==MPI_BOTTOM ? nullptr : buf, count, datatype, src, dst, tag, - comm, RMA | NON_PERSISTENT | RECV | PREPARED); + request = new Request(buf == MPI_BOTTOM ? nullptr : buf, count, datatype, src, dst, tag, comm, + RMA | NON_PERSISTENT | RECV | PREPARED); }else{ - request = new Request(buf==MPI_BOTTOM ? nullptr : buf, count, datatype, src, dst, tag, - comm, RMA | NON_PERSISTENT | RECV | PREPARED | ACCUMULATE); + request = new Request(buf == MPI_BOTTOM ? nullptr : buf, count, datatype, src, dst, tag, comm, + RMA | NON_PERSISTENT | RECV | PREPARED | ACCUMULATE); request->op_ = op; } return request; @@ -233,16 +233,16 @@ MPI_Request Request::rma_recv_init(void *buf, int count, MPI_Datatype datatype, MPI_Request Request::irecv_init(void *buf, int count, MPI_Datatype datatype, int src, int tag, MPI_Comm comm) { - return new Request(buf==MPI_BOTTOM ? nullptr : buf, count, datatype, src == MPI_ANY_SOURCE ? MPI_ANY_SOURCE : - comm->group()->index(src), smpi_process()->index(), tag, - comm, PERSISTENT | RECV | PREPARED); + return new Request(buf == MPI_BOTTOM ? nullptr : buf, count, datatype, + src == MPI_ANY_SOURCE ? MPI_ANY_SOURCE : comm->group()->index(src), smpi_process()->index(), tag, + comm, PERSISTENT | RECV | PREPARED); } MPI_Request Request::isend(void *buf, int count, MPI_Datatype datatype, int dst, int tag, MPI_Comm comm) { MPI_Request request = nullptr; /* MC needs the comm to be set to nullptr during the call */ - request = new Request(buf==MPI_BOTTOM ? nullptr : buf, count, datatype, smpi_process()->index(), - comm->group()->index(dst), tag, comm, NON_PERSISTENT | ISEND | SEND); + request = new Request(buf == MPI_BOTTOM ? nullptr : buf, count, datatype, smpi_process()->index(), + comm->group()->index(dst), tag, comm, NON_PERSISTENT | ISEND | SEND); request->start(); return request; } @@ -250,8 +250,8 @@ MPI_Request Request::isend(void *buf, int count, MPI_Datatype datatype, int dst, MPI_Request Request::issend(void *buf, int count, MPI_Datatype datatype, int dst, int tag, MPI_Comm comm) { MPI_Request request = nullptr; /* MC needs the comm to be set to nullptr during the call */ - request = new Request(buf==MPI_BOTTOM ? nullptr : buf, count, datatype, smpi_process()->index(), - comm->group()->index(dst), tag,comm, NON_PERSISTENT | ISEND | SSEND | SEND); + request = new Request(buf == MPI_BOTTOM ? nullptr : buf, count, datatype, smpi_process()->index(), + comm->group()->index(dst), tag, comm, NON_PERSISTENT | ISEND | SSEND | SEND); request->start(); return request; } @@ -260,9 +260,9 @@ MPI_Request Request::issend(void *buf, int count, MPI_Datatype datatype, int dst MPI_Request Request::irecv(void *buf, int count, MPI_Datatype datatype, int src, int tag, MPI_Comm comm) { MPI_Request request = nullptr; /* MC needs the comm to be set to nullptr during the call */ - request = new Request(buf==MPI_BOTTOM ? nullptr : buf, count, datatype, src == MPI_ANY_SOURCE ? MPI_ANY_SOURCE : - comm->group()->index(src), smpi_process()->index(), tag, comm, - NON_PERSISTENT | RECV); + request = new Request(buf == MPI_BOTTOM ? nullptr : buf, count, datatype, + src == MPI_ANY_SOURCE ? MPI_ANY_SOURCE : comm->group()->index(src), smpi_process()->index(), + tag, comm, NON_PERSISTENT | RECV); request->start(); return request; } @@ -278,8 +278,8 @@ void Request::recv(void *buf, int count, MPI_Datatype datatype, int src, int tag void Request::send(void *buf, int count, MPI_Datatype datatype, int dst, int tag, MPI_Comm comm) { MPI_Request request = nullptr; /* MC needs the comm to be set to nullptr during the call */ - request = new Request(buf==MPI_BOTTOM ? nullptr : buf, count, datatype, smpi_process()->index(), - comm->group()->index(dst), tag, comm, NON_PERSISTENT | SEND); + request = new Request(buf == MPI_BOTTOM ? nullptr : buf, count, datatype, smpi_process()->index(), + comm->group()->index(dst), tag, comm, NON_PERSISTENT | SEND); request->start(); wait(&request, MPI_STATUS_IGNORE); @@ -289,8 +289,8 @@ void Request::send(void *buf, int count, MPI_Datatype datatype, int dst, int tag void Request::ssend(void *buf, int count, MPI_Datatype datatype, int dst, int tag, MPI_Comm comm) { MPI_Request request = nullptr; /* MC needs the comm to be set to nullptr during the call */ - request = new Request(buf==MPI_BOTTOM ? nullptr : buf, count, datatype, smpi_process()->index(), - comm->group()->index(dst), tag, comm, NON_PERSISTENT | SSEND | SEND); + request = new Request(buf == MPI_BOTTOM ? nullptr : buf, count, datatype, smpi_process()->index(), + comm->group()->index(dst), tag, comm, NON_PERSISTENT | SSEND | SEND); request->start(); wait(&request,MPI_STATUS_IGNORE); diff --git a/src/surf/FileImpl.cpp b/src/surf/FileImpl.cpp index e454c3a073..ff0a830369 100644 --- a/src/surf/FileImpl.cpp +++ b/src/surf/FileImpl.cpp @@ -17,9 +17,10 @@ FileImpl::FileImpl(sg_storage_t st, std::string path, std::string mount) : path_ location_ = st->getImpl(); std::map* content = location_->getContent(); // if file does not exist create an empty file - try { - size_ = content->at(path); - } catch (std::out_of_range& unfound) { + auto sz = content->find(path); + if (sz != content->end()) { + size_ = sz->second; + } else { size_ = 0; content->insert({path, size_}); XBT_DEBUG("File '%s' was not found, file created.", path.c_str()); @@ -91,22 +92,23 @@ int FileImpl::unlink() } } -void FileImpl::move(const char* fullpath) +void FileImpl::move(std::string fullpath) { /* Check if the new full path is on the same mount point */ - if (not strncmp(mount_point_.c_str(), fullpath, mount_point_.size())) { + if (not strncmp(mount_point_.c_str(), fullpath.c_str(), mount_point_.size())) { std::map* content = location_->getContent(); - try { // src file exists - sg_size_t new_size = content->at(path_); + auto sz = content->find(path_); + if (sz != content->end()) { // src file exists + sg_size_t new_size = sz->second; content->erase(path_); - std::string path = std::string(fullpath).substr(mount_point_.size(), strlen(fullpath)); + std::string path = fullpath.substr(mount_point_.length(), fullpath.length()); content->insert({path.c_str(), new_size}); - XBT_DEBUG("Move file from %s to %s, size '%llu'", path_.c_str(), fullpath, new_size); - } catch (std::out_of_range& unfound) { + XBT_DEBUG("Move file from %s to %s, size '%llu'", path_.c_str(), fullpath.c_str(), new_size); + } else { XBT_WARN("File %s doesn't exist", path_.c_str()); } } else { - XBT_WARN("New full path %s is not on the same mount point: %s.", fullpath, mount_point_.c_str()); + XBT_WARN("New full path %s is not on the same mount point: %s.", fullpath.c_str(), mount_point_.c_str()); } } } diff --git a/src/surf/FileImpl.hpp b/src/surf/FileImpl.hpp index 0478ea1097..6b2f9cce25 100644 --- a/src/surf/FileImpl.hpp +++ b/src/surf/FileImpl.hpp @@ -28,7 +28,7 @@ public: sg_size_t tell() { return current_position_; } int seek(sg_offset_t offset, int origin); int unlink(); - void move(const char* fullpath); + void move(std::string fullpath); Action* read(sg_size_t size); Action* write(sg_size_t size); diff --git a/src/surf/PropertyHolder.cpp b/src/surf/PropertyHolder.cpp index a6696aee1e..e7657a277d 100644 --- a/src/surf/PropertyHolder.cpp +++ b/src/surf/PropertyHolder.cpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2015. The SimGrid Team. All rights reserved. */ +/* Copyright (c) 2015-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ @@ -16,11 +16,8 @@ PropertyHolder::~PropertyHolder() { const char *PropertyHolder::getProperty(const char*key) { if (properties_ == nullptr) return nullptr; - try { - return properties_->at(key).c_str(); - } catch (std::out_of_range& unfound) { - return nullptr; - } + auto prop = properties_->find(key); + return prop == properties_->end() ? nullptr : prop->second.c_str(); } /** @brief Change the value of a given key in the property set */ diff --git a/src/surf/StorageImpl.cpp b/src/surf/StorageImpl.cpp index b162a6e2e9..122bfde350 100644 --- a/src/surf/StorageImpl.cpp +++ b/src/surf/StorageImpl.cpp @@ -32,7 +32,7 @@ simgrid::xbt::signal storage std::unordered_map* StorageImpl::storages = new std::unordered_map(); -StorageImpl* StorageImpl::byName(const char* name) +StorageImpl* StorageImpl::byName(std::string name) { if (storages->find(name) == storages->end()) return nullptr; @@ -89,7 +89,7 @@ std::map* StorageImpl::parseContent(std::string filename std::map* parse_content = new std::map(); - std::ifstream* fs = surf_ifsopen(filename.c_str()); + std::ifstream* fs = surf_ifsopen(filename); std::string line; std::vector tokens; diff --git a/src/surf/StorageImpl.hpp b/src/surf/StorageImpl.hpp index 18835f3696..b74d624435 100644 --- a/src/surf/StorageImpl.hpp +++ b/src/surf/StorageImpl.hpp @@ -91,7 +91,7 @@ public: /** @brief Public interface */ s4u::Storage piface_; - static StorageImpl* byName(const char* name); + static StorageImpl* byName(std::string name); /** @brief Check if the Storage is used (if an action currently uses its resources) */ bool isUsed() override; @@ -120,7 +120,7 @@ public: /** * @brief Get the content of the current Storage * - * @return A xbt_dict_t with path as keys and size in bytes as values + * @return A map with path as keys and size in bytes as values */ virtual std::map* getContent(); diff --git a/src/surf/cpu_interface.cpp b/src/surf/cpu_interface.cpp index 955de3a1bc..caf30b7b20 100644 --- a/src/surf/cpu_interface.cpp +++ b/src/surf/cpu_interface.cpp @@ -210,6 +210,18 @@ void CpuAction::updateRemainingLazy(double now) simgrid::xbt::signal CpuAction::onStateChange; +void CpuAction::suspend(){ + Action::State previous = getState(); + onStateChange(this, previous); + Action::suspend(); +} + +void CpuAction::resume(){ + Action::State previous = getState(); + onStateChange(this, previous); + Action::resume(); +} + void CpuAction::setState(Action::State state){ Action::State previous = getState(); Action::setState(state); diff --git a/src/surf/cpu_interface.hpp b/src/surf/cpu_interface.hpp index 920f92f6f0..fc67386f29 100644 --- a/src/surf/cpu_interface.hpp +++ b/src/surf/cpu_interface.hpp @@ -168,6 +168,9 @@ static simgrid::xbt::signal onShareChange; void updateRemainingLazy(double now) override; std::list cpus(); + + void suspend() override; + void resume() override; }; } diff --git a/src/surf/instr_routing.cpp b/src/surf/instr_routing.cpp index 588afb731b..c69b4d85e2 100644 --- a/src/surf/instr_routing.cpp +++ b/src/surf/instr_routing.cpp @@ -111,7 +111,7 @@ static void linkContainers (container_t src, container_t dst, xbt_dict_t filter) } //register EDGE types for triva configuration - xbt_dict_set (trivaEdgeTypes, link_type->name, xbt_strdup("1"), nullptr); + trivaEdgeTypes.insert(link_type->name); //create the link static long long counter = 0; diff --git a/src/surf/instr_surf.cpp b/src/surf/instr_surf.cpp index df986ab2ad..4d45aa1169 100644 --- a/src/surf/instr_surf.cpp +++ b/src/surf/instr_surf.cpp @@ -9,16 +9,6 @@ XBT_LOG_NEW_DEFAULT_SUBCATEGORY(instr_surf, instr, "Tracing Surf"); -void TRACE_surf_alloc() -{ - TRACE_surf_resource_utilization_alloc(); -} - -void TRACE_surf_release() -{ - TRACE_surf_resource_utilization_release(); -} - void TRACE_surf_host_set_speed(double date, const char *resource, double speed) { if (TRACE_categorized() || TRACE_uncategorized() || TRACE_platform()) { diff --git a/src/surf/network_cm02.cpp b/src/surf/network_cm02.cpp index 5a4eb90592..54418ed840 100644 --- a/src/surf/network_cm02.cpp +++ b/src/surf/network_cm02.cpp @@ -13,7 +13,6 @@ XBT_LOG_EXTERNAL_DEFAULT_CATEGORY(surf_network); -double sg_sender_gap = 0.0; double sg_latency_factor = 1.0; /* default value; can be set by model or from command line */ double sg_bandwidth_factor = 1.0; /* default value; can be set by model or from command line */ double sg_weight_S_parameter = 0.0; /* default value; can be set by model or from command line */ @@ -314,14 +313,6 @@ Action* NetworkCm02Model::communicate(s4u::Host* src, s4u::Host* dst, double siz action->latCurrent_ = action->latency_; action->latency_ *= latencyFactor(size); action->rate_ = bandwidthConstraint(action->rate_, bandwidth_bound, size); - if (haveGap_) { - xbt_assert(not route->empty(), - "Using a model with a gap (e.g., SMPI) with a platform without links (e.g. vivaldi)!!!"); - - gapAppend(size, route->at(0), action); - XBT_DEBUG("Comm %p: %s -> %s gap=%f (lat=%f)", action, src->getCname(), dst->getCname(), action->senderGap_, - action->latency_); - } int constraints_per_variable = route->size(); if (back_route != nullptr) @@ -364,10 +355,6 @@ Action* NetworkCm02Model::communicate(s4u::Host* src, s4u::Host* dst, double siz return action; } -void NetworkCm02Model::gapAppend(double size, const LinkImpl* link, NetworkAction* action){ - // Nothing -}; - /************ * Resource * ************/ @@ -514,8 +501,5 @@ void NetworkCm02Action::updateRemainingLazy(double now) lastValue_ = lmm_variable_getvalue(getVariable()); } -void NetworkCm02Link::gapAppend(double size, const LinkImpl* link, NetworkAction* action){ - // Nothing -}; } } diff --git a/src/surf/network_cm02.hpp b/src/surf/network_cm02.hpp index a04f21518f..8ffa8ef3b0 100644 --- a/src/surf/network_cm02.hpp +++ b/src/surf/network_cm02.hpp @@ -43,10 +43,6 @@ public: void updateActionsStateLazy(double now, double delta) override; void updateActionsStateFull(double now, double delta) override; Action* communicate(s4u::Host* src, s4u::Host* dst, double size, double rate) override; - virtual void gapAppend(double size, const LinkImpl* link, NetworkAction* action); - -protected: - bool haveGap_ = false; }; /************ @@ -61,7 +57,6 @@ public: void apply_event(tmgr_trace_event_t event, double value) override; void setBandwidth(double value) override; void setLatency(double value) override; - virtual void gapAppend(double size, const LinkImpl* link, NetworkAction* action); }; /********** @@ -75,9 +70,6 @@ public: NetworkCm02Action(Model* model, double cost, bool failed) : NetworkAction(model, cost, failed){}; virtual ~NetworkCm02Action() = default; void updateRemainingLazy(double now) override; - -protected: - double senderGap_; }; } } diff --git a/src/surf/network_ib.cpp b/src/surf/network_ib.cpp index 6018e1d2dc..7756239812 100644 --- a/src/surf/network_ib.cpp +++ b/src/surf/network_ib.cpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2014-2015. The SimGrid Team. +/* Copyright (c) 2014-2017. The SimGrid Team. *All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -52,15 +52,17 @@ static void IB_action_init_callback(simgrid::surf::NetworkAction* action, simgri simgrid::surf::IBNode* act_src; simgrid::surf::IBNode* act_dst; - try { - act_src = ibModel->active_nodes.at(src->getName()); - } catch (std::out_of_range& unfound) { + auto asrc = ibModel->active_nodes.find(src->getName()); + if (asrc != ibModel->active_nodes.end()) { + act_src = asrc->second; + } else { throw std::out_of_range(std::string("Could not find '") + src->getCname() + "' active comms !"); } - try { - act_dst = ibModel->active_nodes.at(dst->getName()); - } catch (std::out_of_range& unfound) { + auto adst = ibModel->active_nodes.find(dst->getName()); + if (adst != ibModel->active_nodes.end()) { + act_dst = adst->second; + } else { throw std::out_of_range(std::string("Could not find '") + dst->getCname() + "' active comms !"); } @@ -102,7 +104,6 @@ namespace surf { NetworkIBModel::NetworkIBModel() : NetworkSmpiModel() { - haveGap_ = false; const char* IB_factors_string = xbt_cfg_get_string("smpi/IB-penalty-factors"); std::vector radical_elements; boost::split(radical_elements, IB_factors_string, boost::is_any_of(";")); diff --git a/src/surf/network_interface.cpp b/src/surf/network_interface.cpp index fdb99fcd11..668f4ed2b4 100644 --- a/src/surf/network_interface.cpp +++ b/src/surf/network_interface.cpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2013-2015. The SimGrid Team. +/* Copyright (c) 2013-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -20,13 +20,10 @@ namespace simgrid { /* List of links */ std::unordered_map* LinkImpl::links = new std::unordered_map(); - LinkImpl* LinkImpl::byName(const char* name) + LinkImpl* LinkImpl::byName(std::string name) { - try { - return links->at(name); - } catch (std::out_of_range& unfound) { - return nullptr; - } + auto link = links->find(name); + return link == links->end() ? nullptr : link->second; } /** @brief Returns the amount of links in the platform */ int LinkImpl::linksCount() diff --git a/src/surf/network_interface.hpp b/src/surf/network_interface.hpp index 59c93ccc0e..be484919a7 100644 --- a/src/surf/network_interface.hpp +++ b/src/surf/network_interface.hpp @@ -171,7 +171,7 @@ private: static std::unordered_map* links; public: - static LinkImpl* byName(const char* name); + static LinkImpl* byName(std::string name); static int linksCount(); static LinkImpl** linksList(); static void linksExit(); diff --git a/src/surf/network_ns3.cpp b/src/surf/network_ns3.cpp index fb0f9cc7d2..34631b53a2 100644 --- a/src/surf/network_ns3.cpp +++ b/src/surf/network_ns3.cpp @@ -54,7 +54,7 @@ NetPointNs3::NetPointNs3() * Callbacks * *************/ -static void clusterCreation_cb(sg_platf_cluster_cbarg_t cluster) +static void clusterCreation_cb(ClusterCreationArgs* cluster) { for (int i : *cluster->radicals) { // Routers don't create a router on the other end of the private link by themselves. @@ -62,19 +62,18 @@ static void clusterCreation_cb(sg_platf_cluster_cbarg_t cluster) NetPointNs3* host_dst = new NetPointNs3(); // Create private link - char* host_id = bprintf("%s%d%s", cluster->prefix, i, cluster->suffix); - NetPointNs3* host_src = sg_host_by_name(host_id)->pimpl_netpoint->extension(); - xbt_assert(host_src, "Cannot find a NS3 host of name %s", host_id); + std::string host_id = cluster->prefix + std::to_string(i) + cluster->suffix; + NetPointNs3* host_src = sg_host_by_name(host_id.c_str())->pimpl_netpoint->extension(); + xbt_assert(host_src, "Cannot find a NS3 host of name %s", host_id.c_str()); // Any NS3 route is symmetrical ns3_add_link(host_src, host_dst, cluster->bw, cluster->lat); delete host_dst; - free(host_id); } //Create link backbone - ns3_add_cluster(cluster->id, cluster->bb_bw, cluster->bb_lat); + ns3_add_cluster(cluster->id.c_str(), cluster->bb_bw, cluster->bb_lat); } static void routeCreation_cb(bool symmetrical, simgrid::kernel::routing::NetPoint* src, diff --git a/src/surf/network_smpi.cpp b/src/surf/network_smpi.cpp index 411a467b6c..b559d234c1 100644 --- a/src/surf/network_smpi.cpp +++ b/src/surf/network_smpi.cpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2013-2015. The SimGrid Team. +/* Copyright (c) 2013-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -18,8 +18,6 @@ XBT_LOG_EXTERNAL_DEFAULT_CATEGORY(surf_network); std::vector smpi_bw_factor; std::vector smpi_lat_factor; -xbt_dict_t gap_lookup = nullptr; - /********* * Model * *********/ @@ -42,73 +40,65 @@ void surf_network_model_init_SMPI() surf_network_model = new simgrid::surf::NetworkSmpiModel(); all_existing_models->push_back(surf_network_model); - xbt_cfg_setdefault_double("network/sender-gap", 10e-6); xbt_cfg_setdefault_double("network/weight-S", 8775); } namespace simgrid { - namespace surf { - - NetworkSmpiModel::NetworkSmpiModel() : NetworkCm02Model() - { - haveGap_ = true; - } - - NetworkSmpiModel::~NetworkSmpiModel() - { - xbt_dict_free(&gap_lookup); - } - - double NetworkSmpiModel::bandwidthFactor(double size) - { - if (smpi_bw_factor.empty()) - smpi_bw_factor = parse_factor(xbt_cfg_get_string("smpi/bw-factor")); - - double current = 1.0; - for (const auto& fact : smpi_bw_factor) { - if (size <= fact.factor) { - XBT_DEBUG("%f <= %zu return %f", size, fact.factor, current); - return current; - } else - current = fact.values.front(); - } - XBT_DEBUG("%f > %zu return %f", size, smpi_bw_factor.back().factor, current); +namespace surf { - return current; - } - - double NetworkSmpiModel::latencyFactor(double size) - { - if (smpi_lat_factor.empty()) - smpi_lat_factor = parse_factor(xbt_cfg_get_string("smpi/lat-factor")); - - double current=1.0; - for (const auto& fact : smpi_lat_factor) { - if (size <= fact.factor) { - XBT_DEBUG("%f <= %zu return %f", size, fact.factor, current); - return current; - }else - current=fact.values.front(); - } - XBT_DEBUG("%f > %zu return %f", size, smpi_lat_factor.back().factor, current); +NetworkSmpiModel::NetworkSmpiModel() : NetworkCm02Model() +{ +} + +NetworkSmpiModel::~NetworkSmpiModel() = default; +double NetworkSmpiModel::bandwidthFactor(double size) +{ + if (smpi_bw_factor.empty()) + smpi_bw_factor = parse_factor(xbt_cfg_get_string("smpi/bw-factor")); + + double current = 1.0; + for (const auto& fact : smpi_bw_factor) { + if (size <= fact.factor) { + XBT_DEBUG("%f <= %zu return %f", size, fact.factor, current); return current; - } + } else + current = fact.values.front(); + } + XBT_DEBUG("%f > %zu return %f", size, smpi_bw_factor.back().factor, current); + + return current; +} - double NetworkSmpiModel::bandwidthConstraint(double rate, double bound, double size) - { - return rate < 0 ? bound : std::min(bound, rate * bandwidthFactor(size)); - } +double NetworkSmpiModel::latencyFactor(double size) +{ + if (smpi_lat_factor.empty()) + smpi_lat_factor = parse_factor(xbt_cfg_get_string("smpi/lat-factor")); - /************ - * Resource * - ************/ + double current = 1.0; + for (const auto& fact : smpi_lat_factor) { + if (size <= fact.factor) { + XBT_DEBUG("%f <= %zu return %f", size, fact.factor, current); + return current; + } else + current = fact.values.front(); + } + XBT_DEBUG("%f > %zu return %f", size, smpi_lat_factor.back().factor, current); + return current; +} +double NetworkSmpiModel::bandwidthConstraint(double rate, double bound, double size) +{ + return rate < 0 ? bound : std::min(bound, rate * bandwidthFactor(size)); +} - /********** - * Action * - **********/ +/************ + * Resource * + ************/ - } +/********** + * Action * + **********/ +} } diff --git a/src/surf/network_smpi.hpp b/src/surf/network_smpi.hpp index 7a2f4db3ec..1ab37a73c2 100644 --- a/src/surf/network_smpi.hpp +++ b/src/surf/network_smpi.hpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2013-2015. The SimGrid Team. +/* Copyright (c) 2013-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -16,7 +16,6 @@ namespace simgrid { NetworkSmpiModel(); ~NetworkSmpiModel(); - using NetworkCm02Model::gapAppend; // Explicit about overloaded method (silence Woverloaded-virtual from clang) double latencyFactor(double size); double bandwidthFactor(double size); double bandwidthConstraint(double rate, double bound, double size); diff --git a/src/surf/sg_platf.cpp b/src/surf/sg_platf.cpp index d0ef69e9f9..474ef71fad 100644 --- a/src/surf/sg_platf.cpp +++ b/src/surf/sg_platf.cpp @@ -35,8 +35,7 @@ XBT_PRIVATE std::vector known_storages; namespace simgrid { namespace surf { -simgrid::xbt::signal on_cluster; - +simgrid::xbt::signal on_cluster; } } @@ -100,18 +99,18 @@ void sg_platf_new_host(sg_platf_host_cbarg_t args) } /** @brief Add a "router" to the network element list */ -simgrid::kernel::routing::NetPoint* sg_platf_new_router(const char* name, const char* coords) +simgrid::kernel::routing::NetPoint* sg_platf_new_router(std::string name, const char* coords) { simgrid::kernel::routing::NetZoneImpl* current_routing = routing_get_current(); if (current_routing->hierarchy_ == simgrid::kernel::routing::NetZoneImpl::RoutingMode::unset) current_routing->hierarchy_ = simgrid::kernel::routing::NetZoneImpl::RoutingMode::base; xbt_assert(nullptr == simgrid::s4u::Engine::getInstance()->getNetpointByNameOrNull(name), - "Refusing to create a router named '%s': this name already describes a node.", name); + "Refusing to create a router named '%s': this name already describes a node.", name.c_str()); simgrid::kernel::routing::NetPoint* netpoint = new simgrid::kernel::routing::NetPoint(name, simgrid::kernel::routing::NetPoint::Type::Router, current_routing); - XBT_DEBUG("Router '%s' has the id %u", name, netpoint->id()); + XBT_DEBUG("Router '%s' has the id %u", name.c_str(), netpoint->id()); if (coords && strcmp(coords, "")) new simgrid::kernel::routing::vivaldi::Coords(netpoint, coords); @@ -149,7 +148,7 @@ void sg_platf_new_link(LinkCreationArgs* link) } } -void sg_platf_new_cluster(sg_platf_cluster_cbarg_t cluster) +void sg_platf_new_cluster(ClusterCreationArgs* cluster) { using simgrid::kernel::routing::ClusterZone; using simgrid::kernel::routing::DragonflyZone; @@ -159,23 +158,23 @@ void sg_platf_new_cluster(sg_platf_cluster_cbarg_t cluster) int rankId=0; // What an inventive way of initializing the AS that I have as ancestor :-( - s_sg_platf_AS_cbarg_t AS; - AS.id = cluster->id; + ZoneCreationArgs zone; + zone.id = cluster->id; switch (cluster->topology) { case SURF_CLUSTER_TORUS: - AS.routing = A_surfxml_AS_routing_ClusterTorus; + zone.routing = A_surfxml_AS_routing_ClusterTorus; break; case SURF_CLUSTER_DRAGONFLY: - AS.routing = A_surfxml_AS_routing_ClusterDragonfly; + zone.routing = A_surfxml_AS_routing_ClusterDragonfly; break; case SURF_CLUSTER_FAT_TREE: - AS.routing = A_surfxml_AS_routing_ClusterFatTree; + zone.routing = A_surfxml_AS_routing_ClusterFatTree; break; default: - AS.routing = A_surfxml_AS_routing_Cluster; + zone.routing = A_surfxml_AS_routing_Cluster; break; } - sg_platf_new_AS_begin(&AS); + sg_platf_new_Zone_begin(&zone); simgrid::kernel::routing::ClusterZone* current_as = static_cast(routing_get_current()); current_as->parse_specific_arguments(cluster); @@ -190,14 +189,14 @@ void sg_platf_new_cluster(sg_platf_cluster_cbarg_t cluster) } for (int i : *cluster->radicals) { - char * host_id = bprintf("%s%d%s", cluster->prefix, i, cluster->suffix); - char * link_id = bprintf("%s_link_%d", cluster->id, i); + std::string host_id = std::string(cluster->prefix) + std::to_string(i) + cluster->suffix; + std::string link_id = std::string(cluster->id) + "_link_" + std::to_string(i); - XBT_DEBUG("", host_id, cluster->speeds.front()); + XBT_DEBUG("", host_id.c_str(), cluster->speeds.front()); s_sg_platf_host_cbarg_t host; memset(&host, 0, sizeof(host)); - host.id = host_id; + host.id = host_id.c_str(); if ((cluster->properties != nullptr) && (not cluster->properties->empty())) { host.properties = new std::map; @@ -212,7 +211,7 @@ void sg_platf_new_cluster(sg_platf_cluster_cbarg_t cluster) sg_platf_new_host(&host); XBT_DEBUG(""); - XBT_DEBUG("", link_id, cluster->bw, cluster->lat); + XBT_DEBUG("", link_id.c_str(), cluster->bw, cluster->lat); // All links are saved in a matrix; // every row describes a single node; every node may have multiple links. @@ -224,17 +223,17 @@ void sg_platf_new_cluster(sg_platf_cluster_cbarg_t cluster) simgrid::surf::LinkImpl* linkUp = nullptr; simgrid::surf::LinkImpl* linkDown = nullptr; if(cluster->loopback_bw > 0 || cluster->loopback_lat > 0){ - std::string tmp_link = std::string(link_id) + "_loopback"; + std::string tmp_link = link_id + "_loopback"; XBT_DEBUG("", tmp_link.c_str(), cluster->loopback_bw); LinkCreationArgs link; - link.id = tmp_link.c_str(); + link.id = tmp_link; link.bandwidth = cluster->loopback_bw; link.latency = cluster->loopback_lat; link.policy = SURF_LINK_FATPIPE; sg_platf_new_link(&link); - linkUp = simgrid::surf::LinkImpl::byName(tmp_link.c_str()); - linkDown = simgrid::surf::LinkImpl::byName(tmp_link.c_str()); + linkUp = simgrid::surf::LinkImpl::byName(tmp_link); + linkDown = simgrid::surf::LinkImpl::byName(tmp_link); auto as_cluster = static_cast(current_as); as_cluster->privateLinks_.insert({rankId * as_cluster->linkCountPerNode_, {linkUp, linkDown}}); @@ -248,12 +247,12 @@ void sg_platf_new_cluster(sg_platf_cluster_cbarg_t cluster) XBT_DEBUG("", tmp_link.c_str(), cluster->limiter_link); LinkCreationArgs link; - link.id = tmp_link.c_str(); + link.id = tmp_link; link.bandwidth = cluster->limiter_link; link.latency = 0; link.policy = SURF_LINK_SHARED; sg_platf_new_link(&link); - linkDown = simgrid::surf::LinkImpl::byName(tmp_link.c_str()); + linkDown = simgrid::surf::LinkImpl::byName(tmp_link); linkUp = linkDown; current_as->privateLinks_.insert( {rankId * current_as->linkCountPerNode_ + current_as->hasLoopback_, {linkUp, linkDown}}); @@ -266,19 +265,16 @@ void sg_platf_new_cluster(sg_platf_cluster_cbarg_t cluster) current_as->create_links_for_node(cluster, i, rankId, rankId*current_as->linkCountPerNode_ + current_as->hasLoopback_ + current_as->hasLimiter_ ); } - xbt_free(link_id); - xbt_free(host_id); rankId++; } delete cluster->properties; // Add a router. XBT_DEBUG(" "); - XBT_DEBUG("", cluster->router_id); - if (not cluster->router_id || not strcmp(cluster->router_id, "")) { - char* newid = bprintf("%s%s_router%s", cluster->prefix, cluster->id, cluster->suffix); - current_as->router_ = sg_platf_new_router(newid, NULL); - free(newid); + XBT_DEBUG("", cluster->router_id.c_str()); + if (cluster->router_id.empty()) { + std::string newid = std::string(cluster->prefix) + cluster->id + "_router" + cluster->suffix; + current_as->router_ = sg_platf_new_router(newid.c_str(), NULL); } else { current_as->router_ = sg_platf_new_router(cluster->router_id, NULL); } @@ -295,11 +291,11 @@ void sg_platf_new_cluster(sg_platf_cluster_cbarg_t cluster) XBT_DEBUG("", link.id.c_str(), cluster->bb_bw, cluster->bb_lat); sg_platf_new_link(&link); - routing_cluster_add_backbone(simgrid::surf::LinkImpl::byName(link.id.c_str())); + routing_cluster_add_backbone(simgrid::surf::LinkImpl::byName(link.id)); } XBT_DEBUG(""); - sg_platf_new_AS_seal(); + sg_platf_new_Zone_seal(); simgrid::surf::on_cluster(cluster); delete cluster->radicals; @@ -317,10 +313,10 @@ void routing_cluster_add_backbone(simgrid::surf::LinkImpl* bb) XBT_DEBUG("Add a backbone to AS '%s'", current_routing->getCname()); } -void sg_platf_new_cabinet(sg_platf_cabinet_cbarg_t cabinet) +void sg_platf_new_cabinet(CabinetCreationArgs* cabinet) { for (int radical : *cabinet->radicals) { - std::string hostname = std::string(cabinet->prefix) + std::to_string(radical) + std::string(cabinet->suffix); + std::string hostname = cabinet->prefix + std::to_string(radical) + cabinet->suffix; s_sg_platf_host_cbarg_t host; memset(&host, 0, sizeof(host)); host.pstate = 0; @@ -336,13 +332,10 @@ void sg_platf_new_cabinet(sg_platf_cabinet_cbarg_t cabinet) link.id = "link_" + hostname; sg_platf_new_link(&link); - s_sg_platf_host_link_cbarg_t host_link; - memset(&host_link, 0, sizeof(host_link)); - std::string tmp_link_up = std::string("link_") + hostname + "_UP"; - std::string tmp_link_down = std::string("link_") + hostname + "_DOWN"; - host_link.id = hostname.c_str(); - host_link.link_up = tmp_link_up.c_str(); - host_link.link_down = tmp_link_down.c_str(); + HostLinkCreationArgs host_link; + host_link.id = hostname; + host_link.link_up = std::string("link_") + hostname + "_UP"; + host_link.link_down = std::string("link_") + hostname + "_DOWN"; sg_platf_new_hostlink(&host_link); } delete cabinet->radicals; @@ -354,9 +347,10 @@ void sg_platf_new_storage(StorageCreationArgs* storage) "Refusing to add a second storage named \"%s\"", storage->id.c_str()); simgrid::surf::StorageType* stype; - try { - stype = storage_types.at(storage->type_id); - } catch (std::out_of_range& unfound) { + auto st = storage_types.find(storage->type_id); + if (st != storage_types.end()) { + stype = st->second; + } else { xbt_die("No storage type '%s'", storage->type_id.c_str()); } @@ -461,7 +455,7 @@ void sg_platf_new_process(sg_platf_process_cbarg_t process) arg->data = nullptr; arg->host = host; arg->kill_time = kill_time; - arg->properties = current_property_set; + arg->properties = process->properties; host->extension()->boot_processes.push_back(arg); @@ -473,7 +467,7 @@ void sg_platf_new_process(sg_platf_process_cbarg_t process) arg->data = nullptr; arg->host = host; arg->kill_time = kill_time; - arg->properties = current_property_set; + arg->properties = process->properties; XBT_DEBUG("Process %s@%s will be started at time %f", arg->name.c_str(), arg->host->getCname(), start_time); SIMIX_timer_set(start_time, [arg, auto_restart]() { @@ -489,7 +483,7 @@ void sg_platf_new_process(sg_platf_process_cbarg_t process) XBT_DEBUG("Starting Process %s(%s) right now", arg->name.c_str(), host->getCname()); smx_actor_t actor = simix_global->create_process_function(arg->name.c_str(), std::move(code), nullptr, host, - current_property_set, nullptr); + arg->properties, nullptr); /* The actor creation will fail if the host is currently dead, but that's fine */ if (actor != nullptr) { @@ -566,18 +560,16 @@ static void surf_config_models_setup() } /** - * \brief Add an AS to the platform + * \brief Add a Zone to the platform * - * Add a new autonomous system to the platform. Any elements (such as host, - * router or sub-AS) added after this call and before the corresponding call - * to sg_platf_new_AS_seal() will be added to this AS. + * Add a new autonomous system to the platform. Any elements (such as host, router or sub-Zone) added after this call + * and before the corresponding call to sg_platf_new_Zone_seal() will be added to this Zone. * - * Once this function was called, the configuration concerning the used - * models cannot be changed anymore. + * Once this function was called, the configuration concerning the used models cannot be changed anymore. * - * @param AS the parameters defining the AS to build. + * @param zone the parameters defining the Zone to build. */ -simgrid::s4u::NetZone* sg_platf_new_AS_begin(sg_platf_AS_cbarg_t AS) +simgrid::s4u::NetZone* sg_platf_new_Zone_begin(ZoneCreationArgs* zone) { if (not surf_parse_models_setup_already_called) { /* Initialize the surf models. That must be done after we got all config, and before we need the models. @@ -595,37 +587,37 @@ simgrid::s4u::NetZone* sg_platf_new_AS_begin(sg_platf_AS_cbarg_t AS) * any further config now that we created some real content */ /* search the routing model */ - simgrid::kernel::routing::NetZoneImpl* new_as = nullptr; - switch(AS->routing){ + simgrid::kernel::routing::NetZoneImpl* new_zone = nullptr; + switch (zone->routing) { case A_surfxml_AS_routing_Cluster: - new_as = new simgrid::kernel::routing::ClusterZone(current_routing, AS->id); + new_zone = new simgrid::kernel::routing::ClusterZone(current_routing, zone->id); break; case A_surfxml_AS_routing_ClusterDragonfly: - new_as = new simgrid::kernel::routing::DragonflyZone(current_routing, AS->id); + new_zone = new simgrid::kernel::routing::DragonflyZone(current_routing, zone->id); break; case A_surfxml_AS_routing_ClusterTorus: - new_as = new simgrid::kernel::routing::TorusZone(current_routing, AS->id); + new_zone = new simgrid::kernel::routing::TorusZone(current_routing, zone->id); break; case A_surfxml_AS_routing_ClusterFatTree: - new_as = new simgrid::kernel::routing::FatTreeZone(current_routing, AS->id); + new_zone = new simgrid::kernel::routing::FatTreeZone(current_routing, zone->id); break; case A_surfxml_AS_routing_Dijkstra: - new_as = new simgrid::kernel::routing::DijkstraZone(current_routing, AS->id, 0); + new_zone = new simgrid::kernel::routing::DijkstraZone(current_routing, zone->id, 0); break; case A_surfxml_AS_routing_DijkstraCache: - new_as = new simgrid::kernel::routing::DijkstraZone(current_routing, AS->id, 1); + new_zone = new simgrid::kernel::routing::DijkstraZone(current_routing, zone->id, 1); break; case A_surfxml_AS_routing_Floyd: - new_as = new simgrid::kernel::routing::FloydZone(current_routing, AS->id); + new_zone = new simgrid::kernel::routing::FloydZone(current_routing, zone->id); break; case A_surfxml_AS_routing_Full: - new_as = new simgrid::kernel::routing::FullZone(current_routing, AS->id); + new_zone = new simgrid::kernel::routing::FullZone(current_routing, zone->id); break; case A_surfxml_AS_routing_None: - new_as = new simgrid::kernel::routing::EmptyZone(current_routing, AS->id); + new_zone = new simgrid::kernel::routing::EmptyZone(current_routing, zone->id); break; case A_surfxml_AS_routing_Vivaldi: - new_as = new simgrid::kernel::routing::VivaldiZone(current_routing, AS->id); + new_zone = new simgrid::kernel::routing::VivaldiZone(current_routing, zone->id); break; default: xbt_die("Not a valid model!"); @@ -635,22 +627,22 @@ simgrid::s4u::NetZone* sg_platf_new_AS_begin(sg_platf_AS_cbarg_t AS) if (current_routing == nullptr) { /* it is the first one */ xbt_assert(simgrid::s4u::Engine::getInstance()->pimpl->netRoot_ == nullptr, "All defined components must belong to a networking zone."); - simgrid::s4u::Engine::getInstance()->pimpl->netRoot_ = new_as; + simgrid::s4u::Engine::getInstance()->pimpl->netRoot_ = new_zone; } else { /* set the father behavior */ if (current_routing->hierarchy_ == simgrid::kernel::routing::NetZoneImpl::RoutingMode::unset) current_routing->hierarchy_ = simgrid::kernel::routing::NetZoneImpl::RoutingMode::recursive; /* add to the sons dictionary */ - current_routing->getChildren()->push_back(static_cast(new_as)); + current_routing->getChildren()->push_back(static_cast(new_zone)); } /* set the new current component of the tree */ - current_routing = new_as; + current_routing = new_zone; - simgrid::s4u::NetZone::onCreation(*new_as); // notify the signal + simgrid::s4u::NetZone::onCreation(*new_zone); // notify the signal - return new_as; + return new_zone; } /** @@ -659,7 +651,7 @@ simgrid::s4u::NetZone* sg_platf_new_AS_begin(sg_platf_AS_cbarg_t AS) * Once you've declared all the content of your AS, you have to seal * it with this call. Your AS is not usable until you call this function. */ -void sg_platf_new_AS_seal() +void sg_platf_new_Zone_seal() { xbt_assert(current_routing, "Cannot seal the current AS: none under construction"); current_routing->seal(); @@ -668,36 +660,36 @@ void sg_platf_new_AS_seal() } /** @brief Add a link connecting an host to the rest of its AS (which must be cluster or vivaldi) */ -void sg_platf_new_hostlink(sg_platf_host_link_cbarg_t hostlink) +void sg_platf_new_hostlink(HostLinkCreationArgs* hostlink) { - simgrid::kernel::routing::NetPoint* netpoint = sg_host_by_name(hostlink->id)->pimpl_netpoint; - xbt_assert(netpoint, "Host '%s' not found!", hostlink->id); + simgrid::kernel::routing::NetPoint* netpoint = sg_host_by_name(hostlink->id.c_str())->pimpl_netpoint; + xbt_assert(netpoint, "Host '%s' not found!", hostlink->id.c_str()); xbt_assert(dynamic_cast(current_routing), "Only hosts from Cluster and Vivaldi ASes can get an host_link."); simgrid::surf::LinkImpl* linkUp = simgrid::surf::LinkImpl::byName(hostlink->link_up); simgrid::surf::LinkImpl* linkDown = simgrid::surf::LinkImpl::byName(hostlink->link_down); - xbt_assert(linkUp, "Link '%s' not found!", hostlink->link_up); - xbt_assert(linkDown, "Link '%s' not found!", hostlink->link_down); + xbt_assert(linkUp, "Link '%s' not found!", hostlink->link_up.c_str()); + xbt_assert(linkDown, "Link '%s' not found!", hostlink->link_down.c_str()); auto as_cluster = static_cast(current_routing); if (as_cluster->privateLinks_.find(netpoint->id()) != as_cluster->privateLinks_.end()) - surf_parse_error("Host_link for '%s' is already defined!",hostlink->id); + surf_parse_error(std::string("Host_link for '") + hostlink->id.c_str() + "' is already defined!"); XBT_DEBUG("Push Host_link for host '%s' to position %u", netpoint->cname(), netpoint->id()); as_cluster->privateLinks_.insert({netpoint->id(), {linkUp, linkDown}}); } -void sg_platf_new_trace(sg_platf_trace_cbarg_t trace) +void sg_platf_new_trace(TraceCreationArgs* trace) { tmgr_trace_t tmgr_trace; - if (trace->file && strcmp(trace->file, "") != 0) { + if (not trace->file.empty()) { tmgr_trace = tmgr_trace_new_from_file(trace->file); } else { - xbt_assert(strcmp(trace->pc_data, ""), - "Trace '%s' must have either a content, or point to a file on disk.",trace->id); + xbt_assert(not trace->pc_data.empty(), "Trace '%s' must have either a content, or point to a file on disk.", + trace->id.c_str()); tmgr_trace = tmgr_trace_new_from_string(trace->id, trace->pc_data, trace->periodicity); } traces_set_list.insert({trace->id, tmgr_trace}); diff --git a/src/surf/storage_n11.cpp b/src/surf/storage_n11.cpp index 5ae236e757..f026497dfb 100644 --- a/src/surf/storage_n11.cpp +++ b/src/surf/storage_n11.cpp @@ -20,8 +20,8 @@ static void check_disk_attachment() for (auto s : *simgrid::surf::StorageImpl::storagesMap()) { simgrid::kernel::routing::NetPoint* host_elm = sg_netpoint_by_name_or_null(s.second->getHost().c_str()); if (not host_elm) - surf_parse_error("Unable to attach storage %s: host %s does not exist.", s.second->cname(), - s.second->getHost().c_str()); + surf_parse_error(std::string("Unable to attach storage ") + s.second->cname() + ": host " + s.second->getHost() + + " does not exist."); else s.second->piface_.attached_to_ = sg_host_by_name(s.second->getHost().c_str()); } diff --git a/src/surf/surf_interface.cpp b/src/surf/surf_interface.cpp index 7973e5ec1f..39c8b9d258 100644 --- a/src/surf/surf_interface.cpp +++ b/src/surf/surf_interface.cpp @@ -39,8 +39,8 @@ simgrid::xbt::signal surfExitCallbacks; } } -#include // FIXME: this plugin should not be linked to the core -#include // FIXME: this plugin should not be linked to the core +#include // FIXME: this plug-in should not be linked to the core +#include // FIXME: this plug-in should not be linked to the core s_surf_model_description_t surf_plugin_description[] = { {"Energy", "Cpu energy consumption.", &sg_host_energy_plugin_init}, @@ -115,10 +115,6 @@ s_surf_model_description_t surf_storage_model_description[] = { {nullptr, nullptr, nullptr} /* this array must be nullptr terminated */ }; -#if HAVE_THREAD_CONTEXTS -static xbt_parmap_t surf_parmap = nullptr; /* parallel map on models */ -#endif - double NOW = 0; double surf_get_clock() @@ -132,12 +128,12 @@ double surf_get_clock() # define FILE_DELIM "/" /* FIXME: move to better location */ #endif -std::ifstream* surf_ifsopen(const char* name) +std::ifstream* surf_ifsopen(std::string name) { std::ifstream* fs = new std::ifstream(); - xbt_assert(name); - if (__surf_is_absolute_file_path(name)) { /* don't mess with absolute file names */ - fs->open(name, std::ifstream::in); + xbt_assert(not name.empty()); + if (__surf_is_absolute_file_path(name.c_str())) { /* don't mess with absolute file names */ + fs->open(name.c_str(), std::ifstream::in); } /* search relative files in the path */ @@ -153,6 +149,7 @@ std::ifstream* surf_ifsopen(const char* name) return fs; } + FILE *surf_fopen(const char *name, const char *mode) { FILE *file = nullptr; @@ -352,9 +349,6 @@ void surf_init(int *argc, char **argv) if (not future_evt_set) future_evt_set = new simgrid::trace_mgr::future_evt_set(); - TRACE_surf_alloc(); - simgrid::surf::surfExitCallbacks.connect(TRACE_surf_release); - sg_config_init(argc, argv); if (MC_is_active()) @@ -388,10 +382,6 @@ void surf_exit() future_evt_set = nullptr; } -#if HAVE_THREAD_CONTEXTS - xbt_parmap_destroy(surf_parmap); -#endif - tmgr_finalize(); sg_platf_exit(); simgrid::s4u::Engine::shutdown(); diff --git a/src/surf/surf_interface.hpp b/src/surf/surf_interface.hpp index 8ab050fd83..f465a21607 100644 --- a/src/surf/surf_interface.hpp +++ b/src/surf/surf_interface.hpp @@ -1,4 +1,4 @@ -/* Copyright (c) 2004-2016. The SimGrid Team. All rights reserved. */ +/* Copyright (c) 2004-2017. The SimGrid Team. All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ @@ -25,7 +25,6 @@ /* user-visible parameters */ extern XBT_PRIVATE double sg_tcp_gamma; -extern XBT_PRIVATE double sg_sender_gap; extern XBT_PRIVATE double sg_latency_factor; extern XBT_PRIVATE double sg_bandwidth_factor; extern XBT_PRIVATE double sg_weight_S_parameter; @@ -42,9 +41,6 @@ XBT_PUBLIC(double) surf_get_clock(); */ XBT_PUBLIC_DATA(std::vector) host_that_restart; - -extern XBT_PRIVATE double sg_sender_gap; - namespace simgrid { namespace surf { diff --git a/src/surf/surf_private.h b/src/surf/surf_private.h index 7d3fa1f45c..fc90b62a9a 100644 --- a/src/surf/surf_private.h +++ b/src/surf/surf_private.h @@ -31,7 +31,7 @@ typedef enum { /* Generic functions common to all models */ XBT_PRIVATE FILE *surf_fopen(const char *name, const char *mode); -XBT_PRIVATE std::ifstream* surf_ifsopen(const char* name); +XBT_PRIVATE std::ifstream* surf_ifsopen(std::string name); /* The __surf_is_absolute_file_path() returns 1 if * file_path is a absolute file path, in the other diff --git a/src/surf/trace_mgr.cpp b/src/surf/trace_mgr.cpp index f100e2e1a0..e9d4f785f3 100644 --- a/src/surf/trace_mgr.cpp +++ b/src/surf/trace_mgr.cpp @@ -24,7 +24,7 @@ XBT_LOG_NEW_DEFAULT_SUBCATEGORY(surf_trace, surf, "Surf trace management"); namespace tmgr = simgrid::trace_mgr; -static std::unordered_map trace_list; +static std::unordered_map trace_list; static inline bool doubleEq(double d1, double d2) { @@ -58,13 +58,13 @@ simgrid::trace_mgr::future_evt_set::~future_evt_set() } } -tmgr_trace_t tmgr_trace_new_from_string(const char* name, std::string input, double periodicity) +tmgr_trace_t tmgr_trace_new_from_string(std::string name, std::string input, double periodicity) { int linecount = 0; tmgr_trace_t trace = new simgrid::trace_mgr::trace(); tmgr::DatedValue* last_event = &(trace->event_list.back()); - xbt_assert(trace_list.find(name) == trace_list.end(), "Refusing to define trace %s twice", name); + xbt_assert(trace_list.find(name) == trace_list.end(), "Refusing to define trace %s twice", name.c_str()); std::vector list; boost::split(list, input, boost::is_any_of("\n\r")); @@ -80,10 +80,10 @@ tmgr_trace_t tmgr_trace_new_from_string(const char* name, std::string input, dou continue; xbt_assert(sscanf(val.c_str(), "%lg %lg\n", &event.date_, &event.value_) == 2, "%s:%d: Syntax error in trace\n%s", - name, linecount, input.c_str()); + name.c_str(), linecount, input.c_str()); xbt_assert(last_event->date_ <= event.date_, - "%s:%d: Invalid trace: Events must be sorted, but time %g > time %g.\n%s", name, linecount, + "%s:%d: Invalid trace: Events must be sorted, but time %g > time %g.\n%s", name.c_str(), linecount, last_event->date_, event.date_, input.c_str()); last_event->date_ = event.date_ - last_event->date_; @@ -98,18 +98,18 @@ tmgr_trace_t tmgr_trace_new_from_string(const char* name, std::string input, dou } } - trace_list.insert({xbt_strdup(name), trace}); + trace_list.insert({name, trace}); return trace; } -tmgr_trace_t tmgr_trace_new_from_file(const char *filename) +tmgr_trace_t tmgr_trace_new_from_file(std::string filename) { - xbt_assert(filename && filename[0], "Cannot parse a trace from the null or empty filename"); - xbt_assert(trace_list.find(filename) == trace_list.end(), "Refusing to define trace %s twice", filename); + xbt_assert(not filename.empty(), "Cannot parse a trace from an empty filename"); + xbt_assert(trace_list.find(filename) == trace_list.end(), "Refusing to define trace %s twice", filename.c_str()); std::ifstream* f = surf_ifsopen(filename); - xbt_assert(not f->fail(), "Cannot open file '%s' (path=%s)", filename, (boost::join(surf_path, ":")).c_str()); + xbt_assert(not f->fail(), "Cannot open file '%s' (path=%s)", filename.c_str(), (boost::join(surf_path, ":")).c_str()); std::stringstream buffer; buffer << f->rdbuf(); @@ -177,10 +177,8 @@ tmgr_trace_event_t simgrid::trace_mgr::future_evt_set::pop_leq(double date, doub void tmgr_finalize() { - for (auto kv : trace_list) { - xbt_free((char*)kv.first); + for (auto kv : trace_list) delete kv.second; - } trace_list.clear(); } diff --git a/src/surf/trace_mgr.hpp b/src/surf/trace_mgr.hpp index d78d4de20d..2cb74ce73a 100644 --- a/src/surf/trace_mgr.hpp +++ b/src/surf/trace_mgr.hpp @@ -33,8 +33,8 @@ XBT_PUBLIC(void) tmgr_trace_event_unref(tmgr_trace_event_t* trace_event); XBT_PUBLIC(void) tmgr_finalize(); -XBT_PUBLIC(tmgr_trace_t) tmgr_trace_new_from_file(const char* filename); -XBT_PUBLIC(tmgr_trace_t) tmgr_trace_new_from_string(const char* id, std::string input, double periodicity); +XBT_PUBLIC(tmgr_trace_t) tmgr_trace_new_from_file(std::string filename); +XBT_PUBLIC(tmgr_trace_t) tmgr_trace_new_from_string(std::string id, std::string input, double periodicity); SG_END_DECL() diff --git a/src/surf/xml/platf.hpp b/src/surf/xml/platf.hpp index 3d29ba49af..17c6f9e155 100644 --- a/src/surf/xml/platf.hpp +++ b/src/surf/xml/platf.hpp @@ -3,12 +3,9 @@ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ -#ifndef SURF_SURFXML_PARSE_H -#define SURF_SURFXML_PARSE_H +#ifndef SURF_SURFXML_PARSE_HPP +#define SURF_SURFXML_PARSE_HPP -#include -#include -#include #include SG_BEGIN_DECL() @@ -19,17 +16,17 @@ XBT_PUBLIC(void) sg_platf_exit(); XBT_PUBLIC(void) surf_parse_open(const char *file); XBT_PUBLIC(void) surf_parse_close(); -XBT_PUBLIC(void) surf_parse_assert(bool cond, const char *fmt, ...) XBT_ATTRIB_PRINTF(2,3); -XBT_PUBLIC(void) XBT_ATTRIB_NORETURN surf_parse_error(const char *msg,...) XBT_ATTRIB_PRINTF(1,2); -XBT_PUBLIC(void) surf_parse_assert_netpoint(char* hostname, const char* pre, const char* post); -XBT_PUBLIC(void) surf_parse_warn(const char *msg,...) XBT_ATTRIB_PRINTF(1,2); - -XBT_PUBLIC(double) surf_parse_get_double(const char *string); -XBT_PUBLIC(int) surf_parse_get_int(const char *string); -XBT_PUBLIC(double) surf_parse_get_time(const char *string, const char *entity_kind, const char *name); -XBT_PUBLIC(double) surf_parse_get_size(const char *string, const char *entity_kind, const char *name); -XBT_PUBLIC(double) surf_parse_get_bandwidth(const char *string, const char *entity_kind, const char *name); -XBT_PUBLIC(double) surf_parse_get_speed(const char *string, const char *entity_kind, const char *name); +XBT_PUBLIC(void) surf_parse_assert(bool cond, std::string msg); +XBT_PUBLIC(void) XBT_ATTRIB_NORETURN surf_parse_error(std::string msg); +XBT_PUBLIC(void) surf_parse_assert_netpoint(std::string hostname, std::string pre, std::string post); +XBT_PUBLIC(void) surf_parse_warn(std::string msg); + +XBT_PUBLIC(double) surf_parse_get_double(std::string s); +XBT_PUBLIC(int) surf_parse_get_int(std::string s); +XBT_PUBLIC(double) surf_parse_get_time(const char* string, const char* entity_kind, std::string name); +XBT_PUBLIC(double) surf_parse_get_size(const char* string, const char* entity_kind, std::string name); +XBT_PUBLIC(double) surf_parse_get_bandwidth(const char* string, const char* entity_kind, std::string name); +XBT_PUBLIC(double) surf_parse_get_speed(const char* string, const char* entity_kind, std::string name); XBT_PUBLIC(int) surf_parse(); /* Entry-point to the parser */ diff --git a/src/surf/xml/platf_private.hpp b/src/surf/xml/platf_private.hpp index b5ae88cec5..565fbe422a 100644 --- a/src/surf/xml/platf_private.hpp +++ b/src/surf/xml/platf_private.hpp @@ -53,12 +53,12 @@ typedef struct { } s_sg_platf_host_cbarg_t; typedef s_sg_platf_host_cbarg_t* sg_platf_host_cbarg_t; -typedef struct { - const char* id; - const char* link_up; - const char* link_down; -} s_sg_platf_host_link_cbarg_t; -typedef s_sg_platf_host_link_cbarg_t* sg_platf_host_link_cbarg_t; +class HostLinkCreationArgs { +public: + std::string id; + std::string link_up; + std::string link_down; +}; class LinkCreationArgs { public: @@ -93,39 +93,39 @@ typedef struct s_sg_platf_route_cbarg { std::vector* link_list; } s_sg_platf_route_cbarg_t; -typedef struct s_sg_platf_cluster_cbarg *sg_platf_cluster_cbarg_t; -typedef struct s_sg_platf_cluster_cbarg { - const char* id; - const char* prefix; - const char* suffix; - std::vector* radicals; +class ClusterCreationArgs { +public: + std::string id; + std::string prefix; + std::string suffix; + std::vector* radicals = nullptr; std::vector speeds; - int core_amount; - double bw; - double lat; - double bb_bw; - double bb_lat; - double loopback_bw; - double loopback_lat; - double limiter_link; + int core_amount = 0; + double bw = 0; + double lat = 0; + double bb_bw = 0; + double bb_lat = 0; + double loopback_bw = 0; + double loopback_lat = 0; + double limiter_link = 0; e_surf_cluster_topology_t topology; - const char* topo_parameters; + std::string topo_parameters; std::map* properties; - const char* router_id; + std::string router_id; e_surf_link_sharing_policy_t sharing_policy; e_surf_link_sharing_policy_t bb_sharing_policy; -} s_sg_platf_cluster_cbarg_t; +}; -typedef struct s_sg_platf_cabinet_cbarg* sg_platf_cabinet_cbarg_t; -typedef struct s_sg_platf_cabinet_cbarg { - const char* id; - const char* prefix; - const char* suffix; +class CabinetCreationArgs { +public: + std::string id; + std::string prefix; + std::string suffix; std::vector* radicals; double speed; double bw; double lat; -} s_sg_platf_cabinet_cbarg_t; +}; class StorageCreationArgs { public: @@ -158,20 +158,20 @@ typedef struct s_sg_platf_prop_cbarg { const char *value; } s_sg_platf_prop_cbarg_t; -typedef struct s_sg_platf_trace_cbarg *sg_platf_trace_cbarg_t; -typedef struct s_sg_platf_trace_cbarg { - const char *id; - const char *file; +class TraceCreationArgs { +public: + std::string id; + std::string file; double periodicity; - const char *pc_data; -} s_sg_platf_trace_cbarg_t; + std::string pc_data; +}; -typedef struct s_sg_platf_trace_connect_cbarg *sg_platf_trace_connect_cbarg_t; -typedef struct s_sg_platf_trace_connect_cbarg { +class TraceConnectCreationArgs { +public: e_surf_trace_connect_kind_t kind; - const char *trace; - const char *element; -} s_sg_platf_trace_connect_cbarg_t; + std::string trace; + std::string element; +}; typedef struct s_sg_platf_process_cbarg *sg_platf_process_cbarg_t; typedef struct s_sg_platf_process_cbarg { @@ -185,13 +185,12 @@ typedef struct s_sg_platf_process_cbarg { e_surf_process_on_failure_t on_failure; } s_sg_platf_process_cbarg_t; -typedef struct s_sg_platf_AS_cbarg *sg_platf_AS_cbarg_t; -typedef struct s_sg_platf_AS_cbarg { - const char *id; +class ZoneCreationArgs { +public: + std::string id; int routing; -} s_sg_platf_AS_cbarg_t; +}; -#define SG_PLATF_AS_INITIALIZER {nullptr,0} /* The default current property receiver. Setup in the corresponding opening callbacks. */ extern std::map* current_property_set; @@ -202,29 +201,29 @@ void routing_cluster_add_backbone(simgrid::surf::LinkImpl* bb); XBT_PUBLIC(void) sg_platf_begin(); // Start a new platform XBT_PUBLIC(void) sg_platf_end(); // Finish the creation of the platform -XBT_PUBLIC(simgrid::s4u::NetZone*) sg_platf_new_AS_begin(sg_platf_AS_cbarg_t AS); // Begin description of new AS -XBT_PUBLIC(void) sg_platf_new_AS_seal(); // That AS is fully described +XBT_PUBLIC(simgrid::s4u::NetZone*) sg_platf_new_Zone_begin(ZoneCreationArgs* zone); // Begin description of new Zone +XBT_PUBLIC(void) sg_platf_new_Zone_seal(); // That Zone is fully described -XBT_PUBLIC(void) sg_platf_new_host (sg_platf_host_cbarg_t host); // Add an host to the currently described AS -XBT_PUBLIC(void) sg_platf_new_hostlink(sg_platf_host_link_cbarg_t h); // Add an host_link to the currently described AS -XBT_PUBLIC(simgrid::kernel::routing::NetPoint*) -sg_platf_new_router(const char* name, const char* coords); // Add a router to the currently described AS -XBT_PUBLIC(void) sg_platf_new_link(LinkCreationArgs* link); // Add a link to the currently described AS -XBT_PUBLIC(void) sg_platf_new_peer(PeerCreationArgs* peer); // Add a peer to the currently described AS -XBT_PUBLIC(void) sg_platf_new_cluster(sg_platf_cluster_cbarg_t clust); // Add a cluster to the currently described AS -XBT_PUBLIC(void) sg_platf_new_cabinet(sg_platf_cabinet_cbarg_t cabinet); // Add a cabinet to the currently described AS +XBT_PUBLIC(void) sg_platf_new_host(sg_platf_host_cbarg_t host); // Add a host to the current Zone +XBT_PUBLIC(void) sg_platf_new_hostlink(HostLinkCreationArgs* h); // Add a host_link to the current Zone +XBT_PUBLIC(void) sg_platf_new_link(LinkCreationArgs* link); // Add a link to the current Zone +XBT_PUBLIC(void) sg_platf_new_peer(PeerCreationArgs* peer); // Add a peer to the current Zone +XBT_PUBLIC(void) sg_platf_new_cluster(ClusterCreationArgs* clust); // Add a cluster to the current Zone +XBT_PUBLIC(void) sg_platf_new_cabinet(CabinetCreationArgs* cabinet); // Add a cabinet to the current Zone +XBT_PUBLIC(simgrid::kernel::routing::NetPoint*) // Add a router to the current Zone +sg_platf_new_router(std::string, const char* coords); XBT_PUBLIC(void) sg_platf_new_route (sg_platf_route_cbarg_t route); // Add a route XBT_PUBLIC(void) sg_platf_new_bypassRoute (sg_platf_route_cbarg_t bypassroute); // Add a bypassRoute -XBT_PUBLIC(void) sg_platf_new_trace(sg_platf_trace_cbarg_t trace); +XBT_PUBLIC(void) sg_platf_new_trace(TraceCreationArgs* trace); -XBT_PUBLIC(void) sg_platf_new_storage(StorageCreationArgs* storage); // Add a storage to the currently described AS +XBT_PUBLIC(void) sg_platf_new_storage(StorageCreationArgs* storage); // Add a storage to the current Zone XBT_PUBLIC(void) sg_platf_new_storage_type(StorageTypeCreationArgs* storage_type); XBT_PUBLIC(void) sg_platf_new_mount(MountCreationArgs* mount); XBT_PUBLIC(void) sg_platf_new_process(sg_platf_process_cbarg_t process); -XBT_PRIVATE void sg_platf_trace_connect(sg_platf_trace_connect_cbarg_t trace_connect); +XBT_PRIVATE void sg_platf_trace_connect(TraceConnectCreationArgs* trace_connect); /* Prototypes of the functions offered by flex */ XBT_PUBLIC(int) surf_parse_lex(); @@ -247,8 +246,7 @@ SG_END_DECL() namespace simgrid { namespace surf { -extern XBT_PRIVATE simgrid::xbt::signal on_cluster; - +extern XBT_PRIVATE simgrid::xbt::signal on_cluster; } } diff --git a/src/surf/xml/surfxml_parseplatf.cpp b/src/surf/xml/surfxml_parseplatf.cpp index 0e905d76dd..055b5fee3b 100644 --- a/src/surf/xml/surfxml_parseplatf.cpp +++ b/src/surf/xml/surfxml_parseplatf.cpp @@ -7,7 +7,6 @@ #include "src/instr/instr_private.h" // TRACE_start(). FIXME: remove by subscribing tracing to the surf signals #include "src/surf/cpu_interface.hpp" #include "src/surf/network_interface.hpp" -#include "xbt/dict.h" #include "xbt/log.h" #include "xbt/misc.h" #include "xbt/str.h" @@ -27,7 +26,6 @@ extern "C" { XBT_LOG_EXTERNAL_DEFAULT_CATEGORY(surf_parse); - /* Trace related stuff */ XBT_PRIVATE std::unordered_map traces_set_list; XBT_PRIVATE std::unordered_map trace_connect_list_host_avail; @@ -37,10 +35,11 @@ XBT_PRIVATE std::unordered_map trace_connect_list_link XBT_PRIVATE std::unordered_map trace_connect_list_link_lat; SG_BEGIN_DECL() -void sg_platf_trace_connect(sg_platf_trace_connect_cbarg_t trace_connect) +void sg_platf_trace_connect(TraceConnectCreationArgs* trace_connect) { xbt_assert(traces_set_list.find(trace_connect->trace) != traces_set_list.end(), - "Cannot connect trace %s to %s: trace unknown", trace_connect->trace, trace_connect->element); + "Cannot connect trace %s to %s: trace unknown", trace_connect->trace.c_str(), + trace_connect->element.c_str()); switch (trace_connect->kind) { case SURF_TRACE_CONNECT_KIND_HOST_AVAIL: @@ -59,8 +58,8 @@ void sg_platf_trace_connect(sg_platf_trace_connect_cbarg_t trace_connect) trace_connect_list_link_lat.insert({trace_connect->trace, trace_connect->element}); break; default: - surf_parse_error("Cannot connect trace %s to %s: kind of trace unknown", trace_connect->trace, - trace_connect->element); + surf_parse_error(std::string("Cannot connect trace ") + trace_connect->trace + " to " + trace_connect->element + + ": unknown kind of trace"); break; } } @@ -169,7 +168,7 @@ void parse_platform_file(const char *file) surf_parse_close(); if (parse_status) - surf_parse_error("Parse error in %s", file); + surf_parse_error(std::string("Parse error in ") + file); } } diff --git a/src/surf/xml/surfxml_sax_cb.cpp b/src/surf/xml/surfxml_sax_cb.cpp index 8d77fca7da..fd457c94e6 100644 --- a/src/surf/xml/surfxml_sax_cb.cpp +++ b/src/surf/xml/surfxml_sax_cb.cpp @@ -30,39 +30,32 @@ std::vector parsed_link_list; /* temporary store of cu /* * Helping functions */ -void surf_parse_assert(bool cond, const char *fmt, ...) { +void surf_parse_assert(bool cond, std::string msg) +{ if (not cond) { - va_list va; - va_start(va,fmt); int lineno = surf_parse_lineno; - char *msg = bvprintf(fmt,va); - va_end(va); cleanup(); - XBT_ERROR("Parse error at %s:%d: %s", surf_parsed_filename, lineno, msg); + XBT_ERROR("Parse error at %s:%d: %s", surf_parsed_filename, lineno, msg.c_str()); surf_exit(); xbt_die("Exiting now"); } } -void surf_parse_error(const char *fmt, ...) { - va_list va; - va_start(va,fmt); + +void surf_parse_error(std::string msg) +{ int lineno = surf_parse_lineno; - char *msg = bvprintf(fmt,va); - va_end(va); cleanup(); - XBT_ERROR("Parse error at %s:%d: %s", surf_parsed_filename, lineno, msg); + XBT_ERROR("Parse error at %s:%d: %s", surf_parsed_filename, lineno, msg.c_str()); surf_exit(); xbt_die("Exiting now"); } -void surf_parse_assert_netpoint(char* hostname, const char* pre, const char* post) + +void surf_parse_assert_netpoint(std::string hostname, std::string pre, std::string post) { - if (sg_netpoint_by_name_or_null(hostname) != nullptr) // found + if (sg_netpoint_by_name_or_null(hostname.c_str()) != nullptr) // found return; - std::string msg = std::string(pre); - msg += hostname; - msg += post; - msg += " Existing netpoints: \n"; + std::string msg = pre + hostname + post + " Existing netpoints: \n"; std::vector list; simgrid::s4u::Engine::getInstance()->getNetpointList(&list); @@ -85,36 +78,36 @@ void surf_parse_assert_netpoint(char* hostname, const char* pre, const char* pos break; } } - surf_parse_error("%s", msg.c_str()); + surf_parse_error(msg); } -void surf_parse_warn(const char *fmt, ...) { - va_list va; - va_start(va,fmt); - char *msg = bvprintf(fmt,va); - va_end(va); - XBT_WARN("%s:%d: %s", surf_parsed_filename, surf_parse_lineno, msg); - free(msg); +void surf_parse_warn(std::string msg) +{ + XBT_WARN("%s:%d: %s", surf_parsed_filename, surf_parse_lineno, msg.c_str()); } -double surf_parse_get_double(const char *string) { - double res; - int ret = sscanf(string, "%lg", &res); - if (ret != 1) - surf_parse_error("%s is not a double", string); - return res; +double surf_parse_get_double(std::string s) +{ + try { + return std::stod(s); + } catch (std::invalid_argument& ia) { + surf_parse_error(s + " is not a double"); + return -1; + } } -int surf_parse_get_int(const char *string) { - int res; - int ret = sscanf(string, "%d", &res); - if (ret != 1) - surf_parse_error("%s is not an integer", string); - return res; +int surf_parse_get_int(std::string s) +{ + try { + return std::stoi(s); + } catch (std::invalid_argument& ia) { + surf_parse_error(s + " is not a double"); + return -1; + } } /* Turn something like "1-4,6,9-11" into the vector {1,2,3,4,6,9,10,11} */ -static std::vector* explodesRadical(const char* radicals) +static std::vector* explodesRadical(std::string radicals) { std::vector* exploded = new std::vector(); @@ -124,18 +117,18 @@ static std::vector* explodesRadical(const char* radicals) for (auto group : radical_elements) { std::vector radical_ends; boost::split(radical_ends, group, boost::is_any_of("-")); - int start = surf_parse_get_int((radical_ends.front()).c_str()); - int end = 0; + int start = surf_parse_get_int(radical_ends.front()); + int end = 0; switch (radical_ends.size()) { case 1: end = start; break; case 2: - end = surf_parse_get_int((radical_ends.back()).c_str()); + end = surf_parse_get_int(radical_ends.back()); break; default: - surf_parse_error("Malformed radical: %s", group.c_str()); + surf_parse_error(std::string("Malformed radical: ") + group); break; } for (int i = start; i <= end; i++) @@ -151,22 +144,23 @@ struct unit_scale { }; /* Note: field `unit' for the last element of parameter `units' should be nullptr. */ -static double surf_parse_get_value_with_unit(const char *string, const struct unit_scale *units, - const char *entity_kind, const char *name, const char *error_msg, const char *default_unit) +static double surf_parse_get_value_with_unit(const char* string, const struct unit_scale* units, + const char* entity_kind, std::string name, const char* error_msg, + const char* default_unit) { char* ptr; int i; errno = 0; double res = strtod(string, &ptr); if (errno == ERANGE) - surf_parse_error("value out of range: %s", string); + surf_parse_error(std::string("value out of range: ") + string); if (ptr == string) - surf_parse_error("cannot parse number: %s", string); + surf_parse_error(std::string("cannot parse number:") + string); if (ptr[0] == '\0') { if (res == 0) return res; // Ok, 0 can be unit-less - XBT_WARN("Deprecated unit-less value '%s' for %s %s. %s",string, entity_kind, name, error_msg); + XBT_WARN("Deprecated unit-less value '%s' for %s %s. %s", string, entity_kind, name.c_str(), error_msg); ptr = (char*)default_unit; } for (i = 0; units[i].unit != nullptr && strcmp(ptr, units[i].unit) != 0; i++); @@ -174,11 +168,11 @@ static double surf_parse_get_value_with_unit(const char *string, const struct un if (units[i].unit != nullptr) res *= units[i].scale; else - surf_parse_error("unknown unit: %s", ptr); + surf_parse_error(std::string("unknown unit: ") + ptr); return res; } -double surf_parse_get_time(const char *string, const char *entity_kind, const char *name) +double surf_parse_get_time(const char* string, const char* entity_kind, std::string name) { const struct unit_scale units[] = { { "w", 7 * 24 * 60 * 60 }, @@ -196,7 +190,7 @@ double surf_parse_get_time(const char *string, const char *entity_kind, const ch "Append 's' to your time to get seconds", "s"); } -double surf_parse_get_size(const char *string, const char *entity_kind, const char *name) +double surf_parse_get_size(const char* string, const char* entity_kind, std::string name) { const struct unit_scale units[] = { { "EiB", pow(1024, 6) }, @@ -231,7 +225,7 @@ double surf_parse_get_size(const char *string, const char *entity_kind, const ch "Append 'B' to get bytes (or 'b' for bits but 1B = 8b).", "B"); } -double surf_parse_get_bandwidth(const char *string, const char *entity_kind, const char *name) +double surf_parse_get_bandwidth(const char* string, const char* entity_kind, std::string name) { const struct unit_scale units[] = { { "EiBps", pow(1024, 6) }, @@ -264,7 +258,7 @@ double surf_parse_get_bandwidth(const char *string, const char *entity_kind, con "Append 'Bps' to get bytes per second (or 'bps' for bits but 1Bps = 8bps)", "Bps"); } -double surf_parse_get_speed(const char *string, const char *entity_kind, const char *name) +double surf_parse_get_speed(const char* string, const char* entity_kind, std::string name) { const struct unit_scale units[] = { { "yottaflops", 1e24 }, @@ -291,7 +285,8 @@ double surf_parse_get_speed(const char *string, const char *entity_kind, const c "Append 'f' or 'flops' to your speed to get flop per second", "f"); } -static std::vector surf_parse_get_all_speeds(char* speeds, const char* entity_kind, const char* id){ +static std::vector surf_parse_get_all_speeds(char* speeds, const char* entity_kind, std::string id) +{ std::vector speed_per_pstate; @@ -323,9 +318,6 @@ std::map* current_property_set = nullptr; std::map* current_model_property_set = nullptr; int ZONE_TAG = 0; // Whether we just opened a zone tag (to see what to do with the properties) -/* dictionary of random generator data */ -xbt_dict_t random_data_list = nullptr; - YY_BUFFER_STATE surf_input_buffer; FILE *surf_file_to_parse = nullptr; @@ -552,8 +544,7 @@ void ETag_surfxml_host() { void STag_surfxml_host___link(){ XBT_DEBUG("Create a Host_link for %s",A_surfxml_host___link_id); - s_sg_platf_host_link_cbarg_t host_link; - memset(&host_link,0,sizeof(host_link)); + HostLinkCreationArgs host_link; host_link.id = A_surfxml_host___link_id; host_link.link_up = A_surfxml_host___link_up; @@ -566,9 +557,8 @@ void STag_surfxml_router(){ } void ETag_surfxml_cluster(){ - s_sg_platf_cluster_cbarg_t cluster; - memset(&cluster,0,sizeof(cluster)); - cluster.properties = current_property_set; + ClusterCreationArgs cluster; + cluster.properties = current_property_set; current_property_set = nullptr; cluster.id = A_surfxml_cluster_id; @@ -604,8 +594,7 @@ void ETag_surfxml_cluster(){ cluster.topology= SURF_CLUSTER_DRAGONFLY ; break; default: - surf_parse_error("Invalid cluster topology for cluster %s", - cluster.id); + surf_parse_error(std::string("Invalid cluster topology for cluster ") + cluster.id); break; } cluster.topo_parameters = A_surfxml_cluster_topo___parameters; @@ -622,7 +611,7 @@ void ETag_surfxml_cluster(){ cluster.sharing_policy = SURF_LINK_FATPIPE; break; default: - surf_parse_error("Invalid cluster sharing policy for cluster %s", cluster.id); + surf_parse_error(std::string("Invalid cluster sharing policy for cluster ") + cluster.id); break; } switch (AX_surfxml_cluster_bb___sharing___policy) { @@ -633,7 +622,7 @@ void ETag_surfxml_cluster(){ cluster.bb_sharing_policy = SURF_LINK_SHARED; break; default: - surf_parse_error("Invalid bb sharing policy in cluster %s", cluster.id); + surf_parse_error(std::string("Invalid bb sharing policy in cluster ") + cluster.id); break; } @@ -648,14 +637,13 @@ void STag_surfxml_cluster(){ void STag_surfxml_cabinet(){ parse_after_config(); - s_sg_platf_cabinet_cbarg_t cabinet; - memset(&cabinet,0,sizeof(cabinet)); + CabinetCreationArgs cabinet; cabinet.id = A_surfxml_cabinet_id; cabinet.prefix = A_surfxml_cabinet_prefix; cabinet.suffix = A_surfxml_cabinet_suffix; - cabinet.speed = surf_parse_get_speed(A_surfxml_cabinet_speed, "speed of cabinet", cabinet.id); - cabinet.bw = surf_parse_get_bandwidth(A_surfxml_cabinet_bw, "bw of cabinet", cabinet.id); - cabinet.lat = surf_parse_get_time(A_surfxml_cabinet_lat, "lat of cabinet", cabinet.id); + cabinet.speed = surf_parse_get_speed(A_surfxml_cabinet_speed, "speed of cabinet", cabinet.id.c_str()); + cabinet.bw = surf_parse_get_bandwidth(A_surfxml_cabinet_bw, "bw of cabinet", cabinet.id.c_str()); + cabinet.lat = surf_parse_get_time(A_surfxml_cabinet_lat, "lat of cabinet", cabinet.id.c_str()); cabinet.radicals = explodesRadical(A_surfxml_cabinet_radical); sg_platf_new_cabinet(&cabinet); @@ -709,35 +697,31 @@ void ETag_surfxml_link(){ link.policy = SURF_LINK_FULLDUPLEX; break; default: - surf_parse_error("Invalid sharing policy in link %s", link.id.c_str()); + surf_parse_error(std::string("Invalid sharing policy in link ") + link.id); break; } sg_platf_new_link(&link); } -void STag_surfxml_link___ctn(){ - +void STag_surfxml_link___ctn() +{ simgrid::surf::LinkImpl* link = nullptr; - char *link_name=nullptr; switch (A_surfxml_link___ctn_direction) { case AU_surfxml_link___ctn_direction: case A_surfxml_link___ctn_direction_NONE: link = simgrid::surf::LinkImpl::byName(A_surfxml_link___ctn_id); break; case A_surfxml_link___ctn_direction_UP: - link_name = bprintf("%s_UP", A_surfxml_link___ctn_id); - link = simgrid::surf::LinkImpl::byName(link_name); + link = simgrid::surf::LinkImpl::byName(std::string(A_surfxml_link___ctn_id) + "_UP"); break; case A_surfxml_link___ctn_direction_DOWN: - link_name = bprintf("%s_DOWN", A_surfxml_link___ctn_id); - link = simgrid::surf::LinkImpl::byName(link_name); + link = simgrid::surf::LinkImpl::byName(std::string(A_surfxml_link___ctn_id) + "_DOWN"); break; default: - surf_parse_error("Invalid direction for link %s", link_name); + surf_parse_error(std::string("Invalid direction for link ") + A_surfxml_link___ctn_id); break; } - xbt_free(link_name); // no-op if it's already nullptr const char* dirname = ""; switch (A_surfxml_link___ctn_direction) { @@ -750,7 +734,7 @@ void STag_surfxml_link___ctn(){ default: dirname = ""; } - surf_parse_assert(link != nullptr, "No such link: '%s'%s", A_surfxml_link___ctn_id, dirname); + surf_parse_assert(link != nullptr, std::string("No such link: '") + A_surfxml_link___ctn_id + "'" + dirname); parsed_link_list.push_back(link); } @@ -912,8 +896,7 @@ void ETag_surfxml_bypassZoneRoute() } void ETag_surfxml_trace(){ - s_sg_platf_trace_cbarg_t trace; - memset(&trace,0,sizeof(trace)); + TraceCreationArgs trace; trace.id = A_surfxml_trace_id; trace.file = A_surfxml_trace_file; @@ -926,8 +909,7 @@ void ETag_surfxml_trace(){ void STag_surfxml_trace___connect() { parse_after_config(); - s_sg_platf_trace_connect_cbarg_t trace_connect; - memset(&trace_connect,0,sizeof(trace_connect)); + TraceConnectCreationArgs trace_connect; trace_connect.element = A_surfxml_trace___connect_element; trace_connect.trace = A_surfxml_trace___connect_trace; @@ -972,20 +954,23 @@ void STag_surfxml_zone() { parse_after_config(); ZONE_TAG = 1; - s_sg_platf_AS_cbarg_t AS = {A_surfxml_zone_id, (int)A_surfxml_zone_routing}; + ZoneCreationArgs zone; + zone.id = A_surfxml_zone_id; + zone.routing = static_cast(A_surfxml_zone_routing); - sg_platf_new_AS_begin(&AS); + sg_platf_new_Zone_begin(&zone); } void ETag_surfxml_zone() { - sg_platf_new_AS_seal(); + sg_platf_new_Zone_seal(); } void STag_surfxml_config() { ZONE_TAG = 0; - xbt_assert(current_property_set == nullptr, "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)"); + xbt_assert(current_property_set == nullptr, + "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)"); XBT_DEBUG("START configuration name = %s",A_surfxml_config_id); if (_sg_cfg_init_status == 2) { surf_parse_error("All tags must be given before any platform elements (such as , , , " @@ -1016,6 +1001,7 @@ void STag_surfxml_process() AX_surfxml_actor_function = AX_surfxml_process_function; STag_surfxml_actor(); } + void STag_surfxml_actor() { ZONE_TAG = 0; @@ -1034,6 +1020,7 @@ void ETag_surfxml_process() AX_surfxml_actor_on___failure = (AT_surfxml_actor_on___failure)AX_surfxml_process_on___failure; ETag_surfxml_actor(); } + void ETag_surfxml_actor() { s_sg_platf_process_cbarg_t actor; @@ -1080,8 +1067,7 @@ void STag_surfxml_model___prop(){ if (not current_model_property_set) current_model_property_set = new std::map(); - current_model_property_set->insert( - {std::string(A_surfxml_model___prop_id), std::string(A_surfxml_model___prop_value)}); + current_model_property_set->insert({A_surfxml_model___prop_id, A_surfxml_model___prop_value}); } void ETag_surfxml_prop(){/* Nothing to do */} diff --git a/src/xbt/config.cpp b/src/xbt/config.cpp index f451949b43..2b1788bf63 100644 --- a/src/xbt/config.cpp +++ b/src/xbt/config.cpp @@ -305,15 +305,17 @@ Config::~Config() inline ConfigurationElement* Config::getDictElement(const char* name) { - try { - return options.at(name); - } catch (std::out_of_range& unfound) { - try { - ConfigurationElement* res = aliases.at(name); + auto opt = options.find(name); + if (opt != options.end()) { + return opt->second; + } else { + auto als = aliases.find(name); + if (als != aliases.end()) { + ConfigurationElement* res = als->second; if (warn_for_aliases) XBT_INFO("Option %s has been renamed to %s. Consider switching.", name, res->getKey().c_str()); return res; - } catch (std::out_of_range& missing_key) { + } else { throw simgrid::config::missing_key_error(std::string("Bad config key: ") + name); } } diff --git a/src/xbt/dynar.cpp b/src/xbt/dynar.cpp index cef1231c79..4c44eb35d9 100644 --- a/src/xbt/dynar.cpp +++ b/src/xbt/dynar.cpp @@ -1,6 +1,6 @@ /* a generic DYNamic ARray implementation. */ -/* Copyright (c) 2004-2015. The SimGrid Team. +/* Copyright (c) 2004-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -614,53 +614,6 @@ extern "C" xbt_dynar_t xbt_dynar_sort_strings(xbt_dynar_t dynar) return dynar; // to enable functional uses } -/** @brief Sorts a dynar according to their color assuming elements can have only three colors. - * Since there are only three colors, it is linear and much faster than a classical sort. - * See for example http://en.wikipedia.org/wiki/Dutch_national_flag_problem - * - * \param dynar the dynar to sort - * \param color the color function of type (int (compar_fn*) (void*) (void*)). The return value of color is assumed to - * be 0, 1, or 2. - * - * At the end of the call, elements with color 0 are at the beginning of the dynar, elements with color 2 are at the - * end and elements with color 1 are in the middle. - * - * Remark: if the elements stored in the dynar are structures, the color function has to retrieve the field to sort - * first. - */ -extern "C" void xbt_dynar_three_way_partition(xbt_dynar_t const dynar, int_f_pvoid_t color) -{ - unsigned long int i; - unsigned long int p = -1; - unsigned long int q = dynar->used; - const unsigned long elmsize = dynar->elmsize; - char* tmp[elmsize]; - void *elm; - - for (i = 0; i < q;) { - void *elmi = _xbt_dynar_elm(dynar, i); - int colori = color(elmi); - - if (colori == 1) { - ++i; - } else { - if (colori == 0) { - ++p; - elm = _xbt_dynar_elm(dynar, p); - ++i; - } else { /* colori == 2 */ - --q; - elm = _xbt_dynar_elm(dynar, q); - } - if (elm != elmi) { - memcpy(tmp, elm, elmsize); - memcpy(elm, elmi, elmsize); - memcpy(elmi, tmp, elmsize); - } - } - } -} - /** @brief Transform a dynar into a nullptr terminated array. * * \param dynar the dynar to transform diff --git a/src/xbt/parmap.cpp b/src/xbt/parmap.cpp index 935149f0ab..c5754e4471 100644 --- a/src/xbt/parmap.cpp +++ b/src/xbt/parmap.cpp @@ -3,506 +3,6 @@ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ -#include -#include - -#include "src/internal_config.h" -#if HAVE_UNISTD_H -#include -#endif - -#ifndef _WIN32 -#include -#endif - -#if HAVE_FUTEX_H -#include -#include -#endif - -#include "xbt/parmap.h" #include "xbt/log.h" -#include "xbt/function_types.h" -#include "xbt/dynar.h" -#include "xbt/xbt_os_thread.h" -#include "xbt/sysdep.h" -#include "src/simix/smx_private.h" XBT_LOG_NEW_DEFAULT_SUBCATEGORY(xbt_parmap, xbt, "parmap: parallel map"); - -typedef enum { - XBT_PARMAP_WORK, - XBT_PARMAP_DESTROY -} e_xbt_parmap_flag_t; - -static void xbt_parmap_set_mode(xbt_parmap_t parmap, e_xbt_parmap_mode_t mode); -static void *xbt_parmap_worker_main(void *parmap); -static void xbt_parmap_work(xbt_parmap_t parmap); - -static void xbt_parmap_posix_master_wait(xbt_parmap_t parmap); -static void xbt_parmap_posix_worker_signal(xbt_parmap_t parmap); -static void xbt_parmap_posix_master_signal(xbt_parmap_t parmap); -static void xbt_parmap_posix_worker_wait(xbt_parmap_t parmap, unsigned round); - -#if HAVE_FUTEX_H -static void xbt_parmap_futex_master_wait(xbt_parmap_t parmap); -static void xbt_parmap_futex_worker_signal(xbt_parmap_t parmap); -static void xbt_parmap_futex_master_signal(xbt_parmap_t parmap); -static void xbt_parmap_futex_worker_wait(xbt_parmap_t parmap, unsigned round); -static void futex_wait(unsigned *uaddr, unsigned val); -static void futex_wake(unsigned *uaddr, unsigned val); -#endif - -static void xbt_parmap_busy_master_wait(xbt_parmap_t parmap); -static void xbt_parmap_busy_worker_signal(xbt_parmap_t parmap); -static void xbt_parmap_busy_master_signal(xbt_parmap_t parmap); -static void xbt_parmap_busy_worker_wait(xbt_parmap_t parmap, unsigned round); - -/** - * \brief Parallel map structure - */ -typedef struct s_xbt_parmap { - e_xbt_parmap_flag_t status; /**< is the parmap active or being destroyed? */ - unsigned work; /**< index of the current round */ - unsigned thread_counter; /**< number of workers that have done the work */ - - unsigned int num_workers; /**< total number of worker threads including the controller */ - xbt_os_thread_t *workers; /**< worker thread handlers */ - void_f_pvoid_t fun; /**< function to run in parallel on each element of data */ - xbt_dynar_t data; /**< parameters to pass to fun in parallel */ - std::atomic index; /**< index of the next element of data to pick */ - - /* posix only */ - xbt_os_cond_t ready_cond; - xbt_os_mutex_t ready_mutex; - xbt_os_cond_t done_cond; - xbt_os_mutex_t done_mutex; - - /* fields that depend on the synchronization mode */ - e_xbt_parmap_mode_t mode; /**< synchronization mode */ - void (*master_wait_f)(xbt_parmap_t); /**< wait for the workers to have done the work */ - void (*worker_signal_f)(xbt_parmap_t); /**< signal the master that a worker has done the work */ - void (*master_signal_f)(xbt_parmap_t); /**< wakes the workers threads to process tasks */ - void (*worker_wait_f)(xbt_parmap_t, unsigned); /**< waits for more work */ -} s_xbt_parmap_t; - -/** - * \brief Thread data transmission structure - */ -typedef struct s_xbt_parmap_thread_data{ - xbt_parmap_t parmap; - int worker_id; -} s_xbt_parmap_thread_data_t; - -typedef s_xbt_parmap_thread_data_t *xbt_parmap_thread_data_t; - -/** - * \brief Creates a parallel map object - * \param num_workers number of worker threads to create - * \param mode how to synchronize the worker threads - * \return the parmap created - */ -xbt_parmap_t xbt_parmap_new(unsigned int num_workers, e_xbt_parmap_mode_t mode) -{ - XBT_DEBUG("Create new parmap (%u workers)", num_workers); - - /* Initialize the thread pool data structure */ - xbt_parmap_t parmap = new s_xbt_parmap_t(); - parmap->workers = xbt_new(xbt_os_thread_t, num_workers); - - parmap->num_workers = num_workers; - parmap->status = XBT_PARMAP_WORK; - xbt_parmap_set_mode(parmap, mode); - - /* Create the pool of worker threads */ - parmap->workers[0] = nullptr; -#if HAVE_PTHREAD_SETAFFINITY - int core_bind = 0; -#endif - for (unsigned int i = 1; i < num_workers; i++) { - xbt_parmap_thread_data_t data = xbt_new0(s_xbt_parmap_thread_data_t, 1); - data->parmap = parmap; - data->worker_id = i; - parmap->workers[i] = xbt_os_thread_create(nullptr, xbt_parmap_worker_main, data, nullptr); -#if HAVE_PTHREAD_SETAFFINITY - xbt_os_thread_bind(parmap->workers[i], core_bind); - if (core_bind != xbt_os_get_numcores() - 1) - core_bind++; - else - core_bind = 0; -#endif - } - return parmap; -} - -/** - * \brief Destroys a parmap - * \param parmap the parmap to destroy - */ -void xbt_parmap_destroy(xbt_parmap_t parmap) -{ - if (not parmap) { - return; - } - - parmap->status = XBT_PARMAP_DESTROY; - parmap->master_signal_f(parmap); - - unsigned int i; - for (i = 1; i < parmap->num_workers; i++) - xbt_os_thread_join(parmap->workers[i], nullptr); - - xbt_os_cond_destroy(parmap->ready_cond); - xbt_os_mutex_destroy(parmap->ready_mutex); - xbt_os_cond_destroy(parmap->done_cond); - xbt_os_mutex_destroy(parmap->done_mutex); - - xbt_free(parmap->workers); - delete parmap; -} - -/** - * \brief Sets the synchronization mode of a parmap. - * \param parmap a parallel map object - * \param mode the synchronization mode - */ -static void xbt_parmap_set_mode(xbt_parmap_t parmap, e_xbt_parmap_mode_t mode) -{ - if (mode == XBT_PARMAP_DEFAULT) { -#if HAVE_FUTEX_H - mode = XBT_PARMAP_FUTEX; -#else - mode = XBT_PARMAP_POSIX; -#endif - } - parmap->mode = mode; - - switch (mode) { - case XBT_PARMAP_POSIX: - parmap->master_wait_f = &xbt_parmap_posix_master_wait; - parmap->worker_signal_f = &xbt_parmap_posix_worker_signal; - parmap->master_signal_f = &xbt_parmap_posix_master_signal; - parmap->worker_wait_f = &xbt_parmap_posix_worker_wait; - - parmap->ready_cond = xbt_os_cond_init(); - parmap->ready_mutex = xbt_os_mutex_init(); - parmap->done_cond = xbt_os_cond_init(); - parmap->done_mutex = xbt_os_mutex_init(); - break; - case XBT_PARMAP_FUTEX: -#if HAVE_FUTEX_H - parmap->master_wait_f = &xbt_parmap_futex_master_wait; - parmap->worker_signal_f = &xbt_parmap_futex_worker_signal; - parmap->master_signal_f = &xbt_parmap_futex_master_signal; - parmap->worker_wait_f = &xbt_parmap_futex_worker_wait; - - xbt_os_cond_destroy(parmap->ready_cond); - xbt_os_mutex_destroy(parmap->ready_mutex); - xbt_os_cond_destroy(parmap->done_cond); - xbt_os_mutex_destroy(parmap->done_mutex); - break; -#else - xbt_die("Futex is not available on this OS."); -#endif - case XBT_PARMAP_BUSY_WAIT: - parmap->master_wait_f = &xbt_parmap_busy_master_wait; - parmap->worker_signal_f = &xbt_parmap_busy_worker_signal; - parmap->master_signal_f = &xbt_parmap_busy_master_signal; - parmap->worker_wait_f = &xbt_parmap_busy_worker_wait; - - xbt_os_cond_destroy(parmap->ready_cond); - xbt_os_mutex_destroy(parmap->ready_mutex); - xbt_os_cond_destroy(parmap->done_cond); - xbt_os_mutex_destroy(parmap->done_mutex); - break; - case XBT_PARMAP_DEFAULT: - THROW_IMPOSSIBLE; - break; - default: - THROW_IMPOSSIBLE; - } -} - -/** - * \brief Applies a list of tasks in parallel. - * \param parmap a parallel map object - * \param fun the function to call in parallel - * \param data each element of this dynar will be passed as an argument to fun - */ -void xbt_parmap_apply(xbt_parmap_t parmap, void_f_pvoid_t fun, xbt_dynar_t data) -{ - /* Assign resources to worker threads (we are maestro here)*/ - parmap->fun = fun; - parmap->data = data; - parmap->index = 0; - parmap->master_signal_f(parmap); // maestro runs futex_wait to wake all the minions (the working threads) - xbt_parmap_work(parmap); // maestro works with its minions - parmap->master_wait_f(parmap); // When there is no more work to do, then maestro waits for the last minion to stop - XBT_DEBUG("Job done"); // ... and proceeds -} - -/** - * \brief Returns a next task to process. - * - * Worker threads call this function to get more work. - * - * \return the next task to process, or nullptr if there is no more work - */ -void* xbt_parmap_next(xbt_parmap_t parmap) -{ - unsigned int index = parmap->index++; - if (index < xbt_dynar_length(parmap->data)) { - return xbt_dynar_get_as(parmap->data, index, void*); - } - return nullptr; -} - -static void xbt_parmap_work(xbt_parmap_t parmap) -{ - unsigned int index = parmap->index++; - while (index < xbt_dynar_length(parmap->data)){ - parmap->fun(xbt_dynar_get_as(parmap->data, index, void*)); - index = parmap->index++; - } -} - -/** - * \brief Main function of a worker thread. - * \param arg the parmap - */ -static void *xbt_parmap_worker_main(void *arg) -{ - xbt_parmap_thread_data_t data = static_cast(arg); - xbt_parmap_t parmap = data->parmap; - unsigned round = 0; - smx_context_t context = SIMIX_context_new(std::function(), nullptr, nullptr); - SIMIX_context_set_current(context); - - XBT_DEBUG("New worker thread created"); - - /* Worker's main loop */ - while (1) { - round++; - parmap->worker_wait_f(parmap, round); - if (parmap->status == XBT_PARMAP_WORK) { - XBT_DEBUG("Worker %d got a job", data->worker_id); - - xbt_parmap_work(parmap); - parmap->worker_signal_f(parmap); - - XBT_DEBUG("Worker %d has finished", data->worker_id); - /* We are destroying the parmap */ - } else { - delete context; - xbt_free(data); - return nullptr; - } - } -} - -#if HAVE_FUTEX_H -static void futex_wait(unsigned *uaddr, unsigned val) -{ - XBT_VERB("Waiting on futex %p", uaddr); - syscall(SYS_futex, uaddr, FUTEX_WAIT_PRIVATE, val, nullptr, nullptr, 0); -} - -static void futex_wake(unsigned *uaddr, unsigned val) -{ - XBT_VERB("Waking futex %p", uaddr); - syscall(SYS_futex, uaddr, FUTEX_WAKE_PRIVATE, val, nullptr, nullptr, 0); -} -#endif - -/** - * \brief Starts the parmap: waits for all workers to be ready and returns. - * - * This function is called by the controller thread. - * - * \param parmap a parmap - */ -static void xbt_parmap_posix_master_wait(xbt_parmap_t parmap) -{ - xbt_os_mutex_acquire(parmap->done_mutex); - if (parmap->thread_counter < parmap->num_workers) { - /* wait for all workers to be ready */ - xbt_os_cond_wait(parmap->done_cond, parmap->done_mutex); - } - xbt_os_mutex_release(parmap->done_mutex); -} - -/** - * \brief Ends the parmap: wakes the controller thread when all workers terminate. - * - * This function is called by all worker threads when they end (not including the controller). - * - * \param parmap a parmap - */ -static void xbt_parmap_posix_worker_signal(xbt_parmap_t parmap) -{ - xbt_os_mutex_acquire(parmap->done_mutex); - parmap->thread_counter++; - if (parmap->thread_counter == parmap->num_workers) { - /* all workers have finished, wake the controller */ - xbt_os_cond_signal(parmap->done_cond); - } - xbt_os_mutex_release(parmap->done_mutex); -} - -/** - * \brief Wakes all workers and waits for them to finish the tasks. - * - * This function is called by the controller thread. - * - * \param parmap a parmap - */ -static void xbt_parmap_posix_master_signal(xbt_parmap_t parmap) -{ - xbt_os_mutex_acquire(parmap->ready_mutex); - parmap->thread_counter = 1; - parmap->work++; - /* wake all workers */ - xbt_os_cond_broadcast(parmap->ready_cond); - xbt_os_mutex_release(parmap->ready_mutex); -} - -/** - * \brief Waits for some work to process. - * - * This function is called by each worker thread (not including the controller) when it has no more work to do. - * - * \param parmap a parmap - * \param round the expected round number - */ -static void xbt_parmap_posix_worker_wait(xbt_parmap_t parmap, unsigned round) -{ - xbt_os_mutex_acquire(parmap->ready_mutex); - /* wait for more work */ - if (parmap->work != round) { - xbt_os_cond_wait(parmap->ready_cond, parmap->ready_mutex); - } - xbt_os_mutex_release(parmap->ready_mutex); -} - -#if HAVE_FUTEX_H -/** - * \brief Starts the parmap: waits for all workers to be ready and returns. - * - * This function is called by the controller thread. - * - * \param parmap a parmap - */ -static void xbt_parmap_futex_master_wait(xbt_parmap_t parmap) -{ - unsigned count = parmap->thread_counter; - while (count < parmap->num_workers) { - /* wait for all workers to be ready */ - futex_wait(&parmap->thread_counter, count); - count = parmap->thread_counter; - } -} - -/** - * \brief Ends the parmap: wakes the controller thread when all workers terminate. - * - * This function is called by all worker threads when they end (not including the controller). - * - * \param parmap a parmap - */ -static void xbt_parmap_futex_worker_signal(xbt_parmap_t parmap) -{ - unsigned count = __sync_add_and_fetch(&parmap->thread_counter, 1); - if (count == parmap->num_workers) { - /* all workers have finished, wake the controller */ - futex_wake(&parmap->thread_counter, INT_MAX); - } -} - -/** - * \brief Wakes all workers and waits for them to finish the tasks. - * - * This function is called by the controller thread. - * - * \param parmap a parmap - */ -static void xbt_parmap_futex_master_signal(xbt_parmap_t parmap) -{ - parmap->thread_counter = 1; - __sync_add_and_fetch(&parmap->work, 1); - /* wake all workers */ - futex_wake(&parmap->work, INT_MAX); -} - -/** - * \brief Waits for some work to process. - * - * This function is called by each worker thread (not including the controller) when it has no more work to do. - * - * \param parmap a parmap - * \param round the expected round number - */ -static void xbt_parmap_futex_worker_wait(xbt_parmap_t parmap, unsigned round) -{ - unsigned work = parmap->work; - /* wait for more work */ - while (work != round) { - futex_wait(&parmap->work, work); - work = parmap->work; - } -} -#endif - -/** - * \brief Starts the parmap: waits for all workers to be ready and returns. - * - * This function is called by the controller thread. - * - * \param parmap a parmap - */ -static void xbt_parmap_busy_master_wait(xbt_parmap_t parmap) -{ - while (parmap->thread_counter < parmap->num_workers) { - xbt_os_thread_yield(); - } -} - -/** - * \brief Ends the parmap: wakes the controller thread when all workers terminate. - * - * This function is called by all worker threads when they end. - * - * \param parmap a parmap - */ -static void xbt_parmap_busy_worker_signal(xbt_parmap_t parmap) -{ - __sync_add_and_fetch(&parmap->thread_counter, 1); -} - -/** - * \brief Wakes all workers and waits for them to finish the tasks. - * - * This function is called by the controller thread. - * - * \param parmap a parmap - */ -static void xbt_parmap_busy_master_signal(xbt_parmap_t parmap) -{ - parmap->thread_counter = 1; - __sync_add_and_fetch(&parmap->work, 1); -} - -/** - * \brief Waits for some work to process. - * - * This function is called by each worker thread (not including the controller) when it has no more work to do. - * - * \param parmap a parmap - * \param round the expected round number - */ -static void xbt_parmap_busy_worker_wait(xbt_parmap_t parmap, unsigned round) -{ - /* wait for more work */ - while (parmap->work != round) { - xbt_os_thread_yield(); - } -} diff --git a/src/xbt/xbt_replay.cpp b/src/xbt/xbt_replay.cpp index 9571091105..4c2fec8c09 100644 --- a/src/xbt/xbt_replay.cpp +++ b/src/xbt/xbt_replay.cpp @@ -38,6 +38,7 @@ public: fs = new std::ifstream(filename, std::ifstream::in); xbt_assert(fs->is_open(), "Cannot read replay file '%s'", filename); } + ReplayReader(const ReplayReader&) = delete; ~ReplayReader() { delete fs; @@ -78,9 +79,10 @@ static ReplayAction* get_action(char* name) } else { // Else, I have to store it for the relevant colleague std::queue* otherqueue = nullptr; - try { - otherqueue = action_queues.at(evtname); - } catch (std::out_of_range& unfound) { // Damn. Create the queue of that guy + auto act = action_queues.find(evtname); + if (act != action_queues.end()) { + otherqueue = act->second; + } else { // Damn. Create the queue of that guy otherqueue = new std::queue(); action_queues.insert({evtname, otherqueue}); } diff --git a/src/xbt/xbt_str.cpp b/src/xbt/xbt_str.cpp index ac5bc37168..bead714bd1 100644 --- a/src/xbt/xbt_str.cpp +++ b/src/xbt/xbt_str.cpp @@ -56,7 +56,7 @@ void xbt_str_rtrim(char *s, const char *char_list) /** @brief Strip whitespace (or other characters) from the beginning of a string. * - * Strips the whitespaces from the begining of s. + * Strips the whitespaces from the beginning of s. * By default (when char_list=nullptr), these characters get stripped: * * - " " (ASCII 32 (0x20)) space. @@ -94,7 +94,7 @@ void xbt_str_ltrim(char *s, const char *char_list) memmove(s, cur, strlen(cur) + 1); } -/** @brief Strip whitespace (or other characters) from the end and the begining of a string. +/** @brief Strip whitespace (or other characters) from the end and the beginning of a string. * * Strips the whitespaces from both the beginning and the end of s. * By default (when char_list=nullptr), these characters get stripped: @@ -123,7 +123,7 @@ void xbt_str_trim(char *s, const char *char_list) * @param str the string to modify * @param from char to search * @param to char to put instead - * @param occurence number of changes to do (=0 means all) + * @param occurrence number of changes to do (=0 means all) */ void xbt_str_subst(char *str, char from, char to, int occurence) { diff --git a/teshsuite/mc/dwarf/dwarf.cpp b/teshsuite/mc/dwarf/dwarf.cpp index 886d410750..02c26eac9c 100644 --- a/teshsuite/mc/dwarf/dwarf.cpp +++ b/teshsuite/mc/dwarf/dwarf.cpp @@ -153,7 +153,7 @@ int main(int argc, char** argv) int lexical_block_variable = 50; test_local_variable(process.binary_info.get(), "main", "lexical_block_variable", &lexical_block_variable, &cursor); - s_foo my_foo; + s_foo my_foo = {0}; test_type_by_name(process, my_foo); _exit(0); diff --git a/teshsuite/msg/CMakeLists.txt b/teshsuite/msg/CMakeLists.txt index 8203287fac..be9c437bdb 100644 --- a/teshsuite/msg/CMakeLists.txt +++ b/teshsuite/msg/CMakeLists.txt @@ -19,8 +19,17 @@ foreach(x task_destroy_cancel task_listen_from) set(teshsuite_src ${teshsuite_src} ${CMAKE_CURRENT_SOURCE_DIR}/${x}/${x}.cpp) endforeach() +add_executable (bittorrent app-bittorrent/bittorrent.c app-bittorrent/messages.c app-bittorrent/peer.c app-bittorrent/tracker.c app-bittorrent/connection.c) +target_link_libraries(bittorrent simgrid) +set_target_properties(bittorrent PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/app-bittorrent) +foreach (file bittorrent connection messages peer tracker) + set(teshsuite_src ${teshsuite_src} ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/${file}.c ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/${file}.h) +endforeach() + + set(teshsuite_src ${teshsuite_src} PARENT_SCOPE) -set(tesh_files ${tesh_files} PARENT_SCOPE) +set(tesh_files ${tesh_files} ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/app-bittorrent.tesh PARENT_SCOPE) +set(bin_files ${bin_files} ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/generate.py PARENT_SCOPE) set(txt_files ${txt_files} ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/actions-comm.txt ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/actions-comm_split_p0.txt ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/actions-comm_split_p1.txt @@ -28,7 +37,8 @@ set(txt_files ${txt_files} ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/acti set(xml_files ${xml_files} ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/actions-comm_d.xml ${CMAKE_CURRENT_SOURCE_DIR}/actions-comm/actions-comm_split_d.xml ${CMAKE_CURRENT_SOURCE_DIR}/actions-storage/actions-storage_d.xml - ${CMAKE_CURRENT_SOURCE_DIR}/trace_integration/test-hbp1.0-hbp1.0-hbp1.0.xml + ${CMAKE_CURRENT_SOURCE_DIR}/app-bittorrent/app-bittorrent_d.xml + ${CMAKE_CURRENT_SOURCE_DIR}/trace_integration/test-hbp1.0-hbp1.0-hbp1.0.xml ${CMAKE_CURRENT_SOURCE_DIR}/trace_integration/test-hbp1.0-hbp3.0-hbp4.0.xml ${CMAKE_CURRENT_SOURCE_DIR}/trace_integration/test-hbp1.5-hbp1.5.xml ${CMAKE_CURRENT_SOURCE_DIR}/trace_integration/test-hbp1-c0s0-c0s1.xml @@ -45,7 +55,7 @@ foreach(x get_sender host_on_off host_on_off_processes host_on_off_recv task_des ADD_TESH_FACTORIES(tesh-msg-${x} "thread;boost;ucontext;raw" --setenv srcdir=${CMAKE_HOME_DIRECTORY}/teshsuite/msg/${x} --cd ${CMAKE_BINARY_DIR}/teshsuite/msg/${x} ${CMAKE_HOME_DIRECTORY}/teshsuite/msg/${x}/${x}.tesh) endforeach() -foreach(x actions-comm actions-storage) +foreach(x actions-comm actions-storage app-bittorrent) ADD_TESH_FACTORIES(tesh-msg-${x} "thread;ucontext;raw;boost" --setenv bindir=${CMAKE_BINARY_DIR}/teshsuite/msg/${x} --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/platforms --cd ${CMAKE_HOME_DIRECTORY}/teshsuite/msg/${x} ${x}.tesh) endforeach() # One context factory is enough for these ones @@ -53,3 +63,5 @@ endforeach() foreach(x cloud-sharing) ADD_TESH(tesh-msg-${x} --setenv srcdir=${CMAKE_HOME_DIRECTORY}/teshsuite/msg/${x} --cd ${CMAKE_BINARY_DIR}/teshsuite/msg/${x} ${CMAKE_HOME_DIRECTORY}/teshsuite/msg/${x}/${x}.tesh) endforeach() + +ADD_TESH_FACTORIES(tesh-app-bittorrent-parallel "thread;ucontext;raw" --cfg contexts/nthreads:4 ${CONTEXTS_SYNCHRO} --setenv bindir=${CMAKE_BINARY_DIR}/teshsuite/msg/app-bittorrent --setenv srcdir=${CMAKE_HOME_DIRECTORY}/examples/platforms --cd ${CMAKE_HOME_DIRECTORY}/teshsuite/msg/app-bittorrent app-bittorrent.tesh) diff --git a/examples/msg/app-bittorrent/app-bittorrent.tesh b/teshsuite/msg/app-bittorrent/app-bittorrent.tesh similarity index 88% rename from examples/msg/app-bittorrent/app-bittorrent.tesh rename to teshsuite/msg/app-bittorrent/app-bittorrent.tesh index dbb9cbccdb..262fce3ae9 100644 --- a/examples/msg/app-bittorrent/app-bittorrent.tesh +++ b/teshsuite/msg/app-bittorrent/app-bittorrent.tesh @@ -4,7 +4,7 @@ p Testing the Bittorrent implementation with MSG ! timeout 10 ! output sort 19 -$ $SG_TEST_EXENV ${bindir:=.}/bittorrent ${srcdir:=.}/cluster.xml ${srcdir:=.}/../msg/app-bittorrent/app-bittorrent_d.xml "--log=root.fmt:[%12.6r]%e(%i:%P@%h)%e%m%n" +$ ${bindir:=.}/bittorrent ${srcdir:=.}/cluster.xml app-bittorrent_d.xml "--log=root.fmt:[%12.6r]%e(%i:%P@%h)%e%m%n" > [ 0.000000] (1:tracker@node-0.acme.org) Tracker launched. > [ 0.000000] (2:peer@node-1.acme.org) Hi, I'm joining the network with id 2 > [ 0.000000] (3:peer@node-2.acme.org) Hi, I'm joining the network with id 3 diff --git a/teshsuite/msg/app-bittorrent/app-bittorrent_d.xml b/teshsuite/msg/app-bittorrent/app-bittorrent_d.xml new file mode 100644 index 0000000000..5460ab1a9d --- /dev/null +++ b/teshsuite/msg/app-bittorrent/app-bittorrent_d.xml @@ -0,0 +1,39 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/examples/msg/app-bittorrent/bittorrent.c b/teshsuite/msg/app-bittorrent/bittorrent.c similarity index 85% rename from examples/msg/app-bittorrent/bittorrent.c rename to teshsuite/msg/app-bittorrent/bittorrent.c index c38162ad2d..93b8e0e183 100644 --- a/examples/msg/app-bittorrent/bittorrent.c +++ b/teshsuite/msg/app-bittorrent/bittorrent.c @@ -11,7 +11,7 @@ #include /** Bittorrent example launcher */ -int main(int argc, char *argv[]) +int main(int argc, char* argv[]) { msg_host_t host; unsigned i; @@ -19,12 +19,12 @@ int main(int argc, char *argv[]) MSG_init(&argc, argv); /* Check the arguments */ - xbt_assert (argc > 2, "Usage: %s platform_file deployment_file", argv[0]); + xbt_assert(argc > 2, "Usage: %s platform_file deployment_file", argv[0]); MSG_create_environment(argv[1]); xbt_dynar_t host_list = MSG_hosts_as_dynar(); - xbt_dynar_foreach(host_list, i, host) { + xbt_dynar_foreach (host_list, i, host) { char descr[512]; snprintf(descr, sizeof descr, "RngSream<%s>", MSG_host_get_name(host)); RngStream stream = RngStream_CreateStream(descr); @@ -38,7 +38,7 @@ int main(int argc, char *argv[]) MSG_main(); - xbt_dynar_foreach(host_list, i, host) { + xbt_dynar_foreach (host_list, i, host) { RngStream stream = (RngStream)MSG_host_get_data(host); RngStream_DeleteStream(&stream); MSG_host_set_data(host, NULL); diff --git a/examples/msg/app-bittorrent/bittorrent.h b/teshsuite/msg/app-bittorrent/bittorrent.h similarity index 84% rename from examples/msg/app-bittorrent/bittorrent.h rename to teshsuite/msg/app-bittorrent/bittorrent.h index b45d688a4c..d88793811e 100644 --- a/examples/msg/app-bittorrent/bittorrent.h +++ b/teshsuite/msg/app-bittorrent/bittorrent.h @@ -1,4 +1,4 @@ -/* Copyright (c) 2012-2014, 2016. The SimGrid Team. +/* Copyright (c) 2012-2014, 2016-2017. The SimGrid Team. * All rights reserved. */ /* This program is free software; you can redistribute it and/or modify it @@ -10,11 +10,11 @@ #define MAILBOX_SIZE 40 #define TRACKER_MAILBOX "tracker_mailbox" /** Max number of pairs sent by the tracker to clients */ -#define MAXIMUM_PAIRS 50 +#define MAXIMUM_PEERS 50 /** Interval of time where the peer should send a request to the tracker */ #define TRACKER_QUERY_INTERVAL 1000 /** Communication size for a task to the tracker */ -#define TRACKER_COMM_SIZE 0.01 +#define TRACKER_COMM_SIZE 1 #define GET_PEERS_TIMEOUT 10000 #define TIMEOUT_MESSAGE 10 #define TRACKER_RECEIVE_TIMEOUT 10 @@ -25,4 +25,4 @@ /** Number of pieces the peer asks for simultaneously */ #define MAX_PIECES 1 -#endif /* BITTORRENT_BITTORRENT_H_ */ +#endif /* BITTORRENT_BITTORRENT_H_ */ diff --git a/examples/msg/app-bittorrent/connection.c b/teshsuite/msg/app-bittorrent/connection.c similarity index 63% rename from examples/msg/app-bittorrent/connection.c rename to teshsuite/msg/app-bittorrent/connection.c index 43140e6fb8..3a53ed9cf6 100644 --- a/examples/msg/app-bittorrent/connection.c +++ b/teshsuite/msg/app-bittorrent/connection.c @@ -13,16 +13,16 @@ connection_t connection_new(int id) { connection_t connection = xbt_new(s_connection_t, 1); - connection->id = id; - connection->mailbox = bprintf("%d", id); - connection->bitfield = 0; - connection->current_piece = -1; - connection->interested = 0; - connection->am_interested = 0; - connection->choked_upload = 1; + connection->id = id; + connection->mailbox = bprintf("%d", id); + connection->bitfield = 0; + connection->current_piece = -1; + connection->interested = 0; + connection->am_interested = 0; + connection->choked_upload = 1; connection->choked_download = 1; - connection->peer_speed = 0; - connection->last_unchoke = 0; + connection->peer_speed = 0; + connection->last_unchoke = 0; return connection; } @@ -32,13 +32,14 @@ void connection_add_speed_value(connection_t connection, double speed) connection->peer_speed = connection->peer_speed * 0.6 + speed * 0.4; } -void connection_free(void *data) +void connection_free(void* data) { - connection_t co = (connection_t) data; + connection_t co = (connection_t)data; xbt_free(co->mailbox); xbt_free(co); } -int connection_has_piece(connection_t connection, unsigned int piece){ - return (connection->bitfield & 1U<bitfield & 1U << piece); } diff --git a/examples/msg/app-bittorrent/connection.h b/teshsuite/msg/app-bittorrent/connection.h similarity index 62% rename from examples/msg/app-bittorrent/connection.h rename to teshsuite/msg/app-bittorrent/connection.h index cd3362014f..2b64528b7e 100644 --- a/examples/msg/app-bittorrent/connection.h +++ b/teshsuite/msg/app-bittorrent/connection.h @@ -9,20 +9,20 @@ /** Contains the connection data of a peer. */ typedef struct s_connection { - int id; //Peer id - unsigned int bitfield; //Fields - char *mailbox; + int id; // Peer id + unsigned int bitfield; // Fields + char* mailbox; int messages_count; double peer_speed; double last_unchoke; int current_piece; - unsigned int am_interested:1; //Indicates if we are interested in something the peer has - unsigned int interested:1; //Indicates if the peer is interested in one of our pieces - unsigned int choked_upload:1; //Indicates if the peer is choked for the current peer - unsigned int choked_download:1; //Indicates if the peer has choked the current peer + unsigned int am_interested : 1; // Indicates if we are interested in something the peer has + unsigned int interested : 1; // Indicates if the peer is interested in one of our pieces + unsigned int choked_upload : 1; // Indicates if the peer is choked for the current peer + unsigned int choked_download : 1; // Indicates if the peer has choked the current peer } s_connection_t; -typedef s_connection_t *connection_t; +typedef s_connection_t* connection_t; /** @brief Build a new connection object from the peer id. * @param id id of the peer @@ -34,6 +34,6 @@ connection_t connection_new(int id); */ void connection_add_speed_value(connection_t connection, double speed); /** Frees a connection object */ -void connection_free(void *data); +void connection_free(void* data); int connection_has_piece(connection_t connection, unsigned int piece); -#endif /* BITTORRENT_CONNECTION_H_ */ +#endif /* BITTORRENT_CONNECTION_H_ */ diff --git a/examples/msg/app-bittorrent/generate.py b/teshsuite/msg/app-bittorrent/generate.py similarity index 100% rename from examples/msg/app-bittorrent/generate.py rename to teshsuite/msg/app-bittorrent/generate.py diff --git a/teshsuite/msg/app-bittorrent/messages.c b/teshsuite/msg/app-bittorrent/messages.c new file mode 100644 index 0000000000..671c289215 --- /dev/null +++ b/teshsuite/msg/app-bittorrent/messages.c @@ -0,0 +1,118 @@ +/* Copyright (c) 2012-2014. The SimGrid Team. + * All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#include "messages.h" +#include "bittorrent.h" + +XBT_LOG_NEW_DEFAULT_CATEGORY(msg_messages, "Messages specific for the message factory"); + +#define BITS_TO_BYTES(x) (((x) / 8 + (x) % 8) ? 1 : 0) + +/** @brief Build a new empty message + * @param type type of the message + * @param issuer_host_name hostname of the issuer, for debugging purposes + * @param mailbox mailbox where the peer should answer + * @param peer_id id of the issuer + * @param size message size in bytes + */ +msg_task_t task_message_new(e_message_type type, const char* issuer_host_name, const char* mailbox, int peer_id, + int size) +{ + message_t message = xbt_new(s_message_t, 1); + message->issuer_host_name = issuer_host_name; + message->peer_id = peer_id; + message->mailbox = mailbox; + message->type = type; + msg_task_t task = MSG_task_create(NULL, 0, size, message); + XBT_DEBUG("type: %d size: %d", (int)type, size); + return task; +} + +/** Builds a message containing an index. */ +msg_task_t task_message_index_new(e_message_type type, const char* issuer_host_name, const char* mailbox, int peer_id, + int index, int varsize) +{ + msg_task_t task = task_message_new(type, issuer_host_name, mailbox, peer_id, task_message_size(type) + varsize); + message_t message = MSG_task_get_data(task); + message->index = index; + return task; +} + +msg_task_t task_message_bitfield_new(const char* issuer_host_name, const char* mailbox, int peer_id, + unsigned int bitfield, int bitfield_size) +{ + msg_task_t task = task_message_new(MESSAGE_BITFIELD, issuer_host_name, mailbox, peer_id, + task_message_size(MESSAGE_BITFIELD) + BITS_TO_BYTES(bitfield_size)); + message_t message = MSG_task_get_data(task); + message->bitfield = bitfield; + return task; +} + +msg_task_t task_message_request_new(const char* issuer_host_name, const char* mailbox, int peer_id, int index, + int block_index, int block_length) +{ + msg_task_t task = task_message_index_new(MESSAGE_REQUEST, issuer_host_name, mailbox, peer_id, index, 0); + message_t message = MSG_task_get_data(task); + message->block_index = block_index; + message->block_length = block_length; + return task; +} + +msg_task_t task_message_piece_new(const char* issuer_host_name, const char* mailbox, int peer_id, int index, + int block_index, int block_length, int block_size) +{ + msg_task_t task = + task_message_index_new(MESSAGE_PIECE, issuer_host_name, mailbox, peer_id, index, block_length * block_size); + message_t message = MSG_task_get_data(task); + message->block_index = block_index; + message->block_length = block_length; + return task; +} + +void task_message_free(void* task) +{ + message_t message = MSG_task_get_data(task); + xbt_free(message); + MSG_task_destroy(task); +} + +int task_message_size(e_message_type type) +{ + int size = 0; + switch (type) { + case MESSAGE_HANDSHAKE: + size = MESSAGE_HANDSHAKE_SIZE; + break; + case MESSAGE_CHOKE: + size = MESSAGE_CHOKE_SIZE; + break; + case MESSAGE_UNCHOKE: + size = MESSAGE_UNCHOKE_SIZE; + break; + case MESSAGE_INTERESTED: + case MESSAGE_NOTINTERESTED: + size = MESSAGE_INTERESTED_SIZE; + break; + case MESSAGE_HAVE: + size = MESSAGE_HAVE_SIZE; + break; + case MESSAGE_BITFIELD: + size = MESSAGE_BITFIELD_SIZE; + break; + case MESSAGE_REQUEST: + size = MESSAGE_REQUEST_SIZE; + break; + case MESSAGE_PIECE: + size = MESSAGE_PIECE_SIZE; + break; + case MESSAGE_CANCEL: + size = MESSAGE_CANCEL_SIZE; + break; + default: + THROW_IMPOSSIBLE; + } + return size; +} diff --git a/examples/msg/app-bittorrent/messages.h b/teshsuite/msg/app-bittorrent/messages.h similarity index 74% rename from examples/msg/app-bittorrent/messages.h rename to teshsuite/msg/app-bittorrent/messages.h index 66129d984b..a86649b411 100644 --- a/examples/msg/app-bittorrent/messages.h +++ b/teshsuite/msg/app-bittorrent/messages.h @@ -40,33 +40,33 @@ typedef enum { /** Message data */ typedef struct s_message { e_message_type type; - const char *mailbox; - const char *issuer_host_name; + const char* mailbox; + const char* issuer_host_name; int peer_id; unsigned int bitfield; int index; int block_index; int block_length; } s_message_t; -typedef s_message_t *message_t; +typedef s_message_t* message_t; /** Builds a new value-less message */ -msg_task_t task_message_new(e_message_type type, const char *issuer_host_name, const char *mailbox, int peer_id, +msg_task_t task_message_new(e_message_type type, const char* issuer_host_name, const char* mailbox, int peer_id, int size); /** Builds a new "have/piece" message */ -msg_task_t task_message_index_new(e_message_type type, const char *issuer_host_name, const char *mailbox, int peer_id, +msg_task_t task_message_index_new(e_message_type type, const char* issuer_host_name, const char* mailbox, int peer_id, int index, int varsize); /** Builds a new bitfield message */ -msg_task_t task_message_bitfield_new(const char *issuer_host_name, const char *mailbox, int peer_id, +msg_task_t task_message_bitfield_new(const char* issuer_host_name, const char* mailbox, int peer_id, unsigned int bitfield, int bitfield_size); /** Builds a new "request" message */ -msg_task_t task_message_request_new(const char *issuer_host_name, const char *mailbox, int peer_id, int index, +msg_task_t task_message_request_new(const char* issuer_host_name, const char* mailbox, int peer_id, int index, int block_index, int block_length); /** Build a new "piece" message */ -msg_task_t task_message_piece_new(const char *issuer_host_name, const char *mailbox, int peer_id, int index, +msg_task_t task_message_piece_new(const char* issuer_host_name, const char* mailbox, int peer_id, int index, int block_index, int block_length, int block_size); /** Free a message task */ -void task_message_free(void *); +void task_message_free(void*); int task_message_size(e_message_type type); -#endif /* BITTORRENT_MESSAGES_H_ */ +#endif /* BITTORRENT_MESSAGES_H_ */ diff --git a/examples/msg/app-bittorrent/peer.c b/teshsuite/msg/app-bittorrent/peer.c similarity index 64% rename from examples/msg/app-bittorrent/peer.c rename to teshsuite/msg/app-bittorrent/peer.c index ea5c370589..69d75df59b 100644 --- a/examples/msg/app-bittorrent/peer.c +++ b/teshsuite/msg/app-bittorrent/peer.c @@ -4,12 +4,12 @@ * under the terms of the license (GNU LGPL) which comes with this package. */ #include "peer.h" -#include "tracker.h" #include "connection.h" #include "messages.h" +#include "tracker.h" +#include #include #include -#include XBT_LOG_NEW_DEFAULT_CATEGORY(msg_peers, "Messages specific for the peers"); @@ -17,9 +17,9 @@ XBT_LOG_NEW_DEFAULT_CATEGORY(msg_peers, "Messages specific for the peers"); * User parameters for transferred file data. For the test, the default values are : * File size: 10 pieces * 5 blocks/piece * 16384 bytes/block = 819200 bytes */ -#define FILE_PIECES 10U -#define PIECES_BLOCKS 5U -#define BLOCK_SIZE 16384 +#define FILE_PIECES 10UL +#define PIECES_BLOCKS 5UL +#define BLOCK_SIZE 16384 static const unsigned long int FILE_SIZE = FILE_PIECES * PIECES_BLOCKS * BLOCK_SIZE; /** Number of blocks asked by each request */ @@ -28,48 +28,52 @@ static const unsigned long int FILE_SIZE = FILE_PIECES * PIECES_BLOCKS * BLOCK_S #define ENABLE_END_GAME_MODE 1 #define SLEEP_DURATION 1 -int count_pieces(unsigned int bitfield){ - int count=0; +int count_pieces(unsigned int bitfield) +{ + int count = 0; unsigned int n = bitfield; - while (n){ + while (n) { count += n & 1U; - n >>= 1U ; + n >>= 1U; } return count; } -int peer_has_not_piece(peer_t peer, unsigned int piece){ - return !(peer->bitfield & 1U<bitfield & 1U << piece); } /** Check that a piece is not currently being download by the peer. */ -int peer_is_not_downloading_piece(peer_t peer, unsigned int piece){ - return !(peer->current_pieces & 1U<current_pieces & 1U << piece); } -void get_status(char **status, unsigned int bitfield){ - for(int i=FILE_PIECES-1; i>=0; i--) +void get_status(char** status, unsigned int bitfield) +{ + for (int i = FILE_PIECES - 1; i >= 0; i--) (*status)[i] = (bitfield & (1U << i)) ? '1' : '0'; (*status)[FILE_PIECES] = '\0'; } /** Peer main function */ -int peer(int argc, char *argv[]) +int peer(int argc, char* argv[]) { - //Check arguments + // Check arguments xbt_assert(argc == 3 || argc == 4, "Wrong number of arguments"); - //Build peer object - peer_t peer = peer_init(xbt_str_parse_int(argv[1],"Invalid ID: %s"), argc==4 ? 1:0); + // Build peer object + peer_t peer = peer_init(xbt_str_parse_int(argv[1], "Invalid ID: %s"), argc == 4 ? 1 : 0); - //Retrieve deadline - double deadline = xbt_str_parse_double(argv[2],"Invalid deadline: %s"); + // Retrieve deadline + double deadline = xbt_str_parse_double(argv[2], "Invalid deadline: %s"); xbt_assert(deadline > 0, "Wrong deadline supplied"); - char *status = xbt_malloc0(FILE_PIECES+1); + char* status = xbt_malloc0(FILE_PIECES + 1); get_status(&status, peer->bitfield); XBT_INFO("Hi, I'm joining the network with id %d", peer->id); - //Getting peer data from the tracker. + // Getting peer data from the tracker. if (get_peers_data(peer)) { XBT_DEBUG("Got %d peers from the tracker. Current status is: %s", xbt_dict_length(peer->peers), status); peer->begin_receive_time = MSG_get_clock(); @@ -121,7 +125,7 @@ void leech_loop(peer_t peer, double deadline) handle_message(peer, peer->task_received); } } else { - //We don't execute the choke algorithm if we don't already have a piece + // We don't execute the choke algorithm if we don't already have a piece if (MSG_get_clock() >= next_choked_update && count_pieces(peer->bitfield) > 0) { update_choked_peers(peer); next_choked_update += UPDATE_CHOKED_INTERVAL; @@ -142,7 +146,7 @@ void seed_loop(peer_t peer, double deadline) { double next_choked_update = MSG_get_clock() + UPDATE_CHOKED_INTERVAL; XBT_DEBUG("Start seeding."); - //start the main seed loop + // start the main seed loop while (MSG_get_clock() < deadline) { if (peer->comm_received == NULL) { peer->task_received = NULL; @@ -158,7 +162,7 @@ void seed_loop(peer_t peer, double deadline) } else { if (MSG_get_clock() >= next_choked_update) { update_choked_peers(peer); - //TODO: Change the choked peer algorithm when seeding. + // TODO: Change the choked peer algorithm when seeding. next_choked_update += UPDATE_CHOKED_INTERVAL; } else { MSG_process_sleep(SLEEP_DURATION); @@ -172,12 +176,12 @@ void seed_loop(peer_t peer, double deadline) */ int get_peers_data(peer_t peer) { - int success = 0; + int success = 0; double timeout = MSG_get_clock() + GET_PEERS_TIMEOUT; - //Build the task to send to the tracker - tracker_task_data_t data = tracker_task_data_new(MSG_host_get_name(MSG_host_self()), peer->mailbox_tracker, - peer->id, 0, 0, FILE_SIZE); + // Build the task to send to the tracker + tracker_task_data_t data = + tracker_task_data_new(MSG_host_get_name(MSG_host_self()), peer->mailbox_tracker, peer->id, 0, 0, FILE_SIZE); msg_task_t task_send = MSG_task_create(NULL, 0, TRACKER_COMM_SIZE, data); while ((success == 0) && MSG_get_clock() < timeout) { XBT_DEBUG("Sending a peer request to the tracker."); @@ -187,22 +191,22 @@ int get_peers_data(peer_t peer) } } - success = 0; + success = 0; msg_task_t task_received = NULL; while ((success == 0) && MSG_get_clock() < timeout) { msg_comm_t comm_received = MSG_task_irecv(&task_received, peer->mailbox_tracker); - msg_error_t status = MSG_comm_wait(comm_received, GET_PEERS_TIMEOUT); + msg_error_t status = MSG_comm_wait(comm_received, GET_PEERS_TIMEOUT); if (status == MSG_OK) { tracker_task_data_t data = MSG_task_get_data(task_received); unsigned i; int peer_id; - //Add the peers the tracker gave us to our peer list. - xbt_dynar_foreach(data->peers, i, peer_id) { + // Add the peers the tracker gave us to our peer list. + xbt_dynar_foreach (data->peers, i, peer_id) { if (peer_id != peer->id) - xbt_dict_set_ext(peer->peers, (char *) &peer_id, sizeof(int), connection_new(peer_id), NULL); + xbt_dict_set_ext(peer->peers, (char*)&peer_id, sizeof(int), connection_new(peer_id), NULL); } success = 1; - //free the communication and the task + // free the communication and the task MSG_comm_destroy(comm_received); tracker_task_data_free(data); MSG_task_destroy(task_received); @@ -219,20 +223,20 @@ int get_peers_data(peer_t peer) */ peer_t peer_init(int id, int seed) { - peer_t peer = xbt_new(s_peer_t,1); - peer->id = id; + peer_t peer = xbt_new(s_peer_t, 1); + peer->id = id; peer->hostname = MSG_host_get_name(MSG_host_self()); - snprintf(peer->mailbox,MAILBOX_SIZE-1, "%d", id); - snprintf(peer->mailbox_tracker,MAILBOX_SIZE-1, "tracker_%d", id); + snprintf(peer->mailbox, MAILBOX_SIZE - 1, "%d", id); + snprintf(peer->mailbox_tracker, MAILBOX_SIZE - 1, "tracker_%d", id); peer->peers = xbt_dict_new_homogeneous(NULL); peer->active_peers = xbt_dict_new_homogeneous(NULL); if (seed) { - peer->bitfield = (1U<bitfield_blocks = (1ULL<<(FILE_PIECES * PIECES_BLOCKS))-1ULL ; + peer->bitfield = (1U << FILE_PIECES) - 1U; + peer->bitfield_blocks = (1ULL << (FILE_PIECES * PIECES_BLOCKS)) - 1ULL; } else { - peer->bitfield = 0; + peer->bitfield = 0; peer->bitfield_blocks = 0; } @@ -251,10 +255,10 @@ peer_t peer_init(int id, int seed) /** Destroys a poor peer object. */ void peer_free(peer_t peer) { - char *key; + char* key; connection_t connection; xbt_dict_cursor_t cursor; - xbt_dict_foreach(peer->peers, cursor, key, connection) { + xbt_dict_foreach (peer->peers, cursor, key, connection) { connection_free(connection); } xbt_dict_free(&peer->peers); @@ -268,16 +272,16 @@ void peer_free(peer_t peer) */ int has_finished(unsigned int bitfield) { - return bitfield == (1U<peers, cursor, key, connection) { + xbt_dict_foreach (peer->peers, cursor, key, connection) { if (connection->interested) nb++; } @@ -287,10 +291,10 @@ int nb_interested_peers(peer_t peer) void update_active_peers_set(peer_t peer, connection_t remote_peer) { if ((remote_peer->interested != 0) && (remote_peer->choked_upload == 0)) { - //add in the active peers set - xbt_dict_set_ext(peer->active_peers, (char *) &remote_peer->id, sizeof(int), remote_peer, NULL); - } else if (xbt_dict_get_or_null_ext(peer->active_peers, (char *) &remote_peer->id, sizeof(int))) { - xbt_dict_remove_ext(peer->active_peers, (char *) &remote_peer->id, sizeof(int)); + // add in the active peers set + xbt_dict_set_ext(peer->active_peers, (char*)&remote_peer->id, sizeof(int), remote_peer, NULL); + } else if (xbt_dict_get_or_null_ext(peer->active_peers, (char*)&remote_peer->id, sizeof(int))) { + xbt_dict_remove_ext(peer->active_peers, (char*)&remote_peer->id, sizeof(int)); } } @@ -300,128 +304,129 @@ void update_active_peers_set(peer_t peer, connection_t remote_peer) */ void handle_message(peer_t peer, msg_task_t task) { - const char* type_names[10] = - {"HANDSHAKE","CHOKE","UNCHOKE","INTERESTED","NOTINTERESTED","HAVE","BITFIELD","REQUEST", "PIECE", "CANCEL" }; + const char* type_names[10] = {"HANDSHAKE", "CHOKE", "UNCHOKE", "INTERESTED", "NOTINTERESTED", + "HAVE", "BITFIELD", "REQUEST", "PIECE", "CANCEL"}; message_t message = MSG_task_get_data(task); XBT_DEBUG("Received a %s message from %s (%s)", type_names[message->type], message->mailbox, - message->issuer_host_name); + message->issuer_host_name); connection_t remote_peer; - remote_peer = xbt_dict_get_or_null_ext(peer->peers, (char *) &message->peer_id, sizeof(int)); + remote_peer = xbt_dict_get_or_null_ext(peer->peers, (char*)&message->peer_id, sizeof(int)); switch (message->type) { - case MESSAGE_HANDSHAKE: - //Check if the peer is in our connection list. - if (remote_peer == 0) { - xbt_dict_set_ext(peer->peers, (char *) &message->peer_id, sizeof(int), connection_new(message->peer_id), NULL); - send_handshake(peer, message->mailbox); - } - //Send our bitfield to the peer - send_bitfield(peer, message->mailbox); - break; - case MESSAGE_BITFIELD: - //Update the pieces list - update_pieces_count_from_bitfield(peer, message->bitfield); - //Store the bitfield - remote_peer->bitfield = message->bitfield; - xbt_assert(!remote_peer->am_interested, "Should not be interested at first"); - if (is_interested(peer, remote_peer)) { - remote_peer->am_interested = 1; - send_interested(peer, message->mailbox); - } - break; - case MESSAGE_INTERESTED: - xbt_assert((remote_peer != NULL), "A non-in-our-list peer has sent us a message. WTH ?"); - //Update the interested state of the peer. - remote_peer->interested = 1; - update_active_peers_set(peer, remote_peer); - break; - case MESSAGE_NOTINTERESTED: - xbt_assert((remote_peer != NULL), "A non-in-our-list peer has sent us a message. WTH ?"); - remote_peer->interested = 0; - update_active_peers_set(peer, remote_peer); - break; - case MESSAGE_UNCHOKE: - xbt_assert((remote_peer != NULL), "A non-in-our-list peer has sent us a message. WTH ?"); - xbt_assert(remote_peer->choked_download); - remote_peer->choked_download = 0; - //Send requests to the peer, since it has unchoked us - if (remote_peer->am_interested) - request_new_piece_to_peer(peer, remote_peer); - break; - case MESSAGE_CHOKE: - xbt_assert((remote_peer != NULL), "A non-in-our-list peer has sent us a message. WTH ?"); - xbt_assert(!remote_peer->choked_download); - remote_peer->choked_download = 1; - if (remote_peer->current_piece != -1) - remove_current_piece(peer, remote_peer, remote_peer->current_piece); - break; - case MESSAGE_HAVE: - XBT_DEBUG("\t for piece %d", message->index); - xbt_assert((message->index >= 0 && message->index < FILE_PIECES), "Wrong HAVE message received"); - remote_peer->bitfield = remote_peer->bitfield | (1U<index); - peer->pieces_count[message->index]++; - //If the piece is in our pieces, we tell the peer that we are interested. - if ((remote_peer->am_interested == 0) && peer_has_not_piece(peer,message->index)) { - remote_peer->am_interested = 1; - send_interested(peer, message->mailbox); - if (remote_peer->choked_download == 0) + case MESSAGE_HANDSHAKE: + // Check if the peer is in our connection list. + if (remote_peer == 0) { + xbt_dict_set_ext(peer->peers, (char*)&message->peer_id, sizeof(int), connection_new(message->peer_id), NULL); + send_handshake(peer, message->mailbox); + } + // Send our bitfield to the peer + send_bitfield(peer, message->mailbox); + break; + case MESSAGE_BITFIELD: + // Update the pieces list + update_pieces_count_from_bitfield(peer, message->bitfield); + // Store the bitfield + remote_peer->bitfield = message->bitfield; + xbt_assert(!remote_peer->am_interested, "Should not be interested at first"); + if (is_interested(peer, remote_peer)) { + remote_peer->am_interested = 1; + send_interested(peer, message->mailbox); + } + break; + case MESSAGE_INTERESTED: + xbt_assert((remote_peer != NULL), "A non-in-our-list peer has sent us a message. WTH ?"); + // Update the interested state of the peer. + remote_peer->interested = 1; + update_active_peers_set(peer, remote_peer); + break; + case MESSAGE_NOTINTERESTED: + xbt_assert((remote_peer != NULL), "A non-in-our-list peer has sent us a message. WTH ?"); + remote_peer->interested = 0; + update_active_peers_set(peer, remote_peer); + break; + case MESSAGE_UNCHOKE: + xbt_assert((remote_peer != NULL), "A non-in-our-list peer has sent us a message. WTH ?"); + xbt_assert(remote_peer->choked_download); + remote_peer->choked_download = 0; + // Send requests to the peer, since it has unchoked us + if (remote_peer->am_interested) request_new_piece_to_peer(peer, remote_peer); - } - break; - case MESSAGE_REQUEST: - xbt_assert(remote_peer->interested); - xbt_assert((message->index >= 0 && message->index < FILE_PIECES), "Wrong request received"); - if (remote_peer->choked_upload == 0) { - XBT_DEBUG("\t for piece %d (%d,%d)", message->index, message->block_index, - message->block_index + message->block_length); - if (!peer_has_not_piece(peer, message->index)) { - send_piece(peer, message->mailbox, message->index, message->block_index, message->block_length); + break; + case MESSAGE_CHOKE: + xbt_assert((remote_peer != NULL), "A non-in-our-list peer has sent us a message. WTH ?"); + xbt_assert(!remote_peer->choked_download); + remote_peer->choked_download = 1; + if (remote_peer->current_piece != -1) + remove_current_piece(peer, remote_peer, remote_peer->current_piece); + break; + case MESSAGE_HAVE: + XBT_DEBUG("\t for piece %d", message->index); + xbt_assert((message->index >= 0 && message->index < FILE_PIECES), "Wrong HAVE message received"); + remote_peer->bitfield = remote_peer->bitfield | (1U << message->index); + peer->pieces_count[message->index]++; + // If the piece is in our pieces, we tell the peer that we are interested. + if ((remote_peer->am_interested == 0) && peer_has_not_piece(peer, message->index)) { + remote_peer->am_interested = 1; + send_interested(peer, message->mailbox); + if (remote_peer->choked_download == 0) + request_new_piece_to_peer(peer, remote_peer); } - } else { - XBT_DEBUG("\t for piece %d but he is choked.", message->peer_id); - } - break; - case MESSAGE_PIECE: - XBT_DEBUG(" \t for piece %d (%d,%d)", message->index, message->block_index, - message->block_index + message->block_length); - xbt_assert(!remote_peer->choked_download); - xbt_assert(remote_peer->am_interested || ENABLE_END_GAME_MODE, - "Can't received a piece if I'm not interested wihtout end-game mode!" - "piece (%d) bitfield(%u) remote bitfield(%u)", message->index, peer->bitfield, remote_peer->bitfield); - xbt_assert(remote_peer->choked_download != 1, "Can't received a piece if I'm choked !"); - xbt_assert((message->index >= 0 && message->index < FILE_PIECES), "Wrong piece received"); - //TODO: Execute à computation. - if (peer_has_not_piece(peer,message->index)) { - update_bitfield_blocks(peer, message->index, message->block_index, message->block_length); - if (piece_complete(peer, message->index)) { - //Removing the piece from our piece list - remove_current_piece(peer, remote_peer, message->index); - //Setting the fact that we have the piece - peer->bitfield = peer->bitfield | (1U<index); - char* status = xbt_malloc0(FILE_PIECES+1); - get_status(&status, peer->bitfield); - XBT_DEBUG("My status is now %s", status); - xbt_free(status); - //Sending the information to all the peers we are connected to - send_have(peer, message->index); - //sending UNINTERESTED to peers that do not have what we want. - update_interested_after_receive(peer); - } else { // piece not completed - send_request_to_peer(peer, remote_peer, message->index); // ask for the next block + break; + case MESSAGE_REQUEST: + xbt_assert(remote_peer->interested); + xbt_assert((message->index >= 0 && message->index < FILE_PIECES), "Wrong request received"); + if (remote_peer->choked_upload == 0) { + XBT_DEBUG("\t for piece %d (%d,%d)", message->index, message->block_index, + message->block_index + message->block_length); + if (!peer_has_not_piece(peer, message->index)) { + send_piece(peer, message->mailbox, message->index, message->block_index, message->block_length); + } + } else { + XBT_DEBUG("\t for piece %d but he is choked.", message->peer_id); } - } else { - XBT_DEBUG("However, we already have it"); - xbt_assert(ENABLE_END_GAME_MODE, "Should not happen because we don't use end game mode !"); - request_new_piece_to_peer(peer, remote_peer); - } - break; - case MESSAGE_CANCEL: - break; - default: - THROW_IMPOSSIBLE; + break; + case MESSAGE_PIECE: + XBT_DEBUG(" \t for piece %d (%d,%d)", message->index, message->block_index, + message->block_index + message->block_length); + xbt_assert(!remote_peer->choked_download); + xbt_assert(remote_peer->am_interested || ENABLE_END_GAME_MODE, + "Can't received a piece if I'm not interested wihtout end-game mode!" + "piece (%d) bitfield(%u) remote bitfield(%u)", + message->index, peer->bitfield, remote_peer->bitfield); + xbt_assert(remote_peer->choked_download != 1, "Can't received a piece if I'm choked !"); + xbt_assert((message->index >= 0 && message->index < FILE_PIECES), "Wrong piece received"); + // TODO: Execute à computation. + if (peer_has_not_piece(peer, message->index)) { + update_bitfield_blocks(peer, message->index, message->block_index, message->block_length); + if (piece_complete(peer, message->index)) { + // Removing the piece from our piece list + remove_current_piece(peer, remote_peer, message->index); + // Setting the fact that we have the piece + peer->bitfield = peer->bitfield | (1U << message->index); + char* status = xbt_malloc0(FILE_PIECES + 1); + get_status(&status, peer->bitfield); + XBT_DEBUG("My status is now %s", status); + xbt_free(status); + // Sending the information to all the peers we are connected to + send_have(peer, message->index); + // sending UNINTERESTED to peers that do not have what we want. + update_interested_after_receive(peer); + } else { // piece not completed + send_request_to_peer(peer, remote_peer, message->index); // ask for the next block + } + } else { + XBT_DEBUG("However, we already have it"); + xbt_assert(ENABLE_END_GAME_MODE, "Should not happen because we don't use end game mode !"); + request_new_piece_to_peer(peer, remote_peer); + } + break; + case MESSAGE_CANCEL: + break; + default: + THROW_IMPOSSIBLE; } - //Update the peer speed. + // Update the peer speed. if (remote_peer) { connection_add_speed_value(remote_peer, 1.0 / (MSG_get_clock() - peer->begin_receive_time)); } @@ -435,7 +440,7 @@ void request_new_piece_to_peer(peer_t peer, connection_t remote_peer) { int piece = select_piece_to_download(peer, remote_peer); if (piece != -1) { - peer->current_pieces|= (1U << (unsigned int) piece); + peer->current_pieces |= (1U << (unsigned int)piece); send_request_to_peer(peer, remote_peer, piece); } } @@ -481,21 +486,21 @@ int select_piece_to_download(peer_t peer, connection_t remote_peer) if (count_pieces(peer->current_pieces) >= (FILE_PIECES - count_pieces(peer->bitfield)) && (is_interested(peer, remote_peer) != 0)) { #if ENABLE_END_GAME_MODE == 0 - return -1; + return -1; #endif int nb_interesting_pieces = 0; // compute the number of interesting pieces for (int i = 0; i < FILE_PIECES; i++) { - if (peer_has_not_piece(peer,i) && connection_has_piece(remote_peer,i)) { + if (peer_has_not_piece(peer, i) && connection_has_piece(remote_peer, i)) { nb_interesting_pieces++; } } xbt_assert(nb_interesting_pieces != 0); // get a random interesting piece int random_piece_index = RngStream_RandInt(peer->stream, 0, nb_interesting_pieces - 1); - int current_index = 0; + int current_index = 0; for (int i = 0; i < FILE_PIECES; i++) { - if (peer_has_not_piece(peer,i) && connection_has_piece(remote_peer,i)) { + if (peer_has_not_piece(peer, i) && connection_has_piece(remote_peer, i)) { if (random_piece_index == current_index) { piece = i; break; @@ -511,16 +516,18 @@ int select_piece_to_download(peer_t peer, connection_t remote_peer) int nb_interesting_pieces = 0; // compute the number of interesting pieces for (int i = 0; i < FILE_PIECES; i++) { - if (peer_has_not_piece(peer, i) && connection_has_piece(remote_peer,i) && peer_is_not_downloading_piece(peer, i)) { + if (peer_has_not_piece(peer, i) && connection_has_piece(remote_peer, i) && + peer_is_not_downloading_piece(peer, i)) { nb_interesting_pieces++; } } xbt_assert(nb_interesting_pieces != 0); // get a random interesting piece int random_piece_index = RngStream_RandInt(peer->stream, 0, nb_interesting_pieces - 1); - int current_index = 0; + int current_index = 0; for (int i = 0; i < FILE_PIECES; i++) { - if (peer_has_not_piece(peer, i) && connection_has_piece(remote_peer,i) && peer_is_not_downloading_piece(peer, i)) { + if (peer_has_not_piece(peer, i) && connection_has_piece(remote_peer, i) && + peer_is_not_downloading_piece(peer, i)) { if (random_piece_index == current_index) { piece = i; break; @@ -530,28 +537,28 @@ int select_piece_to_download(peer_t peer, connection_t remote_peer) } xbt_assert(piece != -1); return piece; - } else { // Rarest first policy - short min = SHRT_MAX; + } else { // Rarest first policy + short min = SHRT_MAX; int nb_min_pieces = 0; int current_index = 0; // compute the smallest number of copies of available pieces for (int i = 0; i < FILE_PIECES; i++) { - if (peer->pieces_count[i] < min && peer_has_not_piece(peer, i) && connection_has_piece(remote_peer,i) && + if (peer->pieces_count[i] < min && peer_has_not_piece(peer, i) && connection_has_piece(remote_peer, i) && peer_is_not_downloading_piece(peer, i)) min = peer->pieces_count[i]; } - xbt_assert(min != SHRT_MAX || (is_interested_and_free(peer, remote_peer) ==0)); + xbt_assert(min != SHRT_MAX || (is_interested_and_free(peer, remote_peer) == 0)); // compute the number of rarest pieces for (int i = 0; i < FILE_PIECES; i++) { - if (peer->pieces_count[i] == min && peer_has_not_piece(peer, i) && connection_has_piece(remote_peer,i) && + if (peer->pieces_count[i] == min && peer_has_not_piece(peer, i) && connection_has_piece(remote_peer, i) && peer_is_not_downloading_piece(peer, i)) nb_min_pieces++; } - xbt_assert(nb_min_pieces != 0 || (is_interested_and_free(peer, remote_peer)==0)); + xbt_assert(nb_min_pieces != 0 || (is_interested_and_free(peer, remote_peer) == 0)); // get a random rarest piece int random_rarest_index = RngStream_RandInt(peer->stream, 0, nb_min_pieces - 1); for (int i = 0; i < FILE_PIECES; i++) { - if (peer->pieces_count[i] == min && peer_has_not_piece(peer, i) && connection_has_piece(remote_peer,i) && + if (peer->pieces_count[i] == min && peer_has_not_piece(peer, i) && connection_has_piece(remote_peer, i) && peer_is_not_downloading_piece(peer, i)) { if (random_rarest_index == current_index) { piece = i; @@ -573,17 +580,17 @@ void update_choked_peers(peer_t peer) if (nb_interested_peers(peer) == 0) return; XBT_DEBUG("(%d) update_choked peers %u active peers", peer->id, xbt_dict_size(peer->active_peers)); - //update the current round + // update the current round peer->round = (peer->round + 1) % 3; - char *key; - char *key_choked=NULL; + char* key; + char* key_choked = NULL; connection_t peer_choosed = NULL; - connection_t peer_choked = NULL; - //remove a peer from the list + connection_t peer_choked = NULL; + // remove a peer from the list xbt_dict_cursor_t cursor = NULL; xbt_dict_cursor_first(peer->active_peers, &cursor); if (xbt_dict_length(peer->active_peers) > 0) { - key_choked = xbt_dict_cursor_get_key(cursor); + key_choked = xbt_dict_cursor_get_key(cursor); peer_choked = xbt_dict_cursor_get_data(cursor); } xbt_dict_cursor_free(&cursor); @@ -593,23 +600,23 @@ void update_choked_peers(peer_t peer) connection_t connection; double unchoke_time = MSG_get_clock() + 1; - xbt_dict_foreach(peer->peers, cursor, key, connection) { - if (connection->last_unchoke < unchoke_time && - (connection->interested != 0) && (connection->choked_upload != 0)) { + xbt_dict_foreach (peer->peers, cursor, key, connection) { + if (connection->last_unchoke < unchoke_time && (connection->interested != 0) && + (connection->choked_upload != 0)) { unchoke_time = connection->last_unchoke; peer_choosed = connection; } } } else { - //Random optimistic unchoking + // Random optimistic unchoking if (peer->round == 0) { int j = 0; do { - //We choose a random peer to unchoke. + // We choose a random peer to unchoke. int id_chosen = RngStream_RandInt(peer->stream, 0, xbt_dict_length(peer->peers) - 1); - int i = 0; + int i = 0; connection_t connection; - xbt_dict_foreach(peer->peers, cursor, key, connection) { + xbt_dict_foreach (peer->peers, cursor, key, connection) { if (i == id_chosen) { peer_choosed = connection; break; @@ -624,15 +631,15 @@ void update_choked_peers(peer_t peer) else XBT_DEBUG("Nothing to do, keep going"); j++; - } while (peer_choosed == NULL && j < MAXIMUM_PAIRS); + } while (peer_choosed == NULL && j < MAXIMUM_PEERS); } else { - //Use the "fastest download" policy. + // Use the "fastest download" policy. connection_t connection; double fastest_speed = 0.0; - xbt_dict_foreach(peer->peers, cursor, key, connection) { - if (connection->peer_speed > fastest_speed && - (connection->choked_upload != 0) && (connection->interested != 0)) { - peer_choosed = connection; + xbt_dict_foreach (peer->peers, cursor, key, connection) { + if (connection->peer_speed > fastest_speed && (connection->choked_upload != 0) && + (connection->interested != 0)) { + peer_choosed = connection; fastest_speed = connection->peer_speed; } } @@ -640,14 +647,14 @@ void update_choked_peers(peer_t peer) } if (peer_choosed != NULL) - XBT_DEBUG("(%d) update_choked peers unchoked (%d) ; int (%d) ; choked (%d) ", - peer->id, peer_choosed->id, peer_choosed->interested, peer_choosed->choked_upload); + XBT_DEBUG("(%d) update_choked peers unchoked (%d) ; int (%d) ; choked (%d) ", peer->id, peer_choosed->id, + peer_choosed->interested, peer_choosed->choked_upload); if (peer_choked != peer_choosed) { if (peer_choked != NULL) { xbt_assert((!peer_choked->choked_upload), "Tries to choked a choked peer"); peer_choked->choked_upload = 1; - xbt_assert((*((int *) key_choked) == peer_choked->id)); + xbt_assert((*((int*)key_choked) == peer_choked->id)); update_active_peers_set(peer, peer_choked); XBT_DEBUG("(%d) Sending a CHOKE to %d", peer->id, peer_choked->id); send_choked(peer, peer_choked->mailbox); @@ -655,7 +662,7 @@ void update_choked_peers(peer_t peer) if (peer_choosed != NULL) { xbt_assert((peer_choosed->choked_upload), "Tries to unchoked an unchoked peer"); peer_choosed->choked_upload = 0; - xbt_dict_set_ext(peer->active_peers, (char *) &peer_choosed->id, sizeof(int), peer_choosed, NULL); + xbt_dict_set_ext(peer->active_peers, (char*)&peer_choosed->id, sizeof(int), peer_choosed, NULL); peer_choosed->last_unchoke = MSG_get_clock(); XBT_DEBUG("(%d) Sending a UNCHOKE to %d", peer->id, peer_choosed->id); update_active_peers_set(peer, peer_choosed); @@ -669,20 +676,20 @@ void update_choked_peers(peer_t peer) */ void update_interested_after_receive(peer_t peer) { - char *key; + char* key; xbt_dict_cursor_t cursor; connection_t connection; - xbt_dict_foreach(peer->peers, cursor, key, connection) { + xbt_dict_foreach (peer->peers, cursor, key, connection) { if (connection->am_interested != 0) { int interested = 0; - //Check if the peer still has a piece we want. + // Check if the peer still has a piece we want. for (int i = 0; i < FILE_PIECES; i++) { - if (peer_has_not_piece(peer, i) && connection_has_piece(connection,i)) { + if (peer_has_not_piece(peer, i) && connection_has_piece(connection, i)) { interested = 1; break; } } - if (!interested) { //no more piece to download from connection + if (!interested) { // no more piece to download from connection connection->am_interested = 0; send_notinterested(peer, connection->mailbox); } @@ -695,7 +702,7 @@ void update_bitfield_blocks(peer_t peer, int index, int block_index, int block_l xbt_assert((index >= 0 && index <= FILE_PIECES), "Wrong piece."); xbt_assert((block_index >= 0 && block_index <= PIECES_BLOCKS), "Wrong block : %d.", block_index); for (int i = block_index; i < (block_index + block_length); i++) { - peer->bitfield_blocks |= (1ULL<<(unsigned int)(index * PIECES_BLOCKS + i)); + peer->bitfield_blocks |= (1ULL << (unsigned int)(index * PIECES_BLOCKS + i)); } } @@ -703,7 +710,7 @@ void update_bitfield_blocks(peer_t peer, int index, int block_index, int block_l int piece_complete(peer_t peer, int index) { for (int i = 0; i < PIECES_BLOCKS; i++) { - if (!(peer->bitfield_blocks & 1ULL<<(index * PIECES_BLOCKS + i))) { + if (!(peer->bitfield_blocks & 1ULL << (index * PIECES_BLOCKS + i))) { return 0; } } @@ -714,7 +721,7 @@ int piece_complete(peer_t peer, int index) int get_first_block(peer_t peer, int piece) { for (int i = 0; i < PIECES_BLOCKS; i++) { - if (!(peer->bitfield_blocks & 1ULL<<(piece * PIECES_BLOCKS + i))) { + if (!(peer->bitfield_blocks & 1ULL << (piece * PIECES_BLOCKS + i))) { return i; } } @@ -724,14 +731,14 @@ int get_first_block(peer_t peer, int piece) /** Indicates if the remote peer has a piece not stored by the local peer */ int is_interested(peer_t peer, connection_t remote_peer) { - return remote_peer->bitfield & (peer->bitfield^((1<bitfield & (peer->bitfield ^ ((1 << FILE_PIECES) - 1)); } /** Indicates if the remote peer has a piece not stored by the local peer nor requested by the local peer */ int is_interested_and_free(peer_t peer, connection_t remote_peer) { for (int i = 0; i < FILE_PIECES; i++) { - if (peer_has_not_piece(peer, i) && connection_has_piece(remote_peer,i) && peer_is_not_downloading_piece(peer, i)) { + if (peer_has_not_piece(peer, i) && connection_has_piece(remote_peer, i) && peer_is_not_downloading_piece(peer, i)) { return 1; } } @@ -756,7 +763,7 @@ int partially_downloaded_piece(peer_t peer, connection_t remote_peer) void send_request_to_peer(peer_t peer, connection_t remote_peer, int piece) { remote_peer->current_piece = piece; - xbt_assert(connection_has_piece(remote_peer,piece)); + xbt_assert(connection_has_piece(remote_peer, piece)); int block_index = get_first_block(peer, piece); if (block_index != -1) { int block_length = MIN(BLOCKS_REQUESTED, PIECES_BLOCKS - block_index); @@ -774,7 +781,7 @@ void send_request_to_peer(peer_t peer, connection_t remote_peer, int piece) * @param peer peer data * @param mailbox destination mailbox */ -void send_interested(peer_t peer, const char *mailbox) +void send_interested(peer_t peer, const char* mailbox) { msg_task_t task = task_message_new(MESSAGE_INTERESTED, peer->hostname, peer->mailbox, peer->id, task_message_size(MESSAGE_INTERESTED)); @@ -786,7 +793,7 @@ void send_interested(peer_t peer, const char *mailbox) * @param peer peer data * @param mailbox destination mailbox */ -void send_notinterested(peer_t peer, const char *mailbox) +void send_notinterested(peer_t peer, const char* mailbox) { msg_task_t task = task_message_new(MESSAGE_NOTINTERESTED, peer->hostname, peer->mailbox, peer->id, task_message_size(MESSAGE_NOTINTERESTED)); @@ -801,8 +808,8 @@ void send_handshake_all(peer_t peer) { connection_t remote_peer; xbt_dict_cursor_t cursor = NULL; - char *key; - xbt_dict_foreach(peer->peers, cursor, key, remote_peer) { + char* key; + xbt_dict_foreach (peer->peers, cursor, key, remote_peer) { msg_task_t task = task_message_new(MESSAGE_HANDSHAKE, peer->hostname, peer->mailbox, peer->id, task_message_size(MESSAGE_HANDSHAKE)); MSG_task_dsend(task, remote_peer->mailbox, task_message_free); @@ -814,7 +821,7 @@ void send_handshake_all(peer_t peer) * @param peer peer data * @param mailbox mailbox where to we send the message */ -void send_handshake(peer_t peer, const char *mailbox) +void send_handshake(peer_t peer, const char* mailbox) { XBT_DEBUG("Sending a HANDSHAKE to %s", mailbox); msg_task_t task = task_message_new(MESSAGE_HANDSHAKE, peer->hostname, peer->mailbox, peer->id, @@ -823,20 +830,20 @@ void send_handshake(peer_t peer, const char *mailbox) } /** Send a "choked" message to a peer. */ -void send_choked(peer_t peer, const char *mailbox) +void send_choked(peer_t peer, const char* mailbox) { XBT_DEBUG("Sending a CHOKE to %s", mailbox); - msg_task_t task = task_message_new(MESSAGE_CHOKE, peer->hostname, peer->mailbox, peer->id, - task_message_size(MESSAGE_CHOKE)); + msg_task_t task = + task_message_new(MESSAGE_CHOKE, peer->hostname, peer->mailbox, peer->id, task_message_size(MESSAGE_CHOKE)); MSG_task_dsend(task, mailbox, task_message_free); } /** Send a "unchoked" message to a peer */ -void send_unchoked(peer_t peer, const char *mailbox) +void send_unchoked(peer_t peer, const char* mailbox) { XBT_DEBUG("Sending a UNCHOKE to %s", mailbox); - msg_task_t task = task_message_new(MESSAGE_UNCHOKE, peer->hostname, peer->mailbox, peer->id, - task_message_size(MESSAGE_UNCHOKE)); + msg_task_t task = + task_message_new(MESSAGE_UNCHOKE, peer->hostname, peer->mailbox, peer->id, task_message_size(MESSAGE_UNCHOKE)); MSG_task_dsend(task, mailbox, task_message_free); } @@ -846,8 +853,8 @@ void send_have(peer_t peer, int piece) XBT_DEBUG("Sending HAVE message to all my peers"); connection_t remote_peer; xbt_dict_cursor_t cursor = NULL; - char *key; - xbt_dict_foreach(peer->peers, cursor, key, remote_peer) { + char* key; + xbt_dict_foreach (peer->peers, cursor, key, remote_peer) { msg_task_t task = task_message_index_new(MESSAGE_HAVE, peer->hostname, peer->mailbox, peer->id, piece, task_message_size(MESSAGE_HAVE)); MSG_task_dsend(task, remote_peer->mailbox, task_message_free); @@ -857,7 +864,7 @@ void send_have(peer_t peer, int piece) /** @brief Send a bitfield message to all the peers the peer has. * @param peer peer data */ -void send_bitfield(peer_t peer, const char *mailbox) +void send_bitfield(peer_t peer, const char* mailbox) { XBT_DEBUG("Sending a BITFIELD to %s", mailbox); msg_task_t task = task_message_bitfield_new(peer->hostname, peer->mailbox, peer->id, peer->bitfield, FILE_PIECES); @@ -865,7 +872,7 @@ void send_bitfield(peer_t peer, const char *mailbox) } /** Send a "request" message to a pair, containing a request for a piece */ -void send_request(peer_t peer, const char *mailbox, int piece, int block_index, int block_length) +void send_request(peer_t peer, const char* mailbox, int piece, int block_index, int block_length) { XBT_DEBUG("Sending a REQUEST to %s for piece %d (%d,%d)", mailbox, piece, block_index, block_length); msg_task_t task = task_message_request_new(peer->hostname, peer->mailbox, peer->id, piece, block_index, block_length); @@ -873,12 +880,12 @@ void send_request(peer_t peer, const char *mailbox, int piece, int block_index, } /** Send a "piece" message to a pair, containing a piece of the file */ -void send_piece(peer_t peer, const char *mailbox, int piece, int block_index, int block_length) +void send_piece(peer_t peer, const char* mailbox, int piece, int block_index, int block_length) { XBT_DEBUG("Sending the PIECE %d (%d,%d) to %s", piece, block_index, block_length, mailbox); xbt_assert(piece >= 0, "Tried to send a piece that doesn't exist."); - xbt_assert(!peer_has_not_piece(peer,piece), "Tried to send a piece that we doesn't have."); - msg_task_t task = task_message_piece_new(peer->hostname, peer->mailbox, peer->id, piece, block_index, block_length, - BLOCK_SIZE); + xbt_assert(!peer_has_not_piece(peer, piece), "Tried to send a piece that we doesn't have."); + msg_task_t task = + task_message_piece_new(peer->hostname, peer->mailbox, peer->id, piece, block_index, block_length, BLOCK_SIZE); MSG_task_dsend(task, mailbox, task_message_free); } diff --git a/examples/msg/app-bittorrent/peer.h b/teshsuite/msg/app-bittorrent/peer.h similarity index 57% rename from examples/msg/app-bittorrent/peer.h rename to teshsuite/msg/app-bittorrent/peer.h index d08f1f2738..a3ac6bf11d 100644 --- a/examples/msg/app-bittorrent/peer.h +++ b/teshsuite/msg/app-bittorrent/peer.h @@ -6,43 +6,43 @@ #ifndef BITTORRENT_PEER_H #define BITTORRENT_PEER_H +#include "bittorrent.h" +#include "connection.h" #include +#include #include #include -#include -#include "connection.h" -#include "bittorrent.h" /** Peer data */ typedef struct s_peer { - int id; //peer id + int id; // peer id - unsigned int bitfield; //list of pieces the peer has. - unsigned long long bitfield_blocks; //list of blocks the peer has. - short *pieces_count; //number of peers that have each piece. + unsigned int bitfield; // list of pieces the peer has. + unsigned long long bitfield_blocks; // list of blocks the peer has. + short* pieces_count; // number of peers that have each piece. - unsigned int current_pieces; //current pieces the peer is downloading + unsigned int current_pieces; // current pieces the peer is downloading - xbt_dict_t peers; //peers list - xbt_dict_t active_peers; //active peers list - int round; //current round for the chocking algorithm. + xbt_dict_t peers; // peers list + xbt_dict_t active_peers; // active peers list + int round; // current round for the chocking algorithm. - char mailbox[MAILBOX_SIZE]; //peer mailbox. - char mailbox_tracker[MAILBOX_SIZE]; //pair mailbox while communicating with the tracker. - const char *hostname; //peer hostname + char mailbox[MAILBOX_SIZE]; // peer mailbox. + char mailbox_tracker[MAILBOX_SIZE]; // pair mailbox while communicating with the tracker. + const char* hostname; // peer hostname - msg_task_t task_received; //current task being received - msg_comm_t comm_received; //current comm + msg_task_t task_received; // current task being received + msg_comm_t comm_received; // current comm - RngStream stream; //RngStream for + RngStream stream; // RngStream for - double begin_receive_time; //time when the receiving communication has begun, useful for calculating host speed. + double begin_receive_time; // time when the receiving communication has begun, useful for calculating host speed. } s_peer_t; -typedef s_peer_t *peer_t; +typedef s_peer_t* peer_t; /** Peer main function */ -int peer(int argc, char *argv[]); -void get_status(char **status, unsigned int bitfield); +int peer(int argc, char* argv[]); +void get_status(char** status, unsigned int bitfield); int get_peers_data(peer_t peer); void leech_loop(peer_t peer, double deadline); @@ -83,16 +83,16 @@ int select_piece_to_download(peer_t peer, connection_t remote_peer); void send_handshake_all(peer_t peer); -void send_interested(peer_t peer, const char *mailbox); +void send_interested(peer_t peer, const char* mailbox); -void send_notinterested(peer_t peer, const char *mailbox); -void send_handshake(peer_t peer, const char *mailbox); -void send_bitfield(peer_t peer, const char *mailbox); -void send_choked(peer_t peer, const char *mailbox); -void send_unchoked(peer_t peer, const char *mailbox); +void send_notinterested(peer_t peer, const char* mailbox); +void send_handshake(peer_t peer, const char* mailbox); +void send_bitfield(peer_t peer, const char* mailbox); +void send_choked(peer_t peer, const char* mailbox); +void send_unchoked(peer_t peer, const char* mailbox); void send_have(peer_t peer, int piece); -void send_request(peer_t peer, const char *mailbox, int piece, int block_index, int block_length); -void send_piece(peer_t peer, const char *mailbox, int piece, int block_index, int block_length); +void send_request(peer_t peer, const char* mailbox, int piece, int block_index, int block_length); +void send_piece(peer_t peer, const char* mailbox, int piece, int block_index, int block_length); -#endif /* BITTORRENT_PEER_H */ +#endif /* BITTORRENT_PEER_H */ diff --git a/examples/msg/app-bittorrent/tracker.c b/teshsuite/msg/app-bittorrent/tracker.c similarity index 72% rename from examples/msg/app-bittorrent/tracker.c rename to teshsuite/msg/app-bittorrent/tracker.c index bdcea7f858..d4025c7be2 100644 --- a/examples/msg/app-bittorrent/tracker.c +++ b/teshsuite/msg/app-bittorrent/tracker.c @@ -8,7 +8,7 @@ #include #include -static void task_free(void *data); +static void task_free(void* data); XBT_LOG_NEW_DEFAULT_CATEGORY(msg_tracker, "Messages specific for the tracker"); /** @@ -16,16 +16,16 @@ XBT_LOG_NEW_DEFAULT_CATEGORY(msg_tracker, "Messages specific for the tracker"); * @param argc number of arguments * @param argv arguments */ -int tracker(int argc, char *argv[]) +int tracker(int argc, char* argv[]) { - //Checking arguments + // Checking arguments xbt_assert(argc == 2, "Wrong number of arguments for the tracker."); - //Retrieving end time - double deadline = xbt_str_parse_double(argv[1],"Invalid deadline: %s"); + // Retrieving end time + double deadline = xbt_str_parse_double(argv[1], "Invalid deadline: %s"); xbt_assert(deadline > 0, "Wrong deadline supplied"); - RngStream stream = (RngStream) MSG_host_get_data(MSG_host_self()); - //Building peers array + RngStream stream = (RngStream)MSG_host_get_data(MSG_host_self()); + // Building peers array xbt_dynar_t peers_list = xbt_dynar_new(sizeof(int), NULL); XBT_INFO("Tracker launched."); @@ -38,28 +38,28 @@ int tracker(int argc, char *argv[]) comm_received = MSG_task_irecv(&task_received, TRACKER_MAILBOX); } if (MSG_comm_test(comm_received)) { - //Check for correct status + // Check for correct status if (MSG_comm_get_status(comm_received) == MSG_OK) { - //Retrieve the data sent by the peer. + // Retrieve the data sent by the peer. tracker_task_data_t data = MSG_task_get_data(task_received); - //Add the peer to our peer list. + // Add the peer to our peer list. if (is_in_list(peers_list, data->peer_id) == 0) { xbt_dynar_push_as(peers_list, int, data->peer_id); } - //Sending peers to the peer + // Sending peers to the peer int next_peer; int peers_length = xbt_dynar_length(peers_list); - for (int i = 0; i < MAXIMUM_PAIRS && i < peers_length; i++) { + for (int i = 0; i < MAXIMUM_PEERS && i < peers_length; i++) { do { next_peer = xbt_dynar_get_as(peers_list, RngStream_RandInt(stream, 0, peers_length - 1), int); } while (is_in_list(data->peers, next_peer)); xbt_dynar_push_as(data->peers, int, next_peer); } - //setting the interval + // setting the interval data->interval = TRACKER_QUERY_INTERVAL; - //sending the task back to the peer. + // sending the task back to the peer. MSG_task_dsend(task_received, data->mailbox, task_free); - //destroy the communication. + // destroy the communication. } MSG_comm_destroy(comm_received); comm_received = NULL; @@ -68,11 +68,11 @@ int tracker(int argc, char *argv[]) MSG_process_sleep(1); } } - //Free the remaining communication if any + // Free the remaining communication if any if (comm_received) { MSG_comm_destroy(comm_received); } - //Free the peers list + // Free the peers list xbt_dynar_free(&peers_list); XBT_INFO("Tracker is leaving"); @@ -84,18 +84,18 @@ int tracker(int argc, char *argv[]) * Build a new task for the tracker. * @param issuer_host_name Hostname of the issuer. For debugging purposes */ -tracker_task_data_t tracker_task_data_new(const char *issuer_host_name, const char *mailbox, int peer_id, - int uploaded, int downloaded, int left) +tracker_task_data_t tracker_task_data_new(const char* issuer_host_name, const char* mailbox, int peer_id, int uploaded, + int downloaded, int left) { tracker_task_data_t task = xbt_new(s_tracker_task_data_t, 1); - task->type = TRACKER_TASK_QUERY; + task->type = TRACKER_TASK_QUERY; task->issuer_host_name = issuer_host_name; - task->mailbox = mailbox; - task->peer_id = peer_id; - task->uploaded = uploaded; - task->downloaded = downloaded; - task->left = left; + task->mailbox = mailbox; + task->peer_id = peer_id; + task->uploaded = uploaded; + task->downloaded = downloaded; + task->left = left; task->peers = xbt_dynar_new(sizeof(int), NULL); @@ -106,7 +106,7 @@ tracker_task_data_t tracker_task_data_new(const char *issuer_host_name, const ch * Free a tracker task that has not successfully been sent. * @param data Task to free */ -static void task_free(void *data) +static void task_free(void* data) { tracker_task_data_t task_data = MSG_task_get_data(data); tracker_task_data_free(task_data); diff --git a/teshsuite/msg/app-bittorrent/tracker.h b/teshsuite/msg/app-bittorrent/tracker.h new file mode 100644 index 0000000000..f00e71ff76 --- /dev/null +++ b/teshsuite/msg/app-bittorrent/tracker.h @@ -0,0 +1,42 @@ +/* Copyright (c) 2012-2014. The SimGrid Team. + * All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#ifndef BITTORRENT_TRACKER_H_ +#define BITTORRENT_TRACKER_H_ +#include "bittorrent.h" +#include +/** + * Tracker main function + */ +int tracker(int argc, char* argv[]); +/** + * Task types exchanged between a node and the tracker + */ +typedef enum { TRACKER_TASK_QUERY, TRACKER_TASK_ANSWER } e_tracker_task_type_t; +/** + * Tasks exchanged between a tracker and peers. + */ +typedef struct s_tracker_task_data { + e_tracker_task_type_t type; // type of the task + const char* mailbox; // mailbox where the tracker should answer + const char* issuer_host_name; // hostname, for debug purposes + // Query data + int peer_id; // peer id + int uploaded; // how much the peer has already uploaded + int downloaded; // how much the peer has downloaded + int left; // how much the peer has left + // Answer data + int interval; // how often the peer should contact the tracker (unused for now) + xbt_dynar_t peers; // the peer list the peer has asked for. +} s_tracker_task_data_t; +typedef s_tracker_task_data_t* tracker_task_data_t; + +tracker_task_data_t tracker_task_data_new(const char* issuer_host_name, const char* mailbox, int peer_id, int uploaded, + int downloaded, int left); +void tracker_task_data_free(tracker_task_data_t task); + +int is_in_list(xbt_dynar_t peers, int id); +#endif /* BITTORRENT_TRACKER_H */ diff --git a/teshsuite/s4u/comm-pt2pt/comm-pt2pt.cpp b/teshsuite/s4u/comm-pt2pt/comm-pt2pt.cpp index 430bb73a21..6b70db209d 100644 --- a/teshsuite/s4u/comm-pt2pt/comm-pt2pt.cpp +++ b/teshsuite/s4u/comm-pt2pt/comm-pt2pt.cpp @@ -50,8 +50,8 @@ static void sender(std::vector args) XBT_INFO("Sender spec: %s", args[0].c_str()); for (unsigned int test = 1; test <= args[0].size(); test++) { this_actor::sleep_until(test * 5 - 5); - char* mboxName = bprintf("Test #%u", test); - simgrid::s4u::MailboxPtr mbox = simgrid::s4u::Mailbox::byName(mboxName); + std::string* mboxName = new std::string("Test #" + std::to_string(test)); + simgrid::s4u::MailboxPtr mbox = simgrid::s4u::Mailbox::byName(mboxName->c_str()); switch (args[0][test - 1]) { case 'r': @@ -97,8 +97,8 @@ static void receiver(std::vector args) XBT_INFO("Receiver spec: %s", args[0].c_str()); for (unsigned int test = 1; test <= args[0].size(); test++) { this_actor::sleep_until(test * 5 - 5); - char* mboxName = bprintf("Test #%u", test); - simgrid::s4u::MailboxPtr mbox = simgrid::s4u::Mailbox::byName(mboxName); + std::string mboxName = "Test #" + std::to_string(test); + simgrid::s4u::MailboxPtr mbox = simgrid::s4u::Mailbox::byName(mboxName.c_str()); void* received = nullptr; switch (args[0][test - 1]) { @@ -146,10 +146,9 @@ static void receiver(std::vector args) default: xbt_die("Unknown receiver spec for test %u: '%c'", test, args[0][test - 1]); } - - xbt_assert(strcmp(static_cast(received), mboxName) == 0); - xbt_free(received); - xbt_free(mboxName); + std::string* receivedStr = static_cast(received); + xbt_assert(*receivedStr == mboxName); + delete receivedStr; XBT_INFO("Test %u OK", test); } simgrid::s4u::this_actor::sleep_for(0.5); diff --git a/teshsuite/simdag/flatifier/flatifier.cpp b/teshsuite/simdag/flatifier/flatifier.cpp index bb2aa74919..60bd850f0b 100644 --- a/teshsuite/simdag/flatifier/flatifier.cpp +++ b/teshsuite/simdag/flatifier/flatifier.cpp @@ -50,10 +50,7 @@ static void create_environment(xbt_os_timer_t parse_time, const char *platformFi static void dump_platform() { int version = 4; - xbt_dict_t props = nullptr; - xbt_dict_cursor_t cursor = nullptr; - char* key; - char* data; + std::map* props = nullptr; std::printf("\n"); std::printf("\n"); @@ -68,20 +65,19 @@ static void dump_platform() for (unsigned int i = 0; i < totalHosts; i++) { std::printf(" getCname(), sg_host_speed(hosts[i])); - props = sg_host_get_properties(hosts[i]); + props = hosts[i]->getProperties(); if (hosts[i]->getCoreCount() > 1) { std::printf(" core=\"%d\"", hosts[i]->getCoreCount()); } - if (props && not xbt_dict_is_empty(props)) { + if (props && not props->empty()) { std::printf(">\n"); - xbt_dict_foreach (props, cursor, key, data) { - std::printf(" \n", key, data); + for (auto kv : *props) { + std::printf(" \n", kv.first.c_str(), kv.second.c_str()); } std::printf(" \n"); } else { std::printf("/>\n"); } - xbt_dict_free(&props); } // Routers diff --git a/teshsuite/xbt/CMakeLists.txt b/teshsuite/xbt/CMakeLists.txt index da0f1a1e88..a8fd74320a 100644 --- a/teshsuite/xbt/CMakeLists.txt +++ b/teshsuite/xbt/CMakeLists.txt @@ -1,4 +1,4 @@ -foreach(x heap_bench log_large log_usage mallocator parallel_log_crashtest parmap_bench parmap_test) +foreach(x heap_bench log_large log_usage mallocator parallel_log_crashtest) add_executable (${x} ${x}/${x}.c) target_link_libraries(${x} simgrid) set_target_properties(${x} PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${x}) @@ -7,6 +7,15 @@ foreach(x heap_bench log_large log_usage mallocator parallel_log_crashtest parma set(teshsuite_src ${teshsuite_src} ${CMAKE_CURRENT_SOURCE_DIR}/${x}/${x}.c) endforeach() +foreach(x parmap_bench parmap_test) + add_executable (${x} ${x}/${x}.cpp) + target_link_libraries(${x} simgrid) + set_target_properties(${x} PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${x}) + + set(tesh_files ${tesh_files} ${CMAKE_CURRENT_SOURCE_DIR}/${x}/${x}.tesh) + set(teshsuite_src ${teshsuite_src} ${CMAKE_CURRENT_SOURCE_DIR}/${x}/${x}.cpp) +endforeach() + if(HAVE_MMALLOC) add_executable (mmalloc_test ${CMAKE_CURRENT_SOURCE_DIR}/mmalloc/mmalloc_test.cpp) target_link_libraries(mmalloc_test simgrid) diff --git a/teshsuite/xbt/parmap_bench/parmap_bench.c b/teshsuite/xbt/parmap_bench/parmap_bench.c deleted file mode 100644 index 3944546a5b..0000000000 --- a/teshsuite/xbt/parmap_bench/parmap_bench.c +++ /dev/null @@ -1,204 +0,0 @@ -/* Copyright (c) 2012-2017. The SimGrid Team. All rights reserved. */ - -/* This program is free software; you can redistribute it and/or modify it - * under the terms of the license (GNU LGPL) which comes with this package. */ - -#include "simgrid/msg.h" -#include "src/internal_config.h" /* HAVE_FUTEX_H */ -#include "xbt/xbt_os_time.h" -#include -#include -#include -#include -#include - -#define MODES_DEFAULT 0x7 -#define TIMEOUT 10.0 -#define ARRAY_SIZE 10007 -#define FIBO_MAX 25 - -void (*fun_to_apply)(void *); - -static const char *parmap_mode_name(e_xbt_parmap_mode_t mode) -{ - static char name[80]; - switch (mode) { - case XBT_PARMAP_POSIX: - snprintf(name, sizeof name, "POSIX"); - break; - case XBT_PARMAP_FUTEX: - snprintf(name, sizeof name, "FUTEX"); - break; - case XBT_PARMAP_BUSY_WAIT: - snprintf(name, sizeof name, "BUSY_WAIT"); - break; - case XBT_PARMAP_DEFAULT: - snprintf(name, sizeof name, "DEFAULT"); - break; - default: - snprintf(name, sizeof name, "UNKNOWN(%d)", (int)mode); - break; - } - return name; -} - -static int parmap_skip_mode(e_xbt_parmap_mode_t mode) -{ -#if !HAVE_FUTEX_H - if (mode == XBT_PARMAP_FUTEX) { - printf("not available\n"); - return 1; - } else -#endif - return 0; -} - -static unsigned fibonacci(unsigned n) -{ - if (n < 2) - return n; - else - return fibonacci(n - 1) + fibonacci(n - 2); -} - -static void fun_small_comp(void *arg) -{ - unsigned *u = arg; - *u = 2 * *u + 1; -} - -static void fun_big_comp(void *arg) -{ - unsigned *u = arg; - *u = fibonacci(*u % FIBO_MAX); -} - -static void array_new(unsigned **a, xbt_dynar_t *data) -{ - *a = xbt_malloc(ARRAY_SIZE * sizeof **a); - *data = xbt_dynar_new(sizeof *a, NULL); - xbt_dynar_shrink(*data, ARRAY_SIZE); - for (int i = 0 ; i < ARRAY_SIZE ; i++) { - (*a)[i] = i; - xbt_dynar_push_as(*data, void*, &(*a)[i]); - } -} - -static void bench_parmap_full(int nthreads, e_xbt_parmap_mode_t mode) -{ - unsigned *a; - xbt_dynar_t data; - xbt_parmap_t parmap; - double elapsed_time; - - printf("** mode = %-15s ", parmap_mode_name(mode)); - fflush(stdout); - - if (parmap_skip_mode(mode)) - return; - - array_new(&a, &data); - - int i = 0; - double start_time = xbt_os_time(); - do { - parmap = xbt_parmap_new(nthreads, mode); - xbt_parmap_apply(parmap, fun_to_apply, data); - xbt_parmap_destroy(parmap); - elapsed_time = xbt_os_time() - start_time; - i++; - } while (elapsed_time < TIMEOUT); - - printf("ran %d times in %g seconds (%g/s)\n", i, elapsed_time, i / elapsed_time); - - xbt_dynar_free(&data); - xbt_free(a); -} - -static void bench_parmap_apply(int nthreads, e_xbt_parmap_mode_t mode) -{ - unsigned *a; - xbt_dynar_t data; - double elapsed_time; - - printf("** mode = %-15s ", parmap_mode_name(mode)); - fflush(stdout); - - if (parmap_skip_mode(mode)) - return; - - array_new(&a, &data); - - xbt_parmap_t parmap = xbt_parmap_new(nthreads, mode); - int i = 0; - double start_time = xbt_os_time(); - do { - xbt_parmap_apply(parmap, fun_to_apply, data); - elapsed_time = xbt_os_time() - start_time; - i++; - } while (elapsed_time < TIMEOUT); - xbt_parmap_destroy(parmap); - - printf("ran %d times in %g seconds (%g/s)\n", i, elapsed_time, i / elapsed_time); - - xbt_dynar_free(&data); - xbt_free(a); -} - -static void bench_all_modes(void (*bench_fun)(int, e_xbt_parmap_mode_t), - int nthreads, unsigned modes) -{ - e_xbt_parmap_mode_t all_modes[] = {XBT_PARMAP_POSIX, XBT_PARMAP_FUTEX, XBT_PARMAP_BUSY_WAIT, XBT_PARMAP_DEFAULT}; - - for (unsigned i = 0 ; i < sizeof all_modes / sizeof all_modes[0] ; i++) { - if (1U << i & modes) - bench_fun(nthreads, all_modes[i]); - } -} - -int main(int argc, char *argv[]) -{ - int nthreads; - unsigned modes = MODES_DEFAULT; - - MSG_init(&argc, argv); - - if (argc != 2 && argc != 3) { - fprintf(stderr, "Usage: %s nthreads [modes]\n" - " nthreads - number of working threads\n" - " modes - bitmask of modes to test\n", - argv[0]); - return EXIT_FAILURE; - } - nthreads = atoi(argv[1]); - if (nthreads < 1) { - fprintf(stderr, "ERROR: invalid thread count: %d\n", nthreads); - return EXIT_FAILURE; - } - if (argc == 3) - modes = strtol(argv[2], NULL, 0); - - printf("Parmap benchmark with %d workers (modes = %#x)...\n\n", nthreads, modes); - - fun_to_apply = &fun_small_comp; - - printf("Benchmark for parmap create+apply+destroy (small comp):\n"); - bench_all_modes(bench_parmap_full, nthreads, modes); - printf("\n"); - - printf("Benchmark for parmap apply only (small comp):\n"); - bench_all_modes(bench_parmap_apply, nthreads, modes); - printf("\n"); - - fun_to_apply = &fun_big_comp; - - printf("Benchmark for parmap create+apply+destroy (big comp):\n"); - bench_all_modes(bench_parmap_full, nthreads, modes); - printf("\n"); - - printf("Benchmark for parmap apply only (big comp):\n"); - bench_all_modes(bench_parmap_apply, nthreads, modes); - printf("\n"); - - return EXIT_SUCCESS; -} diff --git a/teshsuite/xbt/parmap_bench/parmap_bench.cpp b/teshsuite/xbt/parmap_bench/parmap_bench.cpp new file mode 100644 index 0000000000..9f74d1cc5f --- /dev/null +++ b/teshsuite/xbt/parmap_bench/parmap_bench.cpp @@ -0,0 +1,186 @@ +/* Copyright (c) 2012-2017. The SimGrid Team. All rights reserved. */ + +/* This program is free software; you can redistribute it and/or modify it + * under the terms of the license (GNU LGPL) which comes with this package. */ + +#include "src/internal_config.h" // HAVE_FUTEX_H +#include +#include +#include + +#include +#include +#include +#include // std::iota +#include +#include + +#define MODES_DEFAULT 0x7 +#define TIMEOUT 10.0 +#define ARRAY_SIZE 10007 +#define FIBO_MAX 25 + +void (*fun_to_apply)(unsigned*); + +static std::string parmap_mode_name(e_xbt_parmap_mode_t mode) +{ + std::string name; + switch (mode) { + case XBT_PARMAP_POSIX: + name = "POSIX"; + break; + case XBT_PARMAP_FUTEX: + name = "FUTEX"; + break; + case XBT_PARMAP_BUSY_WAIT: + name = "BUSY_WAIT"; + break; + case XBT_PARMAP_DEFAULT: + name = "DEFAULT"; + break; + default: + name = "UNKNOWN(" + std::to_string(mode) + ")"; + break; + } + return name; +} + +static bool parmap_skip_mode(e_xbt_parmap_mode_t mode) +{ + if (mode == XBT_PARMAP_FUTEX && not HAVE_FUTEX_H) { + std::cout << "not available\n"; + return true; + } else { + return false; + } +} + +static unsigned fibonacci(unsigned n) +{ + if (n < 2) + return n; + else + return fibonacci(n - 1) + fibonacci(n - 2); +} + +static void fun_small_comp(unsigned* arg) +{ + *arg = 2 * *arg + 1; +} + +static void fun_big_comp(unsigned* arg) +{ + *arg = fibonacci(*arg % FIBO_MAX); +} + +static void bench_parmap_full(int nthreads, e_xbt_parmap_mode_t mode) +{ + std::cout << "** mode = " << std::left << std::setw(15) << parmap_mode_name(mode) << " "; + std::cout.flush(); + + if (parmap_skip_mode(mode)) + return; + + std::vector a(ARRAY_SIZE); + std::vector data(ARRAY_SIZE); + std::iota(begin(a), end(a), 0); + std::iota(begin(data), end(data), &a[0]); + + int i = 0; + double start_time = xbt_os_time(); + double elapsed_time; + do { + { + simgrid::xbt::Parmap parmap(nthreads, mode); + parmap.apply(fun_to_apply, data); + } // enclosing block to ensure that the parmap is destroyed here. + elapsed_time = xbt_os_time() - start_time; + i++; + } while (elapsed_time < TIMEOUT); + + std::cout << "ran " << i << " times in " << elapsed_time << " seconds (" << (i / elapsed_time) << "/s)\n"; +} + +static void bench_parmap_apply(int nthreads, e_xbt_parmap_mode_t mode) +{ + std::cout << "** mode = " << std::left << std::setw(15) << parmap_mode_name(mode) << " "; + std::cout.flush(); + + if (parmap_skip_mode(mode)) + return; + + std::vector a(ARRAY_SIZE); + std::vector data(ARRAY_SIZE); + std::iota(begin(a), end(a), 0); + std::iota(begin(data), end(data), &a[0]); + + simgrid::xbt::Parmap parmap(nthreads, mode); + int i = 0; + double start_time = xbt_os_time(); + double elapsed_time; + do { + parmap.apply(fun_to_apply, data); + elapsed_time = xbt_os_time() - start_time; + i++; + } while (elapsed_time < TIMEOUT); + + std::cout << "ran " << i << " times in " << elapsed_time << " seconds (" << (i / elapsed_time) << "/s)\n"; +} + +static void bench_all_modes(void (*bench_fun)(int, e_xbt_parmap_mode_t), int nthreads, unsigned modes) +{ + std::vector all_modes = {XBT_PARMAP_POSIX, XBT_PARMAP_FUTEX, XBT_PARMAP_BUSY_WAIT, + XBT_PARMAP_DEFAULT}; + + for (unsigned i = 0; i < all_modes.size(); i++) { + if (1U << i & modes) + bench_fun(nthreads, all_modes[i]); + } +} + +int main(int argc, char* argv[]) +{ + int nthreads; + unsigned modes = MODES_DEFAULT; + + MSG_init(&argc, argv); + + if (argc != 2 && argc != 3) { + std::cerr << "Usage: " << argv[0] << " nthreads [modes]\n" + << " nthreads - number of working threads\n" + << " modes - bitmask of modes to test\n"; + return EXIT_FAILURE; + } + nthreads = atoi(argv[1]); + if (nthreads < 1) { + std::cerr << "ERROR: invalid thread count: " << nthreads << "\n"; + return EXIT_FAILURE; + } + if (argc == 3) + modes = strtol(argv[2], NULL, 0); + + std::cout << "Parmap benchmark with " << nthreads << " workers (modes = " << std::hex << modes << std::dec + << ")...\n\n"; + + fun_to_apply = &fun_small_comp; + + std::cout << "Benchmark for parmap create+apply+destroy (small comp):\n"; + bench_all_modes(bench_parmap_full, nthreads, modes); + std::cout << std::endl; + + std::cout << "Benchmark for parmap apply only (small comp):\n"; + bench_all_modes(bench_parmap_apply, nthreads, modes); + std::cout << std::endl; + + fun_to_apply = &fun_big_comp; + + std::cout << "Benchmark for parmap create+apply+destroy (big comp):\n"; + bench_all_modes(bench_parmap_full, nthreads, modes); + std::cout << std::endl; + + std::cout << "Benchmark for parmap apply only (big comp):\n"; + bench_all_modes(bench_parmap_apply, nthreads, modes); + std::cout << std::endl; + + return EXIT_SUCCESS; +} diff --git a/teshsuite/xbt/parmap_test/parmap_test.c b/teshsuite/xbt/parmap_test/parmap_test.cpp similarity index 51% rename from teshsuite/xbt/parmap_test/parmap_test.c rename to teshsuite/xbt/parmap_test/parmap_test.cpp index b520b5d8c5..cfb1684f1c 100644 --- a/teshsuite/xbt/parmap_test/parmap_test.c +++ b/teshsuite/xbt/parmap_test/parmap_test.cpp @@ -5,45 +5,40 @@ /* This program is free software; you can redistribute it and/or modify it * under the terms of the license (GNU LGPL) which comes with this package. */ -#include "simgrid/msg.h" -#include "src/internal_config.h" -#include "xbt.h" -#include "xbt/ex.h" -#include "xbt/xbt_os_time.h" +#include "src/internal_config.h" // HAVE_FUTEX_H +#include +#include +#include + +#include +#include +#include // std::iota +#include XBT_LOG_NEW_DEFAULT_CATEGORY(parmap_test, "Test for parmap"); -static void fun_double(void *arg) +static void fun_double(unsigned* arg) { - unsigned *u = arg; - *u = 2 * *u + 1; + *arg = 2 * *arg + 1; } static int test_parmap_basic(e_xbt_parmap_mode_t mode) { int ret = 0; - unsigned num_workers; - for (num_workers = 1 ; num_workers <= 16 ; num_workers *= 2) { + for (unsigned num_workers = 1; num_workers <= 16; num_workers *= 2) { const unsigned len = 1033; const unsigned num = 5; - unsigned *a; - xbt_dynar_t data; - xbt_parmap_t parmap; - unsigned i; - - parmap = xbt_parmap_new(num_workers, mode); - - a = xbt_malloc(len * sizeof *a); - data = xbt_dynar_new(sizeof a, NULL); - for (i = 0; i < len; i++) { - a[i] = i; - xbt_dynar_push_as(data, void *, &a[i]); - } - for (i = 0; i < num; i++) - xbt_parmap_apply(parmap, fun_double, data); + simgrid::xbt::Parmap parmap(num_workers, mode); + std::vector a(len); + std::vector data(len); + std::iota(begin(a), end(a), 0); + std::iota(begin(data), end(data), &a[0]); + + for (unsigned i = 0; i < num; i++) + parmap.apply(fun_double, data); - for (i = 0; i < len; i++) { + for (unsigned i = 0; i < len; i++) { unsigned expected = (1U << num) * (i + 1) - 1; if (a[i] != expected) { XBT_CRITICAL("with %u threads, a[%u]: expected %u, got %u", num_workers, i, expected, a[i]); @@ -51,62 +46,36 @@ static int test_parmap_basic(e_xbt_parmap_mode_t mode) break; } } - - xbt_dynar_free(&data); - xbt_free(a); - xbt_parmap_destroy(parmap); } return ret; } -static void fun_get_id(void *arg) +static void fun_get_id(uintptr_t* arg) { - *(uintptr_t *)arg = (uintptr_t)xbt_os_thread_self(); + *arg = (uintptr_t)xbt_os_thread_self(); xbt_os_sleep(0.05); } -static int fun_compare(const void *pa, const void *pb) -{ - uintptr_t a = *(uintptr_t *)pa; - uintptr_t b = *(uintptr_t *)pb; - return a < b ? -1 : a > b ? 1 : 0; -} - static int test_parmap_extended(e_xbt_parmap_mode_t mode) { int ret = 0; - unsigned num_workers; - for (num_workers = 1 ; num_workers <= 16 ; num_workers *= 2) { + for (unsigned num_workers = 1; num_workers <= 16; num_workers *= 2) { const unsigned len = 2 * num_workers; - uintptr_t *a; - xbt_parmap_t parmap; - xbt_dynar_t data; - unsigned i; - unsigned count; - - parmap = xbt_parmap_new(num_workers, mode); - - a = xbt_malloc(len * sizeof *a); - data = xbt_dynar_new(sizeof a, NULL); - for (i = 0; i < len; i++) - xbt_dynar_push_as(data, void *, &a[i]); - - xbt_parmap_apply(parmap, fun_get_id, data); - - qsort(a, len, sizeof a[0], fun_compare); - count = 1; - for (i = 1; i < len; i++) - if (a[i] != a[i - 1]) - count++; + + simgrid::xbt::Parmap parmap(num_workers, mode); + std::vector a(len); + std::vector data(len); + std::iota(begin(data), end(data), &a[0]); + + parmap.apply(fun_get_id, data); + + std::sort(begin(a), end(a)); + unsigned count = std::distance(begin(a), std::unique(begin(a), end(a))); if (count != num_workers) { XBT_CRITICAL("only %u/%u threads did some work", count, num_workers); ret = 1; } - - xbt_dynar_free(&data); - xbt_free(a); - xbt_parmap_destroy(parmap); } return ret; } diff --git a/tools/cmake/DefinePackages.cmake b/tools/cmake/DefinePackages.cmake index 958684d51b..7280d5da8e 100644 --- a/tools/cmake/DefinePackages.cmake +++ b/tools/cmake/DefinePackages.cmake @@ -5,14 +5,15 @@ set(EXTRA_DIST src/include/instr/instr_interface.h src/include/mc/datatypes.h src/include/mc/mc.h - src/mc/mc_mmu.h - src/mc/PageStore.hpp - src/mc/mc_record.h src/include/simgrid/sg_config.h src/include/smpi/smpi_utils.hpp src/include/surf/datatypes.h src/include/surf/maxmin.h src/include/surf/surf.h + src/include/xbt/parmap.hpp + src/mc/mc_mmu.h + src/mc/mc_record.h + src/mc/PageStore.hpp src/msg/msg_private.h src/simdag/dax.dtd src/simdag/dax_dtd.c @@ -698,31 +699,29 @@ set(headers_to_install include/smpi/smpi_extended_traces_fortran.h include/smpi/forward.hpp include/xbt.h - include/xbt/RngStream.h + include/xbt/algorithm.hpp include/xbt/asserts.h include/xbt/automaton.h include/xbt/automaton.hpp + include/xbt/backtrace.h + include/xbt/backtrace.hpp include/xbt/base.h include/xbt/config.h include/xbt/config.hpp include/xbt/cunit.h include/xbt/dict.h - include/xbt/string.hpp - include/xbt/signal.hpp include/xbt/dynar.h include/xbt/dynar.hpp include/xbt/ex.h include/xbt/ex.hpp include/xbt/exception.hpp - include/xbt/backtrace.h - include/xbt/backtrace.hpp + include/xbt/Extendable.hpp include/xbt/file.h - include/xbt/function_types.h include/xbt/functional.hpp + include/xbt/function_types.h include/xbt/future.hpp include/xbt/graph.h include/xbt/heap.h - include/xbt/Extendable.hpp include/xbt/log.h include/xbt/log.hpp include/xbt/mallocator.h @@ -732,7 +731,10 @@ set(headers_to_install include/xbt/parmap.h include/xbt/range.hpp include/xbt/replay.hpp + include/xbt/RngStream.h + include/xbt/signal.hpp include/xbt/str.h + include/xbt/string.hpp include/xbt/swag.h include/xbt/synchro.h include/xbt/sysdep.h