From: Martin Quinson Date: Thu, 20 Jan 2022 17:15:52 +0000 (+0100) Subject: Convert the MasterWorker to python, in preparation to the tutorial PyConvertion X-Git-Tag: v3.30~72 X-Git-Url: http://info.iut-bm.univ-fcomte.fr/pub/gitweb/simgrid.git/commitdiff_plain/d546cc6688c4ec0100b9fbd30382c235876bf33a Convert the MasterWorker to python, in preparation to the tutorial PyConvertion --- diff --git a/docs/source/Tutorial_Algorithms.rst b/docs/source/Tutorial_Algorithms.rst index 3ccdde3c97..1c28510845 100644 --- a/docs/source/Tutorial_Algorithms.rst +++ b/docs/source/Tutorial_Algorithms.rst @@ -4,8 +4,8 @@ Simulating Algorithms ===================== SimGrid was conceived as a tool to study distributed algorithms. Its -modern :ref:`S4U interface ` makes it easy to assess Cloud, -P2P, HPC, IoT, and similar settings. +:ref:`S4U interface ` makes it easy to assess Cloud, +P2P, HPC, IoT, and other similar settings (:ref:`more info `). A typical SimGrid simulation is composed of several |Actors|_, that execute user-provided functions. The actors have to explicitly use the @@ -18,7 +18,7 @@ completion of these activities. Each actor executes a user-provided function on a simulated |Host|_ with which it can interact. Communications are not directly sent to -actors, but posted onto a |Mailbox|_ that serves as a rendezvous point +actors, but posted onto a |Mailbox|_ that serves as a rendez-vous point between communicating actors. .. |Actors| replace:: **Actors** @@ -62,8 +62,8 @@ fully-functioning example of SimGrid simulation: the Master/Workers application. We will detail each part of the code and the necessary configuration to make it work. After this tour, several exercises are proposed to let you discover some of the SimGrid features, hands -on the keyboard. This practical session will be given in C++, which you -are supposed to know beforehand. +on the keyboard. This practical session will be given in C++ or Python, +which you are supposed to know beforehand. Discover the Master/Workers diff --git a/docs/source/index.rst b/docs/source/index.rst index 79d021492a..a184df2484 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,5 +1,7 @@ .. SimGrid documentation master file +.. _index: + The Modern Age of Computer Systems Simulation ============================================= diff --git a/examples/cpp/app-masterworkers/s4u-app-masterworkers-class.cpp b/examples/cpp/app-masterworkers/s4u-app-masterworkers-class.cpp index bb9f17965e..88612b43cb 100644 --- a/examples/cpp/app-masterworkers/s4u-app-masterworkers-class.cpp +++ b/examples/cpp/app-masterworkers/s4u-app-masterworkers-class.cpp @@ -4,7 +4,7 @@ * under the terms of the license (GNU LGPL) which comes with this package. */ /* ************************************************************************* */ -/* Take this tutorial online: https://simgrid.frama.io/simgrid/tuto_s4u.html */ +/* Take this tutorial online: https://simgrid.org/doc/latest/Tutorial_Algorithms.html */ /* ************************************************************************* */ #include diff --git a/examples/cpp/app-masterworkers/s4u-app-masterworkers-fun.cpp b/examples/cpp/app-masterworkers/s4u-app-masterworkers-fun.cpp index 0cd4681146..3710195193 100644 --- a/examples/cpp/app-masterworkers/s4u-app-masterworkers-fun.cpp +++ b/examples/cpp/app-masterworkers/s4u-app-masterworkers-fun.cpp @@ -4,7 +4,7 @@ * under the terms of the license (GNU LGPL) which comes with this package. */ /* ************************************************************************* */ -/* Take this tutorial online: https://simgrid.frama.io/simgrid/tuto_s4u.html */ +/* Take this tutorial online: https://simgrid.org/doc/latest/Tutorial_Algorithms.html */ /* ************************************************************************* */ #include diff --git a/examples/python/CMakeLists.txt b/examples/python/CMakeLists.txt index 7a96e1887e..157f7be4ea 100644 --- a/examples/python/CMakeLists.txt +++ b/examples/python/CMakeLists.txt @@ -1,4 +1,5 @@ foreach(example actor-create actor-daemon actor-join actor-kill actor-migrate actor-suspend actor-yield actor-lifetime + app-masterworkers comm-wait comm-waitall comm-waitany exec-async exec-basic exec-dvfs exec-remote network-nonlinear clusters-multicpu io-degradation exec-cpu-nonlinear) diff --git a/examples/python/app-masterworkers/app-masterworkers.py b/examples/python/app-masterworkers/app-masterworkers.py new file mode 100644 index 0000000000..7f93acad36 --- /dev/null +++ b/examples/python/app-masterworkers/app-masterworkers.py @@ -0,0 +1,70 @@ +# Copyright (c) 2010-2022. The SimGrid Team. All rights reserved. + +# This program is free software; you can redistribute it and/or modify it +# under the terms of the license (GNU LGPL) which comes with this package. + +# ################################################################################## +# Take this tutorial online: https://simgrid.org/doc/latest/Tutorial_Algorithms.html +# ################################################################################## + +from simgrid import Actor, Engine, Host, Mailbox, this_actor +import sys + +def master(*args): + if len(args) < 2: + raise AssertionError( + f"Actor master requires 3 parameters plus the workers' names, but got only {len(args)}") + tasks_count = int(args[0]) + compute_cost = int(args[1]) + communicate_cost = int(args[2]) + workers = [] + for i in range(3, len(args)): + workers.append(Mailbox.by_name(args[i])) + this_actor.info(f"Got {len(workers)} workers and {tasks_count} tasks to process") + + for i in range(tasks_count): # For each task to be executed: + # - Select a worker in a round-robin way + mailbox = workers[i % len(workers)] + + # - Send the computation amount to the worker + if (tasks_count < 10000 or (tasks_count < 100000 and i % 10000 == 0) or i % 100000 == 0): + this_actor.info(f"Sending task {i} of {tasks_count} to mailbox '{mailbox.name}'") + mailbox.put(compute_cost, communicate_cost) + + this_actor.info("All tasks have been dispatched. Request all workers to stop.") + for i in range (len(workers)): + # The workers stop when receiving a negative compute_cost + mailbox = workers[i] + mailbox.put(-1, 0) + +def worker(*args): + assert len(args) == 0, "The worker expects to not get any argument" + + mailbox = Mailbox.by_name(this_actor.get_host().name) + done = False + while not done: + compute_cost = mailbox.get() + if compute_cost > 0: # If compute_cost is valid, execute a computation of that cost + this_actor.execute(compute_cost) + else: # Stop when receiving an invalid compute_cost + done = True + + this_actor.info("Exiting now.") + +if __name__ == '__main__': + assert len(sys.argv) > 2, f"Usage: python app-masterworkers.py platform_file deployment_file" + + e = Engine(sys.argv) + + # Register the classes representing the actors + e.register_actor("master", master) + e.register_actor("worker", worker) + + # Load the platform description and then deploy the application + e.load_platform(sys.argv[1]) + e.load_deployment(sys.argv[2]) + + # Run the simulation + e.run() + + this_actor.info("Simulation is over") diff --git a/examples/python/app-masterworkers/app-masterworkers.tesh b/examples/python/app-masterworkers/app-masterworkers.tesh new file mode 100644 index 0000000000..4c3f219871 --- /dev/null +++ b/examples/python/app-masterworkers/app-masterworkers.tesh @@ -0,0 +1,34 @@ +#!/usr/bin/env tesh + +p Testing a simple master/workers example application + +! output sort 19 +$ ${pythoncmd:=python3} ${PYTHON_TOOL_OPTIONS:=} ${bindir:=.}/app-masterworkers.py ${platfdir}/small_platform.xml app-masterworkers_d.xml --trace "--log=root.fmt:[%10.6r]%e(%a@%h)%e%m%n" +> [ 0.000000] (master@Tremblay) Got 5 workers and 20 tasks to process +> [ 0.000000] (master@Tremblay) Sending task 0 of 20 to mailbox 'Tremblay' +> [ 0.002265] (master@Tremblay) Sending task 1 of 20 to mailbox 'Jupiter' +> [ 0.171420] (master@Tremblay) Sending task 2 of 20 to mailbox 'Fafard' +> [ 0.329817] (master@Tremblay) Sending task 3 of 20 to mailbox 'Ginette' +> [ 0.453549] (master@Tremblay) Sending task 4 of 20 to mailbox 'Bourassa' +> [ 0.586168] (master@Tremblay) Sending task 5 of 20 to mailbox 'Tremblay' +> [ 0.588433] (master@Tremblay) Sending task 6 of 20 to mailbox 'Jupiter' +> [ 0.995917] (master@Tremblay) Sending task 7 of 20 to mailbox 'Fafard' +> [ 1.154314] (master@Tremblay) Sending task 8 of 20 to mailbox 'Ginette' +> [ 1.608379] (master@Tremblay) Sending task 9 of 20 to mailbox 'Bourassa' +> [ 1.749885] (master@Tremblay) Sending task 10 of 20 to mailbox 'Tremblay' +> [ 1.752150] (master@Tremblay) Sending task 11 of 20 to mailbox 'Jupiter' +> [ 1.921304] (master@Tremblay) Sending task 12 of 20 to mailbox 'Fafard' +> [ 2.079701] (master@Tremblay) Sending task 13 of 20 to mailbox 'Ginette' +> [ 2.763209] (master@Tremblay) Sending task 14 of 20 to mailbox 'Bourassa' +> [ 2.913601] (master@Tremblay) Sending task 15 of 20 to mailbox 'Tremblay' +> [ 2.915867] (master@Tremblay) Sending task 16 of 20 to mailbox 'Jupiter' +> [ 3.085021] (master@Tremblay) Sending task 17 of 20 to mailbox 'Fafard' +> [ 3.243418] (master@Tremblay) Sending task 18 of 20 to mailbox 'Ginette' +> [ 3.918038] (master@Tremblay) Sending task 19 of 20 to mailbox 'Bourassa' +> [ 4.077318] (master@Tremblay) All tasks have been dispatched. Request all workers to stop. +> [ 4.077513] (worker@Tremblay) Exiting now. +> [ 4.096528] (worker@Jupiter) Exiting now. +> [ 4.122236] (worker@Fafard) Exiting now. +> [ 4.965689] (worker@Ginette) Exiting now. +> [ 5.133855] (maestro@) Simulation is over +> [ 5.133855] (worker@Bourassa) Exiting now. diff --git a/examples/python/app-masterworkers/app-masterworkers_d.xml b/examples/python/app-masterworkers/app-masterworkers_d.xml new file mode 100644 index 0000000000..46ff999456 --- /dev/null +++ b/examples/python/app-masterworkers/app-masterworkers_d.xml @@ -0,0 +1,22 @@ + + + + + + + + + + + + + + + + + + + + + +