+++ /dev/null
-/*! \page faq Frequently Asked Questions
-
-\htmlinclude .FAQ.doc.toc
-
-\section faq_simgrid I'm new to SimGrid. I have some questions. Where should I start?
-
-You are at the right place... Having a look to these
-<a href="http://www.loria.fr/~quinson/blog/2010/06/28/Tutorial_at_HPCS/">the slides of the HPCS'10 tutorial</a>
-(or to these <a href="http://graal.ens-lyon.fr/~alegrand/articles/slides_g5k_simul.pdf">ancient
-slides</a>, or to these
-<a href="http://graal.ens-lyon.fr/~alegrand/articles/Simgrid-Introduction.pdf">"obsolete" slides</a>)
-may give you some insights on what SimGrid can help you to do and what
-are its limitations. Then you definitely should read the \ref
-MSG_examples. The \ref GRAS_tut can also help you.
-
-If you are stuck at any point and if this FAQ cannot help you, please drop us a
-mail to the user mailing list: <simgrid-user@lists.gforge.inria.fr>.
-
-\subsection faq_interfaces What is the difference between MSG, SimDag, and GRAS? Do they serve the same purpose?
-
-It depend on how you define "purpose", I guess ;)
-
-They all allow you to build a prototype of application which you can run
-within the simulator afterward. They all share the same simulation kernel,
-which is the core of the SimGrid project. They differ by the way you express
-your application.
-
-With SimDag, you express your code as a collection of interdependent
-parallel tasks. So, in this model, applications can be seen as a DAG of
-tasks. This is the interface of choice for people wanting to port old
-code designed for SimGrid v1 or v2 to the framework current version.
-
-With both GRAS and MSG, your application is seen as a set of communicating
-processes, exchanging data by the way of messages and performing computation
-on their own.
-
-The difference between both is that MSG is somehow easier to use, but GRAS
-is not limited to the simulator. Once you're done writing your GRAS code,
-you can run your code both in the simulator or on a real platform. For this,
-there is two implementations of the GRAS interface, one for simulation, one
-for real execution. So, you just have to relink your code to chose one of
-both world.
-
-\subsection faq_visualization Visualizing and analyzing the results
-
-It is sometime convenient to "see" how the agents are behaving. If you
-like colors, you can use <tt>tools/MSG_visualization/colorize.pl </tt>
-as a filter to your MSG outputs. It works directly with INFO. Beware,
-INFO() prints on stderr. Do not forget to redirect if you want to
-filter (e.g. with bash):
-\verbatim
-./msg_test small_platform.xml small_deployment.xml 2>&1 | ../../tools/MSG_visualization/colorize.pl
-\endverbatim
-
-We also have a more graphical output. Have a look at section \ref options_tracing.
-
-\subsection faq_C Argh! Do I really have to code in C?
-
-Up until now, there is no binding for other languages. If you use C++,
-you should be able to use the SimGrid library as a standard C library
-and everything should work fine (simply <i>link</i> against this
-library; recompiling SimGrid with a C++ compiler won't work and it
-wouldn't help if you could).
-
-In fact, we are currently working on Java bindings of MSG to allow
-all the undergrad students of the world to use this tool. This is a
-little more tricky than I would have expected, but the work is moving
-fast forward [2006/05/13]. More languages are evaluated, but for now,
-we do not feel a real demand for any other language. Please speak up!
-
-\section faq_howto Feature related questions
-
-\subsection faq_MIA "Could you please add (your favorite feature here) to SimGrid?"
-
-Here is the deal. The whole SimGrid project (MSG, SURF, GRAS, ...) is
-meant to be kept as simple and generic as possible. We cannot add
-functions for everybody's needs when these functions can easily be
-built from the ones already in the API. Most of the time, it is
-possible and when it was not possible we always have upgraded the API
-accordingly. When somebody asks us a question like "How to do that?
-Is there a function in the API to simply do this?", we're always glad
-to answer and help. However if we don't need this code for our own
-need, there is no chance we're going to write it... it's your job! :)
-The counterpart to our answers is that once you come up with a neat
-implementation of this feature (task duplication, RPC, thread
-synchronization, ...), you should send it to us and we will be glad to
-add it to the distribution. Thus, other people will take advantage of
-it (and we don't have to answer this question again and again ;).
-
-You'll find in this section a few "Missing In Action" features. Many
-people have asked about it and we have given hints on how to simply do
-it with MSG. Feel free to contribute...
-
-\subsection faq_MIA_MSG MSG features
-
-\subsubsection faq_MIA_examples I want some more complex MSG examples!
-
-Many people have come to ask me a more complex example and each time,
-they have realized afterward that the basics were in the previous three
-examples.
-
-Of course they have often been needing more complex functions like
-MSG_process_suspend(), MSG_process_resume() and
-MSG_process_isSuspended() (to perform synchronization), or
-MSG_task_Iprobe() and MSG_process_sleep() (to avoid blocking
-receptions), or even MSG_process_create() (to design asynchronous
-communications or computations). But the examples are sufficient to
-start.
-
-We know. We should add some more examples, but not really some more
-complex ones... We should add some examples that illustrate some other
-functionalists (like how to simply encode asynchronous
-communications, RPC, process migrations, thread synchronization, ...)
-and we will do it when we will have a little bit more time. We have
-tried to document the examples so that they are understandable. Tell
-us if something is not clear and once again feel free to participate!
-:)
-
-\subsubsection faq_MIA_taskdup Missing in action: MSG Task duplication/replication
-
-There is no task duplication in MSG. When you create a task, you can
-process it or send it somewhere else. As soon as a process has sent
-this task, he doesn't have this task anymore. It's gone. The receiver
-process has got the task. However, you could decide upon receiving to
-create a "copy" of a task but you have to handle by yourself the
-semantic associated to this "duplication".
-
-As we already told, we prefer keeping the API as simple as
-possible. This kind of feature is rather easy to implement by users
-and the semantic you associate really depends on people. Having a
-*generic* task duplication mechanism is not that trivial (in
-particular because of the data field). That is why I would recommand
-that you write it by yourself even if I can give you advice on how to
-do it.
-
-You have the following functions to get informations about a task:
-MSG_task_get_name(), MSG_task_get_compute_duration(),
-MSG_task_get_remaining_computation(), MSG_task_get_data_size(),
-and MSG_task_get_data().
-
-You could use a dictionary (#xbt_dict_t) of dynars (#xbt_dynar_t). If
-you still don't see how to do it, please come back to us...
-
-\subsubsection faq_MIA_asynchronous I want to do asynchronous communications in MSG
-
-In the past (version <= 3.4), there was no function to perform asynchronous communications.
-It could easily be implemented by creating new process when needed though. Since version 3.5,
-we have introduced the following functions:
- - MSG_task_isend()
- - MSG_task_irecv()
- - MSG_comm_test()
- - MSG_comm_wait()
- - MSG_comm_waitall()
- - MSG_comm_waitany()
- - MSG_comm_destroy()
-
-We refer you to the description of these functions for more details on their usage as well
-as to the exemple section on \ref MSG_ex_asynchronous_communications.
-
-\subsubsection faq_MIA_thread_synchronization I need to synchronize my MSG processes
-
-You obviously cannot use pthread_mutexes of pthread_conds since we handle every
-scheduling related decision within SimGrid.
-
-In the past (version <=3.3.4) you could do it by playing with
-MSG_process_suspend() and MSG_process_resume() or with fake communications (using MSG_task_get(),
-MSG_task_put() and MSG_task_Iprobe()).
-
-Since version 3.4, you can use classical synchronization structures. See page \ref XBT_synchro or simply check in
-include/xbt/synchro_core.h.
-
-\subsubsection faq_MIA_host_load Where is the get_host_load function hidden in MSG?
-
-There is no such thing because its semantic wouldn't be really
-clear. Of course, it is something about the amount of host throughput,
-but there is as many definition of "host load" as people asking for
-this function. First, you have to remember that resource availability
-may vary over time, which make any load notion harder to define.
-
-It may be instantaneous value or an average one. Moreover it may be only the
-power of the computer, or may take the background load into account, or may
-even take the currently running tasks into account. In some SURF models,
-communications have an influence on computational power. Should it be taken
-into account too?
-
-First of all, it's near to impossible to predict the load beforehands in the
-simulator since it depends on too much parameters (background load
-variation, bandwidth sharing algorithmic complexity) some of them even being
-not known beforehands (other task starting at the same time). So, getting
-this information is really hard (just like in real life). It's not just that
-we want MSG to be as painful as real life. But as it is in some way
-realistic, we face some of the same problems as we would face in real life.
-
-How would you do it for real? The most common option is to use something
-like NWS that performs active probes. The best solution is probably to do
-the same within MSG, as in next code snippet. It is very close from what you
-would have to do out of the simulator, and thus gives you information that
-you could also get in real settings to not hinder the realism of your
-simulation.
-
-\verbatim
-double get_host_load() {
- m_task_t task = MSG_task_create("test", 0.001, 0, NULL);
- double date = MSG_get_clock();
-
- MSG_task_execute(task);
- date = MSG_get_clock() - date;
- MSG_task_destroy(task);
- return (0.001/date);
-}
-\endverbatim
-
-Of course, it may not match your personal definition of "host load". In this
-case, please detail what you mean on the mailing list, and we will extend
-this FAQ section to fit your taste if possible.
-
-\subsubsection faq_MIA_communication_time How can I get the *real* communication time?
-
-Communications are synchronous and thus if you simply get the time
-before and after a communication, you'll only get the transmission
-time and the time spent to really communicate (it will also take into
-account the time spent waiting for the other party to be
-ready). However, getting the *real* communication time is not really
-hard either. The following solution is a good starting point.
-
-\verbatim
-int sender()
-{
- m_task_t task = MSG_task_create("Task", task_comp_size, task_comm_size,
- calloc(1,sizeof(double)));
- *((double*) task->data) = MSG_get_clock();
- MSG_task_put(task, slaves[i % slaves_count], PORT_22);
- XBT_INFO("Send completed");
- return 0;
-}
-int receiver()
-{
- m_task_t task = NULL;
- double time1,time2;
-
- time1 = MSG_get_clock();
- a = MSG_task_get(&(task), PORT_22);
- time2 = MSG_get_clock();
- if(time1<*((double *)task->data))
- time1 = *((double *) task->data);
- XBT_INFO("Communication time : \"%f\" ", time2-time1);
- free(task->data);
- MSG_task_destroy(task);
- return 0;
-}
-\endverbatim
-
-\subsection faq_MIA_SimDag SimDag related questions
-
-\subsubsection faq_SG_comm Implementing communication delays between tasks.
-
-A classic question of SimDag newcomers is about how to express a
-communication delay between tasks. The thing is that in SimDag, both
-computation and communication are seen as tasks. So, if you want to
-model a data dependency between two DAG tasks t1 and t2, you have to
-create 3 SD_tasks: t1, t2 and c and add dependencies in the following
-way:
-
-\verbatim
-SD_task_dependency_add(NULL, NULL, t1, c);
-SD_task_dependency_add(NULL, NULL, c, t2);
-\endverbatim
-
-This way task t2 cannot start before the termination of communication c
-which in turn cannot start before t1 ends.
-
-When creating task c, you have to associate an amount of data (in bytes)
-corresponding to what has to be sent by t1 to t2.
-
-Finally to schedule the communication task c, you have to build a list
-comprising the workstations on which t1 and t2 are scheduled (w1 and w2
-for example) and build a communication matrix that should look like
-[0;amount ; 0; 0].
-
-\subsubsection faq_SG_DAG How to implement a distributed dynamic scheduler of DAGs.
-
-Distributed is somehow "contagious". If you start making distributed
-decisions, there is no way to handle DAGs directly anymore (unless I
-am missing something). You have to encode your DAGs in term of
-communicating process to make the whole scheduling process
-distributed. Here is an example of how you could do that. Assume T1
-has to be done before T2.
-
-\verbatim
- int your_agent(int argc, char *argv[] {
- ...
- T1 = MSG_task_create(...);
- T2 = MSG_task_create(...);
- ...
- while(1) {
- ...
- if(cond) MSG_task_execute(T1);
- ...
- if((MSG_task_get_remaining_computation(T1)=0.0) && (you_re_in_a_good_mood))
- MSG_task_execute(T2)
- else {
- /* do something else */
- }
- }
- }
-\endverbatim
-
-If you decide that the distributed part is not that much important and that
-DAG is really the level of abstraction you want to work with, then you should
-give a try to \ref SD_API.
-
-\subsection faq_MIA_generic Generic features
-
-\subsubsection faq_more_processes Increasing the amount of simulated processes
-
-Here are a few tricks you can apply if you want to increase the amount
-of processes in your simulations.
-
- - <b>A few thousands of simulated processes</b> (soft tricks)\n
- SimGrid can use either pthreads library or the UNIX98 contextes. On
- most systems, the number of pthreads is limited and then your
- simulation may be limited for a stupid reason. This is especially
- true with the current linux pthreads, and I cannot get more than
- 2000 simulated processes with pthreads on my box. The UNIX98
- contexts allow me to raise the limit to 25,000 simulated processes
- on my laptop.\n\n
- The <tt>--with-context</tt> option of the <tt>./configure</tt>
- script allows you to choose between UNIX98 contextes
- (<tt>--with-context=ucontext</tt>) and the pthread version
- (<tt>--with-context=pthread</tt>). The default value is ucontext
- when the script detect a working UNIX98 context implementation. On
- Windows boxes, the provided value is discarded and an adapted
- version is picked up.\n\n
- We experienced some issues with contextes on some rare systems
- (solaris 8 and lower or old alpha linuxes comes to mind). The main
- problem is that the configure script detect the contextes as being
- functional when it's not true. If you happen to use such a system,
- switch manually to the pthread version, and provide us with a good
- patch for the configure script so that it is done automatically ;)
-
- - <b>Hundred thousands of simulated processes</b> (hard-core tricks)\n
- As explained above, SimGrid can use UNIX98 contextes to represent
- and handle the simulated processes. Thanks to this, the main
- limitation to the number of simulated processes becomes the
- available memory.\n\n
- Here are some tricks I had to use in order to run a token ring
- between 25,000 processes on my laptop (1Gb memory, 1.5Gb swap).\n
- - First of all, make sure your code runs for a few hundreds
- processes before trying to push the limit. Make sure it's
- valgrind-clean, ie that valgrind does not report neither memory
- error nor memory leaks. Indeed, numerous simulated processes
- result in *fat* simulation hindering debugging.
- - It was really boring to write 25,000 entries in the deployment
- file, so I wrote a little script
- <tt>examples/gras/mutual_exclusion/simple_token/make_deployment.pl</tt>, which you may
- want to adapt to your case. You could also think about hijacking
- the SURFXML parser (have look at \ref faq_flexml_bypassing).
- - The deployment file became quite big, so I had to do what is in
- the FAQ entry \ref faq_flexml_limit
- - Each UNIX98 context has its own stack entry. As debugging this is
- quite hairly, the default value is a bit overestimated so that
- user don't get into trouble about this. You want to tune this
- size to increse the number of processes. This is the
- <tt>STACK_SIZE</tt> define in
- <tt>src/xbt/xbt_context_sysv.c</tt>, which is 128kb by default.
- Reduce this as much as you can, but be warned that if this value
- is too low, you'll get a segfault. The token ring example, which
- is quite simple, runs with 40kb stacks.
- - You may tweak the logs to reduce the stack size further. When
- logging something, we try to build the string to display in a
- char array on the stack. The size of this array is constant (and
- equal to XBT_LOG_BUFF_SIZE, defined in include/xbt/log/h). If the
- string is too large to fit this buffer, we move to a dynamically
- sized buffer. In which case, we have to traverse one time the log
- event arguments to compute the size we need for the buffer,
- malloc it, and traverse the argument list again to do the actual
- job.\n
- The idea here is to move XBT_LOG_BUFF_SIZE to 1, forcing the logs
- to use a dynamic array each time. This allows us to lower further
- the stack size at the price of some performance loss...\n
- This allowed me to run the reduce the stack size to ... 4k. Ie,
- on my 1Gb laptop, I can run more than 250,000 processes!
-
-\subsubsection faq_MIA_batch_scheduler Is there a native support for batch schedulers in SimGrid?
-
-No, there is no native support for batch schedulers and none is
-planned because this is a very specific need (and doing it in a
-generic way is thus very hard). However some people have implemented
-their own batch schedulers. Vincent Garonne wrote one during his PhD
-and put his code in the contrib directory of our SVN so that other can
-keep working on it. You may find inspiring ideas in it.
-
-\subsubsection faq_MIA_checkpointing I need a checkpointing thing
-
-Actually, it depends on whether you want to checkpoint the simulation, or to
-simulate checkpoints.
-
-The first one could help if your simulation is a long standing process you
-want to keep running even on hardware issues. It could also help to
-<i>rewind</i> the simulation by jumping sometimes on an old checkpoint to
-cancel recent calculations.\n
-Unfortunately, such thing will probably never exist in SG. One would have to
-duplicate all data structures because doing a rewind at the simulator level
-is very very hard (not talking about the malloc free operations that might
-have been done in between). Instead, you may be interested in the Libckpt
-library (http://www.cs.utk.edu/~plank/plank/www/libckpt.html). This is the
-checkpointing solution used in the condor project, for example. It makes it
-easy to create checkpoints (at the OS level, creating something like core
-files), and rerunning them on need.
-
-If you want to simulate checkpoints instead, it means that you want the
-state of an executing task (in particular, the progress made towards
-completion) to be saved somewhere. So if a host (and the task executing on
-it) fails (cf. #MSG_HOST_FAILURE), then the task can be restarted
-from the last checkpoint.\n
-
-Actually, such a thing does not exists in SimGrid either, but it's just
-because we don't think it is fundamental and it may be done in the user code
-at relatively low cost. You could for example use a watcher that
-periodically get the remaining amount of things to do (using
-MSG_task_get_remaining_computation()), or fragment the task in smaller
-subtasks.
-
-\subsection faq_platform Platform building and Dynamic resources
-
-\subsubsection faq_platform_example Where can I find SimGrid platform files?
-
-There is several little examples in the archive, in the examples/msg
-directory. From time to time, we are asked for other files, but we
-don't have much at hand right now.
-
-You should refer to the Platform Description Archive
-(http://pda.gforge.inria.fr) project to see the other platform file we
-have available, as well as the Simulacrum simulator, meant to generate
-SimGrid platforms using all classical generation algorithms.
-
-\subsubsection faq_platform_alnem How can I automatically map an existing platform?
-
-We are working on a project called ALNeM (Application-Level Network
-Mapper) which goal is to automatically discover the topology of an
-existing network. Its output will be a platform description file
-following the SimGrid syntax, so everybody will get the ability to map
-their own lab network (and contribute them to the catalog project).
-This tool is not ready yet, but it move quite fast forward. Just stay
-tuned.
-
-\subsubsection faq_platform_synthetic Generating synthetic but realistic platforms
-
-The third possibility to get a platform file (after manual or
-automatic mapping of real platforms) is to generate synthetic
-platforms. Getting a realistic result is not a trivial task, and
-moreover, nobody is really able to define what "realistic" means when
-speaking of topology files. You can find some more thoughts on this
-topic in these
-<a href="http://graal.ens-lyon.fr/~alegrand/articles/Simgrid-Introduction.pdf">slides</a>.
-
-If you are looking for an actual tool, there we have a little tool to
-annotate Tiers-generated topologies. This perl-script is in
-<tt>tools/platform_generation/</tt> directory of the SVN. Dinda et Al.
-released a very comparable tool, and called it GridG.
-
-\subsubsection faq_SURF_multicore Modeling multi-core resources
-
-There is currently no native support for multi-core or SMP machines in
-SimGrid. We are currently working on it, but coming up with the right
-model is very hard: Cores share caches and bus to access memory and
-thus interfere with each others. Memory contention is a crucial
-component of multi-core modeling.
-
-In the meanwhile, some user-level tricks can reveal sufficient for
-you. For example, you may model each core by a CPU and add some very
-high speed links between them. This complicates a bit the user code
-since you have to remember that when you assign something to a (real)
-host, it can be any of the (fake) hosts representing the cores of a
-given machine. For that, you can use the prop tag of the XML files as
-follows. Your code should then look at the ‘machine’ property
-associated with each workstation, and run parallel tasks over all
-cores of the machine.
-
-\verbatim
- <host id="machine0/core0" power="91500E6">
- <prop id="machine" value="machine0"/>
- <prop id="core" value="0"/>
- </host>
- <host id="machine0/core1" power="91500E6">
- <prop id="machine" value="machine0"/>
- <prop id="core" value="1"/>
-</host>
-
-
-\endverbatim
-
-\subsubsection faq_SURF_dynamic Modeling dynamic resource availability
-
-A nice feature of SimGrid is that it enables you to seamlessly have
-resources whose availability change over time. When you build a
-platform, you generally declare hosts like that:
-
-\verbatim
- <host id="host A" power="100.00"/>
-\endverbatim
-
-If you want the availability of "host A" to change over time, the only
-thing you have to do is change this definition like that:
-
-\verbatim
- <host id="host A" power="100.00" availability_file="trace_A.txt" state_file="trace_A_failure.txt"/>
-\endverbatim
-
-For hosts, availability files are expressed in fraction of available
-power. Let's have a look at what "trace_A.txt" may look like:
-
-\verbatim
-PERIODICITY 1.0
-0.0 1.0
-11.0 0.5
-20.0 0.9
-\endverbatim
-
-At time 0, our host will deliver 100 flop/s. At time 11.0, it will
-deliver only 50 flop/s until time 20.0 where it will will start
-delivering 90 flop/s. Last at time 21.0 (20.0 plus the periodicity
-1.0), we'll be back to the beginning and it will deliver 100 flop/s.
-
-Now let's look at the state file:
-\verbatim
-PERIODICITY 10.0
-1.0 -1.0
-2.0 1.0
-\endverbatim
-
-A negative value means "off" while a positive one means "on". At time
-1.0, the host is on. At time 1.0, it is turned off and at time 2.0, it
-is turned on again until time 12 (2.0 plus the periodicity 10.0). It
-will be turned on again at time 13.0 until time 23.0, and so on.
-
-Now, let's look how the same kind of thing can be done for network
-links. A usual declaration looks like:
-
-\verbatim
- <link id="LinkA" bandwidth="10.0" latency="0.2"/>
-\endverbatim
-
-You have at your disposal the following options: bandwidth_file,
-latency_file and state_file. The only difference with hosts is that
-bandwidth_file and latency_file do not express fraction of available
-power but are expressed directly in bytes per seconds and seconds.
-
-\subsubsection faq_platform_multipath How to express multipath routing in platform files?
-
-It is unfortunately impossible to express the fact that there is more
-than one routing path between two given hosts. Let's consider the
-following platform file:
-
-\verbatim
-<route src="A" dst="B">
- <link:ctn id="1"/>
-</route>
-<route src="B" dst="C">
- <link:ctn id="2"/>
-</route>
-<route src="A" dst="C">
- <link:ctn id="3"/>
-</route>
-\endverbatim
-
-Although it is perfectly valid, it does not mean that data traveling
-from A to C can either go directly (using link 3) or through B (using
-links 1 and 2). It simply means that the routing on the graph is not
-trivial, and that data do not following the shortest path in number of
-hops on this graph. Another way to say it is that there is no implicit
-in these routing descriptions. The system will only use the routes you
-declare (such as <route src="A" dst="C"><link:ctn
-id="3"/></route>), without trying to build new routes by aggregating
-the provided ones.
-
-You are also free to declare platform where the routing is not
-symmetric. For example, add the following to the previous file:
-
-\verbatim
-<route src="C" dst="A">
- <link:ctn id="2"/>
- <link:ctn id="1"/>
-</route>
-\endverbatim
-
-This makes sure that data from C to A go through B where data from A
-to C go directly. Don't worry about realism of such settings since
-we've seen ways more weird situation in real settings (in fact, that's
-the realism of very regular platforms which is questionable, but
-that's another story).
-
-\subsubsection faq_flexml_bypassing Bypassing the XML parser with your own C functions
-
-So you want to bypass the XML files parser, uh? Maybe doing some parameter
-sweep experiments on your simulations or so? This is possible, and
-it's not even really difficult (well. Such a brutal idea could be
-harder to implement). Here is how it goes.
-
-For this, you have to first remember that the XML parsing in SimGrid is done
-using a tool called FleXML. Given a DTD, this gives a flex-based parser. If
-you want to bypass the parser, you need to provide some code mimicking what
-it does and replacing it in its interactions with the SURF code. So, let's
-have a look at these interactions.
-
-FleXML parser are close to classical SAX parsers. It means that a
-well-formed SimGrid platform XML file might result in the following
-"events":
-
- - start "platform_description" with attribute version="2"
- - start "host" with attributes id="host1" power="1.0"
- - end "host"
- - start "host" with attributes id="host2" power="2.0"
- - end "host"
- - start "link" with ...
- - end "link"
- - start "route" with ...
- - start "link:ctn" with ...
- - end "link:ctn"
- - end "route"
- - end "platform_description"
-
-The communication from the parser to the SURF code uses two means:
-Attributes get copied into some global variables, and a surf-provided
-function gets called by the parser for each event. For example, the event
- - start "host" with attributes id="host1" power="1.0"
-
-let the parser do something roughly equivalent to:
-\verbatim
- strcpy(A_host_id,"host1");
- A_host_power = 1.0;
- STag_host();
-\endverbatim
-
-In SURF, we attach callbacks to the different events by initializing the
-pointer functions to some the right surf functions. Since there can be
-more than one callback attached to the same event (if more than one
-model is in use, for example), they are stored in a dynar. Example in
-workstation_ptask_L07.c:
-\verbatim
- /* Adding callback functions */
- surf_parse_reset_parser();
- surfxml_add_callback(STag_surfxml_host_cb_list, &parse_cpu_init);
- surfxml_add_callback(STag_surfxml_prop_cb_list, &parse_properties);
- surfxml_add_callback(STag_surfxml_link_cb_list, &parse_link_init);
- surfxml_add_callback(STag_surfxml_route_cb_list, &parse_route_set_endpoints);
- surfxml_add_callback(ETag_surfxml_link_c_ctn_cb_list, &parse_route_elem);
- surfxml_add_callback(ETag_surfxml_route_cb_list, &parse_route_set_route);
-
- /* Parse the file */
- surf_parse_open(file);
- xbt_assert(!surf_parse(), "Parse error in %s", file);
- surf_parse_close();
-\endverbatim
-
-So, to bypass the FleXML parser, you need to write your own version of the
-surf_parse function, which should do the following:
- - Fill the A_<tag>_<attribute> variables with the wanted values
- - Call the corresponding STag_<tag>_fun function to simulate tag start
- - Call the corresponding ETag_<tag>_fun function to simulate tag end
- - (do the same for the next set of values, and loop)
-
-Then, tell SimGrid that you want to use your own "parser" instead of the stock one:
-\verbatim
- surf_parse = surf_parse_bypass_environment;
- MSG_create_environment(NULL);
- surf_parse = surf_parse_bypass_application;
- MSG_launch_application(NULL);
-\endverbatim
-
-A set of macros are provided at the end of
-include/surf/surfxml_parse.h to ease the writing of the bypass
-functions. An example of this trick is distributed in the file
-examples/msg/masterslave/masterslave_bypass.c
-
-\section faq_troubleshooting Troubleshooting
-
-\subsection faq_trouble_lib_compil SimGrid compilation and installation problems
-
-\subsubsection faq_trouble_lib_config cmake fails!
-
-We know only one reason for the configure to fail:
-
- - <b>You are using a broken build environment</b>\n
- If symptom is that the configury magic complains about gcc not being able to build
- executables, you are probably missing the libc6-dev package. Damn Ubuntu.
-
-If you experience other kind of issue, please get in touch with us. We are
-always interested in improving our portability to new systems.
-
-\subsubsection faq_trouble_distcheck Dude! "ctest" fails on my machine!
-
-Don't assume we never run this target, because we do. Check
-http://cdash.inria.fr/CDash/index.php?project=Simgrid (click on
-previous if there is no result for today: results are produced only by
-11am, French time) and
-https://buildd.debian.org/status/logs.php?pkg=simgrid if you don't believe us.
-
-If it's failing on your machine in a way not experienced by the
-autobuilders above, please drop us a mail on the mailing list so that
-we can check it out. Make sure to read \ref faq_bugrepport before you
-do so.
-
-\subsection faq_trouble_compil User code compilation problems
-
-\subsubsection faq_trouble_err_logcat "gcc: _simgrid_this_log_category_does_not_exist__??? undeclared (first use in this function)"
-
-This is because you are using the log mecanism, but you didn't created
-any default category in this file. You should refer to \ref XBT_log
-for all the details, but you simply forgot to call one of
-XBT_LOG_NEW_DEFAULT_CATEGORY() or XBT_LOG_NEW_DEFAULT_SUBCATEGORY().
-
-\subsubsection faq_trouble_pthreadstatic "gcc: undefined reference to pthread_key_create"
-
-This indicates that one of the library SimGrid depends on (libpthread
-here) was missing on the linking command line. Dependencies of
-libsimgrid are expressed directly in the dynamic library, so it's
-quite impossible that you see this message when doing dynamic linking.
-
-If you compile your code statically (and if you use a pthread version
-of SimGrid -- see \ref faq_more_processes), you must absolutely
-specify <tt>-lpthread</tt> on the linker command line. As usual, this should
-come after <tt>-lsimgrid</tt> on this command line.
-
-\subsection faq_trouble_errors Runtime error messages
-
-\subsubsection faq_flexml_limit "surf_parse_lex: Assertion `next limit' failed."
-
-This is because your platform file is too big for the parser.
-
-Actually, the message comes directly from FleXML, the technology on top of
-which the parser is built. FleXML has the bad idea of fetching the whole
-document in memory before parsing it. And moreover, the memory buffer size
-must be determined at compilation time.
-
-We use a value which seems big enough for our need without bloating the
-simulators footprints. But of course your mileage may vary. In this case,
-just edit src/surf/surfxml.l modify the definition of
-FLEXML_BUFFERSTACKSIZE. E.g.
-
-\verbatim
-#define FLEXML_BUFFERSTACKSIZE 1000000000
-\endverbatim
-
-Then recompile and everything should be fine, provided that your version of
-Flex is recent enough (>= 2.5.31). If not the compilation process should
-warn you.
-
-A while ago, we worked on FleXML to reduce a bit its memory consumption, but
-these issues remain. There is two things we should do:
-
- - use a dynamic buffer instead of a static one so that the only limit
- becomes your memory, not a stupid constant fixed at compilation time
- (maybe not so difficult).
- - change the parser so that it does not need to get the whole file in
- memory before parsing
- (seems quite difficult, but I'm a complete newbe wrt flex stuff).
-
-These are changes to FleXML itself, not SimGrid. But since we kinda hijacked
-the development of FleXML, I can grant you that any patches would be really
-welcome and quickly integrated.
-
-<b>Update:</b> A new version of FleXML (1.7) was released. Most of the work
-was done by William Dowling, who use it in his own work. The good point is
-that it now use a dynamic buffer, and that the memory usage was greatly
-improved. The downside is that William also changed some things internally,
-and it breaks the hack we devised to bypass the parser, as explained in
-\ref faq_flexml_bypassing. Indeed, this is not a classical usage of the
-parser, and Will didn't imagine that we may have used (and even documented)
-such a crude usage of FleXML. So, we now have to repair the bypassing
-functionality to use the lastest FleXML version and fix the memory usage in
-SimGrid.
-
-\subsubsection faq_trouble_gras_transport GRAS spits networking error messages
-
-Gras, on real platforms, naturally use regular sockets to communicate. They
-are deeply hidden in the gras abstraction, but when things go wrong, you may
-get some weird error messages. Here are some example, with the probable
-reason:
-
- - <b>Transport endpoint is not connected</b>: several processes try to open
- a server socket on the same port number of the same machine. This is
- naturally bad and each process should pick its own port number for this.\n
- Maybe, you just have some processes remaining from a previous experiment
- on your machine.\n
- Killing them may help, but again if you kill -KILL them, you'll have to
- wait for a while: they didn't close there sockets properly and the system
- needs a while to notice that this port is free again.
-
- - <b>Socket closed by remote side</b>: if the remote process is not
- supposed to close the socket at this point, it may be dead.
-
- - <b>Connection reset by peer</b>: I found this on Internet about this
- error. I think it's what's happening here, too:\n
- <i>This basically means that a network error occurred while the client was
- receiving data from the server. But what is really happening is that the
- server actually accepts the connection, processes the request, and sends
- a reply to the client. However, when the server closes the socket, the
- client believes that the connection has been terminated abnormally
- because the socket implementation sends a TCP reset segment telling the
- client to throw away the data and report an error.\n
- Sometimes, this problem is caused by not properly closing the
- input/output streams and the socket connection. Make sure you close the
- input/output streams and socket connection properly. If everything is
- closed properly, however, and the problem persists, you can work around
- it by adding a one-second sleep before closing the streams and the
- socket. This technique, however, is not reliable and may not work on all
- systems.</i>\n
- Since GRAS sockets are closed properly (repeat after me: there is no bug
- in GRAS), it is either that you are closing your sockets on server side
- before the client get a chance to read them (use gras_os_sleep() to delay
- the server), or the server died awfully before the client got the data.
-
-\subsubsection faq_trouble_errors_big_fat_warning I'm told that my XML files are too old.
-
-The format of the XML platform description files is sometimes
-improved. For example, we decided to change the units used in SimGrid
-from MBytes, MFlops and seconds to Bytes, Flops and seconds to ease
-people exchanging small messages. We also reworked the route
-descriptions to allow more compact descriptions.
-
-That is why the XML files are versionned using the 'version' attribute
-of the root tag. Currently, it should read:
-\verbatim
- <platform version="2">
-\endverbatim
-
-If your files are too old, you can use the simgrid_update_xml.pl
-script which can be found in the tools directory of the archive.
-
-\subsection faq_trouble_valgrind Valgrind-related and other debugger issues
-
-If you don't, you really should use valgrind to debug your code, it's
-almost magic.
-
-\subsubsection faq_trouble_vg_longjmp longjmp madness in valgrind
-
-This is when valgrind starts complaining about longjmp things, just like:
-
-\verbatim ==21434== Conditional jump or move depends on uninitialised value(s)
-==21434== at 0x420DBE5: longjmp (longjmp.c:33)
-==21434==
-==21434== Use of uninitialised value of size 4
-==21434== at 0x420DC3A: __longjmp (__longjmp.S:48)
-\endverbatim
-
-This is the sign that you didn't used the exception mecanism well. Most
-probably, you have a <tt>return;</tt> somewhere within a <tt>TRY{}</tt>
-block. This is <b>evil</b>, and you must not do this. Did you read the section
-about \ref XBT_ex??
-
-\subsubsection faq_trouble_vg_libc Valgrind spits tons of errors about backtraces!
-
-It may happen that valgrind, the memory debugger beloved by any decent C
-programmer, spits tons of warnings like the following :
-\verbatim ==8414== Conditional jump or move depends on uninitialised value(s)
-==8414== at 0x400882D: (within /lib/ld-2.3.6.so)
-==8414== by 0x414EDE9: (within /lib/tls/i686/cmov/libc-2.3.6.so)
-==8414== by 0x400B105: (within /lib/ld-2.3.6.so)
-==8414== by 0x414F937: _dl_open (in /lib/tls/i686/cmov/libc-2.3.6.so)
-==8414== by 0x4150F4C: (within /lib/tls/i686/cmov/libc-2.3.6.so)
-==8414== by 0x400B105: (within /lib/ld-2.3.6.so)
-==8414== by 0x415102D: __libc_dlopen_mode (in /lib/tls/i686/cmov/libc-2.3.6.so)
-==8414== by 0x412D6B9: backtrace (in /lib/tls/i686/cmov/libc-2.3.6.so)
-==8414== by 0x8076446: xbt_dictelm_get_ext (dict_elm.c:714)
-==8414== by 0x80764C1: xbt_dictelm_get (dict_elm.c:732)
-==8414== by 0x8079010: xbt_cfg_register (config.c:208)
-==8414== by 0x806821B: MSG_config (msg_config.c:42)
-\endverbatim
-
-This problem is somewhere in the libc when using the backtraces and there is
-very few things we can do ourselves to fix it. Instead, here is how to tell
-valgrind to ignore the error. Add the following to your ~/.valgrind.supp (or
-create this file on need). Make sure to change the obj line according to
-your personnal mileage (change 2.3.6 to the actual version you are using,
-which you can retrieve with a simple "ls /lib/ld*.so").
-
-\verbatim {
- name: Backtrace madness
- Memcheck:Cond
- obj:/lib/ld-2.3.6.so
- fun:dl_open_worker
- fun:_dl_open
- fun:do_dlopen
- fun:dlerror_run
- fun:__libc_dlopen_mode
-}\endverbatim
-
-Then, you have to specify valgrind to use this suppression file by passing
-the <tt>--suppressions=$HOME/.valgrind.supp</tt> option on the command line.
-You can also add the following to your ~/.bashrc so that it gets passed
-automatically. Actually, it passes a bit more options to valgrind, and this
-happen to be my personnal settings. Check the valgrind documentation for
-more information.
-
-\verbatim export VALGRIND_OPTS="--leak-check=yes --leak-resolution=high --num-callers=40 --tool=memcheck --suppressions=$HOME/.valgrind.supp" \endverbatim
-
-\subsubsection faq_trouble_backtraces Truncated backtraces
-
-When debugging SimGrid, it's easier to pass the
---disable-compiler-optimization flag to the configure if valgrind or
-gdb get fooled by the optimization done by the compiler. But you
-should remove these flag when everything works before going in
-production (before launching your 1252135 experiments), or everything
-will run only one half of the true SimGrid potential.
-
-\subsection faq_deadlock There is a deadlock in my code!!!
-
-Unfortunately, we cannot debug every code written in SimGrid. We
-furthermore believe that the framework provides ways enough
-information to debug such informations yourself. If the textual output
-is not enough, Make sure to check the \ref faq_visualization FAQ entry to see
-how to get a graphical one.
-
-Now, if you come up with a really simple example that deadlocks and
-you're absolutely convinced that it should not, you can ask on the
-list. Just be aware that you'll be severely punished if the mistake is
-on your side... We have plenty of FAQ entries to redact and new
-features to implement for the impenitents! ;)
-
-\subsection faq_surf_network_latency I get weird timings when I play with the latencies.
-
-OK, first of all, remember that units should be Bytes, Flops and
-Seconds. If you don't use such units, some SimGrid constants (e.g. the
-SG_TCP_CTE_GAMMA constant used in most network models) won't have the
-right unit and you'll end up with weird results.
-
-Here is what happens with a single transfer of size L on a link
-(bw,lat) when nothing else happens.
-
-\verbatim
-0-----lat--------------------------------------------------t
-|-----|**** real_bw =min(bw,SG_TCP_CTE_GAMMA/(2*lat)) *****|
-\endverbatim
-
-In more complex situations, this min is the solution of a complex
-max-min linear system. Have a look
-<a href="http://lists.gforge.inria.fr/pipermail/simgrid-devel/2006-April/thread.html">here</a>
-and read the two threads "Bug in SURF?" and "Surf bug not
-fixed?". You'll have a few other examples of such computations. You
-can also read "A Network Model for Simulation of Grid Application" by
-Henri Casanova and Loris Marchal to have all the details. The fact
-that the real_bw is smaller than bw is easy to understand. The fact
-that real_bw is smaller than SG_TCP_CTE_GAMMA/(2*lat) is due to the
-window-based congestion mechanism of TCP. With TCP, you can't exploit
-your huge network capacity if you don't have a good round-trip-time
-because of the acks...
-
-Anyway, what you get is t=lat + L/min(bw,SG_TCP_CTE_GAMMA/(2*lat)).
-
- * if I you set (bw,lat)=(100 000 000, 0.00001), you get t = 1.00001 (you fully
-use your link)
- * if I you set (bw,lat)=(100 000 000, 0.0001), you get t = 1.0001 (you're on the
-limit)
- * if I you set (bw,lat)=(100 000 000, 0.001), you get t = 10.001 (ouch!)
-
-This bound on the effective bandwidth of a flow is not the only thing
-that may make your result be unexpected. For example, two flows
-competing on a saturated link receive an amount of bandwidth inversely
-proportional to their round trip time.
-
-\subsection faq_bugrepport So I've found a bug in SimGrid. How to report it?
-
-We do our best to make sure to hammer away any bugs of SimGrid, but this is
-still an academic project so please be patient if/when you find bugs in it.
-If you do, the best solution is to drop an email either on the simgrid-user
-or the simgrid-devel mailing list and explain us about the issue. You can
-also decide to open a formal bug report using the
-<a href="https://gforge.inria.fr/tracker/?atid=165&group_id=12&func=browse">relevant
-interface</a>. You need to login on the server to get the ability to submit
-bugs.
-
-We will do our best to solve any problem repported, but you need to help us
-finding the issue. Just telling "it segfault" isn't enough. Telling "It
-segfaults when running the attached simulator" doesn't really help either.
-You may find the following article interesting to see how to repport
-informative bug repports:
-http://www.chiark.greenend.org.uk/~sgtatham/bugs.html (it is not SimGrid
-specific at all, but it's full of good advices).
-
-\author Arnaud Legrand (arnaud.legrand::imag.fr)
-\author Martin Quinson (martin.quinson::loria.fr)
-
-
-*/
-
-******************************************************************
-* OLD CRUFT NOT USED ANYMORE *
-******************************************************************
-
-
-subsection faq_crosscompile Cross-compiling a Windows DLL of SimGrid from linux
-
-At the moment, we do not distribute Windows pre-compiled version of SimGrid
-because the support for this platform is still experimental. We know that
-some parts of the GRAS environment do not work, and we think that the others
-environments (MSG and SD) have good chances to work, but we didn't test
-ourselves. This section explains how we generate the SimGrid DLL so that you
-can build it for yourself. First of all, you need to have a version more
-recent than 3.1 (ie, a SVN version as time of writting).
-
-In order to cross-compile the package to windows from linux, you need to
-install mingw32 (minimalist gnu win32). On Debian, you can do so by
-installing the packages mingw32 (compiler), mingw32-binutils (linker and
-so), mingw32-runtime.
-
-You can use the VPATH support of configure to compile at the same time for
-linux and windows without dupplicating the source nor cleaning the tree
-between each. Just run bootstrap (if you use the SVN) to run the autotools.
-Then, create a linux and a win directories. Then, type:
-\verbatim cd linux; ../configure --srcdir=.. <usual configure flags>; make; cd ..
-cd win; ../configure --srcdir=.. --host=i586-mingw32msvc <flags>; make; cd ..
-\endverbatim
-The trick to VPATH builds is to call configure from another directory,
-passing it an extra --srcdir argument to tell it where all the sources are.
-It will understand you want to use VPATH. Then, the trick to cross-compile
-is simply to add a --host argument specifying the target you want to build
-for. The i586-mingw32msvc string is what you have to pass to use the mingw32
-environment as distributed in Debian.
-
-After that, you can run all make targets from both directories, and test
-easily that what you change for one arch does not break the other one.
-
-It is possible that this VPATH build thing breaks from time to time in the
-SVN since it's quite fragile, but it's granted to work in any released
-version. If you experience problems, drop us a mail.
-
-Another possible source of issue is that at the moment, building the
-examples request to use the gras_stub_generator tool, which is a compiled
-program, not a script. In cross-compilation, you need to cross-execute with
-wine for example, which is not really pleasant. We are working on this, but
-in the meanwhile, simply don't build the examples in cross-compilation
-(<tt>cd src</tt> before running make).
-
-Program (cross-)compiled with mingw32 do request an extra DLL at run-time to be
-usable. For example, if you want to test your build with wine, you should do
-the following to put this library where wine looks for DLLs.
-\verbatim
-cp /usr/share/doc/mingw32-runtime/mingwm10.dll.gz ~/.wine/c/windows/system/
-gunzip ~/.wine/c/windows/system/mingwm10.dll.gz
-\endverbatim
-
-The DLL is built in src/.libs, and installed in the <i>prefix</i>/bin directory
-when you run make install.
-
-If you want to use it in a native project on windows, you need to use
-simgrid.dll and mingwm10.dll. For each DLL, you need to build .def file
-under linux (listing the defined symbols), and convert it into a .lib file
-under windows (specifying this in a way that windows compilers like). To
-generate the def files, run (under linux):
-\verbatim echo "LIBRARY libsimgrid-0.dll" > simgrid.def
-echo EXPORTS >> simgrid.def
-nm libsimgrid-0.dll | grep ' T _' | sed 's/.* T _//' >> simgrid.def
-nm libsimgrid-0.dll | grep ' D _' | sed 's/.* D _//' | sed 's/$/ DATA/' >> simgrid.def
-
-echo "LIBRARY mingwm10.dll" > mingwm10.def
-echo EXPORTS >> mingwm10.def
-nm mingwm10.dll | grep ' T _' | sed 's/.* T _//' >> mingwm10.def
-nm mingwm10.dll | grep ' D _' | sed 's/.* D _//' | sed 's/$/ DATA/' >> mingwm10.def
-\endverbatim
-
-To create the import .lib files, use the <tt>lib</tt> windows tool (from
-MSVC) the following way to produce simgrid.lib and mingwm10.lib
-\verbatim lib /def:simgrid.def
-lib /def:mingwm10.def
-\endverbatim
-
-If you happen to use Borland C Builder, the right command line is the
-following (note that you don't need any file.def to get this working).
-\verbatim implib simgrid.lib libsimgrid-0.dll
-implib mingwm10.lib mingwm10.dll
-\endverbatim
-
-Then, set the following parameters in Visual C++ 2005:
-Linker -> Input -> Additional dependencies = simgrid.lib mingwm10.lib
-
-Just in case you wonder how to generate a DLL from libtool in another
-project, we added -no-undefined to any lib*_la_LDFLAGS variables so that
-libtool accepts to generate a dynamic library under windows. Then, to make
-it true, we pass any dependencies (such as -lws2 under windows or -lpthread
-on need) on the linking line. Passing such deps is a good idea anyway so
-that they get noted in the library itself, avoiding the users to know about
-our dependencies and put them manually on their compilation line. Then we
-added the AC_LIBTOOL_WIN32_DLL macro just before AC_PROG_LIBTOOL in the
-configure.ac. It means that we exported any symbols which need to be.
-Nowadays, functions get automatically exported, so we don't need to load our
-header files with tons of __declspec(dllexport) cruft. We only need to do so
-for data, but there is no public data in SimGrid so we are good.
