1 /*! \page faq Frequently Asked Questions
3 \section faq_installation Installing the SimGrid library
5 Many people have been asking me questions on how to use SimGrid. Quite
6 often, the questions were not really about SimGrid but on the
7 installation process. This section is intended to help people that are
8 not familiar with compiling C files under UNIX. If you follow these
9 instructions and still have some troubles, drop an e-mail to
10 <simgrid-user@lists.gforge.inria.fr>.
12 \subsection faq_compiling Compiling SimGrid
14 Suppose you have uncompressed SimGrid in some temporary location of
15 your home directory (say <tt>/home/joe/tmp/simgrid-2.18.2 </tt>). The
16 simplest way to use SimGrid is to install it in your home
17 directory. Change your directory to
18 <tt>/home/joe/tmp/simgrid-2.18.2</tt> and type
20 \verbatim./configure --prefix=$HOME
25 If at some point, something fails, you can report me this problem but,
26 please, avoid sending a laconic mail like "There is a problem. Is it
27 normal ?". Send me the config.log file which is automatically
28 generated by configure. Try to capture both the standard output and
29 the error output of the <tt>make</tt> command. There is no way for me
30 to help you if you do not give me a little bit of information.
32 Now, the following directory should have been created :
34 \li <tt>/home/joe/doc/simgrid/html/</tt>
35 \li <tt>/home/joe/lib/</tt>
36 \li <tt>/home/joe/include/</tt>
38 SimGrid is not a binary, it is a library. Both a static and a dynamic
39 version are available. Here is what you can find if you try a <tt>ls
42 \verbatim libsimgrid.a libsimgrid.la libsimgrid.so libsimgrid.so.0 libsimgrid.so.0.0.1
45 Thus, there is two ways to link your program with SimGrid:
46 \li Either you use the static version, e.g
47 \verbatim gcc libsimgrid.a -o MainProgram MainProgram.c
49 In this case, all the SimGrid functions are directly
50 included in <tt>MainProgram</tt> (hence a bigger binary).
51 \li Either you use the dynamic version (the preferred method)
52 \verbatim gcc -lsimgrid -o MainProgram MainProgram.c
54 In this case, the SimGrid functions are not included in
55 <tt>MainProgram</tt> and you need to set your environment
56 variable in such a way that <tt>libsimgrid.so</tt> will be
57 found at runtime. This can be done by adding the following
58 line in your .bashrc (if you use bash and if you have
59 installed the SimGrid libraries in your home directory):
60 \verbatim export LD_LIBRARY_PATH=$HOME/lib/:$LD_LIBRARY_PATH
63 \subsection faq_setting Setting up your own code
65 Do not build your simulator by modifying the SimGrid examples. Go
66 outside the SimGrid source tree and create your own working directory
67 (say <tt>/home/joe/SimGrid/MyFirstScheduler/</tt>).
69 Suppose your simulation has the following structure (remember it is
70 just an example to illustrate a possible way to compile everything;
71 feel free to organize it as you want).
73 \li <tt>sched.h</tt>: a description of the core of the
74 scheduler (i.e. which functions are can be used by the
75 agents). For example we could find the following functions
76 (master, forwarder, slave).
78 \li <tt>sched.c</tt>: a C file including <tt>sched.h</tt> and
79 implementing the core of the scheduler. Most of these
80 functions use the MSG functions defined in section \ref
83 \li <tt>masterslave.c</tt>: a C file with the main function, i.e.
84 the MSG initialization (MSG_global_init()), the platform
85 creation (e.g. with MSG_create_environment()), the
86 deployment phase (e.g. with MSG_function_register() and
87 MSG_launch_application()) and the call to
90 To compile such a program, I suggest to use the following Makefile. It
91 is a generic Makefile that I generally use with my students when I
96 masterslave: masterslave.o sched.o
100 PEDANTIC_PARANOID_FREAK = -O0 -Wshadow -Wcast-align \
101 -Waggregate-return -Wmissing-prototypes -Wmissing-declarations \
102 -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations \
103 -Wmissing-noreturn -Wredundant-decls -Wnested-externs \
104 -Wpointer-arith -Wwrite-strings -finline-functions
105 REASONABLY_CAREFUL_DUDE = -Wall
106 NO_PRAYER_FOR_THE_WICKED = -w -O2
107 WARNINGS = $(REASONABLY_CAREFUL_DUDE)
108 CFLAGS = -g $(WARNINGS)
110 INCLUDES = -I$(INSTALL_PATH)/include
111 DEFS = -L$(INSTALL_PATH)/lib/
112 LDADD = -lm -lsimgrid
116 $(CC) $(INCLUDES) $(DEFS) $(CFLAGS) $^ $(LIBS) $(LDADD) -o $@
119 $(CC) $(INCLUDES) $(DEFS) $(CFLAGS) -c -o $@ $<
122 rm -f $(BIN_FILES) *.o *~
128 The first two lines indicates what should be build when typing make
129 (<tt>masterslave</tt>) and of which files it is to be made of
130 (<tt>masterslave.o</tt> and <tt>sched.o</tt>). This makefile assumes
131 that you have set up correctly your <tt>LD_LIBRARY_PATH</tt> variable
132 (look, there is a <tt>LDADD = -lm -lsimgrid</tt>). If you prefer using
133 the static version, remove the <tt>-lsimgrid</tt> and add a
134 <tt>$(INSTALL_PATH)/lib/libsimgrid.a</tt> on the next line, right
135 after the <tt>LIBS = </tt>.
137 More generally, if you have never written a Makefile by yourself, type
138 in a terminal : <tt>info make</tt> and read the introduction. The
139 previous example should be enough for a first try but you may want to
140 perform some more complex compilations...
142 \section faq_simgrid I'm new to SimGrid. I have some questions. Where should I start ?
144 You are at the right place... Having a look to these
145 <a href="http://graal.ens-lyon.fr/~alegrand/articles/Simgrid-Introduction.pdf">slides</a>
146 may give you some insights on what SimGrid can help you to do and what
147 are its limitations. Then you definitely should read the \ref
148 MSG_examples. There is also a mailing list: <simgrid-user@lists.gforge.inria.fr>.
150 \subsection faq_generic Building a generic simulator
152 Please read carefully the \ref MSG_examples. You'll find in \ref
153 MSG_ex_master_slave a very simple consisting of a master (that owns a bunch of
154 tasks and distributes them) , some slaves (that process tasks whenever
155 they receive one) and some forwarder agents (that simply pass the
156 tasks they receive to some slaves).
158 \subsection faq_examples I want some more complex examples !
160 Many people have come to ask me a more complex example and each time,
161 they have realized afterward that the basics were in the previous three
164 Of course they have often been needing more complex functions like
165 MSG_process_suspend(), MSG_process_resume() and
166 MSG_process_isSuspended() (to perform synchronization), or
167 MSG_task_Iprobe() and MSG_process_sleep() (to avoid blocking
168 receptions), or even MSG_process_create() (to design asynchronous
169 communications or computations). But the examples are sufficient to
172 I know I should add some more examples, but not some more complex
173 ones... I should add some examples that illustrate some other
174 functionalities (like how to simply encode asynchronous
175 communications, RPC, process migrations, thread synchronization, ...)
176 and I will do it when I will have a little bit more time. I have tried
177 to document the examples so that they are understandable. I know it is
178 not really satisfying but it is the best I have managed to do yet.
180 \subsection faq_platform Building a realistic platform
182 I can speak more than an hour on this subject and I still do not have
183 the right answer, just some ideas. You can read the following
184 <a href="http://graal.ens-lyon.fr/~alegrand/articles/Simgrid-Introduction.pdf">slides</a>.
185 It may give you some hints. You can also have a look at the
186 <tt>tools/platform_generation/</tt> directory. There is a perl-script
187 I use to annotate a Tiers generated platform (may not be up-to-date
190 \subsection faq_visualization Visualizing the schedule
192 It is sometime convenient to "see" how the agents are behaving. If you
193 like colors, you can use <tt>tools/MSG_visualization/colorize.pl </tt>
194 as a filter to your MSG outputs. It works directly with INFO. Beware,
195 INFO() prints on stderr. Do not forget to redirect if you want to
196 filter (e.g. with bash):
198 ./msg_test small_platform.xml small_deployment.xml 2>&1 | ../../tools/MSG_visualization/colorize.pl
201 We also have a more graphical output. Have a look at MSG_paje_output(). It
202 generates an input to <a href="http://www-id.imag.fr/Logiciels/paje/">Paje</a>.
205 <a href="Paje_MSG_screenshot.jpg"><img src="Paje_MSG_screenshot_thn.jpg"></a>
209 \subsection faq_context I have tons of process and it is limiting my simulation.
211 MSG can use either pthreads or the GNU context library. On most
212 systems, the number of pthreads is limited (especially with the
213 current linux pthreads) and then your simulation may be limited for a
214 stupid reason. If you enable the context option
215 (<tt>--enable-context</tt> in the <tt>./configure</tt> phase), you
216 will not use the pthread anymore and the context switching will be
217 done manually, which enables us to have as much agents as your memory
218 can hold and should be much faster... So think about it if your
219 simulation is getting really big.
221 Nevertheless, be aware that this code does not work on some system. It
222 is not very clean. As usual, as soon as I will have a little bit more
223 time, I will recode it in a cleaner way.
225 \section faq_SG Where has SG disappeared ?!?
227 OK, it's time to explain what's happening to the SimGrid project. Let's
228 start with a little bit of history.
230 * Historically, SimGrid was a low-level toolkit for scheduling with
231 classical models such as DAGs. That was SimGrid v.1.* aka SG, written by
232 Henri Casanova. I had been using it in its earliest versions during an
235 Then we have realized that encoding distributed algorithm in SG was a
238 * So we have built MSG on top of SG and have released SimGrid v.2.*. MSG
239 offered a very basic API to encode a distributed application easily.
240 However encoding MSG on top of SG was not really convenient and did not
241 use the DAG part since the control of the task synchronization was done
242 on top of MSG and no more in SG. We have been playing a little bit with
243 MSG. We have realized that:
245 \li 1) the platform modeling was quite flexible and could be "almost"
246 automated (e.g. using random generator and post-annotations);
248 \li 2) SG was the bottleneck because of the way we were using
249 it. We needed to simulate concurrent transfers, complex load
250 sharing mechanisms. Many optimizations (e.g. trace integration)
251 were totally inefficient when combined with MSG and made extending SG
252 to implement new sharing policies, parallel tasks models, or failures
253 (many people were asking for these kind of features) a real pain;
255 \li 3) the application modeling was not really easy. Even though the
256 application modeling depends on people's applications, we thought
257 we could improve things here. One of our target here was realistic
258 distributed applications ranging from computer sensor networks like
259 the NWS to peer-to-peer applications;
261 * So we have been planning mainly two things for SimGrid 3:
263 \li 1) I have proposed to get rid of SG and to re-implement a new kernel
264 that would be faster and more flexible. That is what I did in the
265 end of 2004: SURF. SURF is based on a fast max-min linear solver
266 using O(1) data-structures. I have quickly replaced SG by SURF in
267 MSG and the result has been that on the MSG example, the new
268 version was more than 10 times faster while we had gain a lot of
269 flexibility. I think I could still easily make MSG faster but I
270 have to work on MSG now (e.g. using some of the O(1)
271 data-structures I've been using to build SURF) since it has become
272 the bottleneck. Some MSG functions have been removed from the API
273 but they were mainly intended to build the platform by hand (they
274 had appeared in the earliest versions of MSG) and were therefore
275 not useful anymore since we are providing a complete mechanism to
276 automatically build the platform and deploy the agents on it.;
278 \li 2) GRAS is a new project Martin and I have come up with. The idea is
279 to have a programming environment that let you program real
280 distributed applications while letting you the ability to run it in
281 the simulator without having to change the slightest line of your
282 code. From the simulation point of view, GRAS performs the
283 application modeling automatically... Up until now, GRAS works on
284 top MSG for historical reasons but I'm going to make it work
285 directly on top of SURF so that it can use all the flex and the
286 speed provided by SURF.
288 Those two things are working, but we want to make everything as clean as
289 possible before releasing SimGrid v.3.
291 So what about those nice DAGs we used to have in SimGrid v.1. ? They're not
292 anymore in SimGrid v.3. Let me recall you the way SimGrid 3 is organized:
306 XBT is our tool box and now, you should have an idea of what the other ones
307 are. As you can see, the primitive SG is not here anymore. However it could
308 still be brought back if people really need it. Here is how it would fit.
311 ______________________
313 |____________________|
314 | | MSG | GRAS | SG |
315 | -------------------|
317 | -------------------|
319 ----------------------
322 Re-implementing SG on top of SURF is really straightforward (it only
323 requires a little bit of time that I really don't have right now)
324 since the only thing that lacks to SURF is the DAG part. But adding it
325 to SURF would slow it down and therefore slow MSG and GRAS which is
326 not a good thing. However it is really not on the top of our TODO
327 list because we have to work on GRAS, and its MPI counterpart, and a
328 parallel task model, and ... Anyway, we finally have migrated our CVS
329 to gforge so people that are interested by helping on this part will
330 have the possibility to do it.
332 \subsection faq_SG_DAG But I wanted to implement a distributed dynamic scheduler of DAGs... How can I do that it SG is not available anymore in the next versions ?
334 Distributed is somehow "contagious". If you start making distributed
335 decisions, there is no way to handle DAGs directly anymore (unless I am
336 missing something). You have to encode your DAGs in term of communicating
337 process to make the whole scheduling process distributed. Believe me, it is
338 worth the effort since you'll then be able to try your algorithms in a very
339 wide variety of conditions.
341 If you decide that the distributed part is not that much important and that
342 DAG is really the level of abstraction you want to work with (but it
343 prevents you from having "realistic" platform modeling), then you should
344 keep using the 2.18.5 versions until somebody has ported SG on top of SURF.
345 Note however that SURF will be slower than the old SG to handle traces with
346 a lots of variations (there is no trace integration anymore).
348 \section faq_flexml_limit I get the message "surf_parse_lex: Assertion `next<limit' failed."
350 This is because your platform file is too big for the parser.
352 Actually, the message comes directly from FleXML, the technology on top of
353 which the parser is built. FleXML has the bad idea of fetching the whole
354 document in memory before parsing it. And moreover, the memory buffer size
355 must be determinded at compilation time.
357 We use a value which seems big enough for our need withour bloating the
358 simulators footprints. But of course your mileage may vary. In this case,
359 just edit src/surf/surfxml.l modify the definition of
360 FLEXML_BUFFERSTACKSIZE. E.g.
363 #define FLEXML_BUFFERSTACKSIZE 1000000000
366 Then recompile and everything should be fine, provided that your version of
367 Flex is recent enough (>= 2.5.31). If not the compilation process should
370 A while ago, we worked on FleXML to reduce a bit its memory consumtion, but
371 these issues remain. There is two things we should do:
373 - use a dynamic buffer instead of a static one so that the only limit
374 becomes your memory, not a stupid constant fixed at compilation time
375 (maybe not so difficult).
376 - change the parser so that it does not need to get the whole file in
377 memory before parsing
378 (seems quite difficult, but I'm a complete newbe wrt flex stuff).
380 These are changes to FleXML itself, not SimGrid. But since we kinda hijacked
381 the development of FleXML, I can grant you that any patches would be really
382 welcome and quickly integrated.
384 \section faq_host_load Where is the get_host_load function hidden in MSG?
386 There is no such thing because its semantic wouldn't be really clear. Of
387 course, it is something about the amount of host throughput, but there is as
388 many definition of "host load" as people asking for this function.
390 It may be instantaneous value or an average one. Moreover it may be only the
391 power of the computer, or may take the background load into account, or may
392 even take the currently running tasks into account. In some SURF models,
393 communications have an influence on computational power. Should it be taken
396 So, we decided not to include such a function into MSG and let people do it
397 thereselves so that they get the value matching exactly what they mean. One
398 possibility is to run active measurement as in next code snippet. It is very
399 close from what you would have to do out of the simulator, and thus gives
400 you information that you could also get in real settings to not hinder the
401 realism of your simulation.
404 double get_host_load() {
405 m_task_t task = MSG_task_create("test", 0.001, 0, NULL);
406 double date = MSG_get_clock();
408 MSG_task_execute(task);
409 date = MSG_get_clock() - date;
410 MSG_task_destroy(task);
415 Of course, it may not match your personal definition of "host load". In this
416 case, please detail what you mean on the mailing list, and we will extend
417 this FAQ section to fit your taste if possible.
421 \author Arnaud Legrand (arnaud.legran::imag.fr)
422 \author Martin Quinson (martin.quinson::loria.fr)