Remake
remake.cpp
Go to the documentation of this file.
1 /* -*- mode: C++; indent-tabs-mode: t; c-basic-offset: 8; -*- */
2 /**
3 @mainpage Remake, a build system that bridges the gap between make and redo.
4 
5 As with <b>make</b>, <b>remake</b> uses a centralized rule file, which is
6 named <b>Remakefile</b>. It contains rules with a <em>make</em>-like
7 syntax:
8 
9 @verbatim
10 target1 target2 ... : prerequisite1 prerequisite2 ...
11  shell script
12  that builds
13  the targets
14 @endverbatim
15 
16 A target is known to be up-to-date if all its prerequisites are. If it
17 has no known prerequisites yet the file already exits, it is assumed to
18 be up-to-date. Obsolete targets are rebuilt thanks to the shell script
19 provided by the rule.
20 
21 As with <b>redo</b>, <b>remake</b> supports dynamic dependencies in
22 addition to these static dependencies. Whenever a script executes
23 `remake prerequisite4 prerequisite5 ...`, these prerequisites are
24 rebuilt if they are obsolete. (So <b>remake</b> acts like
25 <b>redo-ifchange</b>.) Moreover, all the dependencies are stored in file
26 <b>.remake</b> so that they are remembered in subsequent runs. Note that
27 dynamic dependencies from previous runs are only used to decide whether a
28 target is obsolete; they are not automatically rebuilt when they are
29 obsolete yet a target depends on them. They will only be rebuilt once the
30 dynamic call to <b>remake</b> is executed.
31 
32 In other words, the following two rules have almost the same behavior.
33 
34 @verbatim
35 target1 target2 ... : prerequisite1 prerequisite2 ...
36  shell script
37 
38 target1 target2 ... :
39  remake prerequisite1 prerequisite2 ...
40  shell script
41 @endverbatim
42 
43 (There is a difference if the targets already exist, have never been
44 built before, and the prerequisites are either younger or obsolete, since
45 the targets will not be rebuilt in the second case.)
46 
47 The above usage of dynamic dependencies is hardly useful. Their strength
48 lies in the fact that they can be computed on the fly:
49 
50 @verbatim
51 %.o : %.c
52  gcc -MMD -MF $@.d -o $@ -c $<
53  remake -r < $@.d
54  rm $@.d
55 
56 %.cmo : %.ml
57  ocamldep $< | remake -r $@
58  ocamlc -c $<
59 
60 after.xml: before.xml rules.xsl
61  xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
62  remake `sed -n -e "\\,//,! s,^.*URL=\"\\([^\"]*\\).*\$,\\1,p" deps`
63  rm deps
64 @endverbatim
65 
66 Note that the first rule fails if any of the header files included by
67 a C source file has to be automatically generated. In that case, one
68 should perform a first call to <b>remake</b> them before calling the
69 compiler. (Dependencies from several calls to <b>remake</b> are
70 cumulative, so they will all be remembered the next time.)
71 
72 \section sec-usage Usage
73 
74 Usage: <tt>remake <i>options</i> <i>targets</i></tt>
75 
76 Options:
77 
78 - `-B`, `--always-make`: Unconditionally make all targets.
79 - `-d`: Echo script commands.
80 - `-f FILE`: Read `FILE` as <b>Remakefile</b>.
81 - `-j[N]`, `--jobs=[N]`: Allow `N` jobs at once;
82  infinite jobs with no argument.
83 - `-k`, `--keep-going`: Keep going when some targets cannot be made.
84 - `-r`: Look up targets from the dependencies on standard input.
85 - `-s`, `--silent`, `--quiet`: Do not echo targets.
86 
87 \section sec-syntax Syntax
88 
89 Lines starting with a space character or a tabulation are assumed to be rule
90 scripts. They are only allowed after a rule header.
91 
92 Lines starting with `#` are considered to be comments and are ignored.
93 They do interrupt rule scripts though.
94 
95 Any other line is either a variable definition or a rule header. If such a
96 line ends with a backslash, the following line break is ignored and the line
97 extends to the next one.
98 
99 Variable definitions are a single name followed by equal followed by a list
100 of names, possibly empty.
101 
102 Rule headers are a nonempty list of names, followed by a colon, followed by
103 another list of names, possibly empty. Basically, the syntax of a rule is as
104 follows:
105 
106 @verbatim
107 targets : prerequisites
108  shell script
109 @endverbatim
110 
111 List of names are space-separated sequences of names. If a name contains
112 a space character, it should be put into double quotes. Names cannot be
113 any of the following special characters `:$(),="`. Again, quotation
114 should be used. Quotation marks can be escaped by a backslash inside
115 quoted names.
116 
117 \subsection sec-variables Variables
118 
119 Variables can be used to factor lists of targets or prerequisites. They are
120 expanded as they are encountered during <b>Remakefile</b> parsing.
121 
122 @verbatim
123 VAR2 = a
124 VAR1 = c d
125 VAR2 += $(VAR1) b
126 $(VAR2) e :
127 @endverbatim
128 
129 Variable assignments can appear instead of prerequisites inside non-generic
130 rules with no script. They are then expanded inside the corresponding
131 generic rule.
132 
133 @verbatim
134 foo.o: CFLAGS += -DBAR
135 
136 %.o : %.c
137  gcc $(CFLAGS) -MMD -MF $@.d -o $@ -c $<
138  remake -r < $@.d
139  rm $@.d
140 @endverbatim
141 
142 Note: contrarily to <b>make</b>, variable names have to be enclosed in
143 parentheses. For instance, `$y` is not a shorthand for <tt>\$(y)</tt> and
144 is left unexpanded.
145 
146 \subsection sec-autovars Automatic variables
147 
148 The following special symbols can appear inside scripts:
149 
150 - `$<` expands to the first static prerequisite of the rule.
151 - `$^` expands to all the static prerequisites of the rule, including
152  duplicates if any.
153 - `$@` expands to the first target of the rule.
154 - `$*` expands to the string that matched `%` in a generic rule.
155 - `$$` expands to a single dollar symbol.
156 
157 Note: contrarily to <b>make</b>, there are no corresponding variables.
158 For instance, `$^` is not a shorthand for `$(^)`. Another difference is
159 that `$@` is always the first target, not the one that triggered the
160 rule.
161 
162 \subsection sec-functions Built-in functions
163 
164 <b>remake</b> also supports a few built-in functions inspired from <b>make</b>.
165 
166 - <tt>$(addprefix <i>prefix</i>, <i>list</i>)</tt> returns the list obtained
167  by prepending its first argument to each element of its second argument.
168 - <tt>$(addsuffix <i>suffix</i>, <i>list</i>)</tt> returns the list obtained
169  by appending its first argument to each element of its second argument.
170 
171 \subsection sec-order Order-only prerequisites
172 
173 If the static prerequisites of a rule contain a pipe symbol, prerequisites
174 on its right do not cause the targets to become obsolete if they are newer
175 (unless they are also dynamically registered as dependencies). They are
176 meant to be used when the targets do not directly depend on them, but the
177 computation of their dynamic dependencies does.
178 
179 @verbatim
180 %.o : %.c | parser.h
181  gcc -MMD -MF $@.d -o $@ -c $<
182  remake -r < $@.d
183  rm $@.d
184 
185 parser.c parser.h: parser.y
186  yacc -d -o parser.c parser.y
187 @endverbatim
188 
189 \subsection sec-static-pattern Static pattern rules
190 
191 A rule with the following structure is expanded into several rules, one
192 per target.
193 
194 @verbatim
195 targets: pattern1 pattern2 ...: prerequisites
196 @endverbatim
197 
198 Every target is matched against one of the patterns containing the `%`
199 character. A rule is then created using the patterns as targets, after
200 having substituted `%` in the patterns and prerequisites. The automatic
201 variable `$*` can be used in the script of the rule.
202 
203 \subsection sec-special-tgt Special targets
204 
205 Target `.PHONY` marks its prerequisites as being always obsolete.
206 
207 \subsection sec-special-var Special variables
208 
209 Variable `.OPTIONS` is handled specially. Its content enables some
210 features of <b>remake</b> that are not enabled by default.
211 
212 - `variable-propagation`: When a variable is set in the prerequisite
213  part of a rule, it is propagated to the rules of all the targets this rule
214  depends on. This option also enables variables to be set on the command
215  line. Note that, as in <b>make</b>, this features introduces non-determinism:
216  the content of some variables will depend on the build order.
217 
218 \section sec-semantics Semantics
219 
220 \subsection src-obsolete When are targets obsolete?
221 
222 A target is obsolete:
223 
224 - if there is no file corresponding to the target, or to one of its siblings
225  in a multi-target rule,
226 - if any of its dynamic prerequisites from a previous run or any of its static
227  prerequisites is obsolete,
228 - if the latest file corresponding to its siblings or itself is older than any
229  of its dynamic prerequisites or static prerequisites.
230 
231 In all the other cases, it is assumed to be up-to-date (and so are all its
232 siblings). Note that the last rule above says "latest" and not "earliest". While
233 it might cause some obsolete targets to go unnoticed in corner cases, it allows
234 for the following kind of rules:
235 
236 @verbatim
237 config.h stamp-config_h: config.h.in config.status
238  ./config.status config.h
239  touch stamp-config_h
240 @endverbatim
241 
242 A `config.status` file generally does not update header files (here
243 `config.h`) if they would not change. As a consequence, if not for the
244 `stamp-config_h` file above, a header would always be considered obsolete
245 once one of its prerequisites is modified. Note that touching `config.h`
246 rather than `stamp-config_h` would defeat the point of not updating it in
247 the first place, since the program files would need to be rebuilt.
248 
249 Once all the static prerequisites of a target have been rebuilt, <b>remake</b>
250 checks whether the target still needs to be built. If it was obsolete only
251 because its prerequisites needed to be rebuilt and none of them changed, the
252 target is assumed to be up-to-date.
253 
254 \subsection sec-rules How are targets (re)built?
255 
256 There are two kinds of rules. If any of the targets or prerequisites contains
257 a `%` character, the rule is said to be <em>generic</em>. All the
258 targets of the rule shall then contain a single `%` character. All the
259 other rules are said to be <em>specific</em>.
260 
261 A rule is said to <em>match</em> a given target:
262 
263 - if it is specific and the target appears inside its target list,
264 - if it is generic and there is a way to replace the `%` character
265  from one of its targets so that it matches the given target.
266 
267 When <b>remake</b> tries to build a given target, it looks for a specific rule
268 that matches it. If there is one and its script is nonempty, it uses it to
269 rebuild the target.
270 
271 Otherwise, it looks for a generic rule that matches the target. If there are
272 several matching rules, it chooses the one with the shortest pattern (and if
273 there are several ones, the earliest one). It then looks for specific rules
274 that match each target of the generic rule. All the prerequisites of these
275 specific rules are added to those of the generic rule. The script of the
276 generic rule is used to build the target.
277 
278 Example:
279 
280 @verbatim
281 t%1 t2%: p1 p%2
282  commands building t%1 and t2%
283 
284 t2z: p4
285  commands building t2z
286 
287 ty1: p3
288 
289 # t2x is built by the first rule (which also builds tx1) and its prerequisites are p1, px2
290 # t2y is built by the first rule (which also builds ty1) and its prerequisites are p1, py2, p3
291 # t2z is built by the second rule and its prerequisite is p4
292 @endverbatim
293 
294 The set of rules from <b>Remakefile</b> is ill-formed:
295 
296 - if any specific rule matching a target of the generic rule has a nonempty script,
297 - if any target of the generic rule is matched by a generic rule with a shorter pattern.
298 
299 \section sec-compilation Compilation
300 
301 - On Linux, MacOSX, and BSD: `g++ -o remake remake.cpp`
302 - On Windows: `g++ -o remake.exe remake.cpp -lws2_32`
303 
304 Installing <b>remake</b> is needed only if <b>Remakefile</b> does not
305 specify the path to the executable for its recursive calls. Thanks to its
306 single source file, <b>remake</b> can be shipped inside other packages and
307 built at configuration time.
308 
309 \section sec-differences Differences with other build systems
310 
311 Differences with <b>make</b>:
312 
313 - Dynamic dependencies are supported.
314 - For rules with multiple targets, the shell script is executed only once
315  and is assumed to build all the targets. There is no need for
316  convoluted rules that are robust enough for parallel builds. For generic
317  rules, this is similar to the behavior of pattern rules from <b>gmake</b>.
318 - As with <b>redo</b>, only one shell is run when executing a script,
319  rather than one per script line. Note that the shells are run with
320  option `-e`, thus causing them to exit as soon as an error is
321  encountered.
322 - The prerequisites of generic rules (known as implicit rules in <b>make</b>
323  lingo) are not used to decide between several of them, which means that
324  <b>remake</b> does not select one for which it could satisfy the dependencies.
325 - Variables and built-in functions are expanded as they are encountered
326  during <b>Remakefile</b> parsing.
327 - Target-specific variables are not propagated, unless specifically enabled,
328  since this causes non-deterministic builds. This is the same for variables
329  set on the command line.
330 
331 Differences with <b>redo</b>:
332 
333 - As with <b>make</b>, it is possible to write the following kind of rules
334  in <b>remake</b>.
335 @verbatim
336 Remakefile: Remakefile.in ./config.status
337  ./config.status Remakefile
338 @endverbatim
339 - If a target is already built the first time <b>remake</b> runs, it still
340  uses the static prerequisites of rules mentioning it to check whether it
341  needs to be rebuilt. It does not assume it to be up-to-date. As with
342  <b>redo</b> though, if its obsolete status would be due to a dynamic
343  prerequisite, it will go unnoticed; it should be removed beforehand.
344 - Multiple targets are supported.
345 - <b>remake</b> has almost no features: no checksum-based dependencies, no
346  compatibility with job servers, etc.
347 
348 \section sec-limitations Limitations
349 
350 - If a rule script calls <b>remake</b>, the current working directory should
351  be the directory containing <b>Remakefile</b> (or the working directory
352  from the original <b>remake</b> if it was called with option `-f`).
353 - As with <b>make</b>, variables passed on the command line should keep
354  the same values, to ensure deterministic builds.
355 - Some cases of ill-formed rules are not caught by <b>remake</b> and can
356  thus lead to unpredictable behaviors.
357 
358 \section sec-links Links
359 
360 @see http://cr.yp.to/redo.html for the philosophy of <b>redo</b> and
361 https://github.com/apenwarr/redo for an implementation and some comprehensive documentation.
362 
363 \section sec-licensing Licensing
364 
365 @author Guillaume Melquiond
366 @version 0.14
367 @date 2012-2020
368 @copyright
369 This program is free software: you can redistribute it and/or modify
370 it under the terms of the GNU General Public License as published by
371 the Free Software Foundation, either version 3 of the License, or
372 (at your option) any later version.
373 \n
374 This program is distributed in the hope that it will be useful,
375 but WITHOUT ANY WARRANTY; without even the implied warranty of
376 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
377 GNU General Public License for more details.
378 
379 \section sec-internals Internals
380 
381 The parent <b>remake</b> process acts as a server. The other ones have a
382 REMAKE_SOCKET environment variable that tells them how to contact the
383 server. They send the content of the REMAKE_JOB_ID environment variable,
384 so that the server can associate the child targets to the jobs that
385 spawned them. They then wait for completion and exit with the status
386 returned by the server. This is handled by #client_mode.
387 
388 The server calls #load_dependencies and #save_dependencies to serialize
389 dynamic dependencies from <b>.remake</b>. It loads <b>Remakefile</b> with
390 #load_rules. It then runs #server_mode, which calls #server_loop.
391 
392 When building a target, the following sequence of events happens:
393 
394 - #start calls #find_rule (and #find_generic_rule) to get the rule.
395 - It then creates a pseudo-client if the rule has static dependencies, or
396  calls #run_script otherwise. In both cases, a new job is created; the
397  rule and the variables are stored into #jobs.
398 - #run_script creates a shell process and stores it in #job_pids. It
399  increases #running_jobs.
400 - The child process possibly calls <b>remake</b> with a list of targets.
401 - #accept_client receives a build request from a child process and adds
402  it to #clients. It also records the new dependencies of the job into
403  #dependencies. It increases #waiting_jobs.
404 - #handle_clients uses #get_status to look up the obsoleteness of the
405  targets.
406 - Once the targets of a request have been built or one of them has failed,
407  #handle_clients calls #complete_request and removes the request from
408  #clients.
409 - If the build targets come from a pseudo-client, #complete_request calls
410  #run_script. Otherwise it sends the reply to the corresponding child
411  process and decreases #waiting_jobs.
412 - When a child process ends, #server_loop calls #finalize_job, which
413  removes the process from #job_pids, decreases #running_jobs, and calls
414  #complete_job.
415 - #complete_job removes the job from #jobs and calls #update_status
416  to change the status of the targets. It also removes the target files in
417  case of failure.
418 */
419 
420 #ifdef _WIN32
421 #define WIN32_LEAN_AND_MEAN
422 #define WINDOWS
423 #endif
424 
425 #include <fstream>
426 #include <iostream>
427 #include <list>
428 #include <map>
429 #include <set>
430 #include <sstream>
431 #include <string>
432 #include <vector>
433 #include <cassert>
434 #include <cstdlib>
435 #include <cstring>
436 #include <ctime>
437 #include <errno.h>
438 #include <fcntl.h>
439 #include <signal.h>
440 #include <unistd.h>
441 #include <sys/stat.h>
442 #include <sys/types.h>
443 
444 #ifdef __APPLE__
445 #define MACOSX
446 #endif
447 
448 #ifdef __linux__
449 #define LINUX
450 #endif
451 
452 #ifdef WINDOWS
453 #include <windows.h>
454 #include <winbase.h>
455 #include <winsock2.h>
456 #define pid_t HANDLE
457 typedef SOCKET socket_t;
458 #else
459 #include <sys/socket.h>
460 #include <sys/un.h>
461 #include <sys/wait.h>
462 typedef int socket_t;
463 enum { INVALID_SOCKET = -1 };
464 extern char **environ;
465 #endif
466 
467 #if defined(WINDOWS) || defined(MACOSX)
468 enum { MSG_NOSIGNAL = 0 };
469 #endif
470 
471 typedef std::list<std::string> string_list;
472 
473 typedef std::set<std::string> string_set;
474 
475 /**
476  * Reference-counted shared object.
477  * @note The default constructor delays the creation of the object until it
478  * is first dereferenced.
479  */
480 template<class T>
481 struct ref_ptr
482 {
483  struct content
484  {
485  size_t cnt;
486  T val;
487  content(): cnt(1) {}
488  content(T const &t): cnt(1), val(t) {}
489  };
490  mutable content *ptr;
491  ref_ptr(): ptr(NULL) {}
492  ref_ptr(T const &t): ptr(new content(t)) {}
493  ref_ptr(ref_ptr const &p): ptr(p.ptr) { if (ptr) ++ptr->cnt; }
494  ~ref_ptr() { if (ptr && --ptr->cnt == 0) delete ptr; }
496  {
497  if (ptr == p.ptr) return *this;
498  if (ptr && --ptr->cnt == 0) delete ptr;
499  ptr = p.ptr;
500  if (ptr) ++ptr->cnt;
501  return *this;
502  }
503  T &operator*() const
504  {
505  if (!ptr) ptr = new content;
506  return ptr->val;
507  }
508  T *operator->() const { return &**this; }
509 };
510 
512 {
515 };
516 
517 typedef std::map<std::string, ref_ptr<dependency_t> > dependency_map;
518 
519 typedef std::map<std::string, string_list> variable_map;
520 
521 /**
522  * Build status of a target.
523  */
525 {
526  Uptodate, ///< Target is up-to-date.
527  Todo, ///< Target is missing or obsolete.
528  Recheck, ///< Target has an obsolete dependency.
529  Running, ///< Target is being rebuilt.
530  RunningRecheck, ///< Static prerequisites are being rebuilt.
531  Remade, ///< Target was successfully rebuilt.
532  Failed ///< Build failed for target.
533 };
534 
535 /**
536  * Build status of a target.
537  */
538 struct status_t
539 {
540  status_e status; ///< Actual status.
541  time_t last; ///< Last-modified date.
542 };
543 
544 typedef std::map<std::string, status_t> status_map;
545 
546 /**
547  * Delayed assignment to a variable.
548  */
549 struct assign_t
550 {
551  bool append;
553 };
554 
555 typedef std::map<std::string, assign_t> assign_map;
556 
557 /**
558  * A rule loaded from Remakefile.
559  */
560 struct rule_t
561 {
562  string_list targets; ///< Files produced by this rule.
563  string_list deps; ///< Dependencies used for an implicit call to remake at the start of the script.
564  string_list wdeps; ///< Like #deps, except that they are not registered as dependencies.
565  assign_map assigns; ///< Assignment of variables.
566  std::string stem; ///< Stem used to instantiate the rule, if any.
567  std::string script; ///< Shell script for building the targets.
568 };
569 
570 typedef std::list<rule_t> rule_list;
571 
572 typedef std::map<std::string, ref_ptr<rule_t> > rule_map;
573 
574 /**
575  * A job created from a set of rules.
576  */
577 
578 struct job_t
579 {
580  rule_t rule; ///< Original rule.
581  variable_map vars; ///< Values of local variables.
582 };
583 
584 typedef std::map<int, job_t> job_map;
585 
586 typedef std::map<pid_t, int> pid_job_map;
587 
588 /**
589  * Client waiting for a request to complete.
590  *
591  * There are two kinds of clients:
592  * - real clients, which are instances of remake created by built scripts,
593  * - pseudo clients, which are created by the server to build specific targets.
594  *
595  * Among pseudo clients, there are two categories:
596  * - original clients, which are created for the targets passed on the
597  * command line by the user or for the initial regeneration of the rule file,
598  * - dependency clients, which are created to handle rules that have
599  * explicit dependencies and thus to emulate a call to remake.
600  */
601 struct client_t
602 {
603  socket_t socket; ///< Socket used to reply to the client (invalid for pseudo clients).
604  int job_id; ///< Job for which the built script called remake and spawned the client (negative for original clients).
605  bool failed; ///< Whether some targets failed in mode -k.
606  string_list pending; ///< Targets not yet started.
607  string_set running; ///< Targets being built.
608  variable_map vars; ///< Variables set on request.
609  bool delayed; ///< Whether it is a dependency client and a script has to be started on request completion.
610  client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(false) {}
611 };
612 
613 typedef std::list<client_t> client_list;
614 
615 /**
616  * Map from variable names to their content.
617  * Initialized with the values passed on the command line.
618  */
620 
621 /**
622  * Map from targets to their known dependencies.
623  */
625 
626 /**
627  * Map from targets to their build status.
628  */
630 
631 /**
632  * Set of generic rules loaded from Remakefile.
633  */
635 
636 /**
637  * Map from targets to specific rules loaded from Remakefile.
638  */
640 
641 /**
642  * Map of jobs being built.
643  */
644 static job_map jobs;
645 
646 /**
647  * Map from jobs to shell pids.
648  */
650 
651 /**
652  * List of clients waiting for a request to complete.
653  * New clients are put to front, so that the build process is depth-first.
654  */
656 
657 /**
658  * Maximum number of parallel jobs (non-positive if unbounded).
659  * Can be modified by the -j option.
660  */
661 static int max_active_jobs = 1;
662 
663 /**
664  * Whether to keep building targets in case of failure.
665  * Can be modified by the -k option.
666  */
667 static bool keep_going = false;
668 
669 /**
670  * Number of jobs currently running:
671  * - it increases when a process is created in #run_script,
672  * - it decreases when a completion message is received in #finalize_job.
673  *
674  * @note There might be some jobs running while #clients is empty.
675  * Indeed, if a client requested two targets to be rebuilt, if they
676  * are running concurrently, if one of them fails, the client will
677  * get a failure notice and might terminate before the other target
678  * finishes.
679  */
680 static int running_jobs = 0;
681 
682 /**
683  * Number of jobs currently waiting for a build request to finish:
684  * - it increases when a build request is received in #accept_client
685  * (since the client is presumably waiting for the reply),
686  * - it decreases when a reply is sent in #complete_request.
687  */
688 static int waiting_jobs = 0;
689 
690 /**
691  * Global counter used to produce increasing job numbers.
692  * @see jobs
693  */
694 static int job_counter = 0;
695 
696 /**
697  * Socket on which the server listens for client request.
698  */
700 
701 /**
702  * Whether the request of an original client failed.
703  */
704 static bool build_failure;
705 
706 #ifndef WINDOWS
707 /**
708  * Name of the server socket in the file system.
709  */
710 static char *socket_name;
711 #endif
712 
713 /**
714  * Name of the first target of the first specific rule, used for default run.
715  */
716 static std::string first_target;
717 
718 /**
719  * Whether a short message should be displayed for each target.
720  */
721 static bool show_targets = true;
722 
723 /**
724  * Whether script commands are echoed.
725  */
726 static bool echo_scripts = false;
727 
728 /**
729  * Time at the start of the program.
730  */
731 static time_t now = time(NULL);
732 
733 /**
734  * Directory with respect to which command-line names are relative.
735  */
736 static std::string working_dir;
737 
738 /**
739  * Directory with respect to which targets are relative.
740  */
741 static std::string prefix_dir;
742 
743 /**
744  * Whether the prefix directory is different from #working_dir.
745  */
746 static bool changed_prefix_dir;
747 
748 /**
749  * Whether target-specific variables are propagated to prerequisites.
750  */
751 static bool propagate_vars = false;
752 
753 /**
754  * Whether targets are unconditionally obsolete.
755  */
756 static bool obsolete_targets = false;
757 
758 #ifndef WINDOWS
759 static sigset_t old_sigmask;
760 
761 static volatile sig_atomic_t got_SIGCHLD = 0;
762 
763 static void sigchld_handler(int)
764 {
765  got_SIGCHLD = 1;
766 }
767 
768 static void sigint_handler(int)
769 {
770  // Child processes will receive the signal too, so just prevent
771  // new jobs from starting and wait for the running jobs to fail.
772  keep_going = false;
773 }
774 #endif
775 
776 struct log
777 {
778  bool active, open;
779  int depth;
780  log(): active(false), open(false), depth(0)
781  {
782  }
783  std::ostream &operator()()
784  {
785  if (open) std::cerr << std::endl;
786  assert(depth >= 0);
787  std::cerr << std::string(depth * 2, ' ');
788  open = false;
789  return std::cerr;
790  }
791  std::ostream &operator()(bool o)
792  {
793  if (o && open) std::cerr << std::endl;
794  if (!o) --depth;
795  assert(depth >= 0);
796  if (o || !open) std::cerr << std::string(depth * 2, ' ');
797  if (o) ++depth;
798  open = o;
799  return std::cerr;
800  }
801 };
802 
803 static log debug;
804 
806 {
809  {
810  }
812  {
813  if (debug.active && still_open) debug(false) << "done\n";
814  }
815 };
816 
817 #define DEBUG if (debug.active) debug()
818 #define DEBUG_open log_auto_close auto_close; if (debug.active) debug(true)
819 #define DEBUG_close if ((auto_close.still_open = false), debug.active) debug(false)
820 
821 /**
822  * Strong typedef for strings that need escaping.
823  * @note The string is stored as a reference, so the constructed object is
824  * meant to be immediately consumed.
825  */
827 {
828  std::string const &input;
829  escape_string(std::string const &s): input(s) {}
830 };
831 
832 /**
833  * Write the string in @a se to @a out if it does not contain any special
834  * characters, a quoted and escaped string otherwise.
835  */
836 static std::ostream &operator<<(std::ostream &out, escape_string const &se)
837 {
838  std::string const &s = se.input;
839  char const *quoted_char = ",: '";
840  char const *escaped_char = "\"\\$!";
841  bool need_quotes = false;
842  char *buf = NULL;
843  size_t len = s.length(), last = 0, j = 0;
844  for (size_t i = 0; i < len; ++i)
845  {
846  if (strchr(escaped_char, s[i]))
847  {
848  need_quotes = true;
849  if (!buf) buf = new char[len * 2];
850  memcpy(&buf[j], &s[last], i - last);
851  j += i - last;
852  buf[j++] = '\\';
853  buf[j++] = s[i];
854  last = i + 1;
855  }
856  if (!need_quotes && strchr(quoted_char, s[i]))
857  need_quotes = true;
858  }
859  if (!need_quotes) return out << s;
860  out << '"';
861  if (!buf) return out << s << '"';
862  out.write(buf, j);
863  out.write(&s[last], len - last);
864  delete[] buf;
865  return out << '"';
866 }
867 
868 /**
869  * @defgroup paths Path helpers
870  *
871  * @{
872  */
873 
874 /**
875  * Initialize #working_dir.
876  */
877 static void init_working_dir()
878 {
879  char buf[1024];
880  char *res = getcwd(buf, sizeof(buf));
881  if (!res)
882  {
883  perror("Failed to get working directory");
884  exit(EXIT_FAILURE);
885  }
886  working_dir = buf;
887 #ifdef WINDOWS
888  for (size_t i = 0, l = working_dir.size(); i != l; ++i)
889  {
890  if (working_dir[i] == '\\') working_dir[i] = '/';
891  }
892 #endif
894 }
895 
896 /**
897  * Initialize #prefix_dir and switch to it.
898  */
899 static void init_prefix_dir()
900 {
901  for (;;)
902  {
903  struct stat s;
904  if (stat((prefix_dir + "/Remakefile").c_str(), &s) == 0)
905  {
906  if (!changed_prefix_dir) return;
907  if (chdir(prefix_dir.c_str()))
908  {
909  perror("Failed to change working directory");
910  exit(EXIT_FAILURE);
911  }
912  if (show_targets)
913  {
914  std::cout << "remake: Entering directory `" << prefix_dir << '\'' << std::endl;
915  }
916  return;
917  }
918  size_t pos = prefix_dir.find_last_of('/');
919  if (pos == std::string::npos)
920  {
921  std::cerr << "Failed to locate Remakefile in the current directory or one of its parents" << std::endl;
922  exit(EXIT_FAILURE);
923  }
924  prefix_dir.erase(pos);
925  changed_prefix_dir = true;
926  }
927 }
928 
929 /**
930  * Normalize an absolute path with respect to @a p.
931  * Paths outside the subtree are left unchanged.
932  */
933 static std::string normalize_abs(std::string const &s, std::string const &p)
934 {
935  size_t l = p.length();
936  if (s.compare(0, l, p)) return s;
937  size_t ll = s.length();
938  if (ll == l) return ".";
939  if (s[l] != '/')
940  {
941  size_t pos = s.rfind('/', l);
942  assert(pos != std::string::npos);
943  return s.substr(pos + 1);
944  }
945  if (ll == l + 1) return ".";
946  return s.substr(l + 1);
947 }
948 
949 /**
950  * Normalize path @a s (possibly relative to @a w) with respect to @a p.
951  *
952  * - If both @a p and @a w are empty, the function just removes ".", "..", "//".
953  * - If only @a p is empty, the function returns an absolute path.
954  */
955 static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
956 {
957 #ifdef WINDOWS
958  char const *delim = "/\\";
959 #else
960  char delim = '/';
961 #endif
962  size_t pos = s.find_first_of(delim);
963  if (pos == std::string::npos && w == p) return s;
964  bool absolute = pos == 0;
965  if (!absolute && w != p && !w.empty())
966  return normalize(w + '/' + s, w, p);
967  size_t prev = 0, len = s.length();
968  string_list l;
969  for (;;)
970  {
971  if (pos != prev)
972  {
973  std::string n = s.substr(prev, pos - prev);
974  if (n == "..")
975  {
976  if (!l.empty()) l.pop_back();
977  else if (!absolute && !w.empty())
978  return normalize(w + '/' + s, w, p);
979  }
980  else if (n != ".")
981  l.push_back(n);
982  }
983  ++pos;
984  if (pos >= len) break;
985  prev = pos;
986  pos = s.find_first_of(delim, prev);
987  if (pos == std::string::npos) pos = len;
988  }
989  string_list::const_iterator i = l.begin(), i_end = l.end();
990  if (i == i_end) return absolute ? "/" : ".";
991  std::string n;
992  if (absolute) n.push_back('/');
993  n.append(*i);
994  for (++i; i != i_end; ++i)
995  {
996  n.push_back('/');
997  n.append(*i);
998  }
999  if (absolute && !p.empty()) return normalize_abs(n, p);
1000  return n;
1001 }
1002 
1003 /**
1004  * Normalize the content of a list of targets.
1005  */
1006 static void normalize_list(string_list &l, std::string const &w, std::string const &p)
1007 {
1008  for (string_list::iterator i = l.begin(),
1009  i_end = l.end(); i != i_end; ++i)
1010  {
1011  *i = normalize(*i, w, p);
1012  }
1013 }
1014 
1015 /** @} */
1016 
1017 /**
1018  * @defgroup lexer Lexer
1019  *
1020  * @{
1021  */
1022 
1023 /**
1024  * Skip spaces.
1025  */
1026 static void skip_spaces(std::istream &in)
1027 {
1028  char c;
1029  while (strchr(" \t", (c = in.get()))) {}
1030  if (in.good()) in.putback(c);
1031 }
1032 
1033 /**
1034  * Skip empty lines.
1035  */
1036 static void skip_empty(std::istream &in)
1037 {
1038  char c;
1039  while (strchr("\r\n", (c = in.get()))) {}
1040  if (in.good()) in.putback(c);
1041 }
1042 
1043 /**
1044  * Skip end of line. If @a multi is true, skip the following empty lines too.
1045  * @return true if there was a line to end.
1046  */
1047 static bool skip_eol(std::istream &in, bool multi = false)
1048 {
1049  char c = in.get();
1050  if (c == '\r') c = in.get();
1051  if (c != '\n' && in.good()) in.putback(c);
1052  if (c != '\n' && !in.eof()) return false;
1053  if (multi) skip_empty(in);
1054  return true;
1055 }
1056 
1057 enum
1058 {
1060  Word = 1 << 1,
1061  Colon = 1 << 2,
1062  Equal = 1 << 3,
1063  Dollarpar = 1 << 4,
1064  Rightpar = 1 << 5,
1065  Comma = 1 << 6,
1066  Plusequal = 1 << 7,
1067  Pipe = 1 << 8,
1068 };
1069 
1070 /**
1071  * Skip spaces and peek at the next token.
1072  * If it is one of @a mask, skip it (if it is not Word) and return it.
1073  * @note For composite tokens allowed by @a mask, input characters might
1074  * have been eaten even for an Unexpected result.
1075  */
1076 static int expect_token(std::istream &in, int mask)
1077 {
1078  while (true)
1079  {
1080  skip_spaces(in);
1081  char c = in.peek();
1082  if (!in.good()) return Unexpected;
1083  int tok;
1084  switch (c)
1085  {
1086  case '\r':
1087  case '\n': return Unexpected;
1088  case ':': tok = Colon; break;
1089  case ',': tok = Comma; break;
1090  case '=': tok = Equal; break;
1091  case ')': tok = Rightpar; break;
1092  case '|': tok = Pipe; break;
1093  case '$':
1094  if (!(mask & Dollarpar)) return Unexpected;
1095  in.ignore(1);
1096  tok = Dollarpar;
1097  if (in.peek() != '(') return Unexpected;
1098  break;
1099  case '+':
1100  if (!(mask & Plusequal)) return Unexpected;
1101  in.ignore(1);
1102  tok = Plusequal;
1103  if (in.peek() != '=') return Unexpected;
1104  break;
1105  case '\\':
1106  in.ignore(1);
1107  if (skip_eol(in)) continue;
1108  in.putback('\\');
1109  return mask & Word ? Word : Unexpected;
1110  default:
1111  return mask & Word ? Word : Unexpected;
1112  }
1113  if (!(tok & mask)) return Unexpected;
1114  in.ignore(1);
1115  return tok;
1116  }
1117 }
1118 
1119 /**
1120  * Read a (possibly quoted) word.
1121  */
1122 static std::string read_word(std::istream &in, bool detect_equal = true)
1123 {
1124  int c = in.peek();
1125  std::string res;
1126  if (!in.good()) return res;
1127  char const *separators = " \t\r\n$(),:";
1128  bool quoted = c == '"';
1129  if (quoted) in.ignore(1);
1130  bool plus = false;
1131  while (true)
1132  {
1133  c = in.peek();
1134  if (!in.good()) return res;
1135  if (quoted)
1136  {
1137  in.ignore(1);
1138  if (c == '\\')
1139  res += in.get();
1140  else if (c == '"')
1141  quoted = false;
1142  else
1143  res += c;
1144  continue;
1145  }
1146  if (detect_equal && c == '=')
1147  {
1148  if (plus) in.putback('+');
1149  return res;
1150  }
1151  if (plus)
1152  {
1153  res += '+';
1154  plus = false;
1155  }
1156  if (strchr(separators, c)) return res;
1157  in.ignore(1);
1158  if (detect_equal && c == '+') plus = true;
1159  else res += c;
1160  }
1161 }
1162 
1163 /** @} */
1164 
1165 /**
1166  * @defgroup stream Token streams
1167  *
1168  * @{
1169  */
1170 
1171 /**
1172  * Possible results from word producers.
1173  */
1175 {
1179 };
1180 
1181 /**
1182  * Interface for word producers.
1183  */
1185 {
1186  virtual ~generator() {}
1187  virtual input_status next(std::string &) = 0;
1188 };
1189 
1190 /**
1191  * Generator for the words of a variable.
1192  */
1194 {
1195  std::string name;
1196  string_list::const_iterator vcur, vend;
1197  variable_generator(std::string const &, variable_map const *);
1198  input_status next(std::string &);
1199 };
1200 
1202  variable_map const *local_variables): name(n)
1203 {
1204  if (local_variables)
1205  {
1206  variable_map::const_iterator i = local_variables->find(name);
1207  if (i != local_variables->end())
1208  {
1209  vcur = i->second.begin();
1210  vend = i->second.end();
1211  return;
1212  }
1213  }
1214  variable_map::const_iterator i = variables.find(name);
1215  if (i == variables.end()) return;
1216  vcur = i->second.begin();
1217  vend = i->second.end();
1218 }
1219 
1221 {
1222  if (vcur != vend)
1223  {
1224  res = *vcur;
1225  ++vcur;
1226  return Success;
1227  }
1228  return Eof;
1229 }
1230 
1231 /**
1232  * Generator for the words of an input stream.
1233  */
1235 {
1236  std::istream &in;
1240  input_generator(std::istream &i, variable_map const *lv, bool e = false)
1241  : in(i), nested(NULL), local_variables(lv), earliest_exit(e), done(false) {}
1242  input_status next(std::string &);
1243  ~input_generator() { assert(!nested); }
1244 };
1245 
1246 static generator *get_function(input_generator const &, std::string const &);
1247 
1249 {
1250  if (nested)
1251  {
1252  restart:
1253  input_status s = nested->next(res);
1254  if (s == Success) return Success;
1255  delete nested;
1256  nested = NULL;
1257  if (s == SyntaxError) return SyntaxError;
1258  }
1259  if (done) return Eof;
1260  if (earliest_exit) done = true;
1261  switch (expect_token(in, Word | Dollarpar))
1262  {
1263  case Word:
1264  res = read_word(in, false);
1265  return Success;
1266  case Dollarpar:
1267  {
1268  std::string name = read_word(in, false);
1269  if (name.empty()) return SyntaxError;
1270  if (expect_token(in, Rightpar))
1272  else
1273  {
1274  nested = get_function(*this, name);
1275  if (!nested) return SyntaxError;
1276  }
1277  goto restart;
1278  }
1279  default:
1280  return Eof;
1281  }
1282 }
1283 
1284 /**
1285  * Read a list of words from an input generator.
1286  * @return false if a syntax error was encountered.
1287  */
1289 {
1290  while (true)
1291  {
1292  res.push_back(std::string());
1293  input_status s = in.next(res.back());
1294  if (s == Success) continue;
1295  res.pop_back();
1296  return s == Eof;
1297  }
1298 }
1299 
1300 static bool read_words(std::istream &in, string_list &res)
1301 {
1302  input_generator gen(in, NULL);
1303  return read_words(gen, res);
1304 }
1305 
1306 /**
1307  * Generator for the result of function addprefix.
1308  */
1310 {
1313  string_list::const_iterator prei;
1314  size_t prej, prel;
1315  std::string suf;
1316  addprefix_generator(input_generator const &, bool &);
1317  input_status next(std::string &);
1318 };
1319 
1321  : gen(top.in, top.local_variables)
1322 {
1323  if (!read_words(gen, pre)) return;
1324  if (!expect_token(gen.in, Comma)) return;
1325  prej = 0;
1326  prel = pre.size();
1327  ok = true;
1328 }
1329 
1331 {
1332  if (prej)
1333  {
1334  produce:
1335  if (prej == prel)
1336  {
1337  res = *prei + suf;
1338  prej = 0;
1339  }
1340  else
1341  {
1342  res = *prei++;
1343  ++prej;
1344  }
1345  return Success;
1346  }
1347  switch (gen.next(res))
1348  {
1349  case Success:
1350  if (!prel) return Success;
1351  prei = pre.begin();
1352  prej = 1;
1353  suf = res;
1354  goto produce;
1355  case Eof:
1356  return expect_token(gen.in, Rightpar) ? Eof : SyntaxError;
1357  default:
1358  return SyntaxError;
1359  }
1360 }
1361 
1362 /**
1363  * Generator for the result of function addsuffix.
1364  */
1366 {
1369  string_list::const_iterator sufi;
1370  size_t sufj, sufl;
1371  std::string pre;
1372  addsuffix_generator(input_generator const &, bool &);
1373  input_status next(std::string &);
1374 };
1375 
1377  : gen(top.in, top.local_variables)
1378 {
1379  if (!read_words(gen, suf)) return;
1380  if (!expect_token(gen.in, Comma)) return;
1381  sufj = 0;
1382  sufl = suf.size();
1383  ok = true;
1384 }
1385 
1387 {
1388  if (sufj)
1389  {
1390  if (sufj != sufl)
1391  {
1392  res = *sufi++;
1393  ++sufj;
1394  return Success;
1395  }
1396  sufj = 0;
1397  }
1398  switch (gen.next(res))
1399  {
1400  case Success:
1401  if (!sufl) return Success;
1402  sufi = suf.begin();
1403  sufj = 1;
1404  res += *sufi++;
1405  return Success;
1406  case Eof:
1407  return expect_token(gen.in, Rightpar) ? Eof : SyntaxError;
1408  default:
1409  return SyntaxError;
1410  }
1411 }
1412 
1413 /**
1414  * Return a generator for function @a name.
1415  */
1416 static generator *get_function(input_generator const &in, std::string const &name)
1417 {
1418  skip_spaces(in.in);
1419  generator *g = NULL;
1420  bool ok = false;
1421  if (name == "addprefix") g = new addprefix_generator(in, ok);
1422  else if (name == "addsuffix") g = new addsuffix_generator(in, ok);
1423  if (!g || ok) return g;
1424  delete g;
1425  return NULL;
1426 }
1427 
1428 /** @} */
1429 
1430 /**
1431  * @defgroup database Dependency database
1432  *
1433  * @{
1434  */
1435 
1436 /**
1437  * Load dependencies from @a in.
1438  */
1439 static void load_dependencies(std::istream &in)
1440 {
1441  if (false)
1442  {
1443  error:
1444  std::cerr << "Failed to load database" << std::endl;
1445  exit(EXIT_FAILURE);
1446  }
1447 
1448  while (!in.eof())
1449  {
1450  string_list targets;
1451  if (!read_words(in, targets)) goto error;
1452  if (in.eof()) return;
1453  if (targets.empty()) goto error;
1454  DEBUG << "reading dependencies of target " << targets.front() << std::endl;
1455  if (in.get() != ':') goto error;
1457  dep->targets = targets;
1458  string_list deps;
1459  if (!read_words(in, deps)) goto error;
1460  dep->deps.insert(deps.begin(), deps.end());
1461  for (string_list::const_iterator i = targets.begin(),
1462  i_end = targets.end(); i != i_end; ++i)
1463  {
1464  dependencies[*i] = dep;
1465  }
1466  skip_empty(in);
1467  }
1468 }
1469 
1470 /**
1471  * Load known dependencies from file `.remake`.
1472  */
1473 static void load_dependencies()
1474 {
1475  DEBUG_open << "Loading database... ";
1476  std::ifstream in(".remake");
1477  if (!in.good())
1478  {
1479  DEBUG_close << "not found\n";
1480  return;
1481  }
1482  load_dependencies(in);
1483 }
1484 
1485 
1486 /**
1487  * Save all the dependencies in file `.remake`.
1488  */
1489 static void save_dependencies()
1490 {
1491  DEBUG_open << "Saving database... ";
1492  std::ofstream db(".remake");
1493  while (!dependencies.empty())
1494  {
1495  ref_ptr<dependency_t> dep = dependencies.begin()->second;
1496  for (string_list::const_iterator i = dep->targets.begin(),
1497  i_end = dep->targets.end(); i != i_end; ++i)
1498  {
1499  db << escape_string(*i) << ' ';
1500  dependencies.erase(*i);
1501  }
1502  db << ':';
1503  for (string_set::const_iterator i = dep->deps.begin(),
1504  i_end = dep->deps.end(); i != i_end; ++i)
1505  {
1506  db << ' ' << escape_string(*i);
1507  }
1508  db << std::endl;
1509  }
1510 }
1511 
1512 /** @} */
1513 
1514 static void merge_rule(rule_t &dest, rule_t const &src);
1515 static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst);
1516 
1517 /**
1518  * @defgroup parser Rule parser
1519  *
1520  * @{
1521  */
1522 
1523 /**
1524  * Register a specific rule with an empty script:
1525  *
1526  * - Check that none of the targets already has an associated rule with a
1527  * nonempty script.
1528  * - Create a new rule with a single target for each target, if needed.
1529  * - Add the prerequisites of @a rule to all these associated rules.
1530  */
1531 static void register_transparent_rule(rule_t const &rule, string_list const &targets)
1532 {
1533  assert(rule.script.empty());
1534  for (string_list::const_iterator i = targets.begin(),
1535  i_end = targets.end(); i != i_end; ++i)
1536  {
1537  std::pair<rule_map::iterator, bool> j =
1538  specific_rules.insert(std::make_pair(*i, ref_ptr<rule_t>()));
1539  ref_ptr<rule_t> &r = j.first->second;
1540  if (j.second)
1541  {
1542  r = ref_ptr<rule_t>(rule);
1543  r->targets = string_list(1, *i);
1544  continue;
1545  }
1546  if (!r->script.empty())
1547  {
1548  std::cerr << "Failed to load rules: " << *i
1549  << " cannot be the target of several rules" << std::endl;
1550  exit(EXIT_FAILURE);
1551  }
1552  assert(r->targets.size() == 1 && r->targets.front() == *i);
1553  merge_rule(*r, rule);
1554  }
1555 
1556  for (string_list::const_iterator i = targets.begin(),
1557  i_end = targets.end(); i != i_end; ++i)
1558  {
1560  if (dep->targets.empty()) dep->targets.push_back(*i);
1561  dep->deps.insert(rule.deps.begin(), rule.deps.end());
1562  }
1563 }
1564 
1565 /**
1566  * Register a specific rule with a nonempty script:
1567  *
1568  * - Check that none of the targets already has an associated rule.
1569  * - Create a single shared rule and associate it to all the targets.
1570  * - Merge the prerequisites of all the targets into a single set and
1571  * add the prerequisites of the rule to it. (The preexisting
1572  * prerequisites, if any, come from a previous run.)
1573  */
1574 static void register_scripted_rule(rule_t const &rule)
1575 {
1576  ref_ptr<rule_t> r(rule);
1577  for (string_list::const_iterator i = rule.targets.begin(),
1578  i_end = rule.targets.end(); i != i_end; ++i)
1579  {
1580  std::pair<rule_map::iterator, bool> j =
1581  specific_rules.insert(std::make_pair(*i, r));
1582  if (j.second) continue;
1583  std::cerr << "Failed to load rules: " << *i
1584  << " cannot be the target of several rules" << std::endl;
1585  exit(EXIT_FAILURE);
1586  }
1587 
1589  dep->targets = rule.targets;
1590  dep->deps.insert(rule.deps.begin(), rule.deps.end());
1591  for (string_list::const_iterator i = rule.targets.begin(),
1592  i_end = rule.targets.end(); i != i_end; ++i)
1593  {
1595  dep->deps.insert(d->deps.begin(), d->deps.end());
1596  d = dep;
1597  }
1598 }
1599 
1600 /**
1601  * Register a specific rule.
1602  */
1603 static void register_rule(rule_t const &rule)
1604 {
1605  if (!rule.script.empty())
1606  {
1607  register_scripted_rule(rule);
1608  }
1609  else
1610  {
1611  // Swap away the targets to avoid costly copies when registering.
1612  rule_t &r = const_cast<rule_t &>(rule);
1613  string_list targets;
1614  targets.swap(r.targets);
1615  register_transparent_rule(r, targets);
1616  targets.swap(r.targets);
1617  }
1618 
1619  // If there is no default target yet, mark it as such.
1620  if (first_target.empty())
1621  first_target = rule.targets.front();
1622 }
1623 
1624 /**
1625  * Read a rule starting with target @a first, if nonempty.
1626  * Store into #generic_rules or #specific_rules depending on its genericity.
1627  */
1628 static void load_rule(std::istream &in, std::string const &first)
1629 {
1630  DEBUG_open << "Reading rule for target " << first << "... ";
1631  if (false)
1632  {
1633  error:
1634  DEBUG_close << "failed\n";
1635  std::cerr << "Failed to load rules: syntax error" << std::endl;
1636  exit(EXIT_FAILURE);
1637  }
1638 
1639  // Read targets and check genericity.
1640  string_list targets;
1641  if (!read_words(in, targets)) goto error;
1642  if (!first.empty()) targets.push_front(first);
1643  else if (targets.empty()) goto error;
1644  else DEBUG << "actual target: " << targets.front() << std::endl;
1645  bool generic = false;
1646  normalize_list(targets, "", "");
1647  for (string_list::const_iterator i = targets.begin(),
1648  i_end = targets.end(); i != i_end; ++i)
1649  {
1650  if (i->empty()) goto error;
1651  if ((i->find('%') != std::string::npos) != generic)
1652  {
1653  if (i == targets.begin()) generic = true;
1654  else goto error;
1655  }
1656  }
1657  skip_spaces(in);
1658  if (in.get() != ':') goto error;
1659 
1660  bool assignment = false, static_pattern = false;
1661 
1662  rule_t rule;
1663  rule.targets.swap(targets);
1664 
1665  // Read dependencies.
1666  {
1667  string_list v;
1668  if (expect_token(in, Word))
1669  {
1670  std::string d = read_word(in);
1671  if (int tok = expect_token(in, Equal | Plusequal))
1672  {
1673  if (!read_words(in, v)) goto error;
1674  assign_t &a = rule.assigns[d];
1675  a.append = tok == Plusequal;
1676  a.value.swap(v);
1677  assignment = true;
1678  goto end_line;
1679  }
1680  v.push_back(d);
1681  }
1682 
1683  if (!read_words(in, v)) goto error;
1684  normalize_list(v, "", "");
1685  rule.deps.swap(v);
1686 
1687  if (expect_token(in, Colon))
1688  {
1689  if (!read_words(in, v)) goto error;
1690  normalize_list(v, "", "");
1691  targets.swap(rule.targets);
1692  rule.targets.swap(rule.deps);
1693  rule.deps.swap(v);
1694  if (rule.targets.empty()) goto error;
1695  for (string_list::const_iterator i = rule.targets.begin(),
1696  i_end = rule.targets.end(); i != i_end; ++i)
1697  {
1698  if (i->find('%') == std::string::npos) goto error;
1699  }
1700  generic = false;
1701  static_pattern = true;
1702  }
1703 
1704  if (expect_token(in, Pipe))
1705  {
1706  if (!read_words(in, v)) goto error;
1707  normalize_list(v, "", "");
1708  rule.wdeps.swap(v);
1709  }
1710  }
1711 
1712  end_line:
1713  skip_spaces(in);
1714  if (!skip_eol(in, true)) goto error;
1715 
1716  // Read script.
1717  std::ostringstream buf;
1718  while (true)
1719  {
1720  char c = in.get();
1721  if (!in.good()) break;
1722  if (c == '\t' || c == ' ')
1723  {
1724  in.get(*buf.rdbuf());
1725  if (in.fail() && !in.eof()) in.clear();
1726  }
1727  else if (c == '\r' || c == '\n')
1728  buf << c;
1729  else
1730  {
1731  in.putback(c);
1732  break;
1733  }
1734  }
1735  rule.script = buf.str();
1736 
1737  // Register phony targets.
1738  if (rule.targets.front() == ".PHONY")
1739  {
1740  for (string_list::const_iterator i = rule.deps.begin(),
1741  i_end = rule.deps.end(); i != i_end; ++i)
1742  {
1743  status[*i].status = Todo;
1744  }
1745  return;
1746  }
1747 
1748  // Add generic rules to the correct set.
1749  if (generic)
1750  {
1751  if (assignment) goto error;
1752  generic_rules.push_back(rule);
1753  return;
1754  }
1755 
1756  if (!static_pattern)
1757  {
1758  if (!rule.script.empty() && assignment) goto error;
1759  register_rule(rule);
1760  return;
1761  }
1762 
1763  for (string_list::const_iterator i = targets.begin(),
1764  i_end = targets.end(); i != i_end; ++i)
1765  {
1766  rule_t r;
1767  instantiate_rule(*i, rule, r);
1768  if (!r.stem.empty()) register_rule(r);
1769  }
1770 }
1771 
1772 /**
1773  * Load rules from @a remakefile.
1774  * If some rules have dependencies and non-generic targets, add these
1775  * dependencies to the targets.
1776  */
1777 static void load_rules(std::string const &remakefile)
1778 {
1779  DEBUG_open << "Loading rules... ";
1780  if (false)
1781  {
1782  error:
1783  std::cerr << "Failed to load rules: syntax error" << std::endl;
1784  exit(EXIT_FAILURE);
1785  }
1786  std::ifstream in(remakefile.c_str());
1787  if (!in.good())
1788  {
1789  std::cerr << "Failed to load rules: no Remakefile found" << std::endl;
1790  exit(EXIT_FAILURE);
1791  }
1792  skip_empty(in);
1793 
1794  string_list options;
1795 
1796  // Read rules
1797  while (in.good())
1798  {
1799  char c = in.peek();
1800  if (c == '#')
1801  {
1802  while (in.get() != '\n') {}
1803  skip_empty(in);
1804  continue;
1805  }
1806  if (c == ' ' || c == '\t') goto error;
1807  if (expect_token(in, Word))
1808  {
1809  std::string name = read_word(in);
1810  if (name.empty()) goto error;
1811  if (int tok = expect_token(in, Equal | Plusequal))
1812  {
1813  DEBUG << "Assignment to variable " << name << std::endl;
1814  string_list value;
1815  if (!read_words(in, value)) goto error;
1816  string_list &dest =
1817  *(name == ".OPTIONS" ? &options : &variables[name]);
1818  if (tok == Equal) dest.swap(value);
1819  else dest.splice(dest.end(), value);
1820  if (!skip_eol(in, true)) goto error;
1821  }
1822  else load_rule(in, name);
1823  }
1824  else load_rule(in, std::string());
1825  }
1826 
1827  // Set actual options.
1828  for (string_list::const_iterator i = options.begin(),
1829  i_end = options.end(); i != i_end; ++i)
1830  {
1831  if (*i == "variable-propagation") propagate_vars = true;
1832  else
1833  {
1834  std::cerr << "Failed to load rules: unrecognized option" << std::endl;
1835  exit(EXIT_FAILURE);
1836  }
1837  }
1838 }
1839 
1840 /** @} */
1841 
1842 /**
1843  * @defgroup rules Rule resolution
1844  *
1845  * @{
1846  */
1847 
1848 static void merge_rule(rule_t &dest, rule_t const &src)
1849 {
1850  dest.deps.insert(dest.deps.end(), src.deps.begin(), src.deps.end());
1851  dest.wdeps.insert(dest.wdeps.end(), src.wdeps.begin(), src.wdeps.end());
1852  for (assign_map::const_iterator i = src.assigns.begin(),
1853  i_end = src.assigns.end(); i != i_end; ++i)
1854  {
1855  if (!i->second.append)
1856  {
1857  new_assign:
1858  dest.assigns[i->first] = i->second;
1859  continue;
1860  }
1861  assign_map::iterator j = dest.assigns.find(i->first);
1862  if (j == dest.assigns.end()) goto new_assign;
1863  j->second.value.insert(j->second.value.end(),
1864  i->second.value.begin(), i->second.value.end());
1865  }
1866 }
1867 
1868 /**
1869  * Substitute a pattern into a list of strings.
1870  */
1871 static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
1872 {
1873  for (string_list::const_iterator i = src.begin(),
1874  i_end = src.end(); i != i_end; ++i)
1875  {
1876  size_t pos = i->find('%');
1877  if (pos == std::string::npos) dst.push_back(*i);
1878  else dst.push_back(i->substr(0, pos) + pat + i->substr(pos + 1));
1879  }
1880 }
1881 
1882 /**
1883  * Instantiate a specific rule, given a target and a generic rule.
1884  * If the rule @a dst already contains a stem longer than the one found,
1885  * it is left unchanged.
1886  */
1887 static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst)
1888 {
1889  size_t tlen = target.length(), plen = dst.stem.length();
1890  for (string_list::const_iterator j = src.targets.begin(),
1891  j_end = src.targets.end(); j != j_end; ++j)
1892  {
1893  size_t len = j->length();
1894  if (tlen < len) continue;
1895  if (plen && plen <= tlen - (len - 1)) continue;
1896  size_t pos = j->find('%');
1897  if (pos == std::string::npos) continue;
1898  size_t len2 = len - (pos + 1);
1899  if (j->compare(0, pos, target, 0, pos) ||
1900  j->compare(pos + 1, len2, target, tlen - len2, len2))
1901  continue;
1902  plen = tlen - (len - 1);
1903  dst = rule_t();
1904  dst.stem = target.substr(pos, plen);
1905  dst.script = src.script;
1906  substitute_pattern(dst.stem, src.targets, dst.targets);
1907  substitute_pattern(dst.stem, src.deps, dst.deps);
1908  substitute_pattern(dst.stem, src.wdeps, dst.wdeps);
1909  break;
1910  }
1911 }
1912 
1913 /**
1914  * Find a generic rule matching @a target:
1915  * - the one leading to shorter matches has priority,
1916  * - among equivalent rules, the earliest one has priority.
1917  */
1918 static void find_generic_rule(job_t &job, std::string const &target)
1919 {
1920  for (rule_list::const_iterator i = generic_rules.begin(),
1921  i_end = generic_rules.end(); i != i_end; ++i)
1922  {
1923  instantiate_rule(target, *i, job.rule);
1924  }
1925 }
1926 
1927 /**
1928  * Find a specific rule matching @a target. Return a generic one otherwise.
1929  * If there is both a specific rule with an empty script and a generic rule, the
1930  * generic one is returned after adding the dependencies of the specific one.
1931  */
1932 static void find_rule(job_t &job, std::string const &target)
1933 {
1934  rule_map::const_iterator i = specific_rules.find(target),
1935  i_end = specific_rules.end();
1936  // If there is a specific rule with a script, return it.
1937  if (i != i_end && !i->second->script.empty())
1938  {
1939  job.rule = *i->second;
1940  return;
1941  }
1942  find_generic_rule(job, target);
1943  // If there is no generic rule, return the specific rule (no script), if any.
1944  if (job.rule.targets.empty())
1945  {
1946  if (i != i_end)
1947  {
1948  job.rule = *i->second;
1949  return;
1950  }
1951  }
1952  // Optimize the lookup when there is only one target (already looked up).
1953  if (job.rule.targets.size() == 1)
1954  {
1955  if (i == i_end) return;
1956  merge_rule(job.rule, *i->second);
1957  return;
1958  }
1959  // Add the dependencies of the specific rules of every target to the
1960  // generic rule. If any of those rules has a nonempty script, error out.
1961  for (string_list::const_iterator j = job.rule.targets.begin(),
1962  j_end = job.rule.targets.end(); j != j_end; ++j)
1963  {
1964  i = specific_rules.find(*j);
1965  if (i == i_end) continue;
1966  if (!i->second->script.empty()) return;
1967  merge_rule(job.rule, *i->second);
1968  }
1969 }
1970 
1971 /** @} */
1972 
1973 /**
1974  * @defgroup status Target status
1975  *
1976  * @{
1977  */
1978 
1979 /**
1980  * Compute and memoize the status of @a target:
1981  * - if the file does not exist, the target is obsolete,
1982  * - if any dependency is obsolete or younger than the file, it is obsolete,
1983  * - otherwise it is up-to-date.
1984  *
1985  * @note For rules with multiple targets, all the targets share the same
1986  * status. (If one is obsolete, they all are.) The second rule above
1987  * is modified in that case: the latest target is chosen, not the oldest!
1988  */
1989 static status_t const &get_status(std::string const &target)
1990 {
1991  std::pair<status_map::iterator,bool> i =
1992  status.insert(std::make_pair(target, status_t()));
1993  status_t &ts = i.first->second;
1994  if (!i.second) return ts;
1995  DEBUG_open << "Checking status of " << target << "... ";
1996  dependency_map::const_iterator j = dependencies.find(target);
1997  if (j == dependencies.end())
1998  {
1999  struct stat s;
2000  if (stat(target.c_str(), &s) != 0)
2001  {
2002  DEBUG_close << "missing\n";
2003  ts.status = Todo;
2004  ts.last = 0;
2005  return ts;
2006  }
2007  DEBUG_close << "up-to-date\n";
2008  ts.status = Uptodate;
2009  ts.last = s.st_mtime;
2010  return ts;
2011  }
2012  if (obsolete_targets)
2013  {
2014  DEBUG_close << "forcefully obsolete\n";
2015  ts.status = Todo;
2016  ts.last = 0;
2017  return ts;
2018  }
2019  dependency_t const &dep = *j->second;
2020  status_e st = Uptodate;
2021  time_t latest = 0;
2022  for (string_list::const_iterator k = dep.targets.begin(),
2023  k_end = dep.targets.end(); k != k_end; ++k)
2024  {
2025  struct stat s;
2026  if (stat(k->c_str(), &s) != 0)
2027  {
2028  if (st == Uptodate) DEBUG_close << *k << " missing\n";
2029  s.st_mtime = 0;
2030  st = Todo;
2031  }
2032  status[*k].last = s.st_mtime;
2033  if (s.st_mtime > latest) latest = s.st_mtime;
2034  }
2035  if (st != Uptodate) goto update;
2036  for (string_set::const_iterator k = dep.deps.begin(),
2037  k_end = dep.deps.end(); k != k_end; ++k)
2038  {
2039  status_t const &ts_ = get_status(*k);
2040  if (latest < ts_.last)
2041  {
2042  DEBUG_close << "older than " << *k << std::endl;
2043  st = Todo;
2044  goto update;
2045  }
2046  if (ts_.status != Uptodate && st != Recheck)
2047  {
2048  DEBUG << "obsolete dependency " << *k << std::endl;
2049  st = Recheck;
2050  }
2051  }
2052  if (st == Uptodate) DEBUG_close << "all siblings up-to-date\n";
2053  update:
2054  for (string_list::const_iterator k = dep.targets.begin(),
2055  k_end = dep.targets.end(); k != k_end; ++k)
2056  {
2057  status[*k].status = st;
2058  }
2059  return ts;
2060 }
2061 
2062 /**
2063  * Change the status of @a target to #Remade or #Uptodate depending on whether
2064  * its modification time changed.
2065  */
2066 static void update_status(std::string const &target)
2067 {
2068  DEBUG_open << "Rechecking status of " << target << "... ";
2069  status_map::iterator i = status.find(target);
2070  assert(i != status.end());
2071  status_t &ts = i->second;
2072  ts.status = Remade;
2073  if (ts.last >= now)
2074  {
2075  DEBUG_close << "possibly remade\n";
2076  return;
2077  }
2078  struct stat s;
2079  if (stat(target.c_str(), &s) != 0)
2080  {
2081  DEBUG_close << "missing\n";
2082  ts.last = 0;
2083  }
2084  else if (s.st_mtime != ts.last)
2085  {
2086  DEBUG_close << "remade\n";
2087  ts.last = s.st_mtime;
2088  }
2089  else
2090  {
2091  DEBUG_close << "unchanged\n";
2092  ts.status = Uptodate;
2093  }
2094 }
2095 
2096 /**
2097  * Check whether all the prerequisites of @a target ended being up-to-date.
2098  */
2099 static bool still_need_rebuild(std::string const &target)
2100 {
2101  status_map::const_iterator i = status.find(target);
2102  assert(i != status.end());
2103  if (i->second.status != RunningRecheck) return true;
2104  DEBUG_open << "Rechecking obsoleteness of " << target << "... ";
2105  dependency_map::const_iterator j = dependencies.find(target);
2106  assert(j != dependencies.end());
2107  dependency_t const &dep = *j->second;
2108  for (string_set::const_iterator k = dep.deps.begin(),
2109  k_end = dep.deps.end(); k != k_end; ++k)
2110  {
2111  if (status[*k].status != Uptodate) return true;
2112  }
2113  for (string_list::const_iterator k = dep.targets.begin(),
2114  k_end = dep.targets.end(); k != k_end; ++k)
2115  {
2116  status[*k].status = Uptodate;
2117  }
2118  DEBUG_close << "no longer obsolete\n";
2119  return false;
2120 }
2121 
2122 /** @} */
2123 
2124 /**
2125  * @defgroup server Server
2126  *
2127  * @{
2128  */
2129 
2130 /**
2131  * Handle job completion.
2132  */
2133 static void complete_job(int job_id, bool success, bool started = true)
2134 {
2135  DEBUG << "Completing job " << job_id << '\n';
2136  job_map::iterator i = jobs.find(job_id);
2137  assert(i != jobs.end());
2138  string_list const &targets = i->second.rule.targets;
2139  if (success)
2140  {
2141  bool show = show_targets && started;
2142  if (show) std::cout << "Finished";
2143  for (string_list::const_iterator j = targets.begin(),
2144  j_end = targets.end(); j != j_end; ++j)
2145  {
2146  update_status(*j);
2147  if (show) std::cout << ' ' << *j;
2148  }
2149  if (show) std::cout << std::endl;
2150  }
2151  else
2152  {
2153  std::cerr << "Failed to build";
2154  for (string_list::const_iterator j = targets.begin(),
2155  j_end = targets.end(); j != j_end; ++j)
2156  {
2157  std::cerr << ' ' << *j;
2158  update_status(*j);
2159  status_e &s = status[*j].status;
2160  if (s != Uptodate)
2161  {
2162  DEBUG << "Removing " << *j << '\n';
2163  remove(j->c_str());
2164  }
2165  s = Failed;
2166  }
2167  std::cerr << std::endl;
2168  }
2169  jobs.erase(i);
2170 }
2171 
2172 /**
2173  * Return the script obtained by substituting variables.
2174  */
2175 static std::string prepare_script(job_t const &job)
2176 {
2177  std::string const &s = job.rule.script;
2178  std::istringstream in(s);
2179  std::ostringstream out;
2180  size_t len = s.size();
2181 
2182  while (!in.eof())
2183  {
2184  size_t pos = in.tellg(), p = s.find('$', pos);
2185  if (p == std::string::npos || p == len - 1) p = len;
2186  out.write(&s[pos], p - pos);
2187  if (p == len) break;
2188  ++p;
2189  switch (s[p])
2190  {
2191  case '$':
2192  out << '$';
2193  in.seekg(p + 1);
2194  break;
2195  case '<':
2196  if (!job.rule.deps.empty())
2197  out << job.rule.deps.front();
2198  in.seekg(p + 1);
2199  break;
2200  case '^':
2201  {
2202  bool first = true;
2203  for (string_list::const_iterator i = job.rule.deps.begin(),
2204  i_end = job.rule.deps.end(); i != i_end; ++i)
2205  {
2206  if (first) first = false;
2207  else out << ' ';
2208  out << *i;
2209  }
2210  in.seekg(p + 1);
2211  break;
2212  }
2213  case '@':
2214  assert(!job.rule.targets.empty());
2215  out << job.rule.targets.front();
2216  in.seekg(p + 1);
2217  break;
2218  case '*':
2219  out << job.rule.stem;
2220  in.seekg(p + 1);
2221  break;
2222  case '(':
2223  {
2224  in.seekg(p - 1);
2225  bool first = true;
2226  input_generator gen(in, &job.vars, true);
2227  while (true)
2228  {
2229  std::string w;
2230  input_status s = gen.next(w);
2231  if (s == SyntaxError)
2232  {
2233  // TODO
2234  return "false";
2235  }
2236  if (s == Eof) break;
2237  if (first) first = false;
2238  else out << ' ';
2239  out << w;
2240  }
2241  break;
2242  }
2243  default:
2244  // Let dollars followed by an unrecognized character
2245  // go through. This differs from Make, which would
2246  // use a one-letter variable.
2247  out << '$';
2248  in.seekg(p);
2249  }
2250  }
2251 
2252  return out.str();
2253 }
2254 
2255 /**
2256  * Execute the script from @a rule.
2257  */
2258 static status_e run_script(int job_id, job_t const &job)
2259 {
2261  dep->targets = job.rule.targets;
2262  dep->deps.insert(job.rule.deps.begin(), job.rule.deps.end());
2263  if (show_targets) std::cout << "Building";
2264  for (string_list::const_iterator i = job.rule.targets.begin(),
2265  i_end = job.rule.targets.end(); i != i_end; ++i)
2266  {
2267  dependencies[*i] = dep;
2268  if (show_targets) std::cout << ' ' << *i;
2269  }
2270  if (show_targets) std::cout << std::endl;
2271 
2272  std::string script = prepare_script(job);
2273 
2274  std::ostringstream job_id_buf;
2275  job_id_buf << job_id;
2276  std::string job_id_ = job_id_buf.str();
2277 
2278  DEBUG_open << "Starting script for job " << job_id << "... ";
2279  if (script.empty())
2280  {
2281  DEBUG_close << "no script\n";
2282  complete_job(job_id, true);
2283  return Remade;
2284  }
2285 
2286  if (false)
2287  {
2288  error:
2289  DEBUG_close << "failed\n";
2290  complete_job(job_id, false);
2291  return Failed;
2292  }
2293 
2294 #ifdef WINDOWS
2295  HANDLE pfd[2];
2296  if (false)
2297  {
2298  error2:
2299  CloseHandle(pfd[0]);
2300  CloseHandle(pfd[1]);
2301  goto error;
2302  }
2303  if (!CreatePipe(&pfd[0], &pfd[1], NULL, 0))
2304  goto error;
2305  if (!SetHandleInformation(pfd[0], HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT))
2306  goto error2;
2307  STARTUPINFO si;
2308  ZeroMemory(&si, sizeof(STARTUPINFO));
2309  si.cb = sizeof(STARTUPINFO);
2310  si.hStdError = GetStdHandle(STD_ERROR_HANDLE);
2311  si.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
2312  si.hStdInput = pfd[0];
2313  si.dwFlags |= STARTF_USESTDHANDLES;
2314  PROCESS_INFORMATION pi;
2315  ZeroMemory(&pi, sizeof(PROCESS_INFORMATION));
2316  if (!SetEnvironmentVariable("REMAKE_JOB_ID", job_id_.c_str()))
2317  goto error2;
2318  char const *argv = echo_scripts ? "SH.EXE -e -s -v" : "SH.EXE -e -s";
2319  if (!CreateProcess(NULL, (char *)argv, NULL, NULL,
2320  true, 0, NULL, NULL, &si, &pi))
2321  {
2322  goto error2;
2323  }
2324  CloseHandle(pi.hThread);
2325  DWORD len = script.length(), wlen;
2326  if (!WriteFile(pfd[1], script.c_str(), len, &wlen, NULL) || wlen < len)
2327  std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2328  CloseHandle(pfd[0]);
2329  CloseHandle(pfd[1]);
2330  ++running_jobs;
2331  job_pids[pi.hProcess] = job_id;
2332  return Running;
2333 #else
2334  int pfd[2];
2335  if (false)
2336  {
2337  error2:
2338  close(pfd[0]);
2339  close(pfd[1]);
2340  goto error;
2341  }
2342  if (pipe(pfd) == -1)
2343  goto error;
2344  if (setenv("REMAKE_JOB_ID", job_id_.c_str(), 1))
2345  goto error2;
2346  if (pid_t pid = vfork())
2347  {
2348  if (pid == -1) goto error2;
2349  ssize_t len = script.length();
2350  if (write(pfd[1], script.c_str(), len) < len)
2351  std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2352  close(pfd[0]);
2353  close(pfd[1]);
2354  ++running_jobs;
2355  job_pids[pid] = job_id;
2356  return Running;
2357  }
2358  // Child process starts here. Notice the use of vfork above.
2359  char const *argv[5] = { "sh", "-e", "-s", NULL, NULL };
2360  if (echo_scripts) argv[3] = "-v";
2361  close(pfd[1]);
2362  if (pfd[0] != 0)
2363  {
2364  dup2(pfd[0], 0);
2365  close(pfd[0]);
2366  }
2367  sigprocmask(SIG_SETMASK, &old_sigmask, NULL);
2368  execve("/bin/sh", (char **)argv, environ);
2369  _exit(EXIT_FAILURE);
2370 #endif
2371 }
2372 
2373 /**
2374  * Create a job for @a target according to the loaded rules.
2375  * Mark all the targets from the rule as running and reset their dependencies.
2376  * Inherit variables from @a current, if enabled.
2377  * If the rule has dependencies, create a new client to build them just
2378  * before @a current, and change @a current so that it points to it.
2379  */
2380 static status_e start(std::string const &target, client_list::iterator &current)
2381 {
2382  int job_id = job_counter++;
2383  DEBUG_open << "Starting job " << job_id << " for " << target << "... ";
2384  job_t &job = jobs[job_id];
2385  find_rule(job, target);
2386  if (job.rule.targets.empty())
2387  {
2388  status[target].status = Failed;
2389  DEBUG_close << "failed\n";
2390  std::cerr << "No rule for building " << target << std::endl;
2391  return Failed;
2392  }
2393  bool has_deps = !job.rule.deps.empty() || !job.rule.wdeps.empty();
2394  status_e st = Running;
2395  if (has_deps && status[target].status == Recheck)
2396  st = RunningRecheck;
2397  for (string_list::const_iterator i = job.rule.targets.begin(),
2398  i_end = job.rule.targets.end(); i != i_end; ++i)
2399  {
2400  status[*i].status = st;
2401  }
2402  if (propagate_vars) job.vars = current->vars;
2403  for (assign_map::const_iterator i = job.rule.assigns.begin(),
2404  i_end = job.rule.assigns.end(); i != i_end; ++i)
2405  {
2406  std::pair<variable_map::iterator, bool> k =
2407  job.vars.insert(std::make_pair(i->first, string_list()));
2408  string_list &v = k.first->second;
2409  if (i->second.append)
2410  {
2411  if (k.second)
2412  {
2413  variable_map::const_iterator j = variables.find(i->first);
2414  if (j != variables.end()) v = j->second;
2415  }
2416  }
2417  else if (!k.second) v.clear();
2418  v.insert(v.end(), i->second.value.begin(), i->second.value.end());
2419  }
2420  if (has_deps)
2421  {
2422  current = clients.insert(current, client_t());
2423  current->job_id = job_id;
2424  current->pending = job.rule.deps;
2425  current->pending.insert(current->pending.end(),
2426  job.rule.wdeps.begin(), job.rule.wdeps.end());
2427  if (propagate_vars) current->vars = job.vars;
2428  current->delayed = true;
2429  return RunningRecheck;
2430  }
2431  return run_script(job_id, job);
2432 }
2433 
2434 /**
2435  * Send a reply to a client then remove it.
2436  * If the client was a dependency client, start the actual script.
2437  */
2438 static void complete_request(client_t &client, bool success)
2439 {
2440  DEBUG_open << "Completing request from client of job " << client.job_id << "... ";
2441  if (client.delayed)
2442  {
2443  assert(client.socket == INVALID_SOCKET);
2444  if (success)
2445  {
2446  job_map::const_iterator i = jobs.find(client.job_id);
2447  assert(i != jobs.end());
2448  if (still_need_rebuild(i->second.rule.targets.front()))
2449  run_script(client.job_id, i->second);
2450  else complete_job(client.job_id, true, false);
2451  }
2452  else complete_job(client.job_id, false);
2453  }
2454  else if (client.socket != INVALID_SOCKET)
2455  {
2456  char res = success ? 1 : 0;
2457  send(client.socket, &res, 1, MSG_NOSIGNAL);
2458  #ifdef WINDOWS
2459  closesocket(client.socket);
2460  #else
2461  close(client.socket);
2462  #endif
2463  --waiting_jobs;
2464  }
2465 
2466  if (client.job_id < 0 && !success) build_failure = true;
2467 }
2468 
2469 /**
2470  * Return whether there are slots for starting new jobs.
2471  */
2472 static bool has_free_slots()
2473 {
2474  if (max_active_jobs <= 0) return true;
2476 }
2477 
2478 /**
2479  * Handle client requests:
2480  * - check for running targets that have finished,
2481  * - start as many pending targets as allowed,
2482  * - complete the request if there are neither running nor pending targets
2483  * left or if any of them failed.
2484  *
2485  * @return true if some child processes are still running.
2486  *
2487  * @post If there are pending requests, at least one child process is running.
2488  *
2489  * @invariant New free slots cannot appear during a run, since the only way to
2490  * decrease #running_jobs is #finalize_job and the only way to
2491  * increase #waiting_jobs is #accept_client. None of these functions
2492  * are called during a run. So breaking out as soon as there are no
2493  * free slots left is fine.
2494  */
2495 static bool handle_clients()
2496 {
2497  DEBUG_open << "Handling client requests... ";
2498  restart:
2499  bool need_restart = false;
2500 
2501  for (client_list::iterator i = clients.begin(), i_next = i,
2502  i_end = clients.end(); i != i_end; i = i_next)
2503  {
2504  if (!has_free_slots()) break;
2505  ++i_next;
2506  DEBUG_open << "Handling client from job " << i->job_id << "... ";
2507 
2508  // Remove running targets that have finished.
2509  for (string_set::iterator j = i->running.begin(), j_next = j,
2510  j_end = i->running.end(); j != j_end; j = j_next)
2511  {
2512  ++j_next;
2513  status_map::const_iterator k = status.find(*j);
2514  assert(k != status.end());
2515  switch (k->second.status)
2516  {
2517  case Running:
2518  case RunningRecheck:
2519  break;
2520  case Failed:
2521  i->failed = true;
2522  if (!keep_going) goto complete;
2523  // fallthrough
2524  case Uptodate:
2525  case Remade:
2526  i->running.erase(j);
2527  break;
2528  case Recheck:
2529  case Todo:
2530  assert(false);
2531  }
2532  }
2533 
2534  // Start pending targets.
2535  while (!i->pending.empty())
2536  {
2537  std::string target = i->pending.front();
2538  i->pending.pop_front();
2539  switch (get_status(target).status)
2540  {
2541  case Running:
2542  case RunningRecheck:
2543  i->running.insert(target);
2544  break;
2545  case Failed:
2546  pending_failed:
2547  i->failed = true;
2548  if (!keep_going) goto complete;
2549  // fallthrough
2550  case Uptodate:
2551  case Remade:
2552  break;
2553  case Recheck:
2554  case Todo:
2555  client_list::iterator j = i;
2556  switch (start(target, i))
2557  {
2558  case Failed:
2559  goto pending_failed;
2560  case Running:
2561  // A shell was started, check for free slots.
2562  j->running.insert(target);
2563  if (!has_free_slots()) return true;
2564  break;
2565  case RunningRecheck:
2566  // Switch to the dependency client that was inserted.
2567  j->running.insert(target);
2568  i_next = j;
2569  break;
2570  case Remade:
2571  // Nothing to run.
2572  need_restart = true;
2573  break;
2574  default:
2575  assert(false);
2576  }
2577  }
2578  }
2579 
2580  // Try to complete the request.
2581  // (This might start a new job if it was a dependency client.)
2582  if (i->running.empty() || i->failed)
2583  {
2584  complete:
2585  complete_request(*i, !i->failed);
2586  DEBUG_close << (i->failed ? "failed\n" : "finished\n");
2587  clients.erase(i);
2588  need_restart = true;
2589  }
2590  }
2591 
2592  if (running_jobs != waiting_jobs) return true;
2593  if (running_jobs == 0 && clients.empty()) return false;
2594  if (need_restart) goto restart;
2595 
2596  // There is a circular dependency.
2597  // Try to break it by completing one of the requests.
2598  assert(!clients.empty());
2599  std::cerr << "Circular dependency detected" << std::endl;
2600  client_list::iterator i = clients.begin();
2601  complete_request(*i, false);
2602  clients.erase(i);
2603  goto restart;
2604 }
2605 
2606 /**
2607  * Create a named unix socket that listens for build requests. Also set
2608  * the REMAKE_SOCKET environment variable that will be inherited by all
2609  * the job scripts.
2610  */
2611 static void create_server()
2612 {
2613  if (false)
2614  {
2615  error:
2616  perror("Failed to create server");
2617 #ifndef WINDOWS
2618  error2:
2619 #endif
2620  exit(EXIT_FAILURE);
2621  }
2622  DEBUG_open << "Creating server... ";
2623 
2624 #ifdef WINDOWS
2625  // Prepare a windows socket.
2626  struct sockaddr_in socket_addr;
2627  socket_addr.sin_family = AF_INET;
2628  socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2629  socket_addr.sin_port = 0;
2630 
2631  // Create and listen to the socket.
2632  socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2633  if (socket_fd == INVALID_SOCKET) goto error;
2634  if (!SetHandleInformation((HANDLE)socket_fd, HANDLE_FLAG_INHERIT, 0))
2635  goto error;
2636  if (bind(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2637  goto error;
2638  int len = sizeof(sockaddr_in);
2639  if (getsockname(socket_fd, (struct sockaddr *)&socket_addr, &len))
2640  goto error;
2641  std::ostringstream buf;
2642  buf << socket_addr.sin_port;
2643  if (!SetEnvironmentVariable("REMAKE_SOCKET", buf.str().c_str()))
2644  goto error;
2645  if (listen(socket_fd, 1000)) goto error;
2646 #else
2647  // Set signal handlers for SIGCHLD and SIGINT.
2648  // Block SIGCHLD (unblocked during select).
2649  sigset_t sigmask;
2650  sigemptyset(&sigmask);
2651  sigaddset(&sigmask, SIGCHLD);
2652  if (sigprocmask(SIG_BLOCK, &sigmask, &old_sigmask) == -1) goto error;
2653  struct sigaction sa;
2654  sa.sa_flags = 0;
2655  sigemptyset(&sa.sa_mask);
2656  sa.sa_handler = &sigchld_handler;
2657  if (sigaction(SIGCHLD, &sa, NULL) == -1) goto error;
2658  sa.sa_handler = &sigint_handler;
2659  if (sigaction(SIGINT, &sa, NULL) == -1) goto error;
2660 
2661  // Prepare a named unix socket in temporary directory.
2662  socket_name = tempnam(NULL, "rmk-");
2663  if (!socket_name) goto error2;
2664  struct sockaddr_un socket_addr;
2665  size_t len = strlen(socket_name);
2666  if (len >= sizeof(socket_addr.sun_path) - 1) goto error2;
2667  socket_addr.sun_family = AF_UNIX;
2668  strcpy(socket_addr.sun_path, socket_name);
2669  len += sizeof(socket_addr.sun_family);
2670  if (setenv("REMAKE_SOCKET", socket_name, 1)) goto error;
2671 
2672  // Create and listen to the socket.
2673 #ifdef LINUX
2674  socket_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
2675  if (socket_fd == INVALID_SOCKET) goto error;
2676 #else
2677  socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2678  if (socket_fd == INVALID_SOCKET) goto error;
2679  if (fcntl(socket_fd, F_SETFD, FD_CLOEXEC) < 0) goto error;
2680 #endif
2681  if (bind(socket_fd, (struct sockaddr *)&socket_addr, len))
2682  goto error;
2683  if (listen(socket_fd, 1000)) goto error;
2684 #endif
2685 }
2686 
2687 /**
2688  * Accept a connection from a client, get the job it spawned from,
2689  * get the targets, and mark them as dependencies of the job targets.
2690  */
2691 static void accept_client()
2692 {
2693  DEBUG_open << "Handling client request... ";
2694 
2695  // Accept connection.
2696 #ifdef WINDOWS
2697  socket_t fd = accept(socket_fd, NULL, NULL);
2698  if (fd == INVALID_SOCKET) return;
2699  if (!SetHandleInformation((HANDLE)fd, HANDLE_FLAG_INHERIT, 0))
2700  {
2701  error2:
2702  std::cerr << "Unexpected failure while setting connection with client" << std::endl;
2703  closesocket(fd);
2704  return;
2705  }
2706  // WSAEventSelect puts sockets into nonblocking mode, so disable it here.
2707  u_long nbio = 0;
2708  if (ioctlsocket(fd, FIONBIO, &nbio)) goto error2;
2709 #elif defined(LINUX)
2710  int fd = accept4(socket_fd, NULL, NULL, SOCK_CLOEXEC);
2711  if (fd < 0) return;
2712 #else
2713  int fd = accept(socket_fd, NULL, NULL);
2714  if (fd < 0) return;
2715  if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0) return;
2716 #endif
2717  clients.push_front(client_t());
2718  client_list::iterator proc = clients.begin();
2719 
2720  if (false)
2721  {
2722  error:
2723  DEBUG_close << "failed\n";
2724  std::cerr << "Received an ill-formed client message" << std::endl;
2725  #ifdef WINDOWS
2726  closesocket(fd);
2727  #else
2728  close(fd);
2729  #endif
2730  clients.erase(proc);
2731  return;
2732  }
2733 
2734  // Receive message. Stop when encountering two nuls in a row.
2735  std::vector<char> buf;
2736  size_t len = 0;
2737  while (len < sizeof(int) + 2 || buf[len - 1] || buf[len - 2])
2738  {
2739  buf.resize(len + 1024);
2740  ssize_t l = recv(fd, &buf[0] + len, 1024, 0);
2741  if (l <= 0) goto error;
2742  len += l;
2743  }
2744 
2745  // Parse job that spawned the client.
2746  int job_id;
2747  memcpy(&job_id, &buf[0], sizeof(int));
2748  proc->socket = fd;
2749  proc->job_id = job_id;
2750  job_map::const_iterator i = jobs.find(job_id);
2751  if (i == jobs.end()) goto error;
2752  DEBUG << "receiving request from job " << job_id << std::endl;
2753  if (propagate_vars) proc->vars = i->second.vars;
2754 
2755  // Parse the targets and the variable assignments.
2756  // Mark the targets as dependencies of the job targets.
2757  dependency_t &dep = *dependencies[i->second.rule.targets.front()];
2758  string_list *last_var = NULL;
2759  char const *p = &buf[0] + sizeof(int);
2760  while (true)
2761  {
2762  len = strlen(p);
2763  if (len == 0)
2764  {
2765  ++waiting_jobs;
2766  break;
2767  }
2768  switch (*p)
2769  {
2770  case 'T':
2771  {
2772  if (len == 1) goto error;
2773  std::string target(p + 1, p + len);
2774  DEBUG << "adding dependency " << target << " to job\n";
2775  proc->pending.push_back(target);
2776  dep.deps.insert(target);
2777  break;
2778  }
2779  case 'V':
2780  {
2781  if (len == 1) goto error;
2782  std::string var(p + 1, p + len);
2783  DEBUG << "adding variable " << var << " to job\n";
2784  last_var = &proc->vars[var];
2785  last_var->clear();
2786  break;
2787  }
2788  case 'W':
2789  {
2790  if (!last_var) goto error;
2791  last_var->push_back(std::string(p + 1, p + len));
2792  break;
2793  }
2794  default:
2795  goto error;
2796  }
2797  p += len + 1;
2798  }
2799 
2800  if (!propagate_vars && !proc->vars.empty())
2801  {
2802  std::cerr << "Assignments are ignored unless 'variable-propagation' is enabled" << std::endl;
2803  proc->vars.clear();
2804  }
2805 }
2806 
2807 /**
2808  * Handle child process exit status.
2809  */
2810 static void finalize_job(pid_t pid, bool res)
2811 {
2812  pid_job_map::iterator i = job_pids.find(pid);
2813  assert(i != job_pids.end());
2814  int job_id = i->second;
2815  job_pids.erase(i);
2816  --running_jobs;
2817  complete_job(job_id, res);
2818 }
2819 
2820 /**
2821  * Loop until all the jobs have finished.
2822  *
2823  * @post There are no client requests left, not even virtual ones.
2824  */
2825 static void server_loop()
2826 {
2827  while (handle_clients())
2828  {
2829  DEBUG_open << "Handling events... ";
2830  #ifdef WINDOWS
2831  size_t len = job_pids.size() + 1;
2832  HANDLE h[len];
2833  int num = 0;
2834  for (pid_job_map::const_iterator i = job_pids.begin(),
2835  i_end = job_pids.end(); i != i_end; ++i, ++num)
2836  {
2837  h[num] = i->first;
2838  }
2839  WSAEVENT aev = WSACreateEvent();
2840  h[num] = aev;
2841  WSAEventSelect(socket_fd, aev, FD_ACCEPT);
2842  DWORD w = WaitForMultipleObjects(len, h, false, INFINITE);
2843  WSAEventSelect(socket_fd, aev, 0);
2844  WSACloseEvent(aev);
2845  if (len <= w)
2846  continue;
2847  if (w == len - 1)
2848  {
2849  accept_client();
2850  continue;
2851  }
2852  pid_t pid = h[w];
2853  DWORD s = 0;
2854  bool res = GetExitCodeProcess(pid, &s) && s == 0;
2855  CloseHandle(pid);
2856  finalize_job(pid, res);
2857  #else
2858  sigset_t emptymask;
2859  sigemptyset(&emptymask);
2860  fd_set fdset;
2861  FD_ZERO(&fdset);
2862  FD_SET(socket_fd, &fdset);
2863  int ret = pselect(socket_fd + 1, &fdset, NULL, NULL, NULL, &emptymask);
2864  if (ret > 0 /* && FD_ISSET(socket_fd, &fdset)*/) accept_client();
2865  if (!got_SIGCHLD) continue;
2866  got_SIGCHLD = 0;
2867  pid_t pid;
2868  int status;
2869  while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
2870  {
2871  bool res = WIFEXITED(status) && WEXITSTATUS(status) == 0;
2872  finalize_job(pid, res);
2873  }
2874  #endif
2875  }
2876 
2877  assert(clients.empty());
2878 }
2879 
2880 /**
2881  * Load dependencies and rules, listen to client requests, and loop until
2882  * all the requests have completed.
2883  * If Remakefile is obsolete, perform a first run with it only, then reload
2884  * the rules, and perform a second with the original clients.
2885  */
2886 static void server_mode(std::string const &remakefile, string_list const &targets)
2887 {
2889  load_rules(remakefile);
2890  create_server();
2891  if (get_status(remakefile).status != Uptodate)
2892  {
2893  clients.push_back(client_t());
2894  clients.back().pending.push_back(remakefile);
2895  server_loop();
2896  if (build_failure) goto early_exit;
2897  variables.clear();
2898  specific_rules.clear();
2899  generic_rules.clear();
2900  first_target.clear();
2901  load_rules(remakefile);
2902  }
2903  clients.push_back(client_t());
2904  if (!targets.empty()) clients.back().pending = targets;
2905  else if (!first_target.empty())
2906  clients.back().pending.push_back(first_target);
2907  server_loop();
2908  early_exit:
2909  close(socket_fd);
2910 #ifndef WINDOWS
2911  remove(socket_name);
2912  free(socket_name);
2913 #endif
2916  {
2917  std::cout << "remake: Leaving directory `" << prefix_dir << '\'' << std::endl;
2918  }
2919  exit(build_failure ? EXIT_FAILURE : EXIT_SUCCESS);
2920 }
2921 
2922 /** @} */
2923 
2924 /**
2925  * @defgroup client Client
2926  *
2927  * @{
2928  */
2929 
2930 /**
2931  * Connect to the server @a socket_name, send a request for building @a targets
2932  * with some @a variables, and exit with the status returned by the server.
2933  */
2934 static void client_mode(char *socket_name, string_list const &targets)
2935 {
2936  if (false)
2937  {
2938  error:
2939  perror("Failed to send targets to server");
2940  exit(EXIT_FAILURE);
2941  }
2942  if (targets.empty()) exit(EXIT_SUCCESS);
2943  DEBUG_open << "Connecting to server... ";
2944 
2945  // Connect to server.
2946 #ifdef WINDOWS
2947  struct sockaddr_in socket_addr;
2948  socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2949  if (socket_fd == INVALID_SOCKET) goto error;
2950  socket_addr.sin_family = AF_INET;
2951  socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2952  socket_addr.sin_port = atoi(socket_name);
2953  if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2954  goto error;
2955 #else
2956  struct sockaddr_un socket_addr;
2957  size_t len = strlen(socket_name);
2958  if (len >= sizeof(socket_addr.sun_path) - 1) exit(EXIT_FAILURE);
2959  socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2960  if (socket_fd == INVALID_SOCKET) goto error;
2961  socket_addr.sun_family = AF_UNIX;
2962  strcpy(socket_addr.sun_path, socket_name);
2963  if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(socket_addr.sun_family) + len))
2964  goto error;
2965 #ifdef MACOSX
2966  int set_option = 1;
2967  if (setsockopt(socket_fd, SOL_SOCKET, SO_NOSIGPIPE, &set_option, sizeof(set_option)))
2968  goto error;
2969 #endif
2970 #endif
2971 
2972  // Send current job id.
2973  char *id = getenv("REMAKE_JOB_ID");
2974  int job_id = id ? atoi(id) : -1;
2975  if (send(socket_fd, (char *)&job_id, sizeof(job_id), MSG_NOSIGNAL) != sizeof(job_id))
2976  goto error;
2977 
2978  // Send targets.
2979  for (string_list::const_iterator i = targets.begin(),
2980  i_end = targets.end(); i != i_end; ++i)
2981  {
2982  DEBUG_open << "Sending target " << *i << "... ";
2983  std::string s = 'T' + *i;
2984  ssize_t len = s.length() + 1;
2985  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2986  goto error;
2987  }
2988 
2989  // Send variables.
2990  for (variable_map::const_iterator i = variables.begin(),
2991  i_end = variables.end(); i != i_end; ++i)
2992  {
2993  DEBUG_open << "Sending variable " << i->first << "... ";
2994  std::string s = 'V' + i->first;
2995  ssize_t len = s.length() + 1;
2996  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2997  goto error;
2998  for (string_list::const_iterator j = i->second.begin(),
2999  j_end = i->second.end(); j != j_end; ++j)
3000  {
3001  std::string s = 'W' + *j;
3002  len = s.length() + 1;
3003  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
3004  goto error;
3005  }
3006  }
3007 
3008  // Send terminating nul and wait for reply.
3009  char result = 0;
3010  if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
3011  if (recv(socket_fd, &result, 1, 0) != 1) exit(EXIT_FAILURE);
3012  exit(result ? EXIT_SUCCESS : EXIT_FAILURE);
3013 }
3014 
3015 /** @} */
3016 
3017 /**
3018  * @defgroup ui User interface
3019  *
3020  * @{
3021  */
3022 
3023 /**
3024  * Display usage and exit with @a exit_status.
3025  */
3026 static void usage(int exit_status)
3027 {
3028  std::cerr << "Usage: remake [options] [target] ...\n"
3029  "Options\n"
3030  " -B, --always-make Unconditionally make all targets.\n"
3031  " -d Echo script commands.\n"
3032  " -d -d Print lots of debugging information.\n"
3033  " -f FILE Read FILE as Remakefile.\n"
3034  " -h, --help Print this message and exit.\n"
3035  " -j[N], --jobs=[N] Allow N jobs at once; infinite jobs with no arg.\n"
3036  " -k, --keep-going Keep going when some targets cannot be made.\n"
3037  " -r Look up targets from the dependencies on stdin.\n"
3038  " -s, --silent, --quiet Do not echo targets.\n";
3039  exit(exit_status);
3040 }
3041 
3042 /**
3043  * This program behaves in two different ways.
3044  *
3045  * - If the environment contains the REMAKE_SOCKET variable, the client
3046  * connects to this socket and sends to the server its build targets.
3047  * It exits once it receives the server reply.
3048  *
3049  * - Otherwise, it creates a server that waits for build requests. It
3050  * also creates a pseudo-client that requests the targets passed on the
3051  * command line.
3052  */
3053 int main(int argc, char *argv[])
3054 {
3055  std::string remakefile;
3056  string_list targets;
3057  bool literal_targets = false;
3058  bool indirect_targets = false;
3059 
3060  // Parse command-line arguments.
3061  for (int i = 1; i < argc; ++i)
3062  {
3063  std::string arg = argv[i];
3064  if (arg.empty()) usage(EXIT_FAILURE);
3065  if (literal_targets) goto new_target;
3066  if (arg == "-h" || arg == "--help") usage(EXIT_SUCCESS);
3067  if (arg == "-d")
3068  if (echo_scripts) debug.active = true;
3069  else echo_scripts = true;
3070  else if (arg == "-k" || arg =="--keep-going")
3071  keep_going = true;
3072  else if (arg == "-s" || arg == "--silent" || arg == "--quiet")
3073  show_targets = false;
3074  else if (arg == "-r")
3075  indirect_targets = true;
3076  else if (arg == "-B" || arg == "--always-make")
3077  obsolete_targets = true;
3078  else if (arg == "-f")
3079  {
3080  if (++i == argc) usage(EXIT_FAILURE);
3081  remakefile = argv[i];
3082  }
3083  else if (arg == "--")
3084  literal_targets = true;
3085  else if (arg.compare(0, 2, "-j") == 0)
3086  max_active_jobs = atoi(arg.c_str() + 2);
3087  else if (arg.compare(0, 7, "--jobs=") == 0)
3088  max_active_jobs = atoi(arg.c_str() + 7);
3089  else
3090  {
3091  if (arg[0] == '-') usage(EXIT_FAILURE);
3092  if (arg.find('=') != std::string::npos)
3093  {
3094  std::istringstream in(arg);
3095  std::string name = read_word(in);
3096  if (name.empty() || !expect_token(in, Equal)) usage(EXIT_FAILURE);
3097  read_words(in, variables[name]);
3098  continue;
3099  }
3100  new_target:
3101  targets.push_back(arg);
3102  DEBUG << "New target: " << arg << '\n';
3103  }
3104  }
3105 
3106  init_working_dir();
3108 
3109  if (indirect_targets)
3110  {
3111  load_dependencies(std::cin);
3112  string_list l;
3113  targets.swap(l);
3114  if (l.empty() && !dependencies.empty())
3115  {
3116  l.push_back(dependencies.begin()->second->targets.front());
3117  }
3118  for (string_list::const_iterator i = l.begin(),
3119  i_end = l.end(); i != i_end; ++i)
3120  {
3121  dependency_map::const_iterator j = dependencies.find(*i);
3122  if (j == dependencies.end()) continue;
3123  dependency_t const &dep = *j->second;
3124  for (string_set::const_iterator k = dep.deps.begin(),
3125  k_end = dep.deps.end(); k != k_end; ++k)
3126  {
3127  targets.push_back(normalize(*k, working_dir, working_dir));
3128  }
3129  }
3130  dependencies.clear();
3131  }
3132 
3133 #ifdef WINDOWS
3134  WSADATA wsaData;
3135  if (WSAStartup(MAKEWORD(2,2), &wsaData))
3136  {
3137  std::cerr << "Unexpected failure while initializing Windows Socket" << std::endl;
3138  return 1;
3139  }
3140 #endif
3141 
3142  // Run as client if REMAKE_SOCKET is present in the environment.
3143  if (char *sn = getenv("REMAKE_SOCKET")) client_mode(sn, targets);
3144 
3145  // Otherwise run as server.
3146  if (remakefile.empty())
3147  {
3148  remakefile = "Remakefile";
3149  init_prefix_dir();
3150  }
3152  server_mode(remakefile, targets);
3153 }
3154 
3155 /** @} */
log::depth
int depth
Definition: remake.cpp:779
variable_generator::name
std::string name
Definition: remake.cpp:1195
ref_ptr::ptr
content * ptr
Definition: remake.cpp:490
first_target
static std::string first_target
Definition: remake.cpp:716
escape_string::escape_string
escape_string(std::string const &s)
Definition: remake.cpp:829
log::log
log()
Definition: remake.cpp:780
clients
static client_list clients
Definition: remake.cpp:655
client_t::socket
socket_t socket
Socket used to reply to the client (invalid for pseudo clients).
Definition: remake.cpp:603
addprefix_generator::addprefix_generator
addprefix_generator(input_generator const &, bool &)
Definition: remake.cpp:1320
save_dependencies
static void save_dependencies()
Definition: remake.cpp:1489
string_set
std::set< std::string > string_set
Definition: remake.cpp:473
job_t
Definition: remake.cpp:578
client_t::pending
string_list pending
Targets not yet started.
Definition: remake.cpp:606
string_list
std::list< std::string > string_list
Definition: remake.cpp:471
input_generator::next
input_status next(std::string &)
Definition: remake.cpp:1248
Recheck
@ Recheck
Target has an obsolete dependency.
Definition: remake.cpp:528
propagate_vars
static bool propagate_vars
Definition: remake.cpp:751
escape_string::input
const std::string & input
Definition: remake.cpp:828
generator
Definition: remake.cpp:1184
log::operator()
std::ostream & operator()()
Definition: remake.cpp:783
normalize
static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
Definition: remake.cpp:955
log_auto_close::~log_auto_close
~log_auto_close()
Definition: remake.cpp:811
variable_generator
Definition: remake.cpp:1193
variable_map
std::map< std::string, string_list > variable_map
Definition: remake.cpp:519
now
static time_t now
Definition: remake.cpp:731
init_prefix_dir
static void init_prefix_dir()
Definition: remake.cpp:899
load_rule
static void load_rule(std::istream &in, std::string const &first)
Definition: remake.cpp:1628
DEBUG
#define DEBUG
Definition: remake.cpp:817
job_t::vars
variable_map vars
Values of local variables.
Definition: remake.cpp:581
Uptodate
@ Uptodate
Target is up-to-date.
Definition: remake.cpp:526
input_status
input_status
Definition: remake.cpp:1174
build_failure
static bool build_failure
Definition: remake.cpp:704
skip_eol
static bool skip_eol(std::istream &in, bool multi=false)
Definition: remake.cpp:1047
job_pids
static pid_job_map job_pids
Definition: remake.cpp:649
ref_ptr::content::content
content(T const &t)
Definition: remake.cpp:488
DEBUG_close
#define DEBUG_close
Definition: remake.cpp:819
dependencies
static dependency_map dependencies
Definition: remake.cpp:624
rule_t::targets
string_list targets
Files produced by this rule.
Definition: remake.cpp:562
variable_generator::next
input_status next(std::string &)
Definition: remake.cpp:1220
status
static status_map status
Definition: remake.cpp:629
register_transparent_rule
static void register_transparent_rule(rule_t const &rule, string_list const &targets)
Definition: remake.cpp:1531
input_generator
Definition: remake.cpp:1234
read_words
static bool read_words(input_generator &in, string_list &res)
Definition: remake.cpp:1288
escape_string
Definition: remake.cpp:826
running_jobs
static int running_jobs
Definition: remake.cpp:680
init_working_dir
static void init_working_dir()
Definition: remake.cpp:877
get_status
static const status_t & get_status(std::string const &target)
Definition: remake.cpp:1989
ref_ptr::ref_ptr
ref_ptr(T const &t)
Definition: remake.cpp:492
generator::~generator
virtual ~generator()
Definition: remake.cpp:1186
Remade
@ Remade
Target was successfully rebuilt.
Definition: remake.cpp:531
job_map
std::map< int, job_t > job_map
Definition: remake.cpp:584
status_t::status
status_e status
Actual status.
Definition: remake.cpp:540
load_rules
static void load_rules(std::string const &remakefile)
Definition: remake.cpp:1777
find_rule
static void find_rule(job_t &job, std::string const &target)
Definition: remake.cpp:1932
normalize_abs
static std::string normalize_abs(std::string const &s, std::string const &p)
Definition: remake.cpp:933
Plusequal
@ Plusequal
Definition: remake.cpp:1066
log::open
bool open
Definition: remake.cpp:778
ref_ptr::content::content
content()
Definition: remake.cpp:487
addsuffix_generator::addsuffix_generator
addsuffix_generator(input_generator const &, bool &)
Definition: remake.cpp:1376
rule_map
std::map< std::string, ref_ptr< rule_t > > rule_map
Definition: remake.cpp:572
max_active_jobs
static int max_active_jobs
Definition: remake.cpp:661
changed_prefix_dir
static bool changed_prefix_dir
Definition: remake.cpp:746
addprefix_generator::prej
size_t prej
Definition: remake.cpp:1314
variable_generator::vend
string_list::const_iterator vend
Definition: remake.cpp:1196
input_generator::done
bool done
Definition: remake.cpp:1239
input_generator::~input_generator
~input_generator()
Definition: remake.cpp:1243
ref_ptr::content::cnt
size_t cnt
Definition: remake.cpp:485
handle_clients
static bool handle_clients()
Definition: remake.cpp:2495
client_t::vars
variable_map vars
Variables set on request.
Definition: remake.cpp:608
rule_t::script
std::string script
Shell script for building the targets.
Definition: remake.cpp:567
rule_list
std::list< rule_t > rule_list
Definition: remake.cpp:570
register_rule
static void register_rule(rule_t const &rule)
Definition: remake.cpp:1603
run_script
static status_e run_script(int job_id, job_t const &job)
Definition: remake.cpp:2258
addsuffix_generator::pre
std::string pre
Definition: remake.cpp:1371
Colon
@ Colon
Definition: remake.cpp:1061
generator::next
virtual input_status next(std::string &)=0
Comma
@ Comma
Definition: remake.cpp:1065
socket_fd
static socket_t socket_fd
Definition: remake.cpp:699
client_t::failed
bool failed
Whether some targets failed in mode -k.
Definition: remake.cpp:605
client_t::client_t
client_t()
Definition: remake.cpp:610
accept_client
static void accept_client()
Definition: remake.cpp:2691
Todo
@ Todo
Target is missing or obsolete.
Definition: remake.cpp:527
got_SIGCHLD
static volatile sig_atomic_t got_SIGCHLD
Definition: remake.cpp:761
finalize_job
static void finalize_job(pid_t pid, bool res)
Definition: remake.cpp:2810
Failed
@ Failed
Build failed for target.
Definition: remake.cpp:532
environ
char ** environ
read_word
static std::string read_word(std::istream &in, bool detect_equal=true)
Definition: remake.cpp:1122
find_generic_rule
static void find_generic_rule(job_t &job, std::string const &target)
Definition: remake.cpp:1918
rule_t
Definition: remake.cpp:560
ref_ptr::operator->
T * operator->() const
Definition: remake.cpp:508
generic_rules
static rule_list generic_rules
Definition: remake.cpp:634
input_generator::input_generator
input_generator(std::istream &i, variable_map const *lv, bool e=false)
Definition: remake.cpp:1240
client_t::delayed
bool delayed
Whether it is a dependency client and a script has to be started on request completion.
Definition: remake.cpp:609
INVALID_SOCKET
@ INVALID_SOCKET
Definition: remake.cpp:463
dependency_t::targets
string_list targets
Definition: remake.cpp:513
update_status
static void update_status(std::string const &target)
Definition: remake.cpp:2066
Equal
@ Equal
Definition: remake.cpp:1062
specific_rules
static rule_map specific_rules
Definition: remake.cpp:639
input_generator::local_variables
const variable_map * local_variables
Definition: remake.cpp:1238
Dollarpar
@ Dollarpar
Definition: remake.cpp:1063
addprefix_generator::gen
input_generator gen
Definition: remake.cpp:1311
ref_ptr::operator*
T & operator*() const
Definition: remake.cpp:503
ref_ptr::content
Definition: remake.cpp:483
addsuffix_generator::next
input_status next(std::string &)
Definition: remake.cpp:1386
client_t::job_id
int job_id
Job for which the built script called remake and spawned the client (negative for original clients).
Definition: remake.cpp:604
addprefix_generator::suf
std::string suf
Definition: remake.cpp:1315
addsuffix_generator
Definition: remake.cpp:1365
complete_job
static void complete_job(int job_id, bool success, bool started=true)
Definition: remake.cpp:2133
status_t::last
time_t last
Last-modified date.
Definition: remake.cpp:541
Success
@ Success
Definition: remake.cpp:1176
prefix_dir
static std::string prefix_dir
Definition: remake.cpp:741
addsuffix_generator::sufi
string_list::const_iterator sufi
Definition: remake.cpp:1369
log
Definition: remake.cpp:776
job_counter
static int job_counter
Definition: remake.cpp:694
dependency_map
std::map< std::string, ref_ptr< dependency_t > > dependency_map
Definition: remake.cpp:517
Running
@ Running
Target is being rebuilt.
Definition: remake.cpp:529
working_dir
static std::string working_dir
Definition: remake.cpp:736
input_generator::in
std::istream & in
Definition: remake.cpp:1236
dependency_t::deps
string_set deps
Definition: remake.cpp:514
assign_t::value
string_list value
Definition: remake.cpp:552
merge_rule
static void merge_rule(rule_t &dest, rule_t const &src)
Definition: remake.cpp:1848
variable_generator::vcur
string_list::const_iterator vcur
Definition: remake.cpp:1196
prepare_script
static std::string prepare_script(job_t const &job)
Definition: remake.cpp:2175
ref_ptr::~ref_ptr
~ref_ptr()
Definition: remake.cpp:494
DEBUG_open
#define DEBUG_open
Definition: remake.cpp:818
Pipe
@ Pipe
Definition: remake.cpp:1067
instantiate_rule
static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst)
Definition: remake.cpp:1887
create_server
static void create_server()
Definition: remake.cpp:2611
log_auto_close
Definition: remake.cpp:805
client_mode
static void client_mode(char *socket_name, string_list const &targets)
Definition: remake.cpp:2934
still_need_rebuild
static bool still_need_rebuild(std::string const &target)
Definition: remake.cpp:2099
jobs
static job_map jobs
Definition: remake.cpp:644
rule_t::assigns
assign_map assigns
Assignment of variables.
Definition: remake.cpp:565
addsuffix_generator::suf
string_list suf
Definition: remake.cpp:1368
normalize_list
static void normalize_list(string_list &l, std::string const &w, std::string const &p)
Definition: remake.cpp:1006
variables
static variable_map variables
Definition: remake.cpp:619
skip_empty
static void skip_empty(std::istream &in)
Definition: remake.cpp:1036
dependency_t
Definition: remake.cpp:511
Word
@ Word
Definition: remake.cpp:1060
status_e
status_e
Definition: remake.cpp:524
addprefix_generator::prei
string_list::const_iterator prei
Definition: remake.cpp:1313
SyntaxError
@ SyntaxError
Definition: remake.cpp:1177
client_list
std::list< client_t > client_list
Definition: remake.cpp:613
keep_going
static bool keep_going
Definition: remake.cpp:667
show_targets
static bool show_targets
Definition: remake.cpp:721
server_mode
static void server_mode(std::string const &remakefile, string_list const &targets)
Definition: remake.cpp:2886
log_auto_close::still_open
bool still_open
Definition: remake.cpp:807
assign_map
std::map< std::string, assign_t > assign_map
Definition: remake.cpp:555
log::operator()
std::ostream & operator()(bool o)
Definition: remake.cpp:791
register_scripted_rule
static void register_scripted_rule(rule_t const &rule)
Definition: remake.cpp:1574
sigint_handler
static void sigint_handler(int)
Definition: remake.cpp:768
obsolete_targets
static bool obsolete_targets
Definition: remake.cpp:756
Eof
@ Eof
Definition: remake.cpp:1178
status_map
std::map< std::string, status_t > status_map
Definition: remake.cpp:544
start
static status_e start(std::string const &target, client_list::iterator &current)
Definition: remake.cpp:2380
usage
static void usage(int exit_status)
Definition: remake.cpp:3026
addsuffix_generator::sufl
size_t sufl
Definition: remake.cpp:1370
rule_t::deps
string_list deps
Dependencies used for an implicit call to remake at the start of the script.
Definition: remake.cpp:563
rule_t::stem
std::string stem
Stem used to instantiate the rule, if any.
Definition: remake.cpp:566
assign_t
Definition: remake.cpp:549
addprefix_generator::next
input_status next(std::string &)
Definition: remake.cpp:1330
get_function
static generator * get_function(input_generator const &, std::string const &)
Definition: remake.cpp:1416
old_sigmask
static sigset_t old_sigmask
Definition: remake.cpp:759
input_generator::nested
generator * nested
Definition: remake.cpp:1237
sigchld_handler
static void sigchld_handler(int)
Definition: remake.cpp:763
expect_token
static int expect_token(std::istream &in, int mask)
Definition: remake.cpp:1076
operator<<
static std::ostream & operator<<(std::ostream &out, escape_string const &se)
Definition: remake.cpp:836
load_dependencies
static void load_dependencies(std::istream &in)
Definition: remake.cpp:1439
Rightpar
@ Rightpar
Definition: remake.cpp:1064
ref_ptr::ref_ptr
ref_ptr()
Definition: remake.cpp:491
client_t
Definition: remake.cpp:601
Unexpected
@ Unexpected
Definition: remake.cpp:1059
socket_t
int socket_t
Definition: remake.cpp:462
log::active
bool active
Definition: remake.cpp:778
echo_scripts
static bool echo_scripts
Definition: remake.cpp:726
main
int main(int argc, char *argv[])
Definition: remake.cpp:3053
assign_t::append
bool append
Definition: remake.cpp:551
skip_spaces
static void skip_spaces(std::istream &in)
Definition: remake.cpp:1026
substitute_pattern
static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
Definition: remake.cpp:1871
RunningRecheck
@ RunningRecheck
Static prerequisites are being rebuilt.
Definition: remake.cpp:530
rule_t::wdeps
string_list wdeps
Like deps, except that they are not registered as dependencies.
Definition: remake.cpp:564
complete_request
static void complete_request(client_t &client, bool success)
Definition: remake.cpp:2438
has_free_slots
static bool has_free_slots()
Definition: remake.cpp:2472
addprefix_generator::prel
size_t prel
Definition: remake.cpp:1314
pid_job_map
std::map< pid_t, int > pid_job_map
Definition: remake.cpp:586
job_t::rule
rule_t rule
Original rule.
Definition: remake.cpp:580
client_t::running
string_set running
Targets being built.
Definition: remake.cpp:607
ref_ptr::content::val
T val
Definition: remake.cpp:486
input_generator::earliest_exit
bool earliest_exit
Definition: remake.cpp:1239
debug
static log debug
Definition: remake.cpp:803
socket_name
static char * socket_name
Definition: remake.cpp:710
addsuffix_generator::sufj
size_t sufj
Definition: remake.cpp:1370
ref_ptr
Definition: remake.cpp:481
ref_ptr::ref_ptr
ref_ptr(ref_ptr const &p)
Definition: remake.cpp:493
addprefix_generator
Definition: remake.cpp:1309
addsuffix_generator::gen
input_generator gen
Definition: remake.cpp:1367
variable_generator::variable_generator
variable_generator(std::string const &, variable_map const *)
Definition: remake.cpp:1201
status_t
Definition: remake.cpp:538
ref_ptr::operator=
ref_ptr & operator=(ref_ptr const &p)
Definition: remake.cpp:495
addprefix_generator::pre
string_list pre
Definition: remake.cpp:1312
waiting_jobs
static int waiting_jobs
Definition: remake.cpp:688
server_loop
static void server_loop()
Definition: remake.cpp:2825
log_auto_close::log_auto_close
log_auto_close()
Definition: remake.cpp:808