shell-like scripting

[xificurC]

Looking at http://www.newlisp.org/downloads/newlisp_manual.html#processes">http://www.newlisp.org/downloads/newlis ... #processes">http://www.newlisp.org/downloads/newlisp_manual.html#processes I see there are several ways to start an external process, namely !, exec, process and spawn. So I thought let's write my typical toy tool - parallel git pull. It's a simple script that gets a set of root folders, looks for subfolders that are git projects and runs a git pull inside them. This should run in parallel since most of the time is spent on waiting for the network. The output to the console should be updated live and each line should be a simple "{project-folder} finished pulling with exit code {exit-code}".



So I need the bash equivalent of something like "git pull &>/dev/null; exitCode=$?", i.e. suppress all output and fetch the exit code. I see a combination of process, exec+which and wait-pid can do this job.



Having this explained my question is - can I retrieve this output live from subprocesses with the Cilk API without resorting to send/receive? I see that the result of a subprocess is stored in the symbol one gives to spawn but I don't see a reasonable way to fetch that information using sync. Am I missing something?

[rrq]

As you know, one way to pass command return code is via ! and wait-pid, the former captures the return code of the invoked shell command, and the latter of a child (though the function return values need downshifting 8 bits to be the actual return code). I made the following toy example to experiment with this

Code Select Expand

(constant 'commands
  '( "sleep 3; exit 1" "sleep 2; exit 2" sleep 2; exit 3" ))

(define (psub x)
  (fork (exit (>> (! (format "%s >/dev/null 2>&1" x)) 8))))

; Set up a table of forked pids and their commands
(setf subs (map list (map psub commands) commands))

; Process child return values as they come about
(dolist (p subs)
    (let ((x (wait-pid -1)))
        (println (lookup (x 0) subs) " ended with " (>> (x 1) 8))))

Perhaps you envisioned something like that?

[xificurC]

Yes, that pretty much does it, thanks! As a nitpick - is there no way to wait-pid for something that is forked only once? Looking at the sources ! uses system which is a fork+exec sh -c and you need to wrap that in another fork to get the pid. And why is the exit code shifted?

[rrq]

The ! function waits for the command to complete, so without the outer fork, only one sub process at a time would be performed. You could redesign it to use process, which includes its own fork+exec, to avoid the explicit outer fork (and the first explicit right-shifting); maybe that'd be more succinct, really. Though in practice you may well need an explicit fork anyhow so as to deal with that all sub processes share stdin/out/err with the parent process.



The right shifting is needed because the return value is that of the wait system call, which in short, combines the process return code with a control byte. See

Code Select Expand

$ man 2 wait for the gory details. (i.e. look up the "wait(2)" man page). Thus, the right shifting discards the control byte and becomes the actual exit code.

[xificurC]

Yeah, I just felt a bit of redundancy, ! is synchronous which means it calls fork, exec and wait (or something similar, not sure how system is coded exactly). Then we call another fork and wait. But as you said process could solve this, although that requires a command-string which means I either have to go {sh -c "..."} or find my executable (e.g. git) with an exec which first. I would probably prefer having access to execlp directly in some cases :)



Anyway, thank you for your help, my script works flawlessly now ;)