threads

[eddier]

Code Select Expand


(context 'DATA)
(setq t1 100 t2 100)

(context 'MAIN)
(define (thread-1 x)
  (while (!= x 0)
    (begin
     (sleep 100)
     (setq DATA:t1 x)
     (dec 'x))))

(define (thread-2 x)
  (while (!= x 0)
    (begin
     (sleep 100)
     (setq DATA:t2 x)
     (dec 'x))))

(define (observer)
  (println DATA:t1 "," DATA:t2)
  (setq i 40)
  (while (!= i 0)
    (println "thread-1 " DATA:t1 "nthread-2 " DATA:t2)
    (sleep 100)
    (dec 'i)))

(fork (observer))
(fork (thread-1 20))
(fork (thread-2 20))

(exit)


Three questions:

 1. Why doesn't fork 1 and fork 2 change DATA:t1 and DATA:t2 respectively?

 2. How do I communicate with a thread?

 3. Why do I have to hit "Enter" to get the main program to exit. This means CGIs with threads won't work.



Eddie

[Lutz]

(1) When a thread is created, it copies the current newLISP environment, but then executes independently, just like a fork() in 'C'. To communicate between threads you could use the newLISP function 'pipe',  the read and write handles (like any other file/socket handles) are available to the child threads. You could create several pipes executing several (pipe) calls.



I use pipe and threads for some of the changes in qa test routines this way:



  (define (test-pipe)

   (set 'channel (pipe))

   (set 'in (first channel))

   (set 'out (last channel))

   (fork (write-line (upper-case (read-line in)) out))

   (write-line "hello there" out)

   (sleep 1000)

   (= (read-line in) "HELLO THERE")

   )



The sleep is necessary on some OS to make this work. (Note that sleep for non-full seconds was broken previously to 8.1.7 on some compiles).



(1) Your main program has exited, it just doesn't show the prompt until you hit enter. Try the following experment:



;; program test



(println "main program")



(fork (dotimes (x 10) (println x) (sleep 1000))) ;; start thread



(println "main now exits")

(exit)



;; eof



Now make a batch file 'batch' containing the 2 lines:



newlisp test

ls



Now do:



./batch



You will see the directory listing right away, then you will see the numbers in the console window printed. newLISP starts the thread and then exists and 'ls' is executed in the batch fle. I tested this on BSD and CGWIN.



Here the test routine I use to test 'pipe' on Linux/UNIXs:





Lutz

[eddier]

Ok. I understand now.

Code Select Expand


(define (count-down-thread x channel)
  (while (!= x 0)
    (begin
     (sleep 100)
     (write-line (string x) channel)
     (dec 'x))))

(define (observer-thread channel)
  (setq i 100)
  (while (!= i 0)
    (println "thread " (setq i (integer (read-line channel))))))

(map set '(in out) (pipe))
(fork (observer-thread in))
(fork (count-down-thread 20 out))

(exit)


Thanks Lutz!

[Lutz]

Nice threads communicatins demo! I'll include this in the manual.



Lutz

[eddier]

Thanks. I think the following would be a bit better though. I didn't like setting i to the arbitrary 100 and there is no need for integer conversion in the example.

Code Select Expand


(define (count-down-thread x channel)
 (while (!= x 0)
  (begin
    (write-line (string x) channel)
    (sleep 100)
    (dec 'x))))

(define (observer-thread channel)
 (setq i "")
 (while (!= i "0")
    (println "thread " (setq i (read-line channel)))))

(map set '(in out) (pipe))
(fork (observer-thread in))
(fork (count-down-thread 20 out))

(exit)


Eddie

[nigelbrown]

Here is a demo using a threaded Seive of Eratosthenes to find primes:

Code Select Expand


; a Seive approach to prime finding with each newly found prime getting
; its own thread to filter out multiples of itself
; a number is passed through the chain of threads
; and if it reaches the last it must be prime

(define (die p)  ; function used to terminate a thread and pass on die signal
              (println "Killing " (string p)) ;scream
              (if out (begin (write-line "0" out) ; pass on the poison pill
                             (close out)))
     (close channel) (exit)) ; die
     
(define (pass-or-fork n)
; if a downstream thread exists then pass on the number
; else fork a thread - a new prime has been found
     (if out (write-line (string n) out)
     (begin
                 (map set '(in out) (pipe))
(fork (prime-thread n in counter)))))

(define (prime-thread p channel , p2 in out)
   (setq p2 (* p p))
   (println "Prime number " (string (inc 'counter)) " is " (string p) )

   (until (=  (setq i (integer (read-line channel))) 0) ;receive a number to test
       ; only do more tests while p<sqrt(i)
       ( if (or (< i p2) (!= (% i p) 0)) (pass-or-fork i)))
   
   ; received 0 is signal to finish
   (die p))

(define (testto nlimit, in out) ;test function
     (map pass-or-fork (append (sequence 2 nlimit) '(0)))
     (exit))

; testing
(setq counter 0) ; initialise prime counter
(testto 600)

For larger number of primes a larger call stack is needed viz

[nigel@p800 nigel]$ ./bin/newlisp -s 6000 primethreads1.lsp





Nigel

List of primes to check against  http://users.argonet.co.uk/oundlesch/alists/prim.html">http://users.argonet.co.uk/oundlesch/alists/prim.html

Seive of Eratosthenes  http://www.math.utah.edu/history/eratosthenes.html">http://www.math.utah.edu/history/eratosthenes.html

[pjot]

Hi Lutz,



I have looked into the possibilities of "fork". The "pipe" to communicate with "fork"-ed functions works OK, but how to communicate with forked external binary's?



Suppose I have this in KSH:



---------------------------

#!/bin/ksh



bc |&



print -p "3*4"

read -p TMP



echo $TMP

---------------------------



The "bc" binary is forked (it's the standard Linux calculator program), after which the calculation "3*4" is sent to a pipe connected to bc (stdin). Now, bc returns the result to the  pipe (stdout), which is then captured by the script.



So how to implement this simple example with newLisp?



For if I do this: (fork "bc"), I will receive the PID all right, but the binary vanishes from the processlist immediately. And if I do this: (fork "bc &") also the PID vanishes. Neither a (process "bc") works, nor an (exec "bc").



Any ideas? So suppose I want to be able to communicate with external binary's, how can I do that?



Peter

[pjot]

Never mind the "fork", I mixed up the functionality of a co-process with a fork. Of course a fork can only fork the main process which invokes the fork; I see you really use the C-function "fork" for that.



I am too pre-occupied with my own programming problems, this made me blind... ;-)



Maybe I should have posted my question in a separate thread. The problem however still remains: how to communicate with a co-process? Is this possible with newLisp?

[Lutz]

You could also use 'pipe' or any network functions. Named pipes would also work. Basically this is what you are already doing in your GTK-server.



Lutz

[pjot]

Network functions only work if the binary involved supports networking. The "bc" binary does not have this.



The "pipe" command creates pipes for IPC, but I only can find examples of IPC with newLisp code. So, how would I use a "pipe" with an external binary like "bc"? Is this possible with newlisp?



In the documentation this is mentioned: "The pipe handles can be passed on to a child process or thread launched via process or fork for inter process communications." So how would I pass these handles to the "bc" binary?

[Lutz]

This will work:



(exec {echo "3 * 4" | bc}) => ("12")



This opens a process-pipe. unfortunately most Linux and most UNIX do not allow a process pipe to be opened for reading and writing at the same time. For that reason the the trick with echo must be used.



Lutz

[pjot]

Thank you for your example; this works. The disadvantage here is that for every new calculation a new pipe have to be started; the "bc" binary cannot keep track of a history. In case of the gtk-server for example, the history of sent GTK calls must be remembered; I cannot use the trick you mention here.



I think the 2-way pipes is not a matter of Unix but of KSH. I have tested Tru64Unix, Solaris, Linux and OpenBSD with the small KSH example above, on all these platforms it works.



I know it (2-way pipes) can work on Windows as well; GNU Prolog supports it for example, they have a command like this:



exec('bc', Out, In, _, _)



The "Out" and "In" are the stdout and stdin for the bc binary. And this is not Cygwinned!



Probably there is a hack with newLisp using a combination of "fork", "process", "pipe" etc. to simulate the same behaviour as the 2-way pipe in KSH, but right now I cannot see how.



So far I think newLisp is the strongest and most powerful interpreted language I've ever seen, and I find it annoying that a simple 4-line example in KSH seem to beat the possibilities of newLisp.



(In CShell 'coproc' is used to start a coprocess; in AWK the symbol '|&' is used as well.)

[Lutz]

There will be a  read/write piped process in the next version



Lutz

[pjot]

As I said before, your support is extremely good! If I find out your address I'll send you a couple of strong Belgian beers... ;-)

[Lutz]

By the time they are here with the mail, they are stale, unless you send them by airmail :-). Actually its not to hard get Belgian beer here in Florida, there are a couple of stores with European beer sections and they sometimes have free tastings, which I take adavnatage of, whenever they let me.



Lutz