development version newLISP v.8.5.9

[pjot]

Well, I got something working.



http://www.turtle.dds.nl/newlisp/winfork.lsp">http://www.turtle.dds.nl/newlisp/winfork.lsp





The original "prodcons.lsp" works in Windows now with some small modifications:



1) Removed the (wait-pid) function, does not exist in newLisp for Windows

2) Load the "winfork.lsp" context first

3) Syntax is: (winfork '(function)) , so calling the symbol of the function instead of the function itself.



The "prodcons.lsp" demo program for Windows can be found here:



http://www.turtle.dds.nl/newlisp/prodcons.lsp">http://www.turtle.dds.nl/newlisp/prodcons.lsp





Anybody who can improve the WINFORK please do so (and let me know...;-) )



Peter

[Lutz]

Excellent! but here 2 small suggestions to make the code even shorter and faster: (1) use 'source' to convert symbols to their source code:

Code Select Expand


(set 'a 123)
(set 'b 456)

(define (foo x y) (+ x y))

(set 'environment "")
(dolist (s '(a b foo)) (write-buffer environment (source s)))

; environment now contains all the source for 'a 'b and 'foo


'source' takes care of all the conversion



(2) when using (write-buffer ...) with a string instead of a file handle, you have a very fast in-memory string append.



Lutz

[pjot]

Nobody can beat the master! ;-)



The WINFORK context is much smaller now and works pretty cool. I uploaded new a version at



http://www.turtle.dds.nl/newlisp/winfork.lsp">http://www.turtle.dds.nl/newlisp/winfork.lsp



Now the 'fork' command can be used in Windows as well!



Peter

[pjot]

Further testing seem to reveal a memory leak in the Windows-version of 'share'.



If you change the 'prodcons.lsp' example to let it start with '(run 100000)' and ALSO comment all semaphore stuff; watch your processlist, the spawned newLisp processes will grow until they stop (at 134mB on my WinXP system).



However, if I change the 'prodcons.lsp' example like this:

Code Select Expand


(load "winfork.lsp")

(constant 'wait -1 'sig 1 'release 0)

(define (consumer n)
(set 'i 0)
(while (< i n)
#(semaphore cons-sem wait)
#(println (set 'i (share data)) " <-")
(println "Consumer")
#(semaphore prod-sem sig)
)
(exit))

(define (producer n)
(for (i 1 n)
#(semaphore prod-sem wait)
#(println "-> " (share data i))
(println "Producer")
#(semaphore cons-sem sig)
)  
(exit))

(define (run n)
(set 'data (share))
(share data 0)

(set 'prod-sem (semaphore)) ; get semaphores
(set 'cons-sem (semaphore))

(set 'prod-pid (winfork '(producer n))) ; start threads
(set 'cons-pid (winfork '(consumer n)))

(semaphore prod-sem sig) ; get producer started

(sleep 10000) ; make sure semaphores are not released too early

(semaphore cons-sem release) ; release semaphores
(semaphore prod-sem release)
)


(run 100000)

(exit)


...then the memory usage is not growing. So it is definitively in the (share).



If I run the original 'prodcons.lsp' in Linux, then the memory usage is OK.



Peter

[pjot]

If I run this on a Win2000 machine the memory usage also increases to 134 Mb (but newlisp does not crash).

[pjot]

Probably the issue is caused by these lines in win32-util.c (lines 224-227):

Code Select Expand


UINT * winMapView(UINT hMemory, int size)
{
return((UINT*)MapViewOfFile((HANDLE)hMemory, FILE_MAP_WRITE, 0, 0, size));
}


Every time the function MapViewOfFile is called, it will cost you some additional memory. Therefore each process should only call it once, and then refer to the same returned address every subsequent time.



I also found this remark:

Note also that MapViewOfFile() increments a usage counter that is decremented by UnmapViewOfFile() or on process termination. The physical named object is not removed from memory until the last process has unmapped the object, and the usage counter is set to zero.

...at  http://www.jps.at/dev/kurs/3-19.html">http://www.jps.at/dev/kurs/3-19.html.



Peter

[Lutz]

Thanks Peter this is fixed for 8.6.0.



Lutz