newLISP Development Release version 10.3.4

[Lutz]

Development release version 10.3.4 features a re-written, faster and enhanced message API and bug fixes.



All files: http://www.newlisp.org/downloads/development/">http://www.newlisp.org/downloads/development/

Release notes: http://www.newlisp.org/downloads/development/newLISP-10.3.4-Release.html">http://www.newlisp.org/downloads/develo ... lease.html">http://www.newlisp.org/downloads/development/newLISP-10.3.4-Release.html

Detailed CHANGES notes: http://www.newlisp.org/downloads/development/CHANGES-10.3.4.txt">http://www.newlisp.org/downloads/develo ... 10.3.4.txt">http://www.newlisp.org/downloads/development/CHANGES-10.3.4.txt

[newdep]

Thumb's up Lutz....!



I already have a new project on the table for newlisp, probably will be pointing towards embedding

newlisp inside hardware but lets see what it brings...

[TedWalther]

Wow. Great improvements.  I'm still a bit disturbed by the need for children to communicate with each other through the parent. That seems like a bottle-neck.  Is that restriction an important element of the Cilk API?

[Lutz]

It's not a restriction of the Cilk API, but not easy to implement without a central registry. I have a version using the shmxxx() API, but it is too slow. It's memory based and the OS kernel has a central registry, which reserves memory segments for each child, but it's too slow. The advantage is, that you can get a handle to a shared memory segment via a PID based key, but again it's too slow. I have yet another version, rolling my own shared memory scheme and with semaphores  (10.3.3 uses shared memory but w/o semaphores) , but it also is much slower than the current socketpair() based version.



The current version 10.3.4 is based on UNIX local domain socket pairs, and it turned out to be the fastest (several times), it is stream/queue based. But you don't want to open socket pairs for all possible connections, that would be 10,000 for a 100 processes and most likely exceed OS limits. With the small size of newLISP, you could easily have a 1000 processes running on a modern machine. So you need some kind of a central registry to open these connections and request the socket numbers via PIDs or some other handle ID. I am still working on this problem.



For now, a parent based message broker/proxy is pretty simple to implement in newLISP, see the new qa-specific-tests/qa-message .



[ following later added ]



Also, a central broker opens a lot of opportunities, e.g. logging, filtering, prioritizing and a zillion of other possibilities. We are just at the beginning of wide spread multi-core computing. Most current languages do concurrent processing with threads in only one CPU core - that is a dead end.