Any plans for a Compiler or JVM mode?

[AlienChild]

Hi,



I really like the idea of newLisp but for me often code execution speed is an issue. Are there any plans afoot to create a compiler for newLisp or perhaps something running on Java Virtual Machine?

[Lutz]

There are no plans for a VM or compiler. Any compilation to either machine code or VM byte code would loose important dynamic aspects of the language. newLISP in the family of scripting languages is fast. Compare some benchmarks here:



http://newlisp.org/benchmarks/">http://newlisp.org/benchmarks/



if parts or your programs need more speed you could write them in C. newLISP as a facility to import C library routines and execute those C rotuines as if they where part of the language. In the manual see:



http://newlisp.org/downloads/newlisp_manual.html#import">http://newlisp.org/downloads/newlisp_manual.html#import



or for a more detailed description see:



http://newlisp.org/DesignPatterns.html#extending">http://newlisp.org/DesignPatterns.html#extending



Lutz

[AlienChild]

I understand that among scripting languages that newLisp easily holds its own or better, but in general the same code running under a compiler will be about 10 times faster (as a very rough general approximation). Using C libraries can of course reduce this by a lot.



However there are good compilers for Common Lisp (SBCL) and Scheme (Larceny, Bigloo, Chicken, Stalin, Gambit)  that produce extremely fast code while still retaining the power and flexibility of the Lisp family of languages.



I am interested however in Neural Network and Genetic Algorithms for non-trivial problems. These types of problems don't lend themselves easily to optimization and really need raw speed throughout. Another problem domain I need to address is Quant programming and games. While often one can get significant performance gains here from libraries I would prefer to largely develop everything in one language/environment.



I realize that this request probably falls outside the original goals for newLisp and I wouldn't use Ruby/Perl/Python either to solve my kind of problems. However I am positive I am not the only only who would love to see newLisp having the option of creating a highly optimised, compiled stand alone .exe form.



That way we could stay with one language, get great scripting ability and great stand alone compiled speeds as well - or are these 2 goals mutually exclusive?

[Lutz]

I am interested however in Neural Network and Genetic Algorithms for non-trivial problems.

some time ago I looked into this library for neural networks, and thougth it would be easy to write a import-module for it in newLISP:



http://lwneuralnet.sourceforge.net/">http://lwneuralnet.sourceforge.net/



For doing such a job one would have to know newLISP a little bit more and be somewhat familiar with C programming to translate C headerfile calling patterns in to the correct newLISP import pattern. The last reference in my last post gives a lot of details.

... while still retaining the power and flexibility of the Lisp family of languages

There are some features of dynamic LISP which cannot be compiled, i.e. treating, manipulating LISP code as data (and reverse) at runtime. I would agree, that on many projects these features are not important, but on some they are, i.e. in Genetic Programming.



Anyway, if you want to take the plunge into newLISP and need help to learn the language or with importing libraries I and others on the board will be glad to help.



Lutz

[nigelbrown]

One area of optimisation with NewLisp to watch is the list copying done as part of memory management - the approach works well if you are avoiding pauses from GC  but you may need to look for places you could use destructive list operations (a bit un-lispy I accept) to avoid copying overhead.



If you find a good matrix library that NewLisp could call for some bulk processing of genetic algorithm steps let us know.

Nigel

[cormullion]

(possibly irrelevant and tangential comment follows...)



Perhaps you could design your approach around the idea of parallel computation using newLISP processes. You could try splitting tasks up into multiple processes running on multiple machines, using (net-eval)...



newLISP seems well-suited for this multiple-process agent-based computing approach. Usually there's up to half a dozen newLISP processes starting up and closing down on my machine at any one time, and even some one-shot scripts bound to editor keystrokes. I suppose it's partly the speed of start-up that makes this possible...

[Lutz]

Your point illustrates very well the philosophy of newLISP computing when building applications. This is exactly what I have in mind and how bigger applications with newLISP are built and should be built.



We have to get away from thinking in big monolithic programs, to thinking of many smaller agent programs collaborating.



Lutz

[m i c h a e l]

> We have to get away from thinking in big monolithic programs, to thinking of

> many smaller agent programs collaborating.



Maybe this is the OO in newLISP? Like Erlang's way of doing objects (communicating through processes). I've always found this way of doing objects a more natural a fit with our mental models of the world (at least mine, anyway :-)



Back into hibernation . . .



m i c h a e l