Newlisp vs Lua

[Astrobe]

Let's Rock'n'Troll!



I've lurked on comp.lang.forth for a long time. One thing I couldn't understand is why people constantly compared Forth with C, which was pointless in my eyes because they where in whole different categories.

It became even more pointless when Lua began to gain traction, because as a lightweight interpreted language, it was a serious contender. That's even more true for Newlisp.



+ Lua and Newlisp are about the same binary size, but Newlisp packs a lot more functionality out of the box

+ Newlisp doesn't rely on garbage collection (but the pass-by-value semantics that ORO requires are sometimes difficult to deal with)

+ Newlisp has macros

+ Newlisp's documentation is better

+ Newlisp is a simpler language, so I believe that parsing speed is better than Lua.

- Lua is faster, even without JIT

- Lua has better debugging support

- Lua has coroutines (Newlisp has IPC but it's heavier and not 100% multi-platform)

- Lua has much more mind share (nothing you can do about it, except make Newlisp better)



All points are not equally important. Lacking lightweight threads isn't a big deal as long as one has non-blocking I/O. Debugging support is important because bug squashing can kill productivity - especially for late-bound dynamic languages in which a simple typo can result in a bug (that's why most such languages have a so-called "strict mode"). Being slow can be acceptable to some extend, as long as interfacing with C (or eventually with external programs but that's really a last resort) is easy.

[TedWalther]

I've been thinking about coroutines over the past 2 years.  To my surprise, I think I've found a way to implement coroutines for newlisp that doesn't involve any assembly language or tricky C.  It is just pure AST manipulation.  However, it definitely wouldn't be lexically scoped. :)  I went back and read the original LISP paper by McCarthy, and the manual for LISP version 1, and it opened my eyes; newLisp is actually a "third way" with Common Lisp and Scheme being the other branches.  I think SmallTalk is also in the family, but just a little TOO different to really count.



I talked to Chuck Moore in person and asked him point blank about the similarities between FORTH and LISP, and he agreed.  Before it got pushed too much in an academic direction, LISP also was small and practical and close to the machine, allowing you to easily drop down into Assembler, and had primitive functions to twiddle with the machine registers, stack frames, etc.  I think SmallTalk originally also had this property and maybe still does, at least if you count the Virtual Machine as the assembly language level.



LISP version 1 was 50kilobytes in size without the built in assembler/compiler, about 150 kilobytes with.  Wonder how hard it would be to implement that today.  But with all the different CPU out there, probably no worthwhile. :(  Unless we go the VM route, then that brings us to Lua, Erlang and Java VMs, which ones to choose.  Or even the TCL byte-code VM.  Or the emacs VM.  And finally, LLVM.

[TedWalther]

About Mind-share, Lua has licensed a bit differently in a way that allows proprietary software.  And it has a university department that has been developing and promoting it for decades.  Guy Steele wrote about the initial user communities for a language being crucial to it taking off.  So, what is the killer app that would make newLisp "take off"?  Also, does Lutz want it to take off more than it already has? :)  To read Guy Steele's paper, look up "Evolution of Lisp" and "History of Lisp" by Guy Steele.

[rickyboy]

About Mind-share, Lua has licensed a bit differently in a way that allows proprietary software.

Good point.  I always try to be mindful of what the licensing is for the tools that I use, and I consider MIT/BSD/ISC and the like to be superior (namely, more free) than any GPL.  Good job, Lua donks!



https://www.lua.org/license.html">https://www.lua.org/license.html



That said, the license of newLISP does not extend to the newLISP code that I write (which I would license as something other than GPL if need be).  I have never contributed, and don't anticipate contributing, to newLISP development per se -- Lutz is doing a fine job by himself.  So, for newLISP, I think that licensing is not a big issue; if not, a non-issue, or rather, shouldn't be considered critical or important.  (But as we all know, perception often wins out over bare facts.  I get it, as sometimes, I'm the cause of such problems/issues myself. :)



Please disabuse me of anything I got wrong or let me know if I forgot something.  Interesting discussion! Thanks, --Rick

[TedWalther]

So, for newLISP, I think that licensing is not a big issue; if not, a non-issue, or rather, shouldn't be considered critical or important.  (But as we all know, perception often wins out over bare facts.  I get it, as sometimes, I'm the cause of such problems/issues myself. :)

License is an issue if you want to distribute a single executable without your client having access to the source code, and if you want to statically link the newlisp DLL into your C/C++ program.  Having a single executable that you can drop onto another computer without any dependancies is very handy at times. There is a current project I'd love to use newLisp for, but I'm grinding it out in C++/Qt instead.  What I'd love is something like Delphi, but LISP instead of Pascal.  With a BSD license, I think I could provide some add-on module for newlisp that would do that.



What Lua got right, and Tcl and Guile also are attempting, is that an application can make itself "programmable" using a standard language.  This way, skills can transfer between applications.  I'd like newLisp to be able to enter that space.  Also, implementing coroutines would be a fun little exercise, wouldn't require any "wierd" C at all.

[rickyboy]

Good point. I wonder if Lutz would be willing to dual license newLISP. Then, you could have some fun on your project. :)

[TedWalther]

A thought on co-routines; how do you implement them so the call stack doesn't blow up?  Is such an implementation possible under the ORO memory manager?  The bash shell uses ORO, and Unix pipes have been compared to coroutines, maybe there is some insight to be gleaned there.  If coroutines require code to avoid stack blowup, then maybe tail calls would make sense too.  But then debuggability goes down the toilet.

[Astrobe]

Actually to me co-routines are not a major feature in Lua. One problem is that a procedure which is not "co-routine aware" cannot be turned in to a "co-routine aware" one. If someone provides a GUI library without a single-step gui.run() function (silly example), you are screwed. So if you have a long-running "task" (GUI, servers, heavy number crunching,...) you have to plan ahead. If you look at the https://www.lua.org/pil/9.html">examples in PIL, doing that requires significant rethinking anyway. For instance, going from the "blunt" permutation generator to the coroutine-aware is not trivial. It seems to me that turning it into a context function would require the same work in Newlisp (and the syntactic equivalence of lists and lambdas in Newlisp could probably allow us to apply a dolist to it).



Since you know Forth, someone on comp.lang.forth a long time ago quoted this definition:

: CO >R >R 2>R ;

It swaps the top elements of the return stack. Since it is a regular definition, when it returns, it returns to the caller-of-the-caller. And when the caller-of-the-call returns, it returns to the call site of CO. I actually never used that myself because it hurts my brain.

A variation is to fetch the return address from the return stack and then execute that - in other words call the return address (this is totally not standard). I think it has some similarities with call/cc.

Of course, Forth is the massive two-stacks cheater it usually is here.

What Lua got right, and Tcl and Guile also are attempting, is that an application can make itself "programmable" using a standard language. This way, skills can transfer between applications. I'd like newLisp to be able to enter that space. Also, implementing coroutines would be a fun little exercise, wouldn't require any "wierd" C at all.

I don't think Lua is "more right" here. It began as a configuration language and naturally evolved into an embeddable scripting language, while others including Newlisp began as a scripting "glue" language. Lua may be better if you want to add scripting to an existing app, but if you start with that idea, both approaches seem equally viable to me. But there are significant differences.



Emacs is an interesting case in this regard. Elisp is specific to Emacs so there's no skill transfer between applications, but the cooperation between applications written in Elisp (org-mode, erc, magit,...) is more than remarkable. Emacs is not an OS for sure, but it is now a full blown application platform. To the point that one could compare it to something like Pharo Smalltalk.

[joejoe]

newLisp vs Lua.



These are two differently licensed projects, which make them incomparable.

I always try to be mindful of what the licensing is for the tools that I use, and I consider MIT/BSD/ISC and the like to be superior (namely, more free) than any GPL.

If you contributed code that your grand-kids could not make use of because your license choice allowed the boxing up and redistribution of your source code without sharing the source code to your grand-kids, that would be a loss, like not really free code (i.e., MIT/BSD).



Dual-licensing allows for deception and deprivation, a novice move. Freedom does not tend towards denying others a freedom you were freely given.



The authors of GPL software want to make sure that any contribution to their codebase remains "free". Free, like code that is not able to be boxed up as unreadable, unredistributable, unusable software.



Everyone here understands how their (copyleft) contributions remain available to everyone else because the GPL requires any redistribution of their code to include the source code, important when you want to know what your computer is doing.



Here's a better explanation than I can offer. Well worth the reading!



https://www.gnu.org/licenses/copyleft.en.html">https://www.gnu.org/licenses/copyleft.en.html



"Copyleft says that anyone who redistributes the software, with or without changes, must pass along the freedom to further copy and change it. Copyleft guarantees that every user has freedom."



Lua is MIT/BSD, newLisp is GPL. These are two different permissions.



If you want your contributions able to be denied to another, go with Lua (MIT licensed software).



If you want to make sure you contributions are available and impermeably free, go w newLisp (copyleft).



MIT/BSD projects do not last over time because too many contributions are absconded with.



License consideration allows for software utility, part of what makes nL better than anything out there!

[TedWalther]

If you contributed code that your grand-kids could not make use of because your license choice allowed the boxing up and redistribution of your source code without sharing the source code to your grand-kids, that would be a loss, like not really free code (i.e., MIT/BSD).



Dual-licensing allows for deception and deprivation, a novice move. Freedom does not tend towards denying others a freedom you were freely given.

You need to look at it from both sides.  Developing software to fill a niche takes time energy and effort, "life force".  A developer doesn't get that back, but he can get paid.  Making everything open source makes a lot of niches impossible to fill.  I have a family to feed.  When my family is fed, and I have spare change, I can contribute back to the ecosystem that supported and enabled me.  The choice I have is, choose a closed ecosystem to contribute back to by paying licensing fees in turn, or contribute back to the open source ecosystem, by paying the developers who work on the ecosystem itself.  GPL takes that choice away from me.



The big issue in the small niches, one client can't pay the whole cost of development.  You need several clients.  And it can take years to get those clients.  But the software already has to be developed, and available under a suitable license.  Otherwise, one client takes the program and runs with it.  A lot of small shops just get by.  Open Source infrastructure helps them; making their own niche software open source by default, does not.  If a client wants something to be open source, the cost goes up.  Most clients aren't interested or willing in paying extra.  Open Source software is too expensive for many niches; with the BSD license, at least the option remains for a client to be provided source if they need it and are willing to pay the premium.  What premium?  The risk premium to the original developer.



For business customers, the real open source that they usually need is open data formats.  If they don't like your software, they should be able to take their data elsewhere.  That also gives them a freedom very much like open source code; the ability to inspect and manipulate their business data with tools they develop in house.



Am seriously looking at FreePascal right now.  But would much rather use Lisp.  Maybe I have to go with ECL (Embedded Common Lisp) which comes with Qt support built in.



As for my grandkids, if you think BSD license will prevent them from accessing my source code, you are wrong.  Even in the unlikely event that said source is still relevant at the time they are ready to program computers.

[TedWalther]

joejoe, you are fundamentally misunderstanding the issue here.  Creating an application, then "compiling" it in into a standalone binary, which means the newlisp executable plus the newlisp source code tacked onto the end, doesn't alter newlisp in any way.  Using the GPL here prevents distributing such a "compiled binary" without also distributing the source to your application.



Even the GNU C compiler allows you to compile a static binary, which includes a substantial part of the libc, without having to give away the source code to your application.  That is why the library exception, the LGPL, was put in place.  Richard Stallman acknowledged the reality that Free Software may be desirable, but isn't possible or practical in all instances.



Since you try to quote GNU to me, who has been developing GNU software for 20 years, read this link:



https://www.gnu.org/licenses/gcc-exception-3.1.en.html">https://www.gnu.org/licenses/gcc-exception-3.1.en.html

[joejoe]

TedWalther,



I have long appreciated your advice and direction.



On multiple occasions you have guided me in the proper direction.



Thank you. Your responses I regard as true mentorship. That is absolute.

You need to look at it from both sides. [...]

The choice I have is, [...] [contribute to closed or open ecosystems].

Yes, the GPL requires that the end user (our own selves included) rank higher than a developer/distributor's personal interests. I would agree and still hold that my spent 'life force' is greater when more people can make more use of it than I can myself alone.

GPL takes that choice away from me.

As it is designed to. Greater good outranks personal profit, any way we dice it.



It does so to insure the allowance to end users, a larger benefit to our society.



Multiplication stacks way higher than does simple addition. It's your choice always.

As for my grandkids, if you think BSD license will prevent them from accessing my source code, you are wrong.

If I grab your BSD code and redistribute it closed, I'd say your code would not be available to your grandkids.



Grandpa, what's my computer doing? 'I used to know'.

Even the GNU C compiler allows you to [...]

Survivability still often supercedes even the greatest of ethic.



I'm not in. Wouldn't want anyone not to know what their computer is doing.