query: threads in newLISP?

[kazemori]

hello Lutz!



you have been rather busy! these past days releasing so many updates to newLISP; i have been reluctant to burden you with additional questions.



here is a general interest query: does newLISP support threads? threads are never mentioned in the newLISP documentation, so one could assume the negative.

(and we all know what happens when we assume ;-)



if threads are not supported, would implementation be difficult (like "never-going-to-happen" difficult), and do you have any plans in this direction?



thanks in advance for your time! -- kazemori

[Lutz]

I thought about this many times, and one day perhaps it will happen. Because of the way memory is managed (no garbage collection) each thread would need it's own stacks for evaluation environments and intermedeate results. They would be able to use the same heap of recycable Lisp cells though and time-wise the overhead would propbably be minimal. I did something similar a decade ago on a Prolog dialect called 'RuleTalk'.



Anyway, you would have an overhead of about 80Kbyte of memory per thread. These threads would share symbols, which includes of course all user defined functions. Threads implemented this way would have no need of threads in the underlying OS and could not block each other except when doing I/O operations (but Ok with  sockets, which can be used in a non blocking fashion in newLISP)..



Are you thinking about a specific application? Perhaps there is a solution running multiple instances of newLISP and letting them communicate via sockets (which also can run in a non-blocking fashion).



Lutz

[kazemori]

at the moment i am only thinking about the future (what i may have to do). certainly for limited problem-spaces, i agree that running multiple instances of newLISP (communicating via sockets) is a straight-forward and acceptable solution.



is it possible to limit the "scope" of a thread, so that it is not necessary for all symbols and user-defined functions to be shared? newLISP already implements contexts, in effect creating multiple namespaces (newLISP is a Lisp-n  ;-) -- is it possible to limit a thread's "scope" to its calling context? consider a single instance of newLISP with at least n = 4 active contexts (SYS, MAIN, user-context-parent and user-context-child), wherein the "child" context is the calling context:



(a) perhaps this would limit resource allocation on a per thread basis (overhead). 80kb per thread is an objectively modest requirement; however, it is subjectively large (comparable to an instance of newLISP).



(b) by extension, demonstrating concurrency across many threads would be analogous to the object-oriented paradigm operating over simultaneous instance methods.



(c) if a thread "needs" to encompass the entire superset of namespaces, this is (more or less) the functional equivalent to a new instance of newLISP, in that a separate instance provides (relatively) equivalent functionality.



of course, i may just be rambling now... ;-)



thanks!  --  kazemori

[Lutz]

Most of the time with processes you want a high degree of separation of computing resources and enable communications or sharing of memory with special means. With threads you want to share resources to make common use of functions or memory as painless as possible.



I would not want to limit thread access to contexts, it also would not save memory overhead. Using contexts, every symbol from a foreign context has to be prefixed by the context name. So an accidental misuse of a context in a thread would be highly improbable.



Whenever you implement separating of resources quickly the demand for ways to work around it rises. Both, C++ and Java developed schemes of various degrees of separating access to functions or variables in objects, which made C++ IMHO to clumsy to use and overloaded it with theoretical concepts, which have to be learned and understood.



Probably some C++ users will vehemently disagree with me. I love Java because it implements a much easer to understand model of separating object resources.



But so much in using programming languages is just personal style and preference. Just look at the vastly different philosophies of doing a LISP from newLISP to Scheme to Common Lisp.



If I implement threads in newLISP I would want to do it as simple to use and understand as possible.



Lutz

[kazemori]

so, i was just rambling after all... (red-faced with embarrassment)



if you do implement threads, please do it in a simple and transparent way.



meanwhile, i will endeavor to keep correct working definitions of processes and threads separated in my mind (so that they will never again spill out so incorrectly into my posts!  ;-)



thanks for your patience!  --  kazemori

[Lutz]

sorry if I sounded like a teacher, I was just looking for an opportunity to use the nice formatting features of this forum software :)



Lutz

[eddier]

I would have to agree with Lutz on Java being much more thread friendly. However, M$ will never embrace it because they can't change Java into a Windoze specific language. So it will slowly fade away ... On the other hand, read a book before implementing threads in C++.



I like Java, but for most of my tasks, I have to be honest here, object oriented programming just gets in the way, i.e. small programs suddenly get complicated.



I would prefer newLisp without threads to a complicated newlisp with complicated features.



Eddie

[kazemori]

Lutz -- no offense taken. i enjoy learning! (so please, teach away! ;-)



i used to say, "you will know that i am dead when a beautiful girl walks by and i do not even move to look at her". i think it is more succinct, more realistic (and even more pc) to say, "you will know that i am dead when i stop learning".



for what it is worth, i do think newLISP should have threads (light-weight, transparent threads). i think newLISP is an excellent Lisp and needs to be more widely adopted. threads will help in this regard, as much of the so-called competition come with threads "out-of-the-box" (e.g., Io, Lua, Ruby, etc.).



perceptions count for a lot in choosing among the various programming languages. just look at java: the language that took over the world through the power of marketing (not because of its intrinsic merits ;-).



kazemori

[Lutz]

>you will know that i am dead when i stop learning



this is very true for me too and one of the enjoyments for me in the computing field



> perceptions count for a lot in choosing among the various programming languages



that is probably so and definetely threads are a nice feature to have in a programming language and could open up the application of newLISP in other fields where this feature is essential, like hardware control, programming games (would make a good demo for the new feature), etc..



The more we talk about, it I think I am going to do it and it seems to me, that it is straightforward to implement.



Lutz

[Lutz]

Yes Eddie!



it will be light and not burden newLISP in any way, size or speed- resource- performance wise. If that is not possible I will not do it.



Lutz

[kazemori]

although i am a physicist by academic training, and started my professional life as a programmer, i have in fact achieved most of my private sector success in the field of sales & marketing. (hiss! boo!)



of course, it was always high-tech sales & marketing in silicon valley. (hiss! boo!)



many of the things you learn while working on the "dark side" have analogous structures in the world of programming, for example: key-value pairs -- dark side equivalent: feature-benefit pairs



implementing threads in newLISP will...



feature: add more capabilities to a versatile language

benefit: encourage more people to download newLISP



feature: justify a major revision number - newLISP 8.0

benefit: implementing a major revision will potentially allow you to implement / re-implement other projects that you may have held back on until now...



i am not remotely competent to help you with C coding; however, i am wizard writer. (yes, i know how to capitalize words ;-) my pen is at your disposal (for starters, i am already scrubbing the newLISP manual for typo's and other gotcha's. (i even scrubbed 50% of newlisp.c last night)



i also propose to retool the introductory paragraph that you have on the O'Reilly open source directory site. (read the Tcl text, and then read newLISP ;-) please let me know if this is okay with you!



evangelist-by-nature  --  kazemori

[eddier]

Thanks Lutz for keeping things simple!



Java did a great job with Applets and with threads. But the Java class hierarchys have gotten too complicated. I have to complete many of my programs in a very short time frame. I gave up on Java for many of my jobs because I became bogged down in learning how to use a bunch of classes to perform a simple task. For instance to open a file for reading, I have to understand the IOException class, InputStream, FileInputStream, FilterInputStream, DataInputStream, etc...  just too find which classes I need to use for a particular job. Many times, I can write a small script in newLisp, Perl, PHP, Ruby, or Python in a day that would otherwise take several days of looking through a dozen or so Java classes to find what I need.



Eddie

[Lutz]

Yes, I will keep it simple ... absolutely! That has always been first priority and it will stay this way. It also has been the most challenging part of newLISP, because requests for new features are abundant and it is not always easy to decide what to leave out or put in. What happened to Java is typical for a tool which gets developed by a group, which has grown to big and managed corporate style, trying to please everybody.



There are also several old things in newLISP, I want to get rid of. Some of the functions are marked deprecated (collapsed into a new one, renamed, etc.). I would like to throw those out in the next round!?. There are other functions, I suspect nobody needs them anymore like 'command-line' for disabling/enabling console input. 'current-line' could be implemented using the same sysvars as in regular expressions input etc., etc., etc.



Any suggestions on this kind of issues is greatly appreciated.



Any input on the introductory paragraph on O'Reilly is very welcome. The current wording is about the same as used on SourceForge; and from there it has been copied by many directory services. Some of these services use paragraphs 5 years old, copied from a submission to AltaVista. O'Reilly is at least current, but perhaps the wording is too dry.



As most of you know I am not a native English speaker and apppreciate any help to 'debug' the manual and make it sound more 'English'. A good manual adds much to the usability of a programming language (at least for me).



Soo much to do ... good evening everybody



Lutz

[kazemori]

> Soo much to do ...



soo true !!!



i will be in touch soon  --  kazemori

[eddier]

Since we have find and slice we could get rid of "member"



I've seen the (cond (bool1 exp1) (bool2 exp2) ....) vs (cond bool1 exp1 bool2 exp2 ...)  debated in several books.  Would it be bad to implement the if or the cond like (? bool1 exp1 bool2 exp2 ...). Maybe even get rid of either "cond" or the "if" altogether?



I know we've tossed around getting rid of starts-with and ends-with string functions, but, since you also provide for lists???



Why have "first" and "nth 0?"



Do we really need let? Or if we need let shouldn't we also have a let! that allows something like

Code Select Expand

(let! ((x 1) (y x)) (func ....))  where the second variable can use the value of the first variable?

Isn't lookup the same as (rest (assoc ....



how about a reverse-assoc?



I know I'm drawing flames here, but, ...



Eddie