Interesting problem for me

[ale870]

@Lutz, I think functors are the best way to manage references. They are easy to be used, fast and readable.



@newdep, that is normal. In fact, while you are studying a programming language, you want to face more and more complex problems, and every problem open a door with many possible solutions inside. But in order to apply a solution, you need to acquire more knowledge. Furthermore, more a language is "flexible", more solutions you can find, and more things you need to learn to apply them. I love this!

I really like progrmaming languages, I study some of them only to get the "phylosofy" that is behind them. Instead I learnt other languages to use them for specific jobs or for every-day-usage. I know more than 15 languages but, of course, I know very well only some of them.

newLisp is one of those languages I started to study just for curiosity, and to learn some basics of functional languages. But in the time, I discovered that it's a powerful tool, and that Lutz made a great job with it.

So I decided to "promote" it to "everyday-language".

But in the time I even discovered that it was a good language not only for scripting, but even for something more complex. So... I'm here to discuss with you, in order to learn every day something more, and to help other guys (this is the reason I opened one italian blog site).

[xytroxon]

Hi all!



This past weekend I was trying to automate the word list, but had a problem on Windows 98se (48 megabytes of memory)...



This code slows down to a dead crawl, unless I add (inc ctr) after (push (string ...and then everything works fine...

Code Select Expand


; Create all 456976 English four letter words...
; ( Warning:  May be offensive to some sensitive programmers ;)
(for (z 97 122)
  (for (y 97 122)
(print (time ; begin timing debug
    (for (x 97 122)
      (for (w 97 122)
        (push (string (char z) (char y) (char x) (char w)) word-list -1)
        ; (inc ctr) ;<<< uncommenting this statement makes time linear
      )
    )
) ",") ; end timing debug
  )
)
(println ctr " - " (length word-list) " words.")

(setq source-list (copy word-list)) ; copy list before each pass
; trust newLISP, but verify... ;)
(println (slice source-list 0 32))
(println (slice source-list -32))
(exit)


Does this work for you on other systems? Or have I found a strange problem with newLISP? May need to add another (for or two for larger memory systems...



-- xytroxon

[cormullion]

I think that must be a bug. It doesn't appear to matter what you insert - a number or string, just anything other than the push result. Older versions of newLISP don't show this behaviour...

[Lutz]

Its the returning of the growing word-list getting longer and longer from the 'for' statement.



Instead of inserting (inc cnt) you can put any thing else i.e. true or nil, which changes the return to a smaller piece of data.



added later:



In older versions 'push' returned the pushed element, so the return value as last fucntion in a 'for' loop was always small.

[DrDave]

I didn't try to figure out your code, but just ran it. As you said, incrementing your counter and it zips through. Comment it out and it crawls, getting slower as it progresses.



newLISP 10.0.0 WIN32 XP pro

[cormullion]

But Lutz I thought it returned a reference rather than a copy?



- ah wait, it's the return of for not push... ?

[Lutz]

There is a list of all of them here:



http://www.newlisp.org/downloads/newLISP-10.0-Release.html">http://www.newlisp.org/downloads/newLIS ... lease.html">http://www.newlisp.org/downloads/newLISP-10.0-Release.html



Its basically everything except user-defined functions and macros and 'for' and all other iterators with a local looping variable, like 'dotimes', 'dolist', 'doargs' and 'dotree'.

[Lutz]

... so the following is possible:

Code Select Expand

> (set 'l '(1 2 3))
(1 2 3)
> (reverse (begin (push 99 l)))
(3 2 1 99)
> l
(3 2 1 99)
>


the reference passes through the 'begin' without a problem.



Note that 'push' in older versions always returned the pushed element, now it returns a reference to the list.



starting version 10.0.1 the 'for' loop will return the reference, so there will be no slowdown.

[Kazimir Majorinc]

Yes, DrDave, that was everything right. Henx.

[ale870]

Using references is a big improvement even to speed up the code.

Lutz, it means now the functors can be substituted in some way in the program, isn't it? In fact, in the past, we used functors to pass values by reference, but now they could be useful only for user-defined functions.



Ones question: Is there any reason (e.g. performance) to still use functors instead native reference-passing?



If I find a situation where I can use either functors or function reference, is there any tip to decide what's the best way to follow?

[Lutz]

All built-in functions automatically pass references in and out for lists, strings and arrays. Only with user-defined functions and macros you need default-functors for reference passing in and out.



If the majority of lists and strings in a program are small, then don't worry about the whole thing. When lists or strings get bigger than a hundred elements then you might consider reference passing, but only if some operation on them occurs frequently.



In older versions everything in newLISP was value passing by default and newLISP was still considered one of the fastest scripting languages (see http://www.newlisp.org/benchmarks/">http://www.newlisp.org/benchmarks/ )



But the reference passing issue is not just about speed. Reference passing also allows you to write destructive user-defined functions without recurring to macros or passing quoted symbols (1).



From a programming style point of view, I would say that value passing and non-destrcuctive user-defined functions are the cleaner, safer and more intuitive way to write bug-free functional programs. But from a practical point of view, you need reference passing to efficiently deal with large data objects, when you want to avoid frequent copying. Then default-functors are the most straight forward method in newLISP.



(1) Traditionally reference passing can also be achieved using either macros or passing quoted symbols to user-defined functions. This method is not recommended because it brings a whole basket of other problems, i.e. variable capture. There are ways around it, but they lead to complex code. Some of Kazimir's blog posts investigate these problems in greater detail.