4 newLISP bugs in 10.1.9-dev and older

[itistoday]

While working on various newLISP code I've encountered several bugs.



Bug #1: context? reporting true for non-context

Code Select Expand

> (new-class 'Foo)
Foo
> (setf f (instantiate Foo))
Foo#1
> (deallocate f)
true
> (context? f)
true
> (symbols f)

ERR: context expected in function symbols : f

You need http://www.taoeffect.com/newlisp/ObjNL.lsp.txt">ObjNL.lsp to run that example.



Bug #2: Memory leak



For this one ObjNL.lsp is also needed, simply because it's much easier to demonstrate the leak by using auto-release pools:

Code Select Expand

> (new-class 'Foo)
Foo
> (define (inflate-memory) (push-autorelease-pool) (dotimes (_ 10000) (autorelease (instantiate Foo))) (pop-autorelease-pool))
(lambda () (push-autorelease-pool)
 (dotimes (_ 10000)
  (autorelease (instantiate Foo)))
 (pop-autorelease-pool))
> (inflate-memory)
true
> (inflate-memory)
true

On my system using the 10.1.9-dev build, each time 'inflate-memory' is called an additional 0.1MB of memory is used up even though 'inflate-memory' cleans up after itself.



Bug #3: nil? evaluates its argument twice



This one is quite significant:

Code Select Expand

> (nil? (inc a))
nil
> a
2

Bug #4: REPL can't properly handle [cmd][/cmd] tags



Take a look at this REPL session:

Code Select Expand

> [cmd]
(define-macro (epln what)
(when (nil? (println "n" what "n -> " (eval what)))
(exit)
)
)[/cmd]
[/cmd]
(lambda-macro (what)
 (when (nil? (println "n" what "n -> " (eval what)))
  (exit)))
nil

That shows one variant of the problem, where the ending [/cmd] tag isn't respected and must be entered again.



Note the 'nil' at the very end. This shows that newLISP is evaluating something... Here's a bizarre variant that shows some more insight to the problem:

Code Select Expand

> [cmd](define-macro (epln what)
(when (nil? (println "n" what "n -> " (eval what)))
(exit)
)
)[/cmd]
[/cmd]

nil
 -> nil
macbookpro:~$

Even though that should work, that will actually causes the 'epln' function to be evaluated, which is why it quits to the shell (because 'what' is nil).

[Lutz]

#1 and #2



Please show these problems in short but complete snippets. Try to boil it down to a minimum sequence of statements and expressions, which still shows the problem. I have no time, I go through user code myself.



#3



is fixed and updated in development/latest 10.1.9-dev



#4



The tags [cmd] and [/cmd] need to go on their own lines. These tags are primarily implemented to be used internally by 'net-eval' and by editors and IDEs interfacing with newLISP's REPL. If you work a lot in the REPL, you could use util/nls from the source distribution and customize it further for multi-line editing. Cormullion has an example on his site how to do this. What nls primarily does, is integratung the BASH and newLISP REPL and giving online syntax help for built-in functions.

[itistoday]

#1 and #2



Please show these problems in short but complete snippets. Try to boil it down to a minimum sequence of statements and expressions, which still shows the problem. I have no time, I go through user code myself.

For #1:

Code Select Expand

> (setf os (sym "blah"))
blah
> (setf o (new Class os))
blah
> (set 'o:self o 'o:os os)
blah
> (let (s o:os) (delete s) (delete s))
true
> (context? o)
true

For #2, here's a simplified version:

Code Select Expand

(define (instantiate class)
(letn ( obj-sym (sym (string class "#" (inc class:@instance-counter)))
obj (new class obj-sym)
)
(set 'obj:@self obj 'obj:@self-sym obj-sym)
obj
)
)

; The existence of these functions causes a memory leak!
(context Class)
(define (Class:Class) true)
(define (Class:dealloc))
(define (Class:equals obj) (= obj @self))
(context MAIN)

(define (inflate-memory)
(push '() autorelease)
(dotimes (_ 10000)
(push (instantiate Class) (first autorelease))
)
(dolist (obj (pop autorelease))
(let (s obj:@self-sym)
(delete s nil)
(delete s nil)
)
)
)

I have figured out what causes it: simply defining functions inside of 'Class' will cause the memory leak. If you comment that part out there will be no memory leak.

#3



is fixed and updated in development/latest 10.1.9-dev

Great! :-)



Actually, there's one more bug that I forgot to mention:



Bug #5:



Take a look at this function:

Code Select Expand

(define (new-class sym-class (super ObjNL) (interfaces '()) , class)
(set 'class            (new super sym-class)
    'class:@super     super
    'class:@class     class
    'class:@self      class
    'class:@self-sym  sym-class
)
; NOTE: newLISP Bug? Why does pushing to the back result in odd behavior?
; (push class class:@interfaces -1)
(push class class:@interfaces)
(dolist (iface interfaces)
(setf iface (eval iface))
(new  iface sym-class)
(push iface class:@interfaces)
)
class
)

Take a look at the comment, it refers to "odd behavior". If the -1 is used then this is what will happen (it's hard to describe):

Code Select Expand

> (define (protocol:test) "hello!")
(lambda () "hello!")
> (new-class 'Foo ObjNL '(protocol))
Foo
> (new Foo 'Bar)
Bar
> Bar:@interfaces
(protocol ObjNL)

That last output should be (protocol Foo ObjNL), not (protocol ObjNL). If you remove the -1 then it works fine.

[Lutz]

#1 is fixed in development/latest dev-9



for #2, #4 and #5, I need more of your help.



Let me show you what I mean with "minimum sequence of statements and expressions producing the bug":



Basically I started with the snippet you gave me and put it in a file. Then I would take out statements and simplify expressions in new file copies until the error would either disappear or I had the minimum form producing the error.



You can see the sequence of elimination and simplification here:

Code Select Expand

; the original
(setf os (sym "blah"))
(setf o (new Class os))
(set 'o:self o 'o:os os)
(let (s o:os) (delete s) (delete s))
(println "->" (context? o) " o ->" o)

; eliminate let
(setf os (sym "blah"))
(setf o (new Class os))
(set 'o:self o 'o:os os)
(set 's o:os) (delete s) (delete s)
(println "->" (context? o) " o ->" o)

; eliminate os:self
(setf os (sym "blah"))
(setf o (new Class os))
(set 'o:os os)
(set 's o:os) (delete s) (delete s)
(println "->" (context? o) " o ->" o)

; eliminate o:os
(setf os (sym "blah"))
(setf o (new Class os))
(set 's os) (delete s) (delete s)
(println "->" (context? o) " o ->" o)

; eliminate indirection of s to os
(setf os (sym "blah"))
(setf o (new Class os))
(delete os) (delete os)
(println "->" (context? o) " o ->" o)

; eliminate second delete
(setf os (sym "blah"))
(setf o (new Class os))
(delete os)
(println "->" (context? o) " o ->" o)

; the final minimal form showing the core of
; the problem: the context cell in ctx does
; not get re-typed to nil, after blah is demoted
; to a normal symbol
(define blah:blah)
(set 'ctx blah)
(delete 'blah)
(context? ctx) ;=> true

; should result in nil, because 'blah gets demoted to
; a normal symbol, so blah is no context anymore


Only after doing this I could exclude programmer error and see that this was a bug. I also could pinpoint the error to the exact location in the code, because now the bug was well defined. Beforeit could have been all kinds of problems: was it the let? or the second delete? ar the double indirection to s? or the additional symbols in the context?



Doing this procedure on somebody else's code is time consuming. If the programmer does it himself, it is quicker, because one understands better the own code written and the critical elements to eliminate. One has also a clear picture of the problem and how to circumvent it.

[Kazimir Majorinc]

I'll add two small bugs on top.



(legal? """)

(legal? "{")



Priority]

[Lutz]

Thanks for catching this. There is a fixed development update 10.1.9-dev in the development/latest directory.

[itistoday]

#1 is fixed in development/latest dev-9

Great!

for #2, #4 and #5, I need more of your help.

I assume you mean for #2 and #5.



For #2:

Code Select Expand

(context Class)
(define (Class:equals obj) (= obj @self))
(context MAIN)

(define (inflate-memory)
(push '() autorelease)
(dotimes (_ 10000)
(setf obj-sym (sym (string "Class#" (inc obj-counter))))
(setf obj (new Class obj-sym))
(setf obj:@self-sym obj-sym)
(push obj (first autorelease))
)
(dolist (obj (pop autorelease))
(let (s obj:@self-sym)
(delete s nil)
(delete s nil)
)
)
)

Enter that code into a REPL session, then run 'inflate-memory' and watch the memory grow each time. You can easily simply that down if you want to, I wrote it that way so that the memory leak is visible.



I've narrowed down the memory leak to the existence of this line:

Code Select Expand

(define (Class:equals obj) (= obj @self))

Bug #5:

Code Select Expand

(setf Class:@interfaces '(Class))
(new Class 'Foo)
(push Foo Foo:@interfaces -1)
(println Foo:@interfaces)

That should print (Class Foo) but it only prints (Class). Removing the '-1' results in correct behavior.

[itistoday]

Hmm... I'm not sure if this is a bug, but I think it should be as it's not what the documentation states and makes it less convenient to use.



'push' does not return a list if the symbol being pushed to does not exist:

Code Select Expand

> (push 1 blah)
1

This means that one must ensure the list exists before combining 'push' with a list manipulating function, i.e.:

Code Select Expand

(setf blah (unique (push foo blah)))

[Lutz]

This and other bugs are fixed in the current development/latest 10.1.9-dev. Currently only the cell-leak #2 is pending when using 'delete' on copied contexts.