memory leak in print?

[Sammo]

Hello Lutz & Hans-Peter,



Using newLisp.dll with NeoBook via the hpwNewLisp plug-in, I am leaking memory at the rate of (I think) two bytes per call. With this test function:



(define (donothing) "test string")



and these calls:



a)  hpwNewLispCall "(donothing)" "[LispResult]"

b)  hpwNewLispCall "(silent (donothing))" "[LispResult]"

c)  hpwNewLispCall "(print (donothing))" "[LispResult]"

d)  hpwNewLispCall "(silent (print (donothing)))" "[LispResult]"



the c) and d) calls (both containing 'print') are leaking memory; the other calls are not.



The '(silent (print (myfunction)))' idiom is necessary to properly return values to NeoBook. I found this problem after writing a NeoBook program that invokes hpwNewLispCall every 200 ms. After running for several hours, Win32 complained about "running low on virtual memory." I found the two-bytes-per-call memory leak by experimentation and by watching Task Manager's Mem Usage window.



Can you help? Is the memory leak in newLISP's 'print' function, in the hpwNewLispCall interface, or elsewhere?



Thanks,

-- Sam

[HPW]

Hello Sam,



from the logical point it seems to me that it should be the 'print' command, since the interface only pass the string through. And this happens with every command in the same way. So when a+b does not leak, then it must be related to the lisp-command. So let's Lutz have a look at it.



Thanks for the observation. This could only be noticed by someone who is a 'bytecounter'. (exact, precise, correct)



Have you cross-tested it with the TK-enviroment?

[Sammo]

Hans-Peter,



Being somewhat unfamiliar with Win32's Task Manager, I misunderstood what it was telling me. The leak is more serious than I reported; I'm actually losing 2K bytes per call! Since my program is calling newLisp.dll 20 times per minute (50 ms intervals instead of 200 ms as previously stated), that's 72,000 calls per hour. It's no wonder that Win32's virtual memory system complained and took a 700MB bite out of my HDD in the course of about five hours.



I tried to but could not duplicate the problem in the newlisp-tk and newlisp command-line environments.

[HPW]

I crosstested BCC and GCC DLL's but both have the problem.



(dotimes (x 100)(print (donothing)))



eats about 192KB.

[HPW]

You can reproduce it with newLISP:

newLISP v.8.2.0 Copyright (c) 2004 Lutz Mueller. All rights reserved.



> (import "newlisp.dll" "newlispEvalStr")



newlispEvalStr <100A150>

> (get-string (newlispEvalStr "(define (donothing) "test string")"))

"(lambda () "test string")"

> (get-string (newlispEvalStr "(donothing)"))

""test string""

> (get-string (newlispEvalStr "(dotimes(x 100)(donothing))"))

""test string""

> (get-string (newlispEvalStr "(dotimes(x 100)(donothing))"))

""test string""

> (get-string (newlispEvalStr "(dotimes(x 100)(donothing))"))

""test string""

> (get-string (newlispEvalStr "(dotimes(x 100)(print(donothing)))"))

[HPW]

Also this way:

> (get-string(newlispEvalStr "(dotimes(x 100)(print "Test"))"))

[HPW]

Problem was also in 7.500 and 8.000!

[Lutz]

This is a known problem internal to Windows loosing system resources, it also happens when opening/closing files. I have now way to fix this in newLISP. The only workaround would be not to 'print' from inside the DLL but only from the parent application.



Instead of using 'print' use 'string' to return a string presentation to the parent process calling the DLL, i.e when calling from newLISP itself:



(print (get-string (newlispEvalStr "(string 1234 "test")")))





Lutz

[Lutz]

print uses a 'stdout' system resource in Windows,



Lutz

[HPW]

Not so good for us on the neobook side.



(string "Test")

returns

"Test"



we use print for formatiing neobook scripts with:



(silent(print "Test"))

returns

Test



Can you think of another command to get the formating of print without this side effekt?

Will also be the case on the other PC-enviroments.

[HPW]

And why only with DLL's?

EXE's print works without this effekt!

[HPW]

The discussion makes me thinking in a different way.



By now I thought that the return string must a ready script-stream for neobook. When only the outer Double-Quote pair is the problem, then i could possibly modify the interface to take away the first and last Double-quote and then feed it into the script-command buffer.



Sam,

what do you think? Should I modify the existing command or add a optional command for this? Then the stuff with (silent(print )) would be obsolet. We could use (string ) as Lutz suggested.



But maybe it is not the wanted behaviour for other PC-enviroments like Powerbasic, VB, VBA etc.

[Lutz]

VBA and VB6 will work fine receiving the string. Using 'print' to pass things back actually sounds exotic to me, you normally would print/display in the host application not from an imported library. But I am not familiar with NeoBook.



Lutz

[HPW]

This is a current return-line in lisp:



(silent(print "nb-action AlertBox "Test" "This is a 1.Action!"nAlertBox "Test" "This is a 2.Action!"n"))



It returns:

nb-action AlertBox "Test" "This is a 1.Action!" LF AlertBox "Test" "This is a 2.Action!" LF



The outer double-quotes are taken away with print and all escape-sequences are resolved /" get " and n gets 'LF'. (chr 10)



So it is not only deleting outer double quotes it is also resolving escape-sequences. A lot more to do in the interface.

[HPW]

I have a working prototype of my neobook-plugin hpwNewLISP which allow a more flexibel handling with the return-string.



So instead of:

(silent(print "nb-action AlertBox "Test" "This is a 1.Action!"nAlertBox "Test" "This is a 2.Action!"n"))

you can use optional:

(string "nb-action AlertBox "Test" "This is a 1.Action!"nAlertBox "Test" "This is a 2.Action!"n")

or at least the string alone:

"nb-action AlertBox "Test" "This is a 1.Action!"nAlertBox "Test" "This is a 2.Action!"n"

or the symbol containing the return-string is the last statement in the function.



I think it is more consistent to get back a string "Text" as 'Text' inside the neobook-return-variable than the current "Text".

But this could cause some changes in current neobook sources.



So what do I do:

I scan the return string for

first and last char = "

or:

first six and last seven = [text][/text]



This text-flags are then deleted.

Then I do a replace:

" to "

r to chr(13)

n to chr(10)

t to chr(9)

\ to



Demo-pub is running so far. I will do some more testing and release it in the evening when I am back home.



Without the need of print we have a workaround for the memory-leak.

(No more exotic thing to pass things back)

As always: The right tool to get the work done!

;-)