Maintenance Release newLISP v.10.1.6

[Lutz]

• This release contains bug fixes and new rewrite macro facility



Release notes: http://www.newlisp.org/downloads/newLISP-10.1-Release.html">http://www.newlisp.org/downloads/newLIS ... lease.html">http://www.newlisp.org/downloads/newLISP-10.1-Release.html

Downloads: http://www.newlisp.org/index.cgi?page=Downloads">http://www.newlisp.org/index.cgi?page=Downloads

[cormullion]

Cool! I think I can have much damageXXXXXX fun with the new reader-event.. :)



I don't think it's as easy to disable it, though:

Code Select Expand

> (define (rewrite expr) nil)
(lambda (expr) nil)
>
> (reader-event rewrite)
$reader-event
>
> (println hello)
nil
> (reader-event nil)
nil
> (println hello)
nil
> (exit)
>nil


I've painted myself into a corner...



(Actually I didn't type this first off. I made a mistake in my rewrite function...)

[itistoday]

I've been getting a crash with versions since at least 10.1.4, and it's still there in 10.1.6.  I'm still working on figuring out how to get it to run, but I've been able to at least get a fairly simple, reproducible crash, as well as a stack trace.



Two files are required:



./foo.lsp

./dragonfly-framework/lib/utils.lsp



./dragonfly-framework/lib/utils.lsp

Code Select Expand

; this declaration needs to be at the top of the file because of newLISP bizarroness with 'load'
; TODO: bug? report on forum
(define-macro (define-subclass)
(new (args 0 1) (args 0 0))
(dolist (method (rest $args))
(setf (method 0 0) (sym $it (args 0 0)))
(eval (push 'define method))
)
)

(context 'Dragonfly)

(define (load-once)
; check if the last argument is a context (to behave like 'load' does)
(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
(doargs (file)
(unless (or (context? file) (find file _loaded))
(push file _loaded)
(saved-load file ctx)
)
)
)
)

; We define our own module function so that we can easily support
; shared hosting services where the modules directory might not be
; in /usr/share/newlisp/modules.
(define (Dragonfly:module module-to-load)
(if-not NEWLISP_DIR (throw-error "need value 'NEWLISP_DIR' config.lsp!"))
(load-once (append NEWLISP_DIR "/modules/" module-to-load))
)

; places the key/value pairs from assoc-list into ctx
(define (into-ctx-assoc ctx assoc-list)
(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
)

; load all .lsp files directory
(define (load-lsp-files-in-dir dir)
(dolist (x (directory dir ".lsp$")) (load-once (append dir "/" x)))
)

(context 'MAIN)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
(def-new  'load 'Dragonfly:saved-load)
(constant 'load Dragonfly:load-once)
(constant 'module Dragonfly:module)
)

./foo.lsp

Code Select Expand

(context 'Dragonfly)
(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")
(context 'MAIN)

Then just do newlisp foo.lsp. Crash (stacktrace for 10.1.6):

Code Select Expand

(gdb) r foo.lsp
Starting program: /usr/bin/newlisp foo.lsp

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
evaluateExpression (cell=0x0) at newlisp.c:1267
1267 switch(cell->type)
(gdb) bt
#0  evaluateExpression (cell=0x0) at newlisp.c:1267
#1  0x000066b8 in p_unless (params=0x0) at newlisp.c:5182
#2  0x00005802 in evaluateExpression (cell=0x819c80) at newlisp.c:1373
#3  0x00006918 in evaluateBlock (cell=0x819c80) at newlisp.c:5100
#4  0x00009e02 in dolist (params=0x819c60, doType=2) at newlisp.c:5597
#5  0x0000a0a9 in p_doargs (params=0x828000) at newlisp.c:5502
#6  0x00005802 in evaluateExpression (cell=0x819cc0) at newlisp.c:1373
#7  0x0000728b in let (params=0x819de0, type=0) at newlisp.c:4782
#8  0x000074e9 in p_let (params=0x828000) at newlisp.c:4653
#9  0x00005802 in evaluateExpression (cell=0x819e10) at newlisp.c:1373
#10 0x0000a8b8 in evaluateLambda (localLst=0x81a1c0, arg=<value temporarily unavailable, due to optimizations>, newContext=0x103870) at newlisp.c:1686
#11 0x0000583e in evaluateExpression (cell=0x81a470) at newlisp.c:1381
#12 0x00006918 in evaluateBlock (cell=0x81a470) at newlisp.c:5100
#13 0x00009e02 in dolist (params=0x81a260, doType=0) at newlisp.c:5597
#14 0x0000a0e9 in p_dolist (params=0x828000) at newlisp.c:5492
#15 0x00005802 in evaluateExpression (cell=0x81a220) at newlisp.c:1373
#16 0x0000a8b8 in evaluateLambda (localLst=0x81a4b0, arg=<value temporarily unavailable, due to optimizations>, newContext=0x103870) at newlisp.c:1686
#17 0x0000583e in evaluateExpression (cell=0x819ba0) at newlisp.c:1381
#18 0x0000d8bb in evaluateStream (stream=0xbfffefac, outDevice=0, flag=1) at newlisp.c:1103
#19 0x0000dc78 in loadFile (fileName=0xbffff2bd "foo.lsp", offset=0, encryptFlag=0, context=0x100570) at newlisp.c:2983
#20 0x0000efcc in main (argc=2, argv=0xbffff1f0) at newlisp.c:719

The problem seems to be related to the 'unless' block at the bottom of 'utils.lsp'. (Actually, it's the other unless block, the one inside of load-once.) Unfortunately I can't even move that into the Dragonfly context as then I'll get this error](unless saved-load
   (def-new  'MAIN:load 'Dragonfly:saved-load)
   (constant 'MAIN:load Dragonfly:load-once)
   (constant 'MAIN:module Dragonfly:module)
)

=> ERR: symbol not in current context in function constant : load[/code]

Also, I was wondering why I have to put 'define-subclass' at the top of utils.lsp? If I try putting it at the bottom (after switching back to the MAIN context), then I get this error:

Code Select Expand

ERR: string expected in function append : module-to-load
called from user defined function Dragonfly:load-once
called from user defined function Dragonfly:load-lsp-files-in-dir

[cormullion]

I can't get that error. With just what you posted:

Code Select Expand

$ newlisp foo.lsp

ERR: invalid function in function unless : (def-new  'load 'Dragonfly:saved-load)


Which figures. Don't understand the def-new function anyway.



The manual says "the current context must not be MAIN". And yet:

Code Select Expand

(context 'MAIN)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
   (def-new  'load 'Dragonfly:saved-load)

Looks like you're in MAIN. ?

[itistoday]

I can't get that error. With just what you posted:

Make sure you have the same directory setup (create a dragonfly-framework folder with a lib folder inside of it with utils.lsp inside of it, etc.).

The manual says "the current context must not be MAIN". And yet:

That's only if you're not specifying the target context.

[itistoday]

BTW, this problem is driving me crazy. I've tried just about everything to fix it, even removing the Dragonfly context from the scenario.



Example:



foo.lsp

Code Select Expand

(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")

./dragonfly-framework/lib/utils.lsp

Code Select Expand

(define-macro (define-subclass)
(new (args 0 1) (args 0 0))
(dolist (method (rest $args))
(setf (method 0 0) (sym $it (args 0 0)))
(eval (push 'define method))
)
)

(define (load-once:load-once)
; check if the last argument is a context (to behave like 'load' does)
(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
(doargs (file)
(unless (or (context? file) (find file _loaded))
(push file _loaded)
(MAIN:__saved-load file ctx)
)
)
)
)

; We define our own module function so that we can easily support
; shared hosting services where the modules directory might not be
; in /usr/share/newlisp/modules.
(define (__module module-to-load)
(if-not NEWLISP_DIR (throw-error "need value 'NEWLISP_DIR' config.lsp!"))
(load-once (append NEWLISP_DIR "/modules/" module-to-load))
)

; places the key/value pairs from assoc-list into ctx
(define (into-ctx-assoc ctx assoc-list)
(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
)

; load all .lsp files directory
(define (load-lsp-files-in-dir dir)
(dolist (x (directory dir ".lsp$")) (load-once (append dir "/" x)))
)

; swap the MAIN functions for ours
(unless __saved-load
(constant '__saved-load load)
(constant 'load load-once)
(constant 'module __module)
(global 'load-lsp-files-in-dir 'into-ctx-assoc)
)

Still crashes at the same place.

[itistoday]

The problem seems to be related to the 'unless' block at the bottom of 'utils.lsp'.

Actually, I'm wrong, it's the other unless block, the one that's inside of load-once.

[itistoday]

Think I've narrowed the crash down to the call to saved-load in load-once. Commenting it out works (but obviously that renders load-once useless). It will crash if it's uncommented, whether saved-load is defined as a symbol in Dragonfly context or a global symbol in main, and whether or not load-once is used as a default function or just a plain old function.

[Lutz]

In foo.lsp

Code Select Expand

(context 'Dragonfly)
(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")
(context 'MAIN)


when the function (load-lsp-files-in-dir ...) is running, it is redefining itself while running because utils.lsp is loaded recursively again. This causes the crash, because after the symbol 'load-lsp-files-in-dir' is redefined loading "utils.lsp" the old contents of symbol 'load-lsp-files-in-dir' is deleted.





PS: Also, why not in the utils.lsp simply redefine the built-in NEWLISPDIR to your custom module directory:

Code Select Expand

(env "NEWLISPDIR" "/home/mypath")

the built-in 'module' function will then work on that path.

[itistoday]

when the function (load-lsp-files-in-dir ...) is running, it is redefining itself while running because utils.lsp is loaded recursively again. This causes the crash, because after the symbol 'load-lsp-files-in-dir' is redefined loading "utils.lsp" the old contents of symbol 'load-lsp-files-in-dir' is deleted.

Ah! Thanks Lutz! That seems to have fixed it.

PS: Also, why not in the utils.lsp simply redefine the built-in NEWLISPDIR to your custom module directory:

Code Select Expand

(env "NEWLISPDIR" "/home/mypath")

the built-in 'module' function will then work on that path.

Good point, I didn't realize that it used NEWLISPDIR like that, but I guess I should have. Done. :-)



Also, it seems like define-subclass now can be put after the (context 'MAIN) switch at the end as well. Not sure why, but it works now.



Here's the updated utils.lsp (sans most comments):

Code Select Expand


(context 'Dragonfly)

; protect against situation where one of the load functions is used to
; load this file, thereby redefining the function itself while it's running
; and causing newlisp to crash.
(unless load-once

(define (load-once)
(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
(doargs (file)
(unless (or (context? file) (find file _loaded))
(push file _loaded)
(saved-load file ctx)
)
)
)
)

(define (into-ctx-assoc ctx assoc-list)
(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
)

(define (load-files-in-dir dir regex-match)
(dolist (x (directory dir regex-match))
(load-once (append dir "/" x))
)
)

)

(context 'MAIN)

(define-macro (define-subclass)
(new (args 0 1) (args 0 0))
(dolist (method (rest $args))
(setf (method 0 0) (sym $it (args 0 0)))
(eval (push 'define method))
)
)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
(def-new 'load 'Dragonfly:saved-load)
(constant 'load Dragonfly:load-once)
)

[Lutz]

newLISP by itself will only define NEWLISPDIR as /usr/share/newlisp, if not defined already. So you could also put this in an Apache .htaccess file

Code Select Expand

setEnv NEWLISPDIR /blah/foo/mypath

now your Apache CGI processes will see NEWLISPDIR as that path, and newLISP when starting up will not change it.

[itistoday]

Hmm... I just remembered the second reason that I wanted to have a custom module function and that was to prevent modules from being loaded twice. Does the current module function already behave this way or will it re-load the module? I'd like to avoid reloads for speed and also safety.

[Lutz]

If you enter 'module' in an interactive console, you see how it is defined (its just embedded as source in the C files):

Code Select Expand

> module
(lambda (m) (load (append (env "NEWLISPDIR") "/modules/" m)))

> (global? 'module)
true
>


Many schemes to avoid double loading are possible. You could redefine 'module' to push every file loaded on a module-list.

Code Select Expand

(define (module m)
    (unless (find m module-list)
        (load (append (env "NEWLISPDIR") "/modules/" m))
        (push m module-list))
)


[itistoday]

If you enter 'module' in an interactive console, you see how it is defined (its just embedded as source in the C files):

Code Select Expand

> module
(lambda (m) (load (append (env "NEWLISPDIR") "/modules/" m)))

> (global? 'module)
true
>


Many schemes to avoid double loading are possible. You could redefine 'module' to push every file loaded on a module-list.

Actually that's what my load-once does, and since I'm replacing newLISP's load with load-once, and module is defined in that way then module should also load modules only once. I should have checked before asking, I thought that because it was a built-in function it would be defined in C, but I'm glad it's not! :-)