newLISP Fan Club

Forum => newLISP newS => Topic started by: Lutz on October 20, 2009, 07:19:30 AM

Title: Maintenance Release newLISP v.10.1.6
Post by: Lutz on October 20, 2009, 07:19:30 AM
• This release contains bug fixes and new rewrite macro facility



Release notes: http://www.newlisp.org/downloads/newLISP-10.1-Release.html

Downloads: http://www.newlisp.org/index.cgi?page=Downloads
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: cormullion on October 20, 2009, 12:32:26 PM
Cool! I think I can have much damageXXXXXX fun with the new reader-event.. :)



I don't think it's as easy to disable it, though:


> (define (rewrite expr) nil)
(lambda (expr) nil)
>
> (reader-event rewrite)
$reader-event
>
> (println hello)
nil
> (reader-event nil)
nil
> (println hello)
nil
> (exit)
>nil


I've painted myself into a corner...



(Actually I didn't type this first off. I made a mistake in my rewrite function...)
Title: Crash
Post by: itistoday on October 21, 2009, 10:06:54 AM
I've been getting a crash with versions since at least 10.1.4, and it's still there in 10.1.6.  I'm still working on figuring out how to get it to run, but I've been able to at least get a fairly simple, reproducible crash, as well as a stack trace.



Two files are required:



./foo.lsp

./dragonfly-framework/lib/utils.lsp



./dragonfly-framework/lib/utils.lsp
; this declaration needs to be at the top of the file because of newLISP bizarroness with 'load'
; TODO: bug? report on forum
(define-macro (define-subclass)
(new (args 0 1) (args 0 0))
(dolist (method (rest $args))
(setf (method 0 0) (sym $it (args 0 0)))
(eval (push 'define method))
)
)

(context 'Dragonfly)

(define (load-once)
; check if the last argument is a context (to behave like 'load' does)
(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
(doargs (file)
(unless (or (context? file) (find file _loaded))
(push file _loaded)
(saved-load file ctx)
)
)
)
)

; We define our own module function so that we can easily support
; shared hosting services where the modules directory might not be
; in /usr/share/newlisp/modules.
(define (Dragonfly:module module-to-load)
(if-not NEWLISP_DIR (throw-error "need value 'NEWLISP_DIR' config.lsp!"))
(load-once (append NEWLISP_DIR "/modules/" module-to-load))
)

; places the key/value pairs from assoc-list into ctx
(define (into-ctx-assoc ctx assoc-list)
(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
)

; load all .lsp files directory
(define (load-lsp-files-in-dir dir)
(dolist (x (directory dir ".lsp$")) (load-once (append dir "/" x)))
)

(context 'MAIN)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
(def-new  'load 'Dragonfly:saved-load)
(constant 'load Dragonfly:load-once)
(constant 'module Dragonfly:module)
)


./foo.lsp
(context 'Dragonfly)
(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")
(context 'MAIN)


Then just do newlisp foo.lsp. Crash (stacktrace for 10.1.6):


(gdb) r foo.lsp
Starting program: /usr/bin/newlisp foo.lsp

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
evaluateExpression (cell=0x0) at newlisp.c:1267
1267 switch(cell->type)
(gdb) bt
#0  evaluateExpression (cell=0x0) at newlisp.c:1267
#1  0x000066b8 in p_unless (params=0x0) at newlisp.c:5182
#2  0x00005802 in evaluateExpression (cell=0x819c80) at newlisp.c:1373
#3  0x00006918 in evaluateBlock (cell=0x819c80) at newlisp.c:5100
#4  0x00009e02 in dolist (params=0x819c60, doType=2) at newlisp.c:5597
#5  0x0000a0a9 in p_doargs (params=0x828000) at newlisp.c:5502
#6  0x00005802 in evaluateExpression (cell=0x819cc0) at newlisp.c:1373
#7  0x0000728b in let (params=0x819de0, type=0) at newlisp.c:4782
#8  0x000074e9 in p_let (params=0x828000) at newlisp.c:4653
#9  0x00005802 in evaluateExpression (cell=0x819e10) at newlisp.c:1373
#10 0x0000a8b8 in evaluateLambda (localLst=0x81a1c0, arg=<value temporarily unavailable, due to optimizations>, newContext=0x103870) at newlisp.c:1686
#11 0x0000583e in evaluateExpression (cell=0x81a470) at newlisp.c:1381
#12 0x00006918 in evaluateBlock (cell=0x81a470) at newlisp.c:5100
#13 0x00009e02 in dolist (params=0x81a260, doType=0) at newlisp.c:5597
#14 0x0000a0e9 in p_dolist (params=0x828000) at newlisp.c:5492
#15 0x00005802 in evaluateExpression (cell=0x81a220) at newlisp.c:1373
#16 0x0000a8b8 in evaluateLambda (localLst=0x81a4b0, arg=<value temporarily unavailable, due to optimizations>, newContext=0x103870) at newlisp.c:1686
#17 0x0000583e in evaluateExpression (cell=0x819ba0) at newlisp.c:1381
#18 0x0000d8bb in evaluateStream (stream=0xbfffefac, outDevice=0, flag=1) at newlisp.c:1103
#19 0x0000dc78 in loadFile (fileName=0xbffff2bd "foo.lsp", offset=0, encryptFlag=0, context=0x100570) at newlisp.c:2983
#20 0x0000efcc in main (argc=2, argv=0xbffff1f0) at newlisp.c:719


The problem seems to be related to the 'unless' block at the bottom of 'utils.lsp'. (Actually, it's the other unless block, the one inside of load-once.) Unfortunately I can't even move that into the Dragonfly context as then I'll get this error](unless saved-load
   (def-new  'MAIN:load 'Dragonfly:saved-load)
   (constant 'MAIN:load Dragonfly:load-once)
   (constant 'MAIN:module Dragonfly:module)
)

=> ERR: symbol not in current context in function constant : load[/code]

Also, I was wondering why I have to put 'define-subclass' at the top of utils.lsp? If I try putting it at the bottom (after switching back to the MAIN context), then I get this error:


ERR: string expected in function append : module-to-load
called from user defined function Dragonfly:load-once
called from user defined function Dragonfly:load-lsp-files-in-dir
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: cormullion on October 21, 2009, 10:23:49 AM
I can't get that error. With just what you posted:


$ newlisp foo.lsp

ERR: invalid function in function unless : (def-new  'load 'Dragonfly:saved-load)


Which figures. Don't understand the def-new function anyway.



The manual says "the current context must not be MAIN". And yet:


(context 'MAIN)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
   (def-new  'load 'Dragonfly:saved-load)


Looks like you're in MAIN. ?
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: itistoday on October 21, 2009, 10:43:48 AM
Quote from: "cormullion"I can't get that error. With just what you posted:


Make sure you have the same directory setup (create a dragonfly-framework folder with a lib folder inside of it with utils.lsp inside of it, etc.).


QuoteThe manual says "the current context must not be MAIN". And yet:


That's only if you're not specifying the target context.
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: itistoday on October 21, 2009, 10:48:25 AM
BTW, this problem is driving me crazy. I've tried just about everything to fix it, even removing the Dragonfly context from the scenario.



Example:



foo.lsp
(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")


./dragonfly-framework/lib/utils.lsp
(define-macro (define-subclass)
(new (args 0 1) (args 0 0))
(dolist (method (rest $args))
(setf (method 0 0) (sym $it (args 0 0)))
(eval (push 'define method))
)
)

(define (load-once:load-once)
; check if the last argument is a context (to behave like 'load' does)
(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
(doargs (file)
(unless (or (context? file) (find file _loaded))
(push file _loaded)
(MAIN:__saved-load file ctx)
)
)
)
)

; We define our own module function so that we can easily support
; shared hosting services where the modules directory might not be
; in /usr/share/newlisp/modules.
(define (__module module-to-load)
(if-not NEWLISP_DIR (throw-error "need value 'NEWLISP_DIR' config.lsp!"))
(load-once (append NEWLISP_DIR "/modules/" module-to-load))
)

; places the key/value pairs from assoc-list into ctx
(define (into-ctx-assoc ctx assoc-list)
(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
)

; load all .lsp files directory
(define (load-lsp-files-in-dir dir)
(dolist (x (directory dir ".lsp$")) (load-once (append dir "/" x)))
)

; swap the MAIN functions for ours
(unless __saved-load
(constant '__saved-load load)
(constant 'load load-once)
(constant 'module __module)
(global 'load-lsp-files-in-dir 'into-ctx-assoc)
)


Still crashes at the same place.
Title: Re: Crash
Post by: itistoday on October 21, 2009, 11:03:22 AM
Quote from: "itistoday"The problem seems to be related to the 'unless' block at the bottom of 'utils.lsp'.


Actually, I'm wrong, it's the other unless block, the one that's inside of load-once.
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: itistoday on October 21, 2009, 11:30:48 AM
Think I've narrowed the crash down to the call to saved-load in load-once. Commenting it out works (but obviously that renders load-once useless). It will crash if it's uncommented, whether saved-load is defined as a symbol in Dragonfly context or a global symbol in main, and whether or not load-once is used as a default function or just a plain old function.
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: Lutz on October 21, 2009, 11:41:15 AM
In foo.lsp


(context 'Dragonfly)
(load "./dragonfly-framework/lib/utils.lsp")
(load-lsp-files-in-dir "./dragonfly-framework/lib")
(context 'MAIN)


when the function (load-lsp-files-in-dir ...) is running, it is redefining itself while running because utils.lsp is loaded recursively again. This causes the crash, because after the symbol 'load-lsp-files-in-dir' is redefined loading "utils.lsp" the old contents of symbol 'load-lsp-files-in-dir' is deleted.





PS: Also, why not in the utils.lsp simply redefine the built-in NEWLISPDIR to your custom module directory:


(env "NEWLISPDIR" "/home/mypath")

the built-in 'module' function will then work on that path.
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: itistoday on October 21, 2009, 12:22:47 PM
Quote from: "Lutz"when the function (load-lsp-files-in-dir ...) is running, it is redefining itself while running because utils.lsp is loaded recursively again. This causes the crash, because after the symbol 'load-lsp-files-in-dir' is redefined loading "utils.lsp" the old contents of symbol 'load-lsp-files-in-dir' is deleted.


Ah! Thanks Lutz! That seems to have fixed it.


QuotePS: Also, why not in the utils.lsp simply redefine the built-in NEWLISPDIR to your custom module directory:


(env "NEWLISPDIR" "/home/mypath")

the built-in 'module' function will then work on that path.


Good point, I didn't realize that it used NEWLISPDIR like that, but I guess I should have. Done. :-)



Also, it seems like define-subclass now can be put after the (context 'MAIN) switch at the end as well. Not sure why, but it works now.



Here's the updated utils.lsp (sans most comments):



(context 'Dragonfly)

; protect against situation where one of the load functions is used to
; load this file, thereby redefining the function itself while it's running
; and causing newlisp to crash.
(unless load-once

(define (load-once)
(let (ctx (let (_ctx (last $args)) (if (context? _ctx) _ctx MAIN)))
(doargs (file)
(unless (or (context? file) (find file _loaded))
(push file _loaded)
(saved-load file ctx)
)
)
)
)

(define (into-ctx-assoc ctx assoc-list)
(dolist (x assoc-list) (ctx (x 0) (x 1))) ; here dolist is slightly faster than map
)

(define (load-files-in-dir dir regex-match)
(dolist (x (directory dir regex-match))
(load-once (append dir "/" x))
)
)

)

(context 'MAIN)

(define-macro (define-subclass)
(new (args 0 1) (args 0 0))
(dolist (method (rest $args))
(setf (method 0 0) (sym $it (args 0 0)))
(eval (push 'define method))
)
)

; swap the MAIN functions for ours
(unless Dragonfly:saved-load
(def-new 'load 'Dragonfly:saved-load)
(constant 'load Dragonfly:load-once)
)
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: Lutz on October 21, 2009, 01:17:13 PM
newLISP by itself will only define NEWLISPDIR as /usr/share/newlisp, if not defined already. So you could also put this in an Apache .htaccess file


setEnv NEWLISPDIR /blah/foo/mypath

now your Apache CGI processes will see NEWLISPDIR as that path, and newLISP when starting up will not change it.
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: itistoday on October 21, 2009, 02:17:24 PM
Hmm... I just remembered the second reason that I wanted to have a custom module function and that was to prevent modules from being loaded twice. Does the current module function already behave this way or will it re-load the module? I'd like to avoid reloads for speed and also safety.
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: Lutz on October 21, 2009, 03:23:43 PM
If you enter 'module' in an interactive console, you see how it is defined (its just embedded as source in the C files):


> module
(lambda (m) (load (append (env "NEWLISPDIR") "/modules/" m)))

> (global? 'module)
true
>


Many schemes to avoid double loading are possible. You could redefine 'module' to push every file loaded on a module-list.


(define (module m)
    (unless (find m module-list)
        (load (append (env "NEWLISPDIR") "/modules/" m))
        (push m module-list))
)
Title: Re: Maintenance Release newLISP v.10.1.6
Post by: itistoday on October 21, 2009, 04:57:13 PM
Quote from: "Lutz"If you enter 'module' in an interactive console, you see how it is defined (its just embedded as source in the C files):


> module
(lambda (m) (load (append (env "NEWLISPDIR") "/modules/" m)))

> (global? 'module)
true
>


Many schemes to avoid double loading are possible. You could redefine 'module' to push every file loaded on a module-list.


Actually that's what my load-once does, and since I'm replacing newLISP's load with load-once, and module is defined in that way then module should also load modules only once. I should have checked before asking, I thought that because it was a built-in function it would be defined in C, but I'm glad it's not! :-)