stack trace in newlisp for debugging?

[TedWalther]

Hi.  I did some searching on Google and this forum for "newlisp stack trace" and "newlisp call stack" and "newlisp traceback".



When debugging, sometimes it is very helpful to introspect and see "what combination of function calls led to where we are right now".  Especially inside of event and error handlers.



Lutz, does newlisp provide some standard way to walk the stack backwards and know how we got to where we are?  I want to make a debug function to printf the call-chain and how it got us to where we are.



Also, such a "traceback" would be helpful in general when newlisp gives an error message and drops into the debugger.



And, if you had such a traceback call, what would make it doubly sweet and miles ahead of Common Lisp and Scheme, is that it would show macros the same as functions.



And finally, it would be nice if the traceback also had an option to show the arguments as they were passed to the function call, and as the function call received them.  Ie, I might do (foo a b c) but foo sees itself being called as (foo 1 2 3).

[rrq]

I'm with you at the wishing well for this one, and mad as I am, I translated this into the attached patch, which modifies the execution procedure ever so slightly, and adds the primitive function (map-stack f) to map function f over the lambda call frames in reverse time order, and collate their results.



This makes use of the "lambda stack", but slightly revised so as to hold on to the "call frames" rather than just the functors, i.e., including the actual call parameters. There's a a bit of a memory management challenge in this, and I resorted to copying the arguments (again), rather than trying to track down how it really should be done.



Note that one can pick up the call chain in an error-event handler, which can be quite useful. Here's an example test code for illustration.

Code Select Expand

(error-event
  (fn () (println (last-error)) (println (map-stack (fn (x) x)))))

(define (recursion x y)
  (amb (rickety bam bam)
       (recursion (- y x) (+ x 3))
       (recursion (+ y x) (+ x 4))
       (recursion (+ y x) (+ x 5)) ))

(recursion 1 1)

The patch is for 10.6.3 as of today, and maybe some weeks or months earlier.



Note that there is an obvious performance hit with this, but it could of course be opted in or out with a command line flag, as a debug mode. It doesn't affect the execution path in any way other than collecting the call frames.

[TedWalther]

Wow.  Ralph, may I see your patch?  My private email address will be in your inbox here on the forum momentarily.  Somehow the patch didn't get attached to the forum.  But the use case looks good.

[rrq]

Yes. I thought I had attached it to the post, but I wasn't observant enough to see that the "attach file" had taken issue with the file name.

[Lutz]

Here is my solution to this:



http://www.newlisp.org/downloads/development/inprogress/">http://www.newlisp.org/downloads/develo ... nprogress/">http://www.newlisp.org/downloads/development/inprogress/



New syntax: (trace device-no) prints an evaluation trace to device-no. Example:

Code Select Expand


(trace (open "trace.txt" "w")) ; open trace file
 ... ; one or more expressions to evaluate
(trace nil) ; close trace file


The file trace.txt  now contains the trace. You could use tail -f trace.txt to watch it in another window. Also nice when working from the REPL in another window.



Ps: the function call level is also shown.

Ps: now also writes error info to trace file when error is thrown.

[rrq]

A map-stack patch variation, which 1) makes map-stack stop when the invoked function returns nil (i.e., similar to collect), and 2) makes the error-event be invoked before system reset also at the interactive prompt. The latter changes this event invocation to be the same as during file loading (which is to reset the system subsequent to the invocation), and I prefer this behaviour, since it allows for stack inspection in the error event handler.

[Lutz]

Thanks Ralph, I incorporated some of your ideas in the code for better error reporting. So now we have a better error reporting with more information and we have the new (trace device-no) mode.



The following interactive session shows how things behave on 10.6.2:

Code Select Expand

> (define (foo x) (bar x))
(lambda (x) (bar x))
> (define (bar x) (baz x))
(lambda (x) (baz x))
> (foo 123)

ERR: invalid function : (baz x)
called from user function bar
called from user function foo
>

The following shows changes included from Ralph in 10.6.3:

Code Select Expand


> (define (foo x) (bar x))
(lambda (x) (bar x))
> (define (bar x) (baz x))
(lambda (x) (baz x))
> (foo 123)

ERR: invalid function : (baz x)
called from user function (bar x)
called from user function (foo 123)
>

An this shows the final 10.6.3 with the new trace mode:

Code Select Expand


> (trace (open "trace.txt" "w"))
3
> (define (foo x) (bar x))
(lambda (x) (bar x))
> (define (bar x) (baz x))
(lambda (x) (baz x))
> (foo 123)

ERR: invalid function : (baz x)
called from user function (bar x)
called from user function (foo 123)
nil
we continue the same session showing the contents from the newLISP command-line using the ! :

Code Select Expand


> !cat trace.txt
1 exit: 3
1 entry: (define (foo x)
 (bar x))
1 exit: (lambda (x) (bar x))
1 entry: (define (bar x)
 (baz x))
1 exit: (lambda (x) (baz x))
1 entry: (foo 123)
2 entry: (bar x)
3 entry: (baz x)
3 ERR: invalid function : (baz x)
called from user function (bar x)
called from user function (foo 123):
3 exit: nil
2 exit: nil
1 exit: nil
>

Ps: I still like Ted's idea to do it all using newLISP but the native implementation could be done with minimum additions in the code and without performance hit when not in trace mode.



Ps: Note that #define NO_DEBUG in newlisp.h will only exclude the debug mode but the new trace mode will still work.

[rickyboy]

Just being a lurker from afar: thank you all -- Ralph, Ted and Lutz -- for your attention to this issue.  Please know it's much appreciated by the rest of newlisperdom.

[xytroxon]

Just being a lurker from afar: thank you all -- Ralph, Ted and Lutz -- for your attention to this issue.  Please know it's much appreciated by the rest of newlisperdom.

Ditto! ;p)



-- xytroxon