fexpr issues

[Astrobe]

Consider this dumbed-down version of do-until:

Code Select Expand


(module "macro.lsp")
(define-macro (DO-UNTIL test B)
 (do-while (not (eval test)) (eval B)))

(macro (DO-UNTIL* T B)
(do-while (not T) B))


(println "test 1")
(define (foo k)
  (DO-UNTIL (<= k 0) (begin (println k) (dec k))))
(foo 3)

(println "test 2")
(define (foo* k)
(DO-UNTIL* (<= k 0) (begin (println k) (dec k))))
(foo* 3)

(println "test 3")
(define (bar*)
  (let (k 0)
(DO-UNTIL* (>= k (args 0)) (begin (println k) (inc k)))))
(bar* 3)

(println "test 4")
(define (bar)
  (let (k 0)
(DO-UNTIL (>= k (args 0)) (begin (println k) (inc k)))))
(bar 3)


First problem: foo doesn't behave correctly under debug; doing (debug (foo 3)) doesn't do the same as (foo 3)



Second problem: bar doesn't work at all, because (args 0) is evaluated in the context fo the macro DO-UNTIL.



Those kind of problems makes it really tricky to use fexprs. On the other hand, rewrite macros provided by macro.lsp don't have these problems, and are also faster because the substitution is done at compile-time. They however increase the code size, but this can be mitigated by factoring out the "big parts" of macros in external functions.



macro.lsp notes that it increases the load-time of scripts. Is the impact significant? What about native and improved support for rewrite macros?

[Lutz]

First problem: foo doesn't behave correctly under debug; doing (debug (foo 3)) doesn't do the same as (foo 3)

I see (debug (foo 3)) behaving correctly doing the same thing as (foo 3). May be if you replace (println k) with (println "=====>" k), you can better follow the debug behavior. There is a lot of output from the debugger in-between the println expressions, so perhaps you just missed it visually, scrolling off the terminal screen.

Second problem: bar doesn't work at all, because (args 0) is evaluated in the context fo the macro DO-UNTIL.

fexprs are functions, so I would argue: the behavior is expected. args should always work in the context of the currently executing function and nested functions/fexprs should all keep their local versions of args, this makes code more readable too.



To your last question: "should macro be implemented natively". I frequently have thought about this and keep looking into it. At least at the moment, I have not found a way to do this efficiently without increasing load speed substantially. Even when implemented natively, for every functor expression symbol parsed, you would have to do the lookup, if a reader-event is defined.

[Astrobe]

May be if you replace (println k) with (println "=====>" k), you can better follow the debug behavior

Indeed I thought about this possibilty and did something like that. It seems I didn't put enough dashes in my println.

fexprs are functions, so I would argue: the behavior is expected. args should always work in the context of the currently executing function and nested functions/fexprs should all keep their local versions of args, this makes code more readable too.

The problem I see is that one can easily forget that some definition is actually a macro and pass it an expression that contains (args) or $args. This situation typically happens when one nests macros, id est when one uses macros inside macros.

Even when implemented natively, for every functor expression symbol parsed, you would have to do the lookup, if a reader-event is defined.

What about doing it like Forth? Macro symbols could have a special mark, that triggers their execution (after the arguments have been parsed) at compile-time.

[TedWalther]

You mean like , and ,@ and ` operators in Common Lisp?

Both Common Lisp and Scheme also support the backquote operator (known as quasiquote in Scheme), entered with the ` character (grave accent). This is almost the same as the plain quote, except it allows expressions to be evaluated and their values interpolated into a quoted list with the comma , unquote and comma-at ,@ splice operators. If the variable snue has the value (bar baz) then `(foo ,snue) evaluates to (foo (bar baz)), while `(foo ,@snue) evaluates to (foo bar baz). The backquote is most frequently used in defining macro expansions.[36][37]

If there are several layers of function calls, how do you know if you need to expand (args 0) with quasiquote or not?

[Lutz]

It would be like the current macro facility from the module macro.lisp but built into newLISP. The goal is to maintain the current macro-method but faster w/o impact on newLISP's speed in loading source code.



To get rid of back quotes and other special characters in LISP was/is one of newLISP's goals ;)



The reader-event function would stay as it has many other possibilities besides using it to implement expansion macros.



See also here: http://www.newlisp.org/downloads/newlisp_manual.html#reader-event">http://www.newlisp.org/downloads/newlis ... ader-event">http://www.newlisp.org/downloads/newlisp_manual.html#reader-event



Ps: I think Astrobe is talking about the way it it implemented, not the way it works for the programmer. When doing it natively, I could handle things at a lower level not possible with the current implementation as shown in the reader-event example.

[TedWalther]

I was referring to this statement by Astrobe:

The problem I see is that one can easily forget that some definition is actually a macro and pass it an expression that contains (args) or $args. This situation typically happens when one nests macros, id est when one uses macros inside macros.

It is a bit of a "gotcha" that fexprs are function calls and not just inline expansions.g  Although it makes it nicer for debugging.

[Lutz]

The expansion macro function is now a built-in primitive :)



http://www.newlisp.org/downloads/development/inprogress/CHANGES-10.5.8.txt">http://www.newlisp.org/downloads/develo ... 10.5.8.txt">http://www.newlisp.org/downloads/development/inprogress/CHANGES-10.5.8.txt



and runs Astrobe's macro examples fine.

[TedWalther]

Very nice.  Did you lift the limitation that macro arguments must be uppercase?

[Astrobe]

Oh, nice!



Do I have to wait 10.5.8 to officially come out or can I grab it right now?

[Lutz]

You can grab a source package right now here:



http://www.newlisp.org/downloads/development/inprogress/">http://www.newlisp.org/downloads/develo ... nprogress/">http://www.newlisp.org/downloads/development/inprogress/



also check the new manual entry:



http://www.newlisp.org/downloads/development/inprogress/newlisp_manual.html#macro">http://www.newlisp.org/downloads/develo ... html#macro">http://www.newlisp.org/downloads/development/inprogress/newlisp_manual.html#macro



Ps: if you cannot compile yourself, I can compile for you an executable for either Windows, OSX or Ubuntu Linux for Windows and Ubunto specify UTF8 or non-UTF8. Let me know.

[rickyboy]

http://i2.wp.com/www.valuerupee.com/wp-content/uploads/2013/04/GRAB-IT-NOW1.jpg">

[TedWalther]

Would it be hard to make delete work on a macro, if I wanted to reuse the symbol as a non-macro?

[bairui]

Lutz, re: the manual entry for (macro ...)



1. The signature still has 'define' instead of 'macro'.

2. "But macro definitions cannot be repeated for the same symbol during the sane newLISP session." (sane)

[Astrobe]

Also, I spotted by chance a misuse of define-macro in the "apply" entry: my-gcd is defined as a macro; but if you try (my-gcd 12 18 (+ 1 2 3)) one gets an error.

[Lutz]

Thanks to Barui and Astrobe. Spelling is fixed and the my_gcd define-macro now evaluates its arguments:



http://www.newlisp.org/downloads/development/inprogress/">http://www.newlisp.org/downloads/develo ... nprogress/">http://www.newlisp.org/downloads/development/inprogress/



There will be a development 10.5.8 release next week and probably an official v.10.6.0 in April.