Text only | Text with Images

newLISP Fan Club

Forum => Whither newLISP? => Topic started by: Astrobe on February 28, 2014, 01:42:33 AM

Title: fexpr issues
Post by: Astrobe on February 28, 2014, 01:42:33 AM
Consider this dumbed-down version of do-until:


Code Select

(module "macro.lsp")
(define-macro (DO-UNTIL test B)
 (do-while (not (eval test)) (eval B)))

(macro (DO-UNTIL* T B)
(do-while (not T) B))


(println "test 1")
(define (foo k)
  (DO-UNTIL (<= k 0) (begin (println k) (dec k))))
(foo 3)

(println "test 2")
(define (foo* k)
(DO-UNTIL* (<= k 0) (begin (println k) (dec k))))
(foo* 3)

(println "test 3")
(define (bar*)
  (let (k 0)
(DO-UNTIL* (>= k (args 0)) (begin (println k) (inc k)))))
(bar* 3)

(println "test 4")
(define (bar)
  (let (k 0)
(DO-UNTIL (>= k (args 0)) (begin (println k) (inc k)))))
(bar 3)


First problem: foo doesn't behave correctly under debug; doing (debug (foo 3)) doesn't do the same as (foo 3)



Second problem: bar doesn't work at all, because (args 0) is evaluated in the context fo the macro DO-UNTIL.



Those kind of problems makes it really tricky to use fexprs. On the other hand, rewrite macros provided by macro.lsp don't have these problems, and are also faster because the substitution is done at compile-time. They however increase the code size, but this can be mitigated by factoring out the "big parts" of macros in external functions.



macro.lsp notes that it increases the load-time of scripts. Is the impact significant? What about native and improved support for rewrite macros?
Title: Re: fexpr issues
Post by: Lutz on February 28, 2014, 09:10:19 AM
QuoteFirst problem: foo doesn't behave correctly under debug; doing (debug (foo 3)) doesn't do the same as (foo 3)




I see (debug (foo 3)) behaving correctly doing the same thing as (foo 3). May be if you replace (println k) with (println "=====>" k), you can better follow the debug behavior. There is a lot of output from the debugger in-between the println expressions, so perhaps you just missed it visually, scrolling off the terminal screen.


QuoteSecond problem: bar doesn't work at all, because (args 0) is evaluated in the context fo the macro DO-UNTIL.


fexprs are functions, so I would argue: the behavior is expected. args should always work in the context of the currently executing function and nested functions/fexprs should all keep their local versions of args, this makes code more readable too.



To your last question: "should macro be implemented natively". I frequently have thought about this and keep looking into it. At least at the moment, I have not found a way to do this efficiently without increasing load speed substantially. Even when implemented natively, for every functor expression symbol parsed, you would have to do the lookup, if a reader-event is defined.
Title: Re: fexpr issues
Post by: Astrobe on February 28, 2014, 11:07:25 AM
Quote May be if you replace (println k) with (println "=====>" k), you can better follow the debug behavior


Indeed I thought about this possibilty and did something like that. It seems I didn't put enough dashes in my println.


Quotefexprs are functions, so I would argue: the behavior is expected. args should always work in the context of the currently executing function and nested functions/fexprs should all keep their local versions of args, this makes code more readable too.


The problem I see is that one can easily forget that some definition is actually a macro and pass it an expression that contains (args) or $args. This situation typically happens when one nests macros, id est when one uses macros inside macros.


QuoteEven when implemented natively, for every functor expression symbol parsed, you would have to do the lookup, if a reader-event is defined.


What about doing it like Forth? Macro symbols could have a special mark, that triggers their execution (after the arguments have been parsed) at compile-time.
Title: Re: fexpr issues
Post by: TedWalther on March 01, 2014, 12:31:06 PM
You mean like , and ,@ and ` operators in Common Lisp?


Quote
Both Common Lisp and Scheme also support the backquote operator (known as quasiquote in Scheme), entered with the ` character (grave accent). This is almost the same as the plain quote, except it allows expressions to be evaluated and their values interpolated into a quoted list with the comma , unquote and comma-at ,@ splice operators. If the variable snue has the value (bar baz) then `(foo ,snue) evaluates to (foo (bar baz)), while `(foo ,@snue) evaluates to (foo bar baz). The backquote is most frequently used in defining macro expansions.[36][37]


If there are several layers of function calls, how do you know if you need to expand (args 0) with quasiquote or not?
Title: Re: fexpr issues
Post by: Lutz on March 01, 2014, 02:03:23 PM
It would be like the current macro facility from the module macro.lisp but built into newLISP. The goal is to maintain the current macro-method but faster w/o impact on newLISP's speed in loading source code.



To get rid of back quotes and other special characters in LISP was/is one of newLISP's goals ;)



The reader-event function would stay as it has many other possibilities besides using it to implement expansion macros.



See also here: http://www.newlisp.org/downloads/newlisp_manual.html#reader-event



Ps: I think Astrobe is talking about the way it it implemented, not the way it works for the programmer. When doing it natively, I could handle things at a lower level not possible with the current implementation as shown in the reader-event example.
Title: Re: fexpr issues
Post by: TedWalther on March 02, 2014, 12:32:51 AM
I was referring to this statement by Astrobe:


Quote
The problem I see is that one can easily forget that some definition is actually a macro and pass it an expression that contains (args) or $args. This situation typically happens when one nests macros, id est when one uses macros inside macros.


It is a bit of a "gotcha" that fexprs are function calls and not just inline expansions.g  Although it makes it nicer for debugging.
Title: Re: fexpr issues
Post by: Lutz on March 03, 2014, 12:47:41 PM
The expansion macro function is now a built-in primitive :)



http://www.newlisp.org/downloads/development/inprogress/CHANGES-10.5.8.txt



and runs Astrobe's macro examples fine.
Title: Re: fexpr issues
Post by: TedWalther on March 03, 2014, 01:30:19 PM
Very nice.  Did you lift the limitation that macro arguments must be uppercase?
Title: Re: fexpr issues
Post by: Astrobe on March 04, 2014, 09:40:35 AM
Oh, nice!



Do I have to wait 10.5.8 to officially come out or can I grab it right now?
Title: Re: fexpr issues
Post by: Lutz on March 04, 2014, 10:21:09 AM
You can grab a source package right now here:



http://www.newlisp.org/downloads/development/inprogress/



also check the new manual entry:



http://www.newlisp.org/downloads/development/inprogress/newlisp_manual.html#macro



Ps: if you cannot compile yourself, I can compile for you an executable for either Windows, OSX or Ubuntu Linux for Windows and Ubunto specify UTF8 or non-UTF8. Let me know.
Title: Re: fexpr issues
Post by: rickyboy on March 04, 2014, 10:43:34 AM
(//%3C/s%3E%3CURL%20url=%22http://i2.wp.com/www.valuerupee.com/wp-content/uploads/2013/04/GRAB-IT-NOW1.jpg%22%3E%3CLINK_TEXT%20text=%22http://i2.wp.com/www.valuerupee.com/wp-%20...%20T-NOW1.jpg%22%3Ehttp://i2.wp.com/www.valuerupee.com/wp-content/uploads/2013/04/GRAB-IT-NOW1.jpg%3C/LINK_TEXT%3E%3C/URL%3E%3Ce%3E)
Title: Re: fexpr issues
Post by: TedWalther on March 04, 2014, 03:32:01 PM
Would it be hard to make delete work on a macro, if I wanted to reuse the symbol as a non-macro?
Title: Re: fexpr issues
Post by: bairui on March 04, 2014, 11:48:25 PM
Lutz, re: the manual entry for (macro ...)



1. The signature still has 'define' instead of 'macro'.

2. "But macro definitions cannot be repeated for the same symbol during the sane newLISP session." (sane)
Title: Re: fexpr issues
Post by: Astrobe on March 05, 2014, 03:54:00 AM
Also, I spotted by chance a misuse of define-macro in the "apply" entry: my-gcd is defined as a macro; but if you try (my-gcd 12 18 (+ 1 2 3)) one gets an error.
Title: Re: fexpr issues
Post by: Lutz on March 05, 2014, 06:46:11 AM
Thanks to Barui and Astrobe. Spelling is fixed and the my_gcd define-macro now evaluates its arguments:



http://www.newlisp.org/downloads/development/inprogress/



There will be a development 10.5.8 release next week and probably an official v.10.6.0 in April.
Title: Re: fexpr issues
Post by: Astrobe on March 05, 2014, 09:26:00 AM
I know this is different from what I initially asked for, but...



if you remove lines from 4912 to 4921 (the ones that insert the expansion code), one gets interesting options for forging the replacement expression.

Like for instance this improved curry:


Code Select

(macro (curry*)
  (case (length $args)
    (2 (letex (F (args 0) A (args 1)) (fn(x) (F A x))))
    (3 (letex (F (args 0) A (args 1) B (args 2)) (fn(x) (F A B x))))))

(set 'K (curry* + 1))
> (lambda (x) (+ 1 x))
(set 'L (curry* + 1 2))
>(lambda (x) (+ 1 2 x))


(copied by hand from another screen; parens might be missing)



What I'm thinking is the functionality of "macro" already exists in a module, and the term "macro" can be confusing together with define-macro, so... What if we call our new feature "inline" (for instance), and we don't restrict it to the functionality of its ancestor?
Title: Re: fexpr issues
Post by: Lutz on March 05, 2014, 01:15:16 PM
The module implementation of macro in macro.lsp added more than 100% of load-time for each macro defined. So when you had 10 macros defined, the whole load time of source files, would increase by more than 10 times.



The new native implementation has no measurable impact on load times, even when defining many macros.



You are still able to do, what you are showing in your post. The default expansion function in C in lines 4912 to 4921 can easily be overwritten once you have marked the symbol as a macro symbol:


Code Select

; let newLISP know that curry* contains a reader expansion function
> (macro (curry*))
(lambda-macro () (expand 'nil))

; now assign your own for that same symbol
(define curry* (lambda-macro ()
  (case (length $args)
    (2 (letex (F (args 0) A (args 1)) (fn(x) (F A x))))
    (3 (letex (F (args 0) A (args 1) B (args 2)) (fn(x) (F A B x)))))))

> (set 'K (curry* + 1))
(lambda (x) (+ 1 x))
> (set 'L (curry* + 1 2))
(lambda (x) (+ 1 2 x))
>


What the native implementation of macro gives us, is more reading speed. But we still have the possibility to play our own tricks. The real work - reader-time expansion - happens in lines 3505 to 3515.



You also could code your example without the technique shown above in one step like this:


Code Select

(macro (curry* F A B)
    (if B (lambda (x) (F A B x)) (lambda (x) (F A x))))

> (curry* + 1)
(lambda (x) (+ 1 x))
> (curry* + 1 2)
(lambda (x) (+ 1 2 x))
Title: Re: fexpr issues
Post by: Lutz on March 07, 2014, 12:03:26 PM
More explanations how the new macro works with map and apply and differences to define-macro:



http://www.newlisp.org/downloads/development/inprogress/newlisp_manual.html#macro



Any corrections are appreciated.
Title: Re: fexpr issues
Post by: bairui on March 08, 2014, 12:53:06 AM
Hi, Lutz,



In the last example box for (macro ...):


Code Select
(define-macro (fexpr-double X) i

Trailing i a vim oopsie? :-)
Text only | Text with Images