mystery quoting

[rickyboy]

suppose newLisp had lexical scoping. Forget macros for a second.

imagine there is a function mydolist that receives a quoted code, and very much behaves like dolist

Code Select Expand

(set 'lst '(1 2 3))
(let (a 3)
    (mydolist '(e lst)
               '(+ a 3)))

when mydolist calls eval on '(+ a 3), what is the value of a? 3, you say. But only with dynamic scope. with lexical scope, if mydolist calls eval, it's evaled in the lexicals scope where mydolist was defined, which could be very different from the actual and where a could be undefined. So, with lexical scope, you have to receive a list and eval it in the current scope. Something like

Code Select Expand

(let (a 3)
   (eval (mydolist '(e lst) '(+ a 3))))
which, by the way, is exactly how common lisp macros work.

Right!  That's why a 'mydolist' CL macro shouldn't call 'eval' in its definition (and one reason why CL and Scheme PL experts discourage its general use in this way), for it would break the semantic model I described in my last post.  Now, because CL macros behave the way you describe, we can write "variable quantifiers" (which bind variables in the scope of their expressions (not their definitions)) in lexically scoped languages.  This is how 'dolist' can respect the outer binding of 'a' by the 'let' in CL.  This boils down to the fact that, even in lexically scoped languages, the bound variables are taken care of and we have to worry about how the free variables get their values (cf. my previous post).

 

about appling macros, ... I think [this] is not the desired behavior.

Ah yes, our original topic.  (Sorry, Fanda.)  What do you think about this?

Code Select Expand

(define-macro (my-or)
  (let ((v (eval (args 0))))
   (cond ((= (length (args)) 1) v) ; base case
         ((empty? (args)) nil)     ; degenerate case
         (true
          (if v
              v
            (apply-macro my-or (rest (args))))))))

(define (apply-macro macro arglist)
  (letex (macapp (cons macro (map quote arglist)))
    (eval macapp)))
I believe apply-macro works when used for nested macro calls where apply would be needed (e.g. in the tail call of a recursive definition of a macro, as in this version of my-or); apply-macro's job, via its call to eval is to prevent the accumulation of successive quotes.  And there may be a better scheme to play than this, since it doesn't seem to have general application (geeky pun intended), as seen in this usage with Fanda's qq macro:

Code Select Expand

> (apply-macro qq '(a b c))
('a 'b 'c)

invalid function : ('a 'b 'c)
called from user defined function apply-macro
It's *almost* there -- it's just doing one eval too many.  :-)  In this case, I'd use apply, as Lutz did

Code Select Expand

> (apply qq '(a b c))
('a 'b 'c)
('a 'b 'c)

It does seem to me that apply is the culprit in Fanda's situation.  I originally thought that it was the nested macro call that caused the problem, but this little test disabused me of that notion.

Code Select Expand

;; Here's a macro with a nested macro call of its argument.
(define-macro (myq)
  (letex (arg1 (args 0))
    (qq arg1)))

> (myq 'x)
('x)
('x)

;; Ah, this worked!  But recall 'q' which uses 'apply':

> (q 'x)
(''x)
(''x)
Lutz, any words of wisdom?  --Rick

[Lutz]

It all boils down to this: does 'apply' evaluate the elements in the arguments list, or does it not?

Code Select Expand

(set 'a 1 'b 2 'c 3)

(+ a b c)  => 6

(apply + '(a b c)) => value expected in function + : 'a


we see that 'apply' does not evaluate a,b and c.

Code Select Expand

(set 'a nil 'x 9)
(my-or nil (cons 2 3)) => (2 3)
(my-or a (cons 2 3)) => (2 3)
(my-or x (cons 2 3))  => 9

(apply-macro my-or '(nil (cons 3 4))) => (3 4)
(apply-macro my-or '(a (cons 2 3))) => nil ; the evaluated a (*)
(apply-macro my-or '(x (cons 2 3))) => 9


'apply-macro' seems to work like this: apply the macro or function on the un-evaluated elements, and then evaluate the result. But this can be accomplished easier like this:

Code Select Expand

(eval (apply my-or '(nil (cons 3 4)))) => (2 3)
(eval (apply my-or '(a (cons 2 3)))) => nil
(eval (apply my-or '(x (cons 2 3)))) => 9


The results are the same as with 'apply-macro', just wrapping the 'apply' with an 'eval':

Code Select Expand

(apply-macro macro argslist)

; the same as

(eval (apply macro argslist))


'apply-macro' works just like 'apply' except it evaluates the result.



My point is this: the whole issue is about the specific evaluation rules 'apply' should follow, and it should apply these rules consistently to functions and macros. When you deal with a big API you don't always know if a function you are using is a macro or normal function.



Lutz



ps: all other 'my-or' in a previous post show the same results



(*) This seems inconsistent with the result of the normal execution of 'my-or' but is consistent with the rule: first apply on unevaluated, then evaluate result.

[nikus80]

it doesnt seems to work.

I need my-or to eval only the arguments needed. my-or as a function is not what I need.



(define (prit)

   (print "printed!"))

(define (ret-nil)

   '())



(define-macro (my-or)

(let (val (eval (args 0)))

(if val val (eval (apply my-or (rest (args)))))))



That doesn't work, because of the mystery quoting/appling problem.



try (my-or nil (ret-nil) (cons 2 3) (prit)), it returns '(). The my-or suggested by ricky returns '() too.



I think what you're receiving is the quoted list '(ret-nil), you eval it and it's still a non-null list, so it returns the list. then its evaled, and it returns the empty list. This is not desired.



About not knowing something is a macro or a function I can't understand why would I want macro and functions to behave identically, especially when I know macros don't have any speed execution advantage. When I use dolist, I know it's a macro. When I use my-or, I know it's a macro and I know it only evals what is needed and that it wont eval (prit).

[Fanda]

My point is this: the whole issue is about the specific evaluation rules 'apply' should follow, and it should apply these rules consistently to functions and macros.

I completely agree. My opinion is that 'apply' works just fine - it needs to be consistent.



Fanda

[Fanda]

'my-or' can be written as this:

Code Select Expand

(define-macro (my-or)
  (if (empty? (args))
    nil
    (if (or (eval (args 0)) (= 1 (length (args))))
      (eval (args 0))
      (eval (cons 'my-or (rest (args)))))))


Code Select Expand

> (my-or nil (ret-nil) (cons 2 3) (prit))
(2 3)


Fanda

[Lutz]

Nikus80: it does work with this changed definition of your macro:

Code Select Expand

(define-macro (my-or)
    (let (val (eval (args 0)))
       (if val val (apply my-or (rest (map eval (args)))))))

; and this better one if all members in the list are nil)

(define (my-or p)
    (if p p (if (args) (apply my-or (args)) p)))


and works with all others I posted together with this one earlier.



Lutz

[Lutz]

... and it works also with Fanda's latest definition:

Code Select Expand

(set 'x 9 'a nil)
> (eval (apply my-or '(nil (cons 3 4))))
(3 4)
> (eval (apply my-or '(a (cons 3 4))))
nil
> (eval (apply my-or '(x (cons 3 4))))
9
>


Lutz

[nikus80]

Fanda, that's cool! that's exactly what I was wanting to do with apply!



for the rest, there is something I'm doing wrong? They don't work!

Code Select Expand

(define (prit)
(print "printed!"))
(define (ret-nil)
'())


(define-macro (my-or)
    (let (val (eval (args 0)))
       (if val val (apply my-or (rest (map eval (args)))))))

(define (my-or2 p)
    (if p p (if (args) (apply my-or2 (args)) p)))

(define-macro (my-or3)
  (if (empty? (args))
    nil
    (if (or (eval (args 0)) (= 1 (length (args))))
      (eval (args 0))
      (eval (cons 'my-or3 (rest (args)))))))

(define (fun-my-or2 )
(let (val (eval (args 0)))
(if val
val
(apply fun-my-or2 (rest (args))))))

(define-macro (my-or4 )
(apply fun-my-or2 (args)))

> (my-or nil (ret-nil) (cons 2 3) (prit))
printed!(2 3)
> (my-or2 nil (ret-nil) (cons 2 3) (prit))
printed!(2 3)
> (my-or3 nil (ret-nil) (cons 2 3) (prit))
(2 3)
> (my-or4 nil (ret-nil) (cons 2 3) (prit))
(2 3)
>


EDIT: (define (my-or p) (if p p (if (args) (apply my-or (args)) p)))  has the same problem

[Lutz]

'my-or3' also double evaluates on this one:

Code Select Expand

>  (my-or3 nil (prit) (cons 2 3) (ret-nil))
printed!printed!"printed!"
>


but this definition finally seems to be ok:

Code Select Expand

(define (prit) (print "printed!"))
(define (ret-nil) '())

(define-macro (my-or)
    (let (v (eval (args 0)))
        (if v v (if (args) (eval (cons 'my-or (rest (args))))))))

> (my-or nil (prit) (cons 2 3) (ret-nil))
printed!"printed!"
> (my-or nil (ret-nil) (cons 2 3) (prit))
(2 3)


Lutz

[rickyboy]

So, what is the current status of this thread?  I will take a shot.



(1) Once you all worked out the my-or argument evaluation issues, when the dust settled, the most progress seemed to be made by Fanda who introduced the (eval (cons 'my-or (rest (args)))) tail call idiom.



(2) We haven't yet found, for my-or, a tail call idiom involving apply which works.  (Of course, I'm not talking about the non-recursive implementation of my-or we figured out many days ago -- and which works fine).  And whether we can find such a call is still an open question.



(3) Fanda's original question, i.e. why does (q 'x) yield (''x), is still an open question.



Have I missed anything?



Finally, I wanted to say (before I forget again) "thank you" for a great discussion thread.  I had to get back to office work the past two days, so I couldn't participate more, but when I had the time, I really enjoyed thinking about the issue.  Thanks,  --Rick

[Lutz]

Here an explanation again for the quote-effect, this time in a different way:



The 'qq' macro is basically a list maker, taking un-evaluated arguments, while 'list' is a list maker evaluating arguments:

Code Select Expand

> (define-macro (qq) (args))

> (list 'a 'b 'c)
(a b c)
> (qq a b c)
(a b c)


If you agree on the outcome of above examples, you have to agree upon the following behavior of 'qq' too.

Code Select Expand

> (apply list '('a 'b 'c))
('a 'b 'c)
> (apply qq '(a b c))
('a 'b 'c)


If the applied 'qq' whould not quote the list members it would behave like 'list', which it is not.





Lutz

[nikus80]

I see what you mean.



(apply list '('a 'b 'c)) is equivalent to (list ''a ''b ''c)

so,

(apply qq '(a b c)) is equivalent to (qq 'a 'b 'c)



the thing is, I can't get rid of the quotes! with list, I can.



(apply list (list a b c)) is equivalent to (list a b c)

(apply qq (list a b c)) is not equivalent to (qq a b c)



In the first one, (list a b c) is evaled, then the resulting list is applied with list, without evaling its arguments since I've already evaled them. This works fine, since list is a function, and all it's arguments are evaled anyway. It works with any function btw.

Now, with apply qq, it isn't very straightforward. Cause (apply qq (list a b c)) is meant to work with functions only, that's what I meant early when I said applying a macro just felt weird. With apply, I can't get something that translates to (qq a b c). so the answer to "We haven't yet found, for my-or, a tail call idiom involving apply which works." is guess is: we won't.



The thing is, it does not matters. Just use fanda's idiom. Or better, make it a macro or something.



at first I thought:

(define-macro (fapply)

    (eval (cons (args 0) (eval (args 1)))))



but it doesn't works. I think is because



(define-macro (my-or)

    (let (v (eval (args 0)))

        (if v v (fapply my-or (rest (args))))))



when fapply evals (rest (args)), it thinks args refers to fapply args, not my-or args. I don't know which is the workaround to that. But for the moment, since there is no need to quote the first argument, just use:



(define (fapply)

    (eval (cons (args 0) (args 1))))



still, like I said, I'm not happy with applying macros unless it's inside another macro, which in that case I probably need fapply, not apply.