Arc Forumnew | comments | leaders | submitlogin
1 point by Pauan 4446 days ago | link | parent

"This apply works for some macros (like Pauan's def example), just not all of them."

I think the fundamental reason for this is that macros don't evaluate their arguments, but functions do. Your example macro is written like as if it were a function, but it's not a function. If you instead wrote it as a macro, like so:

  (mac foo (a b) `(cons ',a ',b))
Then it should work just fine. As I said in my e-mail, I think this demonstrates a mistake in the foo macro, not a mistake in apply. Do you have a better example of a macro that breaks with apply, a macro that's actually written as a macro, rather than as an almost-function?

---

"This apply works for some macros (like Pauan's def example), just not all of them."

It still works if you call it like this: (apply foo '(1 '(2 3)))

As I mentioned in my e-mail, I agree that is an inconsistency when trying to make apply work with fexprs, which is part of why I prefer Kernel's approach of requiring you to wrap the fexpr before calling apply on it... but that doesn't mean that a version of apply that works on fexprs is necessarily broken, just a bit hacky and inconsistent.

I wouldn't call apply broken for the reason you demonstrate above (with the foo macro), because foo really should be written as a function, or its definition should be changed to make use of the non-evaluating nature of its operands.

---

It seems to me that when you talk about apply "working correctly" you're expecting it to somehow guess that the foo macro is behaving like a function, so applying it should treat it like a function, even though it's a macro. But precisely because macros/fexprs might or might not evaluate their arguments, apply cannot guess that.

apply can know whether its argument is a function or fexpr, but not whether the fexpr will actually evaluate its operands or not... so it seems your concept of apply "working correctly" would require apply to essentially read your mind, or do some sort of crazy static analysis and make guesses.

Let me put it like this. When calling apply on something, let's pretend that apply maps quote over the argument list, so that these two are equivalent:

  (foo 1 '(2 3))
  (apply foo '(1 (2 3)))
Okay, but that doesn't work for things like def, which expect unquoted unevaluated operands. So in the case of def, the following must be equivalent:

  (foo 1 (2 3))
  (apply foo '(1 (2 3)))
This is easy enough to do: just change apply so it does one thing for functions and something different for fexprs. Using unwrap is a clean and easy way to do that. But your foo macro is a fexpr, so how is apply supposed to know that it's supposed to treat it like a function, even though it's a fexpr? How is it supposed to know that it's supposed to treat the foo macro different from the def macro?

Really, if you want to solve this, I would suggest having two operators, one for the function-like behavior, and one for the fexpr-like behavior. I'll call these two "apply" and "operate", respectively. So, assuming foo is a function, then the following are equivalent:

  (foo 1 '(2 3))
  (apply foo '(1 (2 3)))
  (operate foo '(1 '(2 3)))
And then you'd use operate for fexprs, and apply for functions. But as already noted in the Kernel Report... operate is always exactly equivalent to calling wrap:

  (apply (wrap foo) '(1 '(2 3)))
So it's rather pointless to provide an "operate" operator since apply + wrap does the same exact thing. And if you want even more fine-grained control, you can use eval directly:

  (eval `(,foo 1 '(2 3)))
Which, incidentally, is sometimes shorter than calling apply, assuming your language has support for quasiquote sugar.

---

"I'd love comments on this way of decomposing things."

Naturally I prefer Kernel's approach of having $vau and then having functions just be a thin wrapper around it. In that case you can create wart's version of fn (quoted args and all) as a fexpr, without needing fn to be built-in.



1 point by akkartik 4446 days ago | link

To me the whole point of macros is that you don't have to worry about whether something is a function or a macro when you call it.

"Do you have a better example of a macro that breaks with apply, a macro that's actually written as a macro, rather than as an almost-function?"

How about and in the previous thread? I tried to use it like this:

  (def all-trues(xs)
    (apply and xs))
This suddenly doesn't work if one of the elements of xs is a list:

  (all-true (list '(1 2))) ; => error
It's pretty clear that 'apply for macros' is an ill-posed, arguably meaningless problem, and I thank you for helping me understand that.

-----

1 point by Pauan 4446 days ago | link

"To me the whole point of macros is that you don't have to worry about whether something is a function or a macro when you call it."

But that's completely incorrect because you always always have to know whether it's a function or a macro, even in a language like Arc or Scheme, because they are fundamentally different things.

Even fexprs aren't the same as functions because they don't evaluate their arguments, and argument evaluation is important, so it's very important that you understand whether something is a function or fexpr, and if it's a fexpr, which arguments are evaluated, and how.

That's a large part of the reason Kernel prefixes fexprs with $ by convention: it really helps you to notice right away the difference between the two, and it's an important difference.

---

"How about and in the previous thread?"

Yeah, I think that qualifies as a good example. It breaks for the exact same reason that your foo macro breaks: it evaluates its arguments, so you have to double-quote them:

  (all-true '('(1 2)))
---

"It's pretty clear that 'apply for macros' is an ill-posed, arguably meaningless problem, and I thank you for helping me understand that."

Sure, I see now that trying to make apply work on macros is quite tricky indeed, especially if you want to "not have to worry about the difference between macros and functions". So providing it a la carte with wrap and unwrap (like Kernel) seems far superior to me.

-----

1 point by akkartik 4446 days ago | link

"it evaluates its arguments, so you have to double-quote them"

But the whole point is that the list is just a list of elements! Imagine it's coming from a whole complex sequence of functions. Don't think of it like this:

  (all-true '((1 2))
Think of it like this:

  (all-true l) ; with l bound to ((1 2))
You want callers of all-true to take the trouble to quote elements of l, to be aware that it's going to go through all-true. Isn't that simply hideous? What if they then have to go through a different path that doesn't eventually pass through a macro? Do they have to recompute l without quotes?

"that's completely incorrect because you always always have to know whether it's a function or a macro, even in a language like Arc or Scheme"

What I called the 'point of lisp' was my sense of the ideal. The promise of a high-level language is that you can create new names and use them like primitives that the language came with.

  (let x 3
    bar.x) ; Does the language come with let? Who cares?
All languages break this abstraction, yes. But lisp is lisp because it takes this abstraction farther than anything else.

You've explained the technical reasons why 'apply for macros' is hard, and that makes sense. But the solution is to not use apply with macros, not tie ourselves up in knots with quote.

-----

1 point by Pauan 4446 days ago | link

"Think of it like this: (all-true l)"

Sure, I already considered that. That is, after all, the most common use for apply. And I agree that's ugly. You'd have to say something like (all-true (wrap-quote l)) or somesuch. Ugly.

---

"Isn't that simply hideous?"

Sure is!

---

"The promise of a high-level language is that you can create new names and use them like primitives that the language came with."

And fexprs/macros let you do that. That wasn't my point. What you said is that the point of macros is being able to call things without caring whether the thing is a macro/function/whatever. That view is incorrect because the meaning of the expression differs depending on whether it's a function or macro.

For instance, in the "let" expression you just gave, it doesn't evaluate the variable "x", but you only know that because "let" is a macro. If it were a function it would throw a "x is undefined" error. So you have to know whether something is a function or macro in order to use it correctly.

The same is true of apply: it has two different (and inconsistent) behaviors depending on whether its argument is a fexpr or a function. And that's a problem because it's confusing and makes it really easy to get it wrong. Hence why Kernel makes you explicitly use wrap to specify your intent.

---

"You've explained the technical reasons why 'apply for macros' is hard, and that makes sense. But the solution is to not use apply with macros, not tie ourselves up in knots with quote."

Sure, and even with that approach, if your language has wrap/unwrap, then you can still use apply on macros, you just need to wrap them first:

  (apply (wrap foo) ...)
And if that breaks, then you bring out eval:

  (eval `(,foo ...))

-----

1 point by akkartik 4446 days ago | link

"For instance, in the "let" expression you just gave, it doesn't evaluate the variable "x", but you only know that because "let" is a macro. If it were a function it would throw a "x is undefined" error."

No, I thought about that when I built let. And then I forgot about it. Now when I see that expression I think, "oh, I'm creating a binding." The point of having power is to not have to derive things from first principles every time we see them.

"And fexprs/macros let you do that. That wasn't my point. What you said is that you want to be able to call things without caring whether the thing is a macro/function/whatever."

Yeah, I'm still struggling to express what I mean by 'use them like language primitives'. I think I'm trying to invoke a sense of fundamental knowledge. When I was taught lisp I didn't think, "wait how does (setq a 3) work if a is not bound". I thought, "Ok, setq is assignment." Functions, macros, fexprs, these are all tools to setup such axiomatic intuitions. That's the whole point.

-----

1 point by Pauan 4446 days ago | link

"No, I thought about that when I built let. And then I forgot about it."

Yes, but you still need to carry around that knowledge in your subconscious, and retrieve that knowledge every time you use "let". That was my point: that the distinction between them is important.

---

"The point of having power is to not have to derive things from first principles every time we see them."

Yes... and I'm agreeing with you.

You said that the point of macros is not having to care whether something is a function/macro or not. But on a very fundamental level you do have to care, just like how you have to care about whether a variable is a string or a number.

I was only disagreeing with that statement, that you don't need to care about the differences between fexprs/functions. All the problems with apply come about because of misunderstandings of the differences between the two:

  ;; functions
  (foo '1 '(2 3))
  (apply foo '(1 (2 3)))

  ;; fexprs
  (foo 1 (2 3))
  (apply foo '(1 (2 3)))
This difference is what causes the confusion and makes it very easy to make mistakes when using apply on fexprs. So it's not so much that apply doesn't work on fexprs, as it is that the behavior of apply for fexprs is different than the behavior of apply for functions... which is awfully confusing and inconsistent.

So choosing to make apply work only on functions is a perfectly reasonable position that discourages mistakes.

-----

1 point by akkartik 4446 days ago | link

"And if that breaks, then you bring out eval"

  wart> = l '((1 2))
  wart> (and @l) ; error
  wart> (wrap.and @l) ; error
  wart> (eval `(,wrap.and ,@l)) ; error
  wart> (eval `(,wrap.and ,@(map (fn(_) (cons quote _)) l))) ; error
I think our aesthetics are in disagreement :) When the second one (or maybe the third one, in extreme extremis) fails, I would simply go back and rewrite all-true to not use apply/splice, like I did all: http://github.com/akkartik/wart/commit/d25410707d

-----

1 point by Pauan 4446 days ago | link

Er, shouldn't it be (list quote _) ...? In any case, you can wrap up the last one into a function:

  (def operate (f args)
    (eval `(,f ,@(map (fn (_) (list quote _)) args))))
Then it would just be (operate and l)

I'm not saying that's necessarily a good idea, but if it comes up often enough, might as well write a helper function for it.

---

And when I said "bring out eval" I meant like this:

  (eval `(,and ,@l))
In other words, not wrapping it.

-----

2 points by akkartik 4446 days ago | link

"In other words, not wrapping it."

Ah, this works :)

  (eval `(,and ,@(map (fn(_) (cons quote _)) l)))
In wart, 'a is just (' . a). That eliminates questions like, "what happens if quote gets multiple args?" :)

-----

1 point by akkartik 4446 days ago | link

Pauan, I'm starting to think it's a win to define mapquote like you suggested:

  def all-true(xs)
    (and @mapquote.xs)
But using quoting to suppress future eval like this makes me nervous. Do you think it's a correct transformation?

-----

1 point by Pauan 4445 days ago | link

My position is quite simple: if you want clean elegance, just use Kernel. Seriously.

If that definition of all-true works and doesn't have any noticeable bugs... then go ahead and use it. It'll be hacky and won't be clean and elegant, but it'll be short and simple and work. And it seems to me that wart is trying to get the power of fexprs in a more hacky Arc-ish way, so that's perfectly fine.

-----

1 point by Pauan 4446 days ago | link

Ah, like PicoLisp. Neat, I like that.

-----

1 point by akkartik 4446 days ago | link

  > > Isn't that simply hideous?
  >
  > Sure is!
I think we're in complete agreement that apply for macros is not well-behaved. The rest is just splitting hairs :)

-----

2 points by Pauan 4446 days ago | link

"I think we're in complete agreement that apply for macros is not well-behaved."

Yes, and as I already said a while ago, I think Kernel's approach is ideal. Except now I actually have some solid reasons to back up that statement, thanks to you.

-----