"you might also decide to keep apply, but have it only work on functions. Then you'd use eval to "apply" fexprs. After looking at the issue, I'm not really opposed to that."

You know, I was tempted to define all-true as a macro:

  mac all-true(xs)
    `(and ,@xs)

I've made two incompatible changes, however.

a) Default values are not inlined during initialization. Before:

  > (inst 'foo)
  #hash((field1 . default))

  > (inst 'foo)
  (tagged tem (foo #hash() ...))

b) Reading from file now includes all fields rather than just the ones included in the template. Does this seem reasonable? I think the reasoning for dropping unknown fields was so you could upgrade or rollback the template and it would work with existing data. But if you delete a field it doesn't do any harm if it's set, right?

Yes, that is the expected result.

---

"Are you saying this should work even if list was a macro/fexpr?"

Yes. But as soon as you try to make it work with fexprs you'll run into the following problem. Here's what I wrote earlier:

"To put it another way, (apply list '((1 2))) would be equivalent to (eval (cons list '((1 2)))) which is equivalent to (eval (list list '(1 2))) which is equivalent to (eval '(list (1 2))) which is equivalent to (list (1 2))."

What I'm saying is that if you changed apply so it worked with fexprs, then (apply list '((1 2))) would be equivalent to (eval '(list (1 2))). If you try evaluating that, you'll get an error saying something like "1 isn't a valid function call" or whatever.

That's because functions evaluate their arguments. So when the "list" function evaluates the argument "(1 2)" it tries to call the "1" function with the argument "2" which obviously fails.

So instead you have to call (eval '(list '(1 2))) to suppress evaluation. Note the extra quote! So what rocketnia said (and I now agree with) is that the "apply" function would have to map quote over each of its arguments. In other words, (apply list '((1 2))) would basically be the same as (eval '(list '(1 2)))

Alternatively, if functions were "wrapped" fexprs, then apply could "unwrap" the function, which would lead to more-or-less the same effect, sans a few details.

In other words, you'd either define "apply" like this...

  (def apply (f args env)
    (eval (cons f (map (fn (x) (list quote x)) args)) env))

  (def apply (f args env)
    (eval (cons (unwrap f) args) env))

This should work with both fexprs and functions in a (hopefully) intuitive way.

---

Oh, yeah, you might also decide to keep apply, but have it only work on functions. Then you'd use eval to "apply" fexprs. After looking at the issue, I'm not really opposed to that. It seems reasonable, provided that supplying arguments as a list is most common with functions. The lack of consistency does bug me a little, though.

---

* [1]: If Kernel already uses that definition, why does apply supposedly only work with functions? I believe (without actually checking) it's because the unwrap function throws an error if the argument isn't a function. So you can't call unwrap on fexprs.

> Absolutely not. It should behave like (list '(1 2)).

sbcl:

  * (apply 'list '((1 2)))
  ((1 2))
  * (list '(1 2))
  ((1 2))

  > (apply list '((1 2)))
  '((1 2))
  > (list '(1 2))
  '((1 2))

The only area I currently disagree with rocketnia is this: "Unfortunately, neither of these approaches does what you want with 'def."

---

"For example your analogy for def sounds exactly like what Pauan is saying. What am I missing?"

Yes, it is exactly like what I was saying. The point was that if you changed apply so it worked with def (and other fexprs) it would then break functions like list. Finding a way so that apply can work on both fexprs and functions is one of the points of unwrapping functions.

---

"But that's how it should behave, right? Is that not what Pauan is suggesting as well?"

Absolutely not. It should behave like (list '(1 2)). That is how it would behave if apply unwrapped functions, or if apply mapped quote over the arguments.

Of course, one could make the argument that you should have to say (apply list '('(1 2))) instead... but in that case I'd rather have two separate operators: one that works on fexprs, and one that works on functions.

For example your analogy for def sounds exactly like what Pauan is saying. What am I missing?

"Meanwhile, with your proposal, the use of 'apply with functions would break: (apply list '((1 2))) would behave like (list (1 2))."

I agree. I rather like that part of Kernel.

---

"Unfortunately, neither of these approaches does what you want with 'def."

---

"But why would apply need to map quote on the argument list? If it simply doesn't evaluate the individual sub-expressions, then it should be the same thing."

Since you (Pauan) would have procedures be fexprs that explicitly eval'd their arguments, using apply on a procedure would cause the procedure to eval the args. Whereas (apply list '((1 2))) results in the list ((1 2)) in Arc, in this hypothetical language, 'list itself would try to eval (1 2) and get an error.

In other words, the Scheme/Arc/Kernel 'apply takes a list of argument values, not expressions, so (from a certain point of view) they're not actually suitable arguments for an fexpr call.

If we want Scheme/Arc/Kernel-like behavior, 'apply can make up for the procedure's own evals by quoting the args, or it can make up for them the way it does in Kernel, by unwrapping an applicative into something that skips the eval-my-args step altogether.

Personally, I think unwrapping is cleaner, but quoting makes (apply and foo) behave like (all idfn foo), which is what people seem to expect. Unfortunately, neither of these approaches does what you want with 'def.

---

"It seems to me that (apply def '(foo (a b c) (+ a b c))) should be exactly equivalent to (def foo (a b c) (+ a b c))."

Pauan, the step-by-step way you comprehended my response to this is great, but I wasn't reasoning in that much detail. :-p I was mostly going by analogy:

  (apply def '(foo (a b c) (+ a b c)))
  (def foo (a b c) (+ a b c))
  *** success ***
  
  (apply list '(foo (a b c) (+ a b c)))
  (list foo (a b c) (+ a b c))
  *** error: unbound variable 'foo ***
  
  (apply list '((1 2)))
  (list (1 2))
  *** error: can't call 1 ***
  
  (all-true (list '(1 2)))
  (apply and '((1 2)))
  (and (1 2))
  *** error: can't call 1 ***

  >
  > (map quote args)
  >

  (def map (f seq)
    ...
    ... (f elem) ...
    ...)

  (def apply (f args env)
    (eval (cons f (map quote args)) env))

---

I didn't at first either. But I walked through the steps one-by-one until I figured it out. Here they are:

apply is a function, so it evaluates all its arguments. When calling (apply list '((1 2))) its arguments evaluate to the function list and the list ((1 2)). It then calls the function list with the argument (1 2). This is equivalent to (list (1 2)), as rocketnia said.

---

To put it another way, (apply list '((1 2))) would be equivalent to (eval (cons list '((1 2)))) which is equivalent to (eval (list list '(1 2))) which is equivalent to (eval '(list (1 2))) which is equivalent to (list (1 2)).

"Meanwhile, with your proposal, the use of 'apply with functions would break: (apply list '((1 2))) would behave like (list (1 2))."

  (def all-true(xs) (and @xs))

  (def all-true(xs) (apply and xs))

  (apply and '((1 2)))

  (and (1 2))

If 'apply literal-wrapped the args, it would work more like this:

  (and '(1 2))

Instead of using write-table directly, use temstore:

  > (deftem foo field1 'default)
  > (temstore 'foo (inst 'foo 'field1 nil) "x") ; field1 = nil
  > ((temload 'foo "x") 'field1)
  nil

But why would apply need to map quote on the argument list? If it simply doesn't evaluate the individual sub-expressions, then it should be the same thing. My current view for an ideal Lisp is that all evaluation should be done explicitly, in fexprs. No implicit eval by apply or the runtime or whatever.

That means that apply would pass the arguments unevaluated to the fexpr, which could then choose to eval them or not. And since a function is defined as a fexpr that evals all its operands first, the semantics are preserved for functions too.

Basically, I think fexprs should be the default (using eval when needed), rather than functions being the default (using quote to suppress evaluation). In other words, not evaluating should be the default.

---

"Meanwhile, (apply def foo), which to me doesn't make sense in the first place"

Why not? It seems to me that...

  (apply def '(foo (a b c) (+ a b c)))

  (def foo (a b c) (+ a b c))

That's why I brought up the case. If a language is designed for a sufficiently smart programmer (as is the case with Arc and Wart), the language designer don't have to worry about ensuring things like "eval always does X"; the programmer will leave these invariants alone if they're really that useful, and will break them if breaking them is even more useful. In fact, I think Arc sacrifices hygiene in favor of implementation simplicity in order to be more intuitive when broken.

Intuitive even after you break it. That could be an interesting take on the Arc philosophy. ^_^

But yeah, Wart could still do this in a way where all the abstraction leaks behave exactly as akkartik expects them to (whether that's hygienically or not).

---

"That was kinda akkartik's point: that apply doesn't work properly on macros."

Right, I meant to allude to a bit more, which I was about to elaborate on. I'll just elaborate here, then. :)

If 'apply wraps up its args as literal expressions, then (apply and foo) does work as everyone keeps trying to say it should work. :-p That is, it works just like (all idfn foo), so I don't see why anyone would even care to use it.

Meanwhile, (apply def foo), which to me doesn't make sense in the first place, may also "work," successfully giving an error since a literal expression isn't an assignable identifier.

I consider these to be inelegant consequences of the implementation approach, rather than useful behavior. But if akkartik sees it differently, I'd like to make sure the other inelegant parts (the potential lack of hygiene) don't come as a surprise. ^_^

I've been playing with this idea:

  (def quotify(f) (fn 'args (f @args)))

  wart> = x '(1 2)
  wart> (and @(quotify.list x))
  (1 2) ; woohoo!

It could use the original definition of those, if that were wanted.

This is actually consistent with Kernel's semantics, since it's not possible to overwrite a built-in variable, you can only shadow it. And because Kernel uses lexical scoping, it should use the binding in the original environment. So you'd get this behavior for free.

Of course, we're talking about wart here, so wart can do whatever the hell it wants to do. I merely mention this to point out that (as far as I know) that's not a problem in Kernel, which also has fexprs. So it shouldn't be a problem in wart either, assuming its implementation of lexical/dynamic variables is solid.

---

"calls 'apply with an fexpr that does something with its args other than immediately evaluating them (e.g. most "macros")."

I think in Kernel, a call to 'apply invokes an applicative with a list of argument values, not expressions it should evaluate. So an applicative isn't exactly an fexpr that calls 'eval on its args, because it doesn't actually eval its args in every instance it's called.

Instead, the arguments to an applicative are evaluated as part of the Kernel core semantics. When applying a value to a list, if the value is an applicative, it's unwrapped, and the value inside is applied to the result of mapping eval over the list.

See, that's what I'm talking about. It seems to me that apply shouldn't eval its arguments. Instead, a function would be a fexpr that would eval all its arguments before doing anything else. Something like this:

  (= fn (fexpr (parms . body) env
          (eval (w/uniq (args env)
                  `(fexpr ,args ,env
                     (let ,parms (map (fexpr (x) nil (eval x ,env)) ,args)
                       ,@body)))
                env)))

Also, it might be a good idea to make a more general "wrap" function that would turn any fexpr into a function:

  (= wrap (fexpr (x) env
            (let x (eval x env)
              (fexpr args env
                (let args (map (fexpr (x) nil (eval x env)) args)
                  (apply x args env))))))

  (= fn (fexpr (parms . body) env
          (wrap (eval `(fexpr ,parms nil ,@body) env))))

  (apply wrap.and list.x)

  (wrap.and @list.x)