Arc Forumnew | comments | leaders | submitlogin
Redefining macros does not effect existing defs?
3 points by brett 6112 days ago | 16 comments
Is this the expected behavior? I'm new to lisp macros.

  arc> (mac foo () "foo")
  #3(tagged mac #<procedure>)
  arc> (def sayfoo () (pr (foo)))
  #<procedure: sayfoo>
  arc> (sayfoo)
  foo"foo"
  arc> (mac foo () "bar")
  *** redefining foo
  #3(tagged mac #<procedure>)
  arc> (sayfoo)
  foo"foo"


6 points by pg 6112 days ago | link

Yes. In most Lisps (Arc included), macro calls are expanded when function definitions are evaluated. So you have to define macros before any code that depends on them.

-----

5 points by NickSmith 6112 days ago | link

I am new to Lisp as well, so please excuse my ignorance as I am probably missing something thats obvious to others here.

Does this mean then, that whenever we make a change to a macro definition we must also redefine every function that already depends on the original macro definition? If this is the case then I would imagine there's a lot of tricky maintenance required for a program that has lots of 'layers' when a 'base' macro is changed.

-----

7 points by soegaard 6112 days ago | link

Yes, which is why DrScheme restarts the REPL everytime the run button is used.

See http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/... for a more elaborate explanation.

-----

1 point by NickSmith 6112 days ago | link

Thanks Soegaard. That is really useful :)

-----

3 points by pg 6112 days ago | link

Yes. The way most Lisp hackers deal with this is to (a) define macros before functions that use them and (b) use the function load, which evaluates all the expressions in a file in order.

-----

3 points by kennytilton 6112 days ago | link

This breaks down on bigger projects spread across multiple source files, at which point one needs some "make" tool such as ASDF for CL where one can pile macros into earlier files and then make other files with code using those macros dependent on the macro-bearing file(s). This can make for a lot of recompilation after minor changes, so I just take my chances and (yep) recompile all when in doubt (which with today's systems goes pretty fast).

-----

4 points by NickSmith 6112 days ago | link

Ah right, I get it now. Thanks. I knew there had to be a workaround somewhere.

BTW, kudos for sticking to your guns on first principles Paul. Arc really is a joy to learn and use.

-----

2 points by brett 6112 days ago | link

Thanks. My hope was to shadow whitepage to add styling to the login page with out messing with login-page.

I'm guessing this is related to the performance cost of expanding the macro on the fly?

-----

4 points by pg 6112 days ago | link

Right. Usually I'm happy to trade speed for increased flexibility, but expanding macros at runtime doesn't seem to let you do all that much more. If anything it seems conceptually cleaner to think of macros as being expanded away at the time a function is evaluated.

-----

1 point by shiro 6112 days ago | link

One idea I'm mulling for some time is that, when you compile a function you mark it with the macros you've used, and when one of the macros is redefined it triggers recompilation of the original function. You can delay the recompilation until the function is called next time, of course.

(Actually it's not only for macros, but it will work well for inlining built-in functions.)

Possible drawback is that the effect of macro redefinition isn't very obvious in such a system. You change a macro in running server, and it can trigger recompilation of large potion of the code...

-----

1 point by em 6112 days ago | link

Interesting idea. However, for that to work, one would need to recompile (or mark as dirty) _anything_ that referenced the macro, which could be lists or even a single variable who's value is simply a symbol.

i.e. (mac foo () ''a) (= bar (foo))

And if a macro expansion used the value of bar, i.e. (mac baz () `(prn ',bar))

then anying using baz also would need to be recompiled.

-----

1 point by shiro 6112 days ago | link

I don't think so. Since Arc (and most Lisp dialects) are applicative order, (foo) is evaluated once when (= bar (foo)) is evaluated. Redefining foo won't affect the value of bar (and it doesn't matter whether foo is a macro or a function).

Of course, if (= bar (foo)) is a part of a function, then the function should be recompiled.

-----

1 point by elibarzilay 6111 days ago | link

A real solution for this will be really hard. For example, I'd expect this work:

  (mac foo () 1)
  (def bar () (let x (foo) (fn () x)))
  (= x (list (bar)))
  (mac foo () 2)
  ((car x))
    --> 2
(And the usual solution is to lower your expectations...)

-----

1 point by shiro 6111 days ago | link

Why? No matter whether foo is a macro or a function, a closed value of 'x' is fixed in the closure returnd by bar, so redefining foo shouldn't affect the closure bound to x. (But the next call of bar returns a closure that closes 2 as 'x'). Am I missing something?

-----

2 points by elibarzilay 6111 days ago | link

Like I said -- the usual solution, which you think about, is to lower your expectations. And I'm not joking when I say that: you really just decide to expect that `x' is not going to changed since it's "closed", so you're happy with the result. Some people set their expectations lower, and they expect a macro redefinition to not change anything about existing bindings, so they won't think about your hack as necessary.

And BTW, this is not just some cooked up academic non-problem. It's a practical issue. Consider this example:

  (mac foo () ...)
  (def bar ()
    ...do something...
    ...change foo...
    ...continue doing stuff...)
If that macro change will (with your suggestion) lead to recompiling `bar', but that will not affect the current active call. This is a very common problem with hacking live servers (as pg often talk about):

* You cannot change the main server handler loop unless you make it call itself via its name, so the recursive call will use the new definition.

* Even if you do, you should still be extremely careful since you might change the code when there's a live thread around which was compiled with the old macros, and might rely on the old macros. For example, `f1' and `f2' use `foo', `f1' is invoked in a thread, now `foo' gets redefined, after that's done `f1' continues to call `f2': old code is calling new one, things break.

-----

1 point by shiro 6111 days ago | link

I still believe you are talking different things. My "expectation" is derived from these axioms:

* Macro is a local source-code transformation that replaces macro call with the macro definition (with some hygienic magic, if you prefer).

* The language adopts applicative order.

If you want redefining foo to affect the value of the closed variable 'x', then either you have a non-trivial definition of macros (macro transformation requires transformation of code surrounding the macro call) or you are adopting different evaluation semantics (like x is bound by call-by-name).

Note that your problem occurs even foo is a procedure. It's not a macro problem. It's a problem of semantics of closed variables.

-----