Arc Forumnew | comments | leaders | submit | CatDancer's commentslogin
2 points by CatDancer 5750 days ago | link | parent | on: Macro expansion/evaluation problem

> Everyone here seems to ignore this eventual issue and focus on the precise problem of 'each, but this was not my point initially.

OK, well, your first example was:

  (if t 1 (amac))
and you complained that the "amac" macro was being expanded at compile time. What if the first argument to 'if isn't known until run time? What if I said

  (if x 1 (amac))
perhaps at run time 'x is going to be false. So we have to expand 'amac, in case that branch of the 'if is invoked.

-----

2 points by palsecam 5750 days ago | link

> So we have to expand 'amac, in case that branch of the 'if is invoked.

But NO!

   cl> (defmacro m () (print "evaluated!"))
   M
   cl> (setf x 2)
   2
   cl> (if x t (m))
   T
   cl> (defun test-m (use-mac?) (if use-mac? (m) nil))
   "evaluated!"     <--- hmmm...
   TEST-M
   cl> (test-m t)
   "evaluated!"
   cl> (test-m nil)
   NIL              <--- ...no it's OK! (no "evaluated!")
Since 'if is a special form we can expect it to do magic. Common Lisp's 'if does, can we agree on that? (I mean seriously. I'm no expert and I don't really know about all this stuff, but I just see some facts. I totally might be wrong).

-----

2 points by CatDancer 5749 days ago | link

Curiosity got the better of me, so I installed Common Lisp.

The (print "evaluated!") itself evaluates to "evaluated!", so here

  cl> (test-m t)
    "evaluated!"
what you're seeing is the value that 'm expands into, not the macro being expanded at that point. If you use something like,

  (defmacro m ()
    (print "evaluated!")
    42)
you'll be able to see what's happening more easily.

-----

1 point by palsecam 5749 days ago | link

Hi hi, nice that you install Common Lisp by curiosity!

However, I don't understand what you want to demonstrate with your macros.

   cl> (defmacro m () (print "evaluated!") 42)
   M
   cl> (if t (m) 43)
   "evaluated"
   42
   cl> (if nil (m) 43)
   43
Ah maybe you mean, when defining the 'test-m function?

   cl> (test-m t)
   42    
Kind of "strange" there is no "evaluated", is this the point? This is a point.

-----

2 points by CatDancer 5749 days ago | link

Yes,

  cl> (test-m t)
  42
is the behavior that I'd expect to see, since the macro was expanded at compile time, when test-m was defined.

The behavior of 'if at the REPL is interesting, perhaps it is to support conditional compilation? However it is a special case, since only at the REPL would we be able to evaluate the test argument to the 'if before compiling the then or else forms. Doing so little as wrapping the 'if statement in a lambda causes the macro to be expanded regardless of the test argument:

  > ((lambda () (if t 3 (m))))
  "evaluated!"
  3

-----

1 point by palsecam 5749 days ago | link

OK here. You're right about the REPL vs normal time eval. The lambda example says everything.

So the 'each of my dreams will be very difficult to have :-D! Maybe with lazy evaluation/JIT compilation [hmmm maybe not needed, see edit below].

Thanks for everything CatDancer!

EDIT:

Lol this stuff has no end. CatDancer I really enjoy all this, but it is late here and I will have to go to bed, seriously!

OK, so no REPL.

In if.lisp:

   (defmacro m () (print "Hello via mac!") 42)

   (defmacro fm ()  ; remember, myeach would also be a macro...
     `(if (eq (read) 'CALLMAC)
        (m) 
        (print "Hello via func")))

   (fm)
On command-line:

   paul@polo-laptop:~ $ echo 'CALLMAC' | sbcl --script if.lisp
   "Hello via mac!" 
   paul@polo-laptop:~ $ echo 'no call' | sbcl --script if.lisp
   "Hello via func"
But yes, if 'fm is a function instead:

   paul@polo-laptop:~ $ echo 'no call' | sbcl --script if.lisp
   "Hello via mac!" 
   "Hello via func"

-----

2 points by CatDancer 5749 days ago | link

Oops, I should have said "top level" not "REPL". My mistake!

"Only at the top level would we be able to evaluate the test argument to the 'if before compiling the then or else forms".

> So the 'each of my dreams will be very difficult to have

Well you know, the Arc compiler knows whether a variable is in the lexical scope. So you could implement your myeach in ac.scm, not using macros, but instead compiling to Scheme.

Or... how about...

  (mac ech args `(each _ ,@args))
:-)

-----

1 point by palsecam 5749 days ago | link

No sorry me because my examples should be better. I use the command line but yes it's still "top level" and this is not enough to test this, right?

But here; OK, if 'fm is a function, the behaviour is "like in Arc". But if it's a macro, it seems to behave "like I want". Or is my example not good? (Sorry I'm really tired here, I can't even realize this :-D).

I mean, don't you think 'myeach as a CL macro would work?

> (mac ech args `(each _ ,@args))

Actually I'm currently using that. I'm just using the name 'each_ instead and define it using the macro currying stuff of twilightsentry (see id=10139, very interesting).

   arc> (= each_ (>_ each '_))
But still you know, one extra character ('_') to type... :-D

-----

2 points by CatDancer 5749 days ago | link

> But if it's a macro, it seems to behave "like I want". Or is my example not good?

Shall I wait until tomorrow to reply so that you can get some sleep? :)

Try

  ((lambda () (fm)))
you can see that only if (fm) is invoked directly from the top level will the macro expansion be avoided.

> I mean, don't you think 'myeach as a CL macro would work?

What you need, in CL or in Arc, is a way to tell if a variable is defined at macro expansion time. Suppose Arc could tell you at macro expansion time if a variable was defined. Then you could say

  (def myeach (first . args)
    (if (defined first)
      `(each _ ,first ,@args)
      `(each ,first ,@args)))
no magic needed in 'if, in either language :)

-----

1 point by palsecam 5749 days ago | link

> Shall I wait until tomorrow to reply so that you can get some sleep? :)

Lol, yes I definitely go to bed now :-)

> ((lambda () (fm))

Nice demonstration indeed.

> What you need, in CL or in Arc, is a way to tell if a variable is defined at macro expansion time.

Right. I may try to modify Arc in this purpose one of these days.

> no magic needed in 'if, in either language :)

Magic definitely sucks :-D!

Thanks a lot CatDancer!

-----

2 points by CatDancer 5750 days ago | link

> can we agree on that?

No, because I don't know what CL is doing.

-----

1 point by palsecam 5750 days ago | link

Well, if I dive deeply into CL one of these days, I'll tell you :-), but this not in my plans for the near future :-/

You are right to refuse "magic".

For Arc, oh this is way too much difficult for me, but if I had to try to solve the problem, I think I'd try to add a level of abstraction between the Scheme runtime and the Arc one (?!) Macros currently, as you know, directly run in the Scheme environment (am I all right?). I'd try to avoid that, and have more control over them by "compiling" them to a special form (`ar-macall') but not directly execute them (?!) And so 'if could then have a "higher priority" than macro expansion (?!)

But these are pure ideas in the wild. I am not able even to try, so don't listen this crap :-D

-----

2 points by CatDancer 5749 days ago | link

> try to solve the problem

which problem, the 'myeach problem or the 'if problem?

-----

1 point by palsecam 5749 days ago | link

Hi hi, this thread is definitely too big now :-D

The 'if/macroexpansion problem.

-----

2 points by CatDancer 5750 days ago | link | parent | on: Macro expansion/evaluation problem

Sounds like what you want is

  (mac myeach (first . rest)
    (if (alist first)
         `(each _ ,first ,@rest)
         `(each ,first ,@rest)))
which works:

  arc> (myeach '(1 2 3) (pr _))
  123nil
though it does defeat list destructuring... with "each", I can say:

  (each (k v) '((a 1) (b 2) (c 3)) (prn k ": " v))

-----

1 point by palsecam 5750 days ago | link

CatDancer, thanks for offering a good solution.

However, as you pointed it out, it does defeat list destructuring, which I would like to keep. Sorry that my words and CL/Arc examples doesn't mention it.

With my 'myeach, it works, because I don't test if first is alist, but if it is defined. If not, I consider it is the variable expression (e.g: v or (k v)) to use.

I.e, with my 'myeach:

    arc> (myeach (k v) (table [for i 1 3 (= _.i (* i i))]) (pr k ":" v " "))  ; would also work with an alist
    2:4 1:1 3:9
Of course, the shorter version I try to be able to use:

   (myeach (table [for i 1 3 (= _.i (* i i))]) (pr (car _) ":" (cadr _) " ")
doesn't work (currently) but it could (if the false branch of the 'if was not expanded. Comment the case for the non-anonymous 'each call in 'myeach, and it'll work).

Btw, 'defined is yet another dirty hack. Oh my god, I know there is a lot of dirt in my stuff, certainly too much. I was ashamed to talk about it but I checked Google and apparently some Lispers sometimes also use something like that, when doing exploratory macro programming:

   (mac defined args   ; typically, args is one or more symbols
    `(errsafe (do ,@args t)))  ; typically, a pedant functional programmer will have an heart attack seeing this
P.S: writing the examples of this post makes me even more hungry to be able to use '_' in more places than just for anonymous 1-param functions.

"If I had a nickel for every time I've written "for i 0 9 ..." in Arc I'd be a millionaire." (what I'd like is writing "for 0 9", the variable being '_' by default. In other words: oh Perl I so love you ;-))

-----

2 points by CatDancer 5750 days ago | link

> list destructuring, which I would like to keep

If I said

  (myeach a b c)
what would you want that to do? Would it set _ to each value in the list 'a and do 'b and 'c in the loop, or set 'a to each value in the list 'b and do 'c in the loop?

To put the question another way, when do you want 'myeach to use "_"?

Suppose it was the number of arguments... two arguments would mean to use "_". That's easy to do:

  (mac myeach args
    (if (is (len args) 2)
      `(each _ ,@args)
      `(each ,@args)))

-----

1 point by palsecam 5750 days ago | link

(myeach a b c): set '_' to each value of 'a, then do 'b and 'c in the loop. Or if 'a is undefined, set 'a to each value in 'b, and do 'c in the loop.

But if 'a is lexically bounded but is not the list/table to traverse, my 'myeach will not work as expected.

   (let v 2 (myeach v '(1 2 3) (prn v)))  
will call (each _ v '(1 2 3) c) because 'v is defined.

And this is a problem. What I try to do, call it "optional first arg" is dirty. I mean, terribly unhygienic ;-), imperfect. Using the kind of tricks I'm trying is subject to discussion, and have a lot of 'cons.

But this is not really the point here. The point is, 'if in Arc doesn't work like in Common Lisp. If this is a bug or just a difference is also subject to discussion, but at least know the fact. I mean it is not that natural that Arc acts like that here.

To come back to your questions. I want to use "_" if the first argument is defined. If it is (defined), it supposes it is the list/table to traverse, else it supposes it is the variable to be bind (call it "context intelligence").

Yes, the two arguments solution is easy to do, like is defeating destructuring, and both solutions could be used (I'll certainly adopt one of them). But 1. adds extra ()s and 2. removes a good feature.

These times I feel adventurous, so I tried another thing :-)

Anyway, great thanks for your help & suggestions.

-----

1 point by CatDancer 5750 days ago | link

How do you tell if 'a is defined? Are you using pg's Arc, or Anarki, or something else?

-----

1 point by palsecam 5750 days ago | link

I'm using this:

   (mac defined args   ; typically, args is one or more symbols
     `(errsafe (do ,@args t)))  ; typically, a pedant functional programmer will have an heart attack seeing this
which is, it's not that I love spamming this term but it must not be forgotten, terribly dirty. I mean using it indicates there is certainly a problem with your (functional) code. Plus here the implementation is tricky.

   arc> (defined a)
   nil
   arc> (let a 2 (defined a))
   t
   arc> (= b 3)
   3
   arc> (defined b)
   t
   arc> (defined (c d) e)
   nil
   arc> (defined 'e)
   t

-----

1 point by CatDancer 5750 days ago | link

At macro expansion time in Arc, there's no lexical scope yet, as Arc macros operate on the input program as lists. So if you say

  (let a 2 (myeach ...))
'myeach can check if 'a is defined using your function, but it will find out that 'a isn't defined, because the lexical scope for 'a isn't created until after the macro expansion is done.

I don't know about Common Lisp in particular, but there are other more powerful macro expansion languages that iirc can give you information such as whether a variable is in scope of not. So I think the issue that you're running into isn't in the behavior of macros and 'if, but that Arc isn't able to tell you at macro expansion time whether a variable is defined or not.

-----

1 point by palsecam 5750 days ago | link

CatDancer, thanks once again, you're full of advice!

> arc macros operate on the input program as lists

I agree.

> (let a 2 (myeach ...)) will see 'a undefined. [...] Lexical scope explanation [...]

I believe you know Arc internals way better than me, and I certainly miss some points in your explanation but it seems to work for me:

   arc> (mac myeach (first expr . body)  
          `(if (defined ,first)  
             (each _ ,first ,expr ,@body)
             (prn "undefined")))
   #3(tagged mac...)
   arc> (let a '(1 2 3) (myeach a (prn _)))
   1
   2
   3
   nil
   arc> (myeach a (prn _))
   undefined
   "undefined"
> More powerful macro systems [...] can give you information such as whether a variable is in scope of not.

Certainly yes. I don't know either.

> So I think the issue that you're running into isn't in the behavior of macros and 'if, but that Arc isn't able to tell you at macro expansion time whether a variable is defined or not.

Being (or not) able to tell me at macexpansion time if a variable is defined is part of what I'd call the behaviour of macros ;-)

But actually, 'defined was just a way I tried to deal with Arc macro expansion stuff, but it was just a consequence of my problems with 'if, not the beginning.

However yes, for the precise case of 'each (but again, this topic was not limited to it), maybe the 'if behaviour would not be the main issue anyway.

-----

2 points by CatDancer 5750 days ago | link

> it seems to work for me

  (mac myeach (first expr . body)  
          `(if (defined ,first)  
             (each _ ,first ,expr ,@body)
             (prn "undefined")))
ah, but now you're doing the 'defined test at run time. What you want is to be able to do the 'defined test at macro expansion time in order to affect how your macro is expanded, and that, as far as I know, Arc isn't able to do for you.

-----

1 point by palsecam 5750 days ago | link

> now you're doing the 'defined test at run time.

Yes, but check the previous messages, I've always wanted to do so. The definition of 'myeach basically never changed.

> What you want is to be able to do the 'defined test at macro expansion time in order to affect how your macro is expanded

Not necessary (maybe yes in the current Arc because of the 'if behaviour, but between making the language works for me [change its behaviour] or works for the language [change my behaviour], you'll guess what I prefer), and no, this is too precise. What I want is 'each to be smart and have an "anonymous form". Which tricks to use to get that, I don't care. I don't care of 'defined, this thing should certainly not exist anyway.

> and that, as far as I know, Arc isn't able to do for you.

You're certainly right here.

-----

2 points by CatDancer 5750 days ago | link | parent | on: Macro expansion/evaluation problem

I don't find this buggy.

Perhaps you want

  (mac amac () `(prn "amac is reached"))

-----


This looks like a shorter example...:

  arc> `(a . b)
  Error: "map: expects type <proper list> as 2nd argument, given: (a . b); other arguments were: #<procedure:/tmp/arc/ac.scm:241:14>"

-----

1 point by conanite 5755 days ago | link

I used slightly longer examples to show the difference when the list members were unquoted. I don't understand at all why it works, because as far as I can tell, the same code branch (that maps over the quasiquoted list) is invoked in both cases.

-----

5 points by CatDancer 5762 days ago | link | parent | on: Newbie arc app just for fun...

the numbers don't look random: http://www.random.org/faq/#Q2.7

-----

1 point by CatDancer 5762 days ago | link | parent | on: Arc vectors?

hmm, come to think of it, I've never used my table literal syntax {...} myself in a program. I wrote it so that I could write out and read back tables, which I use all the time, but that would work just as well with some long syntax like (literal-table k1 v1 k2 v2 ...) which doesn't use up precious syntax characters like { or #{.

-----


You could extend the compiler to save/restore information when a scope is introduced.

In your example though, at the end, 'a is set to 16, but you're reporting that its source is "(= a 42)". Is that what you want?

-----

1 point by shader 5766 days ago | link

whoops, good point. I was intending to make a a new local variable and redefine it using '= to a new value. In that case, the value of a would obviously be local, so redefining its source in the global table would obviously not make sense.

The problem with save/restore is that it isn't thread safe. Hmmm.

-----

2 points by CatDancer 5772 days ago | link | parent | on: How to upload files?

As far as I know, no one has written code for Arc yet to decode multipart/form-data.

I was too lazy to work on a multipart/form-data decoder myself... since I run a number of servers, it was easy enough for me to have the form posted to another one of my servers (Perl in my case) to handle the file upload, and then I have it redirect the user back to my Arc server to continue after the file upload.

-----

1 point by coconutrandom 5771 days ago | link

ok, that explains that. thank you

-----

1 point by CatDancer 5776 days ago | link | parent | on: New: inc, expanded range

Yes, this parallels my experience that it was complex and difficult to use a version control system with the independent hack model.

-----

1 point by rntz 5776 days ago | link

What other choice is there if I want to maintain independence of hacks? If I just use plain diffs, how do I update my hacks when a new arc3.tar comes out and breaks them? Manually? That's even more of a pain.

-----

1 point by CatDancer 5776 days ago | link

I have some ideas that appear to be promising, but no solution to offer you yet. For example, I look at one of my patches and say, "why do I need a patch? Why isn't Arc hackable enough to let me implement this by just loading some code?" and, if I can, see if I can make Arc more hackable instead. And then, if it works, my patches become much smaller, just extending Arc to become more hackable instead of implementing my whole hack, small enough so that often the patch will still apply in new versions of Arc. Now when a new version of Arc comes out I don't always have to come up with a new patch, instead what I'm doing is simply testing my patches to see if they still apply.

-----


I tried a port (http://hacks.catdancer.ws/plt4-port.html) but it's running 3.5 times slower... any suggestions?

-----

More