Arc Forumnew | comments | leaders | submitlogin
1 point by akkartik 4418 days ago | link | parent

"What is 'caller-scope bound to at the top level?"

  wart> caller-scope
  007scope.cc:122 No binding for caller-scope
:)


2 points by Pauan 4418 days ago | link

I'd rather it be bound to the global environment, but that's just me nitpicking.

-----

2 points by rocketnia 4418 days ago | link

That's already the current scope, though.

If I just saw 'caller-scope in pseudocode somewhere, I'd implement it as an anaphoric variable of 'fn (and I'd implement a non-anaphoric variant of 'fn to base it on, so we'd basically be at vau again :-p ). So at the global scope, my implementation would just treat 'current-scope as a global variable.

For a third option, maybe 'caller-scope at the top level should be the scope of eval's caller. I'm not sure the point of that, and it might make the language even harder to optimize, but I'm just throwing it out there.

-----

1 point by Pauan 4417 days ago | link

Yeah, but then you can say (eval ... caller-scope) at the top-level, and if your language has a facility for setting variables in a particular scope, you can say this:

  (env= caller-scope foo 5)
And you can say this:

  (let global caller-scope
    ...
    (env= global foo 5)
    ...
    (eval ... global))
Kernel already does this via the "get-current-environment" function:

  ($let ((global (get-current-environment)))
    ...
    ($set! global foo 5)
    ...
    (eval ... global))
Just to be clear: Kernel's "get-current-environment" is not the same as the environment argument passed to fexprs. But since wart is implementing the environment argument as a global hard-coded "caller-scope" variable, I think it makes sense to merge the two in wart, even though I think it's cleaner to have them be two separate things, as in Kernel.

-----

2 points by Pauan 4417 days ago | link

Which reminds me... rocketnia, you mentioned being able to define vau in terms of wart's magical fn. You're mostly right, except... there will be two ways to grab the environment: the env parameter and the implicit global caller-scope. So that's a bit of a leaky abstraction.

On the other hand, it's quite easy to define wart's magical fn in terms of vau... with no leaky abstractions! The only catch is that caller-scope will always refer to the global scope inside vau forms... that could be changed too, by extending/overwriting vau, but I didn't do that:

  (= get-current-environment (wrap (vau () e e)))

  ;; don't define this if you don't want caller-scope at the global level
  (= caller-scope (get-current-environment))
  
  ;; simple fn, without magical quoted args or caller-scope
  (= fn (vau (parms . body) env
          (wrap (eval `(,vau ,parms nil ,@body)))))

  ;; omitting other stuff like afn, no, caris, list, etc.
  ;; but they can all be defined in terms of simple fn
  ... 
          
  ;; makes these variables hygienic
  (with (with  with
         eval  eval
         let   let)
         
    ;; this recreates the part of wart that is currently handled in C++
    (= parse-fn (fn (parms env)
                  (if (caris x 'quote)
                      (list (cdr parms) nil)
                      ((afn (x vars parms)
                         (if (no x)
                               (list (rev parms)
                                     (rev vars))
                             (caris (car x) 'quote)
                               (self (cdr x)
                                     vars
                                     (cons (cdar x) parms))
                             (let c (car x)
                               (self (cdr x)
                                     `((,eval ,c ,env) ,c ,@vars)
                                     (cons c parms)))))
                       parms nil nil))))
    
    ;; overwrite with the complex fn which has quoted args and caller-scope
    (= fn (vau (parms . body) env
            (w/uniq new-env
              (let (parms vars) (parse-fn parms new-env)
                (eval `(,vau ,parms ,new-env
                         (,with ,vars
                           (,let caller-scope ,new-env
                             ,@body)))
                      env))))))
I have not run the above code, but it should give the general idea... Basically, what it does is it takes this...

  (fn (a 'b . c)
    ...)
...and converts it into this (where "g1" is a gensym and "with", "eval", and "let" are hygienic):

  (vau (a b . c) g1
    (with (a (eval a g1)
           c (eval c g1))
      (let caller-scope g1
        ...)))
I know the above code looks verbose, but keep in mind that quoted args and implicit caller-scope is currently handled in C++, whereas I recreated it from scratch using vau + simple functions (defined with wrap).

One more caveat: I'm not sure how you implemented caller-scope. If it's hardcoded into every function such that functions can't have an argument called "caller-scope" then the above will work the same way. But if it's a global implicit parameter like in Nu and ar, then you'll need to search the arg list for an argument called "caller-scope", which is doable but would require some more code, especially for handling argument destructuring.

---

Anyways, I'm not saying you should necessarily do it this way in wart, but... I think it's cleaner, conceptually, for functions to be defined in terms of vau, rather than vau being defined in terms of functions.

Of course, as I already said, if you care about cleanliness, I'd suggest just using Kernel, which doesn't support the magical fn like wart, which I consider to be a good thing.

-----

3 points by rocketnia 4417 days ago | link

"Which reminds me... rocketnia, you mentioned being able to define vau in terms of wart's magical fn. You're mostly right, except... there will be two ways to grab the environment: the env parameter and the implicit global caller-scope. So that's a bit of a leaky abstraction."

Hmm... yeah, it's a pretty leaky abstraction. It's one of these features with ambient authority:

  (if designed like a global variable assigned at the beginning of each
  call)
  
  Right now, get the most recently started call's caller's environment.
  
  
  (if designed like a continuation mark)
  
  Right now, get the deepest-on-the-stack call's caller's environment.
  
  
  (if designed like a local variable bound by each (fn ...))
  
  Given a non-global lexical environment, its local variables must have
  been initially bound by some call, so get the lexical environment
  that was used to evaluate that call.
---change of topic--

...Actually, the first two of these designs are broken! Say I define a simple fexpr like this one:

  (def idfn '(result)
    (eval result caller-scope))
Now if I run the code ((fn () (idfn caller-scope))), the call stack at the evaluation of 'caller-scope looks something like this:

  ((fn (a) ((fn () (idfn caller-scope)))) 1)  ; eval'd in global scope
  ((fn () (idfn caller-scope)))  ; eval'd in local scope of (fn (a) ...)
  (idfn caller-scope)            ; eval'd in local scope of (fn () ...)
  (eval result caller-scope)     ; eval'd in local scope of idfn
  caller-scope                   ; eval'd in local scope of (fn () ...)
IMO, the value we get should be the local scope of (fn (a) ...), so that it doesn't matter if we replace ((fn () (idfn caller-scope))) with ((fn () caller-scope)). However, the calls to 'idfn and 'eval are deeper on the stack and their start times are more recent, so we get one of those caller scopes instead.

Does Wart have this awkward behavior, akkartik?

---/change of topic---

To get back to the point, the local-variable-style version of the feature (the only version that works?) doesn't strike me as completely undesirable. It lets the programmer evaluate expressions in arbitrary ancestor stack frames, and that isn't a shocking feature to put in a debugger.

Sure, it makes it almost impossible to hide implementation details and keep privilege from leaking to all areas of the program, but it could be useful in a language where all code that runs in a single instance of the language runtime has been personally scrutinized (or even written) by a single programmer. That's the primary use case akkartik has in mind anyway.

-----

1 point by akkartik 4417 days ago | link

the value we get should be the local scope of (fn (a) ...)

Wart returns the scope with a set to 1. Is that the one you mean? It's different from the value of:

  ((fn (a) caller-scope) 1)

-----

1 point by rocketnia 4417 days ago | link

That's all what I was hoping for. ^_^

I'm impressed you're managing to keep track of that with mutation. Are you setting 'caller-scope on every entry and exit of a function and on every entry and exit of an expression passed to 'eval?

-----

1 point by akkartik 4417 days ago | link

caller-scope acts like an implicit param of all functions:

http://github.com/akkartik/wart/blob/98685057cd/008eval.cc#L...

Perhaps you're concerned about something I haven't even thought of :)

-----

1 point by rocketnia 4416 days ago | link

Oh, Pauan had mentioned 'caller-scope being a "global hard-coded" variable, and I didn't see you disagreeing, so I assumed you were just assigning to it a lot behind the scenes. :-p

-----

2 points by akkartik 4417 days ago | link

It's likely that as I reflect on vau, wart will start to look more like Kernel.

But I don't think it will become Kernel. Kernel seems really elegant as long as you give up quote and quasiquote. I suspect if you bolt them onto Kernel it'll have slid from its sweet-spot peak of elegance. A language that encourages quoting and macros will diverge significantly from Kernel.

One simplistic way to put it: Kernel is scheme with fexprs done right, while wart tries to be lisp with fexprs done right. It's far from that local extremum, though. It's even possible the wisdom of caring about hygiene in a lisp-1 is deeper than I realize. Perhaps quasiquote needs to go the way of dynamic scope.

-----

1 point by akkartik 4417 days ago | link

"I'm not sure how you implemented caller-scope. If it's hardcoded into every function such that functions can't have an argument called "caller-scope" then the above will work the same way."

http://github.com/akkartik/wart/blob/98685057cd/008eval.cc#L...

It tries to bind caller-scope after the params, so this fails:

  wart> def foo(caller-scope) 34
  wart> foo 3
  007scope.cc:77 Can't rebind within a lexical scope
  dying
Anyways, I haven't thought about these corner cases at the moment.

-----

2 points by rocketnia 4417 days ago | link

If you want 'caller-scope to be built into the language and you want to skirt the issue of name collisions, you could devote a separate reader syntax like "#caller-scope" to it and make it a type of its own.

-----

1 point by akkartik 4417 days ago | link

"..if your language has a facility for setting variables in a particular scope.."

In wart scopes are just tables. It's not a good idea to modify them, but nobody will stop ya :)

"Kernel's "get-current-environment" is not the same as the environment argument passed to fexprs."

Can you elaborate? In

  ($vau foo env
    ..)
(get-current-environment) would return the callee scope, and env contains the caller scope, right?

Wart doesn't currently have an easy way to get at the callee scope (you could peek inside (globals)), but it's on my radar to change that if necessary.

-----

1 point by Pauan 4417 days ago | link

  ($let ((env1 (get-current-environment)))
    ...
    ($vau foo env2
      ...
      ($let ((env3 (get-current-environment)))
        ...))
In the above, env1 is the lexical scope where the $vau is defined. It does not include bindings for foo, env1, env2, or env3, but contains all the bindings on the outside of the $let, where the $vau is defined.

env2 is the dynamic scope where the $vau is called, it does not include bindings for foo, env1, env2, or env3.

env3 is the environment of the $vau itself. This environment inherits from env1, and thus is lexical just as env1 is. And so, it contains all the bindings of env1, in addition to bindings for foo, env1, and env2, but not env3.

So, by using this combination of lexical and dynamic scope, you can express a huge variety of different things in Kernel.

---

Let me explain via a more elaborate example:

  ($let ((global (get-current-environment)))
    (display (eval 'x global))
    
    ($set! global $foo
      ($vau (x) env
        (display (eval 'x env))
        (display (eval 'env env))
        
        ($let ((inner (get-current-environment)))
          (display (eval 'x inner))
          (display (eval 'env inner))
          (display (eval 'global inner))))))
          
  ($let ((x 10))
    ($foo 20))
First, we use $let to bind the variable 'global to the global environment using get-current-environment. Then we try to evaluate the symbol 'x in that environment. The variable 'x of course does not exist in the global environment, so it throws an error.

Then, we create a $vau form and assign it to the variable $foo in the global environment. This is the same as saying ($define! foo ...) at the top level, but is necessary when inside a $let.

Now, at the bottom, we use $let to bind the variable 'x to 10 and then call $foo with the single argument 20. First, $foo tries to evaluate the variable 'x in the dynamic environment "env". This should display 10, because it's using the binding in the $let where $foo is called.

We then evaluate the variable 'env in the $vau's dynamic environment. But the variable 'env does not exist in the dynamic environment, so it throws an error.

Then, we use a $let inside the $vau to grab a hold of the $vau's inner environment and bind it to the variable 'inner. We then evaluate the variable 'x in the $vau's inner environment. This should display 20, because it's using the binding inside the $vau, which is bound to the $vau's argument 'x which was 20 when calling the $vau.

We then evaluate the variable 'env in the $vau's inner environment, which evaluates to the dynamic environment, because inside the $vau, the variable 'env is bound to the dynamic environment.

Lastly, we evaluate the variable 'global in the $vau's inner environment. This evaluates to the global environment because the $vau's inner environment inherits from the top-level $let where the variable 'global is bound.

--

Here is a picture showing the environments in the above code:

http://img42.imageshack.us/img42/1870/vauenvironments.png

The global environment is pink, the first blue environment is the $let's environment that inherits from the global environment, plus a binding for the variable 'global. The green is the $vau's environment which inherits from the $let's environment, plus a binding for the variables 'x and 'env. And the yellow is the $let's environment that inherits from the $vau's environment.

Lastly, the second blue environment is the dynamic environment that is bound to the 'env variable inside the $vau when it is called.

-----

1 point by akkartik 4417 days ago | link

Thanks. So what Kernel calls the dynamic environment is identical to caller-scope, right?

I think I find this confusing because when I hear 'dynamic environment' I keep thinking of dynamic scope (special variables in common lisp). If we only create bindings using let and combiner parameters, the dynamic environment is just the caller's static environment, is that right?

-----

1 point by Pauan 4417 days ago | link

Yes, it is identical to caller-scope.

In Kernel, the dynamic environment is exactly analogous to dynamic scope. The difference is that lexical scope is the default, and you have to explicitly ask to evaluate things with dynamic scope, rather than earlier Lisps which used dynamic scope by default and didn't even have lexical scope.

Special variables in Common Lisp are similar to dynamic scope, except you're flagging only certain variables to be dynamic, whereas the dynamic environment in Kernel lets you evaluate any variable with dynamic scope.

---

"If we only create bindings using let and combiner parameters, the dynamic environment is just the caller's static environment, is that right?"

Yes. In fact, in Kernel, $let is defined just like it is in Arc, using $lambda. And $lambda uses $vau, and $vau is the only way to create new environments in Kernel, as far as I know. So, environments are all about $vau.

---

By the way, I just edited the post you're replying to, and added a picture.

-----

2 points by Pauan 4417 days ago | link

In fact, I like to think of environments in a similar way as continuations. Consider this, for instance:

  ($define! $foo
    ($vau (x) env
      (list x env)))
      
  ($define! $bar
    ($vau (x) env
      ($foo x)))
      
  ($bar 10)
The above should return a list containing two elements: the number 10 and $bar's environment. The way I think of it is like this:

  ($define! $bar
    ($vau (x) env
      ($foo x env)))
That is, the $bar fexpr implicitly passes its own internal environment to the $foo fexpr. And $let's work this way as well, so that this:

  ($let ((x 10))
    ($bar x))
Would be equivalent to this:

  ((wrap ($vau (x) env
           ($bar x env)))
   10)
So it's really all about $vau's passing their own internal environment whenever they call something else, similar to the idea of all functions implicitly passing their own continuation when calling something else.

-----

1 point by akkartik 4417 days ago | link

Very cool. While struggling with the $vau concept I thought about whether I could build lazy eval directly using $vau, without first building force and delay (SICP 4.2). But it's not quite that powerful, because lazy eval also needs the ability to stop computing the cdr of a value. Your intuition tying $vau to continuations is a similar you-see-it-if-you-squint-a-little correspondence.

-----

1 point by Pauan 4417 days ago | link

I somewhat remember either the Kernel Report or John's dissertation mentioning the similarities between continuations and environments, but I might be mistaken on that...

---

"I thought about whether I could build lazy eval directly using $vau"

Directly? Not sure, but ยง9.1.3 of the Kernel Report does show how to build promise?, memoize, $lazy, and force as a library using existing Kernel features.

-----

1 point by akkartik 4417 days ago | link

"Special variables in Common Lisp are similar to dynamic scope, except you're flagging only certain variables to be dynamic, whereas the dynamic environment in Kernel lets you evaluate any variable with dynamic scope."

Ah! I considered this approach when I was building dynamic variables in wart, but it seemed less confusing to attach the scope to the binding. Wart is unlike common lisp in this respect. SBCL:

  * (defvar foo 34)
  * (defun bar() (foo))
  * (bar)
  34
  * (let ((foo 33)) (bar))
  33 ; foo is still dynamic. WEIRD.
But:

  wart> = foo 34
  wart> def bar() foo.
  wart> bar.
  34
  wart> let foo 33 bar.
  34 
  wart> making foo 33 bar.
  33 ; dynamic scope
I'm starting to appreciate that $vau and first-class environments give the programmer a lot more power (aka rope to hang himself with :) than caller-scope.

-----

1 point by Pauan 4417 days ago | link

"foo is still dynamic. WEIRD."

The reason for this is that Common Lisp explicitly states that if you use "let" on a dynamic variable, it dynamically changes that variable for the scope of the "let". So Common Lisp combines both "let" and Racket's "parameterize" into a single form, rather than keeping them separate.

-----

2 points by akkartik 4417 days ago | link

Yes, of course. I prefer racket's approach in this case.

-----

1 point by akkartik 4417 days ago | link

I'm struggling to wrap my head around the idea that you can define variables to have dynamic scope using just $vau or caller-scope..

-----

1 point by Pauan 4417 days ago | link

You don't define variables to have dynamic scope. They automatically have dynamic scope when you evaluate them in the dynamic environment:

  ($vau () env
    (eval 'x env))
The above, when called, will evaluate the variable 'x in whatever scope it was called in. Thus, the above $vau has just "made" the variable 'x dynamic. But of course this ability to evaluate things in the dynamic scope is only available to fexprs.

---

If you're talking about something similar to Racket's parameterize, you should be able to build that using dynamic-wind. Then something like this...

  ($parameterize ((foo 10))
    ...)
...would be equivalent to this:

  ($let ((env   (get-current-environment))
         (orig  foo))
    (dynamic-wind ($lambda ()
                    ($set! env foo 10))
                  ($lambda ()
                    ...)
                  ($lambda ()
                    ($set! env foo orig))))
---

In fact, I'm gonna write a $parameterize form right now:

  ($define! $parameterize-redirect
    ($vau (target tree . body) env
      ($let ((inner (get-current-environment))
             (env   (eval target env)))
        (dynamic-wind
          ($lambda ()
            ($set! inner orig
              (map ($lambda ((var expr))
                     (list var (eval var env)))
                   tree))
            (for-each ($lambda ((var expr))
                        (eval (list $define! var expr) env))
                      tree))
          ($lambda ()
            (eval (cons $sequence body) env))
          ($lambda ()
            (for-each ($lambda ((var expr))
                        (eval (list $define! var expr) env))
                      orig))))))
                      
  ($define! $parameterize
    ($vau (tree . body) env
      (eval (list* $parameterize-redirect env tree body) env)))
Warning: untested and may not work. However, if it does work then it will work for all variables, global or local, without declaring them as dynamic before-hand. So this will work:

  ($let ((x 5))
    ... x is 5 ...

    ($parameterize ((x 10))
      ... x is 10 in here ...
      )

    ... x is 5 again ...
    )
The catch is that in order to parameterize global variables, the $parameterize form itself needs to be at the top level, because it's mutating the immediate ancestor environment.

If you want to parameterize something other than the immediate environment, use $parameterize-redirect which accepts an environment as the first argument, and then evaluates in that environment instead.

-----

2 points by rocketnia 4417 days ago | link

"Wart doesn't currently have an easy way to get at the callee scope"

What about this?

  (def get-current-environment ()
    caller-scope)

-----

1 point by akkartik 4417 days ago | link

Lol! Yes, that works :)

-----

3 points by rocketnia 4416 days ago | link

If you define (thunk ...) ==> (fn () ...), then using 'get-current-environment is actually more verbose than writing the implementation directly:

  get-current-environment.
  thunk.caller-scope.
Actually, 'thunk and infix syntax isn't necessary, and Kernel has the same quirk:

  (get-current-environment)
  ((fn () caller-scope))     ; Wart
  (($vau () e e))            ; Kernel

-----

1 point by akkartik 4418 days ago | link

Good point!

-----