Arc Forumnew | comments | leaders | submit | Pauan's commentslogin
2 points by Pauan 53 days ago | link | parent | on: Equality in Pyret

I really like the names "equal-now" and "equal-always".

The "equal-always" operator is the same as "egal"[1], and "equal-now" is the same as "iso" in Arc.

If I ever add an "equal-now" operator to Nulan, now I know what to call it!


There's also various "-now" methods for mutable data types[2].

And there's an EqualityResult type, which reminds me of NULL in SQL[3].


* [1]:

* [2]:

* [3]:


That's really cool! I'm a fan of all three of those things (Clojure syntax, ML types, and Rust memory management).

Although it's intended to be low level, one concern I have is that it's missing some kind of extension system (such as typeclasses, first class ML modules, Rust traits, etc.)

Even static function overloading (like in C++) could work.


That sounds super strange. From what I understand, Racket automatically caches modules, so they are only loaded once.

But then again, Racket does have multiple phases, so it's possible that it's loading the "compiler" file in two different phases, causing duplicate variable definitions.

In addition, Racket doesn't allow for mutually recursive modules, but Arc/Nu is using "namespace-require", which apparently bypasses that restriction. It's possible that there is a race condition that is causing the "arc/3.1/main" file to be loaded before the "compiler" file is finished loading, which then triggers a second load of the "compiler" file.

Just to clarify: have you tried inserting a "displayln" in the "compiler" file to verify that it is being loaded twice?


4 points by rocketnia 72 days ago | link

That is exactly how I determined it was being loaded twice.

I was meaning to try on other versions of Racket but I've gotten distracted.


3 points by Pauan 65 days ago | link

Okay, I just tried upgrading to Racket 6.5, but it still works correctly for me.

I wonder if it's a bug with the Windows version of Racket?


Hey, I'm the author of Arc/Nu.

Thanks for letting me know about this.

In my opinion, returning t/nil makes more sense than returning the file path, but Arc/Nu is supposed to be backwards compatible with Arc 3.1, so I fixed this. I also fixed "dir-exists", which has the same issue.

I just pushed out the fix to the Arc/Nu GitHub repository, please try it out and let me know if it solves the problem for you.


4 points by Oscar-Belletti 71 days ago | link

I've just tried running the news site with the new version of Arc/Nu and it runs fine.


3 points by Pauan 71 days ago | link

Thanks for letting me know, I'm glad it's working now.


2 points by Pauan 159 days ago | link | parent | on: An alternative worldview to 'modularity'

I would like to point out that type systems (including dynamic and static type systems) are another way to carve out the territory.

Especially in languages like Haskell, static types are used all over the place to define intentions and boundaries. Languages like Idris and Agda go even further, with dependent types and proof systems.

Unit tests are great, but there are many useful properties that unit tests cannot test, but static type systems can test. And vice versa: there are things that unit tests can test that static types cannot.

Also, things like QuickCheck[1] allow us to encode our intentions in very concise ways (much more concise than traditional unit testing).

Rather than saying this:

  (reverse (reverse [])) is the same as []
  (reverse (reverse [1])) is the same as [1]
  (reverse (reverse [1 2])) is the same as [1 2]
  (reverse (reverse [1 2 3])) is the same as [1 2 3]
  (reverse (reverse [3 2 1])) is the same as [3 2 1]
  (reverse (reverse [1 2 3 4 5])) is the same as [1 2 3 4 5]
You instead say this:

  for every list, (reverse (reverse list)) is the same as list
And it will then generate the unit tests for you.

So, in general I agree with you. I just disagree with the idea that unit testing is the only way we have to specify our intentions. There are many ways, and we should use every useful tool which is available to us.

I think that the more we can specify our intentions (in a testable way), the better our programs will be.


* [1]:


2 points by akkartik 159 days ago | link

Yes, you're right. I'm not very familiar with Haskell, but I was thinking about it when I called the title "An alternative.."

I think of Haskell as being to tests what up-front design is to Agile. It's great if you're trying to carve out right and wrong behaviors along the dimensions it's been designed to help with (and my vague sense is that linear types and other such advances are trying to expand that set of dimensions). But how would you ensure that your program meets its performance guarantees? Or is guaranteed not to block between these two memory barriers? Tests are less rigorous and more work, but if you're willing to put in the elbow grease they'll work for anything you throw at them.


2 points by Pauan 159 days ago | link

That sounds about right. Static types can only guarantee a certain subset of behavior. Unit tests work for a huge amount of behavior, because they run at run-time. They are very versatile. But there are still some things that unit tests cannot test.

In particular, unit tests cannot guarantee the absence of behavior. As an example, Haskell can use its static type system to say, "this function (and any functions it uses) is guaranteed to be pure". Or you can say, "this function (and any functions it uses) is guaranteed to only use certain side effects, but other side effects do not happen".

Languages like Idris go even further and can guarantee that a function terminates (never goes into an infinite loop)!

As another example, Haskell has Software Transactional Memory, and it uses the static type system to guarantee that arbitrary I/O cannot occur, which is important because the transaction might be retried multiple times.

I don't remember which language it was in, but I remember reading about a language that used static types to guarantee that exceptions are always handled. In other words, you will never have an uncaught exception.

Nulan uses its static type system to make a distinction between blocking impure functions and non-blocking impure functions. In JavaScript, there's a big difference between blocking and non-blocking, both in terms of behavior and performance. By using the static type system, it's possible to say "this function (and any functions it uses) is always blocking, and never non-blocking".

As you are aware, Rust uses its static type system to guarantee that pointers are always used correctly, and that data races cannot occur, even with concurrency.

But the biggest reason I like static types is because they make refactoring easier. Let's say you have a type that represents a person. So you have a name property, age property, etc.

Now let's say you want to add in a new property. Some of the existing functions will need to change to accommodate this new property. Without static types, you need to carefully go through your entire code base and find the functions which use the type, and then change their behavior to work with the new property.

Unit tests won't help you with that, because the unit tests are all testing the old properties, not the new property. So in the end, you have no choice but to just bite the bullet and spend a significant amount of time searching through the code and making the changes.

But with static types, it's as simple as adding the new property, compiling, fixing any type errors, compiling, fixing any type errors, etc.

Each type error tells you the file and line where the error occurs, making it easy to fix. You don't need to search through your code to find the places which need fixing: the type system does that for you. And after all the type errors are fixed, it's basically guaranteed that your code is correct.

Another example: a function used to always return a result, but now you need to change it so that it can potentially fail. So anybody that uses that function now has to account for the possibility of failure. With a static type system, you simply make the change and fix the type errors.

With unit tests, you better hope that you have 100% code coverage, because you're going to need it to find all of the functions which need to be changed. And don't forget that the unit tests need to be updated, and you probably need to add in new tests as well. That takes time, in addition to the time spent fixing the code.

So the end result is that it's faster and easier to refactor with a static type system, and it's more likely to be correct as well.

So I think static types and unit tests complement each other: they can both do things that the other cannot. Static types can make guarantees that unit tests cannot, and static types also remove some of the burden from unit tests, which speeds up development without sacrificing correctness.

On the other hand, unit tests can make certain guarantees that static types cannot. So in the end, I think you need both.

P.S. Static typing also helps tremendously with things like smart IDEs. I think Unison is a really cool example of that:


2 points by Pauan 159 days ago | link

Oh, and I completely forgot about typeclasses.

Typeclasses are amazing. They're similar to multimethods, but more powerful, and they have the same performance as regular functions.

You can use them to solve the expression problem, giving you a huge amount of flexibility, power, and conciseness. Or you can use them to create dynamically scoped variables which actually work correctly (unlike dynamic variables in most languages).

The thing is, typeclasses are only possible with static types. They're the main reason I changed Nulan to be statically typed.


By the way, I would like to point out that the static type systems in languages like Haskell is very different from the static type systems in other languages like C, C++, or Java. So even if you like/dislike one of them, you might end up liking the other. I used to lump all of the static type systems together, but I think that's a mistake.

There are bugs which are not caught by C, C++, or Java, but which are caught by Haskell. And there are programs which do not type check in C, C++, or Java, but which do type check in Haskell.


2 points by akkartik 158 days ago | link

Yes, I wanted to say something similar in response to your phrasing that "type systems (including dynamic and static type systems) are another way to carve out the territory". When you say "dynamic type systems", in particular, I think Python and Ruby, and the modularity guarantees there seem much inferior to tests. But Haskell and the more advanced type systems might well disprove my thesis.


2 points by Pauan 158 days ago | link

You're right: dynamic type systems tend to provide very weak guarantees. But I'll still take those guarantees over silent failures.

In my opinion, static guarantees (including static types) are the best, followed by unit tests, followed by dynamic guarantees (including dynamic types).


2 points by akkartik 158 days ago | link

Yes, that's exactly the trade-off I've been thinking of. When I program I like to start out writing tests at the start when I don't yet know what properties I should try to enforce. Over time I start finding places where tests aren't good enough, and start feeling the need for more rigorous checking. But by then I've painted myself into a corner with my choice of language, and it's too hard to switch to a more rigorous platform.

In Mu my plan to break this dichotomy between coverage and soundness is to allow arbitrary metadata to be attached to instruction operands. As a result you aren't stuck with the type annotations and checking that I build into the system. You can have arbitrary metadata and will be able to write programs that can deduce and check this metadata to enforce all sorts of properties. Arbitrary checkers feels to me like a generalization of Lisp macros.


2 points by Pauan 158 days ago | link

Arbitrary metadata is cool. Clojure has arbitrary metadata, which the compiler uses to optimize function performance, but you can use it for other purposes as well:


1 point by akkartik 158 days ago | link

I did not know that!


4 points by Pauan 159 days ago | link | parent | on: ASK: Axioms of Arc?

I took a look through ac.scm, and here are the axioms I found:

* These built-in types: cons sym fn char string int num table output input socket exception thread

* Function calls

* First class lexically scoped functions, including closures (this one has a very complex implementation)

* Required function arguments, optional function arguments, rest function arguments, list destructuring

* Tail call elimination

* Macros (which are just functions annotated with a type of 'mac)

* First-class undelimited continuations (this one has a very complex implementation)

* These special forms: quote quasiquote if fn assign

* Special compiler optimizations for these functions: compose complement andf

* Ssyntax ~ : & (which expands to complement, compose, andf)

* Ssyntax . ! (which expands to function call, function call + quote)

* [ _ ] partial application

* Racket S-expression syntax () [] . ' ` , @ " ; #\ #| |#

* If enabled, you can use atstrings like "foo@bar" which is equivalent to (+ "foo" bar)

* These built-in functions: is < > + - * / mod expt sqrt apply cons car cdr err len annotate type rep uniq ccc infile outfile instring outstring inside stdout stdin stderr call-w/stdin call-w/stdout readc readb peekc writec writeb write disp sread coerce open-socket socket-accept setuid new-thread kill-thread break-thread current-thread sleep pipe-from table maptable protect rand dir file-exists dir-exists rmfile mvfile macex macex1 eval on-err details scar scdr sref bound newstring trunc exact msec current-process-milliseconds current-gc-milliseconds seconds client-ip atomic-invoke dead flushout ssyntax ssexpand quit close force-close memory declare timedate sin cos tan asin acos atan log

* These built-in variables: sig nil t

* Threads and atomics

* Sockets and file I/O

* Exception handling (err and on-error)

I might have missed a few things, but that covers most of it.


1 point by kinnard 159 days ago | link

Very thorough. That's 19 though, and some of them don't seem very axiomatic.


2 points by Pauan 159 days ago | link

It's far more than 19.

13 types + 5 special forms + 1 global variable + 92 built-in functions.

And that's not including things such as first-class functions, lexical scope, first-class continuations, tail call elimination, or macros. I don't even know how to count those.

The fact is that practical programming languages need a lot of axioms. If you look at the list of built-in functions, they are all useful, and most of them are for I/O, which cannot be written in an axiomatic way.

I think it would be interesting to write an "eval" function in Arc, and see how few functions it needs in order to accomplish it. But I don't see much practical benefit to it.


2 points by kinnard 158 days ago | link

I thought that was the whole thing that pg was arguing/exploring. 'What's the least number of axioms necessary for a practical programming language?' And that arc was the product of that exercise.

"Of course, as soon as McCarthy's spec fell into the hands of hackers, all this theorizing was cut short. In Lisp 1.5, read and print were not written in Lisp. Given the hardware available at the time, there is no way they could have been. But things are different now. With present-day hardware you can continue till you have a runnable spec for a complete programming language. So that's what I've been doing.

The question I'm trying to answer at the moment is, what operators do you have to add to the original seven in order to be able to write an eval for a complete programming language? I'm not finished yet with this exercise, but so far I've been surprised by how few primitives you need to add to the core in order to make these things work. I think all you need to define new types is three new primitives (plus assignment and lexical scope). One of the new primitives replaces the original atom, so you still only end up with nine total."


3 points by Pauan 158 days ago | link

There are two questions here:

1) What is the least number of axioms needed for a practical language?

2) What is the least number of axioms needed to write an evaluator for the language in the language itself?

As I demonstrated, you need a lot of axioms to support practical programming, because practical programming involves I/O, threads, sockets, exceptions, etc.

Trying to find the smallest axioms necessary for I/O is a cool idea. But any I/O axioms will be intimately tied to the hardware, and the hardware is currently more C based than Lisp based. So the result won't be very elegant.

If you ignore practical programming and I/O, and only care about mathematical elegance, then McCarthy's original Lisp is already a quite good answer for question number 2.

Arc is quite a bit more elegant than most other programming languages, but at the end of the day it is still a practical language.

So my question to you is: what are you looking for?


2 points by kinnard 158 days ago | link

I'm wondering first, exactly what axioms pg settled on. And I'm also curious as you've described it, if the hardware were built for the language and rather than the language being built for the hardware, "what is the least number of axioms need for a practical language".


3 points by Pauan 158 days ago | link

I can't really answer that. Somebody would need to write an "eval" function in Arc. That would give you a pretty good starting point for figuring out how many axioms you need.


With the current implementation of Arc, it is possible but rather tricky to write hygienic macros.

However, there is a very simple change that can be made to the Arc compiler which would make hygienic macros easy to write.

The Arc compiler would have a big hash table which maps from global symbols to gensyms.

Whenever you define a global variable, it will create a new gensym and store it in the hash table.

When using the quasiquote syntax, it will lookup the variable in the hash table and will use the gensym rather than the symbol.

As an example, if you have this Arc program:

  (= foo 1)

  (mac bar ()
    `(+ foo 2))
The Arc compiler would expand it to this code instead:

  (= #<foo:1> 1)

  (mac #<bar:1> ()
    `(#<+:1> #<foo:1> 2))
In this case, #<foo:1>, #<+:1>, and #<bar:1> are gensyms, rather than ordinary symbols.

Now the macro bar will always expand to the correct global variable foo, and therefore it is 100% hygienic.

If you want to intentionally break hygiene, you can simply do this:

  (mac bar ()
    `(+ ,'foo 2))
The above code uses "," to insert the symbol "foo", rather than the gensym for foo.

This change is very easy to make in the Arc compiler, and it only requires a few minor changes to the language.


3 points by Pauan 191 days ago | link | parent | on: ASK: is arc used in production?

> static typing seems to preclude true macros

I may be wrong on this, but it seems to me that static typing does not prevent macros at all (true or otherwise).

I switched Nulan to use static typing, yet it's using the same kind of macro system that it used when it was dynamically typed.

As far as I can tell, a macro is simply a compile-time function that accepts code and returns code. If so, then its static type is "Code -> Code".

In Nulan, the Code type might be defined like this:

  (TYPE Code
  | (*integer Integer)
  | (*number Number)
  | (*string String)
  | (*symbol String)
  | (*gensym Integer)
  | (*list (List Code)))
In other words, it can be an integer, number, string, symbol, gensym, or list of Code.

Within the macro's body, you can pattern match on the Code, you can map/filter on it just like in Arc, you can dynamically return Code, etc.

In Nulan, this is made easy with the & syntax, which is just a shorthand for the Code type:

  # These two are equivalent

  (*integer 1)

  # These two are equivalent
  &(foo 1 2)

  (*list [ (*symbol "foo") (*integer 1) (*integer 2) ])

  # These two are equivalent
  &(foo ~a ~b)

  (*list [ (*symbol "foo") a b ])
And the & syntax works when pattern matching as well:

  # These two are equivalent
  (MATCH foo
  | &(foo 1 2)

  (MATCH foo
  | (*list [ (*symbol "foo") (*integer 1) (*integer 2) ])

  # These two are equivalent
  (MATCH foo
  | &(foo ~a ~@b)

  (MATCH foo
  | (*list [ (*symbol "foo") a @b ])
As far as I can tell, this allows Nulan macros to do everything that Arc macros can do, even with a static type system.


3 points by rocketnia 191 days ago | link

Yeah! There's a certain way that it's really clear that macros and statically typed code can work together; just run the macros before interpreting the static types. Thanks for demonstrating that. I do think this can do everything Arc can do, like you say.

There are a couple of ways I think that can get more complicated:

In languages that resolve overloaded names in a type-directed way, I think the reconciling of static types and macros gets quite a bit more challenging due to the underlying challenge of reconciling name resolution with macros.

For instance, I think Idris has different meanings for the name (,) depending on if it occurs as a value of type (Type -> Type -> Type) or a value of type (a -> b -> (a, b)). In one case it's a constructor of tuple types (a, b), and in another case it's a constructor of tuple values (a, b).

Idris's macro system actually seems to have some special support for invoking the typechecker explicitly from macro code, which I think helps resolve names like those. I haven't actually succeeded in writing macros in Idris yet, and it seems to be a complicated system, so I'm not sure about what I'm saying.

Secondly, one potential goal of a macro system is that all the language's existing primitive syntaxes can turn out to be macros. That way, future versions of the language don't have to inherit all the crufty syntaxes of the versions before; they can stuff those into an optional library. If the language has sophisticated compile-time systems for type inference, term elaboration, name resolution, syntax highlighting, autocompletion, etc., then hopefully the macro system is expressive enough that the built-in syntaxes are indistinguishable from well-written macros.

Arc already gives us things like (macex ...) that can distinguish macros from built-in special forms, so maybe seamlessness isn't a priority in Arc. But if it were, and Arc had static types, we would probably want Arc to have type inference as well, which could complicate the macro system.

A lot depends on what Paul Graham expects from "true macros."


3 points by Pauan 191 days ago | link

I don't think name overloading is a problem, at least not in Nulan.

Macros deal with things at the syntax level, far earlier than any kind of type overloading.

As far as the macro is concerned, it simply sees the syntax

  (*list [ (*symbol ",") (*integer 1) (*integer 2) ])
It doesn't care what the type is, or the meaning, or anything like that. In some cases, the macro doesn't even know whether the symbol is bound or not!

Basically, I view type inference/checking/overloading occurring in a phase after macro expansion.

Is there any benefit to mixing the type overloading phase and the macro expansion phase?


It occurs to me that you might not be referring to a macro which expands to syntax, but instead using the syntax within the macro body itself.

In Nulan, that isn't a problem either. A macro is essentially a function from Code to Code, so the macro body is compiled/macro expanded/type checked just as if it were a function.

Thus any type overloading will happen inside the macro body, before the macro is ever run.

That does mean that the type checker must be able to overload based solely on the types found within the macro body. In other words, it cannot dynamically dispatch.


I don't think it's possible for the primitives to be macros, for the simple reason that the language needs axioms.

If you mean that the primitives should appear to be macros (even though they're implemented with compiler magic), then I agree with you.


3 points by rocketnia 190 days ago | link

"Macros deal with things at the syntax level, far earlier than any kind of type overloading."

If the macro itself is being referred to by an overloaded name, then the name needs to be resolved before we know which macro to call.

(I'm not sure if macros' names can be overloaded in Idris; I'm just guessing. Actually, I don't know if Idris lets users define overloaded names. I just know a few built-in names like (,) are overloaded.)


"If you mean that the primitives should appear to be macros (even though they're implemented with compiler magic), then I agree with you."

Yeah, that's what I mean. :)


2 points by Pauan 190 days ago | link

You're right, if you allow for users to overload multiple macros onto the same name based upon type, then it gets very messy.

But I think if your language allows for that, it's inconsistent with the way that macros work.

If you want powerful Arc-style macros, then the macro must be run before any types are known.

Any system which allows for that kind of type-based dispatch is different from a Lisp-style macro system, and is probably less powerful.

For example, you won't be able to create macros which create new variable bindings (like "let", "with", etc.) because the type of the variable is not known until after the macro is run.

I'm not sure what a system like that would look like, or if I would even call it a macro system.

I think it makes more sense to have two separate systems: a macro system that deals with syntax only, and an overloading system that deals with types. That's what Nulan does.

But perhaps there is room for a third system, somewhere in between the two: that third system could do type-directed syntax manipulation.


2 points by Pauan 187 days ago | link

I thought about this some more.

I have no experience with Idris or dependent types, so I may be completely wrong.

But from my understanding, a dependent type system allows for types to include values (which are evaluated at compile-time).

If so, then you might not even need macros at all to create the "," behavior.

Instead, you create a typeclass which will dispatch to the appropriate behavior:

  data Pair a b = pair a b

  interface Comma a b c where
    comma : a -> b -> c

  Comma Type Type Type where
    comma = Pair

  Comma a b (Pair a b) where
    comma = pair
Now whenever you use the "comma" function, it will dispatch depending on whether its arguments are Types or not:

  -- Type
  comma Integer Integer

  -- (Pair Integer Integer)
  comma 1 2
And since types may include arbitrary expressions, you can of course use the "comma" function inside of a type:

  foo : a -> b -> (comma a b)
Note: all of the above code is completely untested and probably wrong, but you get the idea.

Basically, your language needs first-class types, and the ability to run arbitrary expressions at compile-time (even within types), and then you can use an ordinary typeclass to get the desired type dispatch. No macros needed.


2 points by rocketnia 185 days ago | link

"If so, then you might not even need macros at all to create the "," behavior."

I never said the overloading of "," was accomplished with macros.

The example I gave was of a macro whose name was overloaded in a type-directed way, similarly to the way "," is overloaded in Idris. (Maybe the macro itself is named ",".) My point is that if the macro system is designed to support that kind of overloading, then sometimes a macro call will depend on intermediate results obtained by the typechecker, so we can't run the macroexpander in a phase of its own.


For the sake of remembering Idris more accurately, I checked out the Idris docs to look for the particular "macro system" I was dealing with.

It turns out what I mean by the "macro system" in Idris is its system of elaborator extensions ( The elaborator's purpose is to fill in holes in the program, like the holes that remain when someone calls a function without supplying its implicit arguments.[1]

It's pretty natural to handle name overloading in the same phase as implicit arguments, because it's effectively a special case. Thanks to dependent types and implicit arguments, instead of having two differently typed functions named (,), you could have one function that implicitly takes a boolean:

  (,) : {isType : Bool} ->
    if isType then (... -> Type) else (... -> (a, b))
  (,) = ...
The elaborator phase is complex enough that I don't understand how to use it yet, but I think almost all that complexity belongs to implicit arguments. Name overloading is probably seamless relative to that.

Another "macro system" I encountered in the docs was Idris's system of syntax extensions ( As far as I can tell, these apply before the elaborator and typechecker, but they might not be able to run arbitrary code or break hygiene. Maybe someday they'll gain those abilities and become a Nulan-like macro system.

[1] Implicit arguments effectively generalize type classes. Idris still has something like type class declarations, which it calls "interfaces," but I bet this is primarily so a programmer can define the interface type and the method lookup functions together in one fell swoop.


5 points by Pauan 546 days ago | link | parent | on: How do you see what a macro expands to?

First off, you're seeing the macro-expansion of the "mac" macro, not the macro-expansion of the "test" macro. Try this instead:

  arc> (mac test a `(list ,a))
  arc> (macex '(test a))
Secondly, you can also use "macex1" to only expand it one time:

  arc> (macex1 '(mac test a `(list ,a)))
Sometimes this produces nicer output.


3 points by jsgrahamus 545 days ago | link

Why does your example return (list (a)) instead of (list a)?



3 points by akkartik 545 days ago | link

Rest arg :)


2 points by jsgrahamus 544 days ago | link



3 points by akkartik 544 days ago | link

  (mac test a `(list ,a))

  (mac test (a) `(list ,a))
Does that help?


3 points by lark 546 days ago | link

Thank you.


3 points by Pauan 551 days ago | link | parent | on: Two useful language extension ideas

You can indent a code block by 2 spaces to format it as code.


Your idea of types is similar to Predicate Dispatch[1]. I believe it's also similar to Racket's contracts[2].

This is an extremely powerful type system in the sense that you can express literally anything with it. However, I believe that a more restrictive type system can actually be better, because it provides more guarantees (especially at compile-time).


I don't think it's possible to pattern match on a function's arguments, body, and closure, simply because Racket doesn't expose those things.

I do approve of pattern matching, though.


* [1]:

* [2]: