Getting back after arc is not the problem; even if I didn't have cd history set up a simple `cd -` would handle that.
Simple problem: I have a file containing a program written in Arc. I want to run it from the shell. How do I do that? I can fire up arc and `(load)` the file, but then the file needs to be in the arc2.3 dir or else I have to load it by absolute path, even if I was just in the same directory it's in.
For now I've just switched to using Anarki; I can run its `arc.sh` start script from anywhere in the filesystem and feed it a locally-relative pathname and it will run. Although it does seem to take quite a lot longer to start up than arc.
Ah, interesting. That's a good point that it's Unix/MacOS-specific. My hope was that it should make Arc interpret paths the same way the default shell on the system does. On Linux, ~/ is the home directory of the current user, so it makes sense that Anarki would work that way on Linux. But Windows doesn't work that way, so it makes sense Anarki wouldn't either.
I don't know of any similar path tricks on Windows, But, if this change hasn't broken anything on Windows, I would be satisfied with that. Can you verify it still works to open an existing file? Are there any other special ways to refer to files or directories on Windows?
"On Linux, ~/ is the home directory of the current user, so it makes sense that Anarki would work that way on Linux. But Windows doesn't work that way, so it makes sense Anarki wouldn't either."
Yeah, I don't know much about what a Windows user would expect ~ to do. I would say `expand-user-path` leaving ~ alone on Windows is probably as good a behavior as any. It coincides with Command Prompt, where ~ just refers to a file or folder named "~". In PowerShell, ~ seems to be expanded to C:\Users\[username]\ somehow, so there's potentially an alternative design there.
"I don't know of any similar path tricks on Windows, But, if this change hasn't broken anything on Windows, I would be satisfied with that. Can you verify it still works to open an existing file?"
On Windows 10 64-bit with Racket 8.1, I've at least run the tests.arc unit tests, the unit-test.arc/tests.arc unit tests, and build-web-help.arc, and they seem to work.
"Are there any other special ways to refer to files or directories on Windows?"
I hardly know where to begin learning about and testing those features, and the documentation makes it look like Racket has explored hat rabbit hole pretty thoroughly already, so I'm inclined to suggest we just piggyback on Racket's work here.
Hrm. The tests definitely were faster. They weren't as fast as I'd wanted, but certainly not this slow. It just took three minutes for me to load that file. It's slightly complicated by that file loading _and_ running the tests, but again, it wasn't that slow before. For reference, I'm on Ubuntu running Racket 8.1.
One thing that definitely has changed is that now, Anarki requires Racket >=7.7. I just had to update my Racket from 7.2, which worked the last time I ran Anarki, Perhaps this is related?
At one point, years ago, I relied on every lambda returning a newly allocated object. I think I brought it up as a bug in Rainbow that the  expressions in my Arc code were generating the same lambda-lifted result instead of creating unique objects each time.
But I think this was never something I could rely on in the original, Racket-based Arc either. The Racket docs don't specify whether a lambda expression returns a new object or reuses an existing one, and in fact they explicitly allow for the possibility of reusing one:
"Similarly, evaluation of a lambda form typically generates a new procedure object, but it may re-use a procedure object previously generated by the same source lambda form."
That sentence has been in the reference since the earliest versions I can find online:
I'm not sure if the bug you're referring to had to do with making the same mistake I was making back then, or if you're talking about making the opposite assumption (expecting two procedures to be equal), but it's probably best not to expect stability either way.
I was doing a deep-`iso` comparison I called `same`, because in Arc, hash tables are not the same. In Arc3.1:
arc> (iso (obj) (obj))
arc> (iso (obj 1 2) (obj 1 2))
So I made a function that iterated over the key/value pairs and compared them. In some unit tests, I used that on a hash table that contained functions. Yesterday, I refactored the unit tests to instead make sure the keys were right.
Interestingly, it seems that Anarki can compare hash tables:
arc> (iso (obj) (obj))
arc> (iso (obj 1 2) (obj 1 2))
Anyway, this should fix the unit test tests in the Anarki repo.
The "racket -f as.scm" is supposed to be done in your terminal, you're right! DrRacket is a different thing that is very cool, but not the tool to run Arc with.
When you type it in your terminal, what does it say? If it says something like "racket not found", try downloading the racket installer from https://download.racket-lang.org/.
If it installs correctly, you should be able to type "racket" (without quotes) in your terminal, and get a message "Welcome to Racket 8.0". To quit, press Ctrl-D, or type (exit) with the parentheses, then hit enter.
If that works, you should be able to run "racket -f as.scm".
Thank you so much, zck :)
I've used a combination of your helpful suggestions, stack overflow, and cd command to get to the right folder, and now it works. I got the arc command prompt (arc>) in my terminal now. I will take my time to explore the tutorial on Arc.
I feel good that this community is really helpful. Thanks everyone!
Super interesting! I like how you can zoom into the system, almost like using a microscope to see how code works.
One thing that isn't quite clear from your examples here (and maybe that wasn't the point, but I am wondering): why have a different stack when a function is called? Your examples would work fine if they used the same stack. That is, if functions used the same stack, `1+` could be defined as `1 +`.
Perhaps it would be more clear if an example used two variables, or used the same variable twice.
Is it that you're trying to reduce stack juggling? That's hard to understand from simple examples (that you need to have to grok the syntax). Maybe compare something like finding the hypotenuse of a right triangle given the two other sides. Without naming arguments, you'd have to do something like:
hypot = dup * swap dup * + sqrt
But with named arguments:
hypot x y = x x * y y * + sqrt
That seems a little easier to read, but even the original isn't so complicated. Is there an example that makes it more obvious why it's better? Even your `square` example is more tokens than with stack juggling (without arguments, `square = dup *` is just four tokens, compared to six with named arguments). I'd say the with-arguments one is easier to read, but I've far less familiar with stack-based postfix programming.
Yeah, some other polynomial might be a better example. The Pythagorean formula has a clean separation when it comes to which args each term uses. How about the roots of a quadratic equation when you start out with a, b and c on the stack?
You're right that in general any such example can be refactored to reduce stack operations. That's even a fun game for many Forth programmers to play. (We lispers have our own equivalents.) But it usually makes the code less comprehensible, in my experience.
I've always implicitly assumed that tables were just an implementation detail for templates, and assumed they were like objects. I think that might be why it never occurred to me to apply `len` on a template, and why a large program like news.arc never ran into this gap.
I have no objection to your approach of making them more like tables. I also have no objection to updating the documentation to stop referring to them as tables. One of those options might be less work than the other :)
> I've always implicitly assumed that tables were just an implementation detail for templates, and assumed they were like objects. I think that might be why it never occurred to me to apply `len` on a template, and why a large program like news.arc never ran into this gap.
Ah, interesting! Here's some background that might help understand why having them act like tables might help. `len` wasn't the first function that I found didn't work with templates; it was just the simplest. I was working on adding teardown functionality to unit-test.arc, and I wanted to look at some of the suites that were created, to see if I was adding tests properly. As the templates end up pretty big (one suite with two tests is about 25 lines), and it was late at night, I figured I'd make it simple on myself, and just look at the keys of the template.
This had nothing to do with the desired "production" code, but only with REPL-hacking introspection.
I want to make them reasonably easy to REPL-hack with; whether they're actually tables or not I don't particularly care right now. The most important table functions are probably len, keys, vals, and maybe maptable/each.
My answer to that question has always been, "it depends." The anti-encapsulation ethos (homoiconicity, using lists where other languages may use objects, the entire compiler fitting in one file and being accessible front-and-center, etc.) means that there's always the ability to peel back another layer of the onion when it becomes relevant.
I think it's a documentation issue. I think I had to search the forums to find out about it when I was playing with JSON interop. Nowhere on the actual template page in the Arc documentation does it tell you this is a thing.
Some things I've only been able to figure out by studying the compiler or arc source code itself. Granted, that's illuminating, but it's also sometimes really annoying.
Templates should just enforce a signature for table fields, but otherwise decompose to tables. I think the issue is that the tables generated by (inst) from a template are annotated as type 'tem when that should (could?) be done with a tag that doesn't actually change the type. Since the template name is passed to the function, you could just build the table with default values without annotating it. You could also delete most of the template functions that just serve as template versions of table functions.
edit: I was also assuming templates type-hinted field values but I don't think they do. Maybe they should?
Also, does anyone else find it a bit odd that a language feature like this is in a lib file rather than being part of the core language?
I disagree with your vision for templates. If you just want something that behaves like tables, why not just use tables? A helper that fills in default values would be pretty easy to write.
Think about the use case of news.arc. There's a list of 'objects' that need to be serialized to disk and unserialized from disk. What should happen if you change the default for a template in code? Should the default update transparently for existing objects? If so, you need some way to distinguish tables that were generated by templates. Which implies something that manages them throughout their life cycle.
>Small languages take it as a mark of pride to move as much as possible into libraries :)
Yeah, I've seen projects that show off the power of a language by doing "x in < 100 lines" that just don't count a remote API call with half a million LOC running on a server or something ;) But with language features like macros and templates that have become ubiquitous, I feel like it's kind of cheating not to just fold them into arc proper.
But that's just me... one thing I've learned being here is that I seem to flow against the culture more than with it, so I can just agree to disagree.
> If you just want something that behaves like tables, why not just use tables?
They are tables, that's what's frustrating. They're tables with metadata. From what I can tell reading earler posts about templates, they used to be something that behaves like tables. Interop between forum data and Racket (and any code where tables are expected) is awkward because that incompatibility has to be worked around, resulting in extra code and extra complexity. Templates need a separate API despite having the same behavior as tables.
>If so, you need some way to distinguish tables that were generated by templates. Which implies something that manages them throughout their life cycle.
Fair enough. But why is it necessary to change their type to do so? Why not make this a feature of tables as a whole, if it's useful? Or tag tables in a way that doesn't change their type, if that's possible?
I understand that tradeoffs have to be made and I'm not trying to be cantankerous or dismiss the value of anyone's work, and no, I couldn't do better myself (yet), which is why I'm commenting on it it rather than making a PR. I'm just wondering if this is the best possible implementation of the concept, given how often I and other people seem to run into issues with it.
Don't worry about sounding dismissive, I totally understand where the questions are coming from.
Tables and objects feel like separate concepts, and they have complementary strengths and weaknesses, and one doesn't subsume the other. To me it seems obvious that if we want to have both, we need them to have different types.
For example, sometimes you want the 'dynamic' ability to set arbitrary keys of metadata on a thing. Sometimes you want the same operation to be an error, by providing a schema. How would a single type do both? No language does so, to my knowledge.
Things should have the same type when they have compatible behavior. When they are incompatible, they shouldn't.
Supporting helpers like len and keys may well still make sense. And as the original story did, this is easy to do.
But in general, having incompatible types easily share functions without sharing too much is an open problem: https://en.wikipedia.org/wiki/Expression_problem A language can easily add a method to many types, or add a new type to many methods. But we don't yet know how to achieve both sides.
And honestly, I think the expression problem isn't important. It doesn't take too much code per method/type. And making it easier just encourages large, bloated codebases.
> ...given how often I and other people seem to run into issues with it.
One thing that might be useful here is a list of issues people have encountered with templates. Maybe we should create a wiki page on GitHub and add to it every time an issue comes up. Then we can have a big-picture view of them and a sense of how many are things people need to learn about Arc, and how many are bugs to be fixed.
I believe Anarki behaves exactly the same as Arc's intent when it comes to templates. The changes that I made here seemed strictly superior to the buggy implementation upstream. But if you disagree you should absolutely feel free to just revert the commits and go back to Arc behavior. I don't use Arc anymore, so my opinions are extremely weakly held, you don't have to bother persuading me. Or, if you have some other specific issue in mind, I'd be happy to be persuaded that I'm wrong.
"But in general, having incompatible types easily share functions without sharing too much is an open problem: https://en.wikipedia.org/wiki/Expression_problem A language can easily add a method to many types, or add a new type to many methods. But we don't yet know how to achieve both sides."
I'm trying to follow, but I think you and I must have different understandings of the expression problem. That article lists several known solutions to the expression problem. The solution Anarki uses is `defextend`.
What do you mean by "sharing too much"?
Is Anarki's `defextend` technique already encouraging a bloated codebase, or is there some other technique you're thinking of that would do that?
Yeah, I suppose you could say the problem is 'solved'. I think of it as a trade-off with costs. We don't know how to achieve zero cost.
For example, I absolutely agree with you that 2 lines per method to extend every table method to some new type constitutes a solution for us. But if we had a thousand such types and a thousand such methods, it may seem like less of a solution. But then `defextend` would be the victim rather than cause of bloat.
Ah, you're imagining us having to write and maintain 1000×1000 individual `defextend` forms someday? Yeah, that does seem like a problem that would not feel solved once we got to it. :-p
I don't think that aspect of the expression problem is solvable in a language design. Instead, it's an ongoing conversation in the community. Sometimes the intent of one feature and the intent of another feature interact, leading people to do a nonzero amount of work to figure out the intent of the two features put together. That work is an essential part of what the community is trying to accomplish together, so it's a cost that can't be eliminated. The intent has to be reflected in the code somewhere, so there will be a nonzero amount of code that serves feature-coordinating purposes.
Regardless, I'm optimistic that although the amount of code will be nonzero, it'll still have a manageable size. To the extent we have any kind of consistency around these feature interaction decisions, those consistent principles can develop into abstractions. The only way we'll have 1000×1000 individual intersections to maintain is if we as a community culture are already maintaining 1,000,000 compelling and distinct justifications for them. :)
I haven't read any more than a few papers on it, and maybe only one of those in depth (which I'll mention below). Mostly I'm going by forum threads, wiki articles, and the design choices certain languages make (like Inform's multimethods and Haskell's type classes).
As far as I understand the history, Philip Wadler's work basically defined the strict parameters of the expression problem and explored solutions for it. Separate compilation and the avoidance of dynamic casts were big deals for Wadler for some reason.
That work was focused on Java, where it's easy to define new classes that implement existing interfaces but impossible to implement new interfaces on existing classes.
The solution I'm most familiar with for Java-style languages is the use of object algebras, as described in Oliveira and Cook's "Extensibility for the Masses: Practical Extensibility with Object Algebras" (https://www.cs.utexas.edu/~wcook/Drafts/2012/ecoop2012.pdf). In this approach, when you extend the system with a new type, you define a new interface with a generic type parameter and a factory method for building that type, and you have that interface inherit all the existing factory methods. So you don't have to solve the unsolvable task of implementing a new interface for an existing class, because you're representing your types as type parameters and methods, not simply as classes.
So I think the main subject of research was how best to represent an extensible program's types and functions in a language like Java where the most obvious choices weren't expressive enough. I think it's more of a "how do we allow extensions to be made at all" problem than a "how do we make all the extensions maintainable" problem.
But then, I've really barely scratched the surface of the research, so I could easily be missing stuff like that.
> ... with language features like macros and templates that have become ubiquitous, I feel like it's kind of cheating not to just fold them into arc proper.
It's totally fine to move something into arc.arc if you want to do that. It's always felt like a non-existent distinction in my mind whether something is under arc.arc or libs/. Is Anarki all language or all standard library? Depends on how you look at it. Why does it matter?
> But that's just me... one thing I've learned being here is that I seem to flow against the culture more than with it, so I can just agree to disagree.
This doesn't feel like a disagreement, more like a language barrier. If I understood better I might know whether I agree or not.
arc> (withs nil 3)
Can't take car of nil
This is a minimal example from something I found in unit-test.arc. It's some macros related to setup code -- if there's no setup, I currently generate something like `(withs nil 3)`. But that errors in Anarki.
At least to me, this is expected. The commit you pointed out above switched the null value to '(). The symbol `nil` still evaluates to (). But the need for evaluation implies that it isn't available in contexts that are not evaluated, such as function arguments or in this particular slot of `withs`.
Like I said, happy to revert it if you don't like it. The whole thing came up because of this conversation: https://github.com/arclanguage/anarki/pull/145#issuecomment-.... The motivation was to simplify the Arc implementation. We already have a nil representation in the underlying Racket; it seems unnecessary to so bend over backwards to switch it to something else.
Yeah, that seems better. I'm still tracking down two test failures, but they're not because of this. I think templates now are of type 'tem, not type 'table.
I tried to make some changes to () instead of nil, and I was not a big fan of how it looked. I found it very unusual that unless quoted, parentheses mean function application. Letting () be the way to write the empty list (and I believe it worked differently quoted from unquoted, but I'm not sure offhand) completely breaks my mental model of how Lisps are parsed.
It's failing because `(type (inst 'foo))` is different in Anarki than Arc. It's a simple change to make it work; I just want to do two things before I stop looking at it:
1. Look deeper into the template inconsistencies. Thanks for the files about this in Anarki.
2. Decide if I want to cut support for Arc, or make this code work in both. This might just involve killing the test, as it's not the _most_ useful test.
Ouch, have the tests for unit-test.arc been failing for the past year? :( :( Very sorry about that. I see the failure now.
I somehow forgot that unit-test.arc has its own tests. Could you post the instructions for running the tests in the Readme? That would also have the salubrious side effect of showing people a way to run a bunch of existing tests.
Once that last one is passing (or maybe even before it's passing), should the top-level tests.arc run these tests too? That way this can be caught not only by Travis CI, but also by people running tests.arc according to the readme.
Anarki isn't really intended to avoid or minimize breaking changes. The unit tests verify only that everything is internally consistent. That boundary around 'internal' should include unit-test.arc, I think.
I _think_ it's some weirdness with the nil/empty list thing. I was getting a case where (str x) resulted in the string "nil", but whatever that object was was not treated as nil, for example in conditionals.
Interesting concept. I feel like I'd be ok with it? I want to say we should bind `nil` to `'()`, so existing code would continue to work, but I might be overindexing on compatability and what I'm used to.
I will admit to not being super sure what the real differences between nil and '() are. Presumably it's more than "what is the human-readable representation of the value that terminates a list/is the false value". But I'm not sure what. Also, is there a difference between the quoted and unquoted version? It feels odd to write () in a repl unquoted -- usually, I expect parens to mean a function or macro call.