Arc Forumnew | comments | leaders | submit | akkartik's commentslogin
2 points by akkartik 6 days ago | link | parent | on: Knark - rewrite in plain Racket?

This is the Rich Hickey talk from 2016: https://www.youtube.com/watch?v=oyLBGkS5ICk. If you listen to it, it was made in an effort to adjust the community's trajectory. I too would like to hear if adjustments did happen.

Basically the way I interpreted it[1] is that "major version" is a meaningless concept. If you're making incompatible changes, rename the package. If that's a disincentive to making incompatible changes -- great!

[1] http://akkartik.name/post/versioning

reply

2 points by hjek 5 days ago | link

Nice blog post.

> Rich Hickey pointed out last year that the convention of bumping the major version of a library to indicate incompatibility conveys no more actionable information than just changing the name of the library.

Yes, I guess Hickey is applying this idea of immutability not only to data structures but also to APIs and even databases[0] with the proprietary database service Datomic. Interestingly Datomic uses Datalog as query language, so it's straight forward to apply at least some of those ideas with the Racket Datalog package[1].

As of now I have a basic web forum working with all data storage done in Datalog (except for file uploads!). I'll post some code once the design is a bit more settled. It's still in the breaking-things-all-the-time phase.

What I find a bit tricky about Datalog is that relations are stored together with other facts, which in my mind feels a bit like storing code in a database, but maybe I just haven't wrapped my head around it yet.

Does anyone here have favourite articles or talks about logic programming?

[0]: https://www.youtube.com/watch?v=EKdV1IgAaFc

[1]: https://docs.racket-lang.org/datalog/interop.html

reply

3 points by i4cu 5 days ago | link

> What I find a bit tricky about Datalog is that relations are stored together with other facts, which in my mind feels a bit like storing code in a database, but maybe I just haven't wrapped my head around it yet.

I can't speak for Datalog, but I've used Datomic.

If it's the same, then a 'fact' is comprised of an entity (the id), + an attribute, + a value.

The relationships are made by storing an entity id into the value slot of another fact. Thus the model is both flat (being a list of facts) and hierarchical (they can point to each other). It pretty much becomes a graph database. Is that what you mean?

reply

2 points by hjek 5 days ago | link

> I can't speak for Datalog, but I've used Datomic.

Cool!

> The relationships are made by storing an entity id into the value slot of another fact. Thus the model is both flat (being a list of facts) and hierarchical (they can point to each other). It pretty much becomes a graph database. Is that what you mean?

Yes, exactly! What I worry about (because I'm fairly new to logic programming) is whether it could potentially be difficult to update a program where storage of business logic and storage of data aren't separated?

If we assume we have a Hacker News web app where we have a fact: One day Alice submits a story with the title "How to peel onions"; and we are thinking "Why on earth did she post that here?!?" So we add this relation to our code: A story is `irrelevant` if it has the word "onion" in the title. Then, the next day we get another fact: Now Bob has submitted a story called "How onion routing works". This new story by Bob then makes us reconsider our definition of `irrelevant`.

In a typical imperative program we'd just edit the code and redefine the `irrelevant` predicate, and it would take effect next time we run the program (or instantly if we enter it at a repl). But here in our logic program we store this `irrelevant` relation in our graph database, so even though we have removed it from our code, it is still sitting there in the database along with all the facts, outside the reach and responsibility of git, or whichever VCS we're using.

Yes, so my question is: How do you practically deal with changes to business logic in logic programming where data storage and relation storage is one and the same? Perhaps Datomic just avoids this issue somehow? I may also be missing or misunderstanding something.

reply

3 points by i4cu 5 days ago | link

Can't say I know what the options are since I don't know Datalog or the DB you're using, but is this reasonable?:

  -----------------------------------------------------------
  Entity              | Attribute | Value
  -----------------------------------------------------------
  person-id-001       | name      | Alice
  person-id-001       | stories   | [story-id-001, story-id-002...]   

  story-id-001        | headline  | "How to peel onions"
  irrelevant-word-001 | stories   | [story-id-001, ...]
  irrelevant-word-001 | word      | onion
  -----------------------------------------------------------
So if you decide that onion is no longer irrelevant then delete the entity 'irrelevant-word-001'. Which seems, at least to me, better than making code pushes.

So all of this assumes a few things:

- Your DB supports a cardinality of 'many' items in the value slot.

- Your query language can perform joins.

Of course none of this helps when someone changes a headline, but only full-text search DB's will help you do that.

Edit: made edits.

reply

3 points by i4cu 5 days ago | link

To be complete (and somehow I edited this out):

  -----------------------------------------------------------
  Entity        | Attribute          | Value
  -----------------------------------------------------------
  globals       | irrelevant-words   | [irrelevant-word-001, ...]   
  -----------------------------------------------------------
So you would also need to remove the value 'irrelevant-word-001' from the above. At least this is how I would do it in Datomic anyway.

What's interesting (at least to me) is that Datomic has a function called 'retractEntity' [1] which auto-magically removes all references of an entity in any value slot when you retract the entity. Man I love Datomic :)

[1] https://docs.datomic.com/on-prem/transactions.html#dbfn-retr...

reply

2 points by hjek 5 days ago | link

> So if you decide that onion is no longer irrelevant then delete the entity 'irrelevant-word-001'. Which seems, at least to me, better than making code pushes.

I can make sense of that when there's just one instance of this app running. Yet imagine the scenario where the web app has been published, and suddenly other people are running this web app. If the business logic is then changed, somehow I'd have to tell those people: "Oh btw, when you're running `git pull` next time, then you just also gotta run this query to retract some of the old relations from the database."

Definitely not a problem for me yet, but I can just smell it coming. I could add those retractions to the code, but they would have to stay there indefinitely, because it's not possible to tell if those retractions have taken place on everyone's databases yet.

Maybe I'm over-thinking this.

> What's interesting (at least to me) is that Datomic has a function called 'retractEntity'

Looks like the one called `~` in Racket's Datalog[0].

[0]: https://docs.racket-lang.org/datalog/interop.html#%28form._%...

reply

2 points by i4cu 5 days ago | link

Quick questions before going further.

Is this irrelevant-word example a real feature you're building into the app or a contrived example to understand Racket DataLog DB use?

If it's a real feature, and I'm assuming it is. Then I'm also assuming that when a story is submitted you're parsing the title and adding the relationship to the current set of irrelevant-words that are stored.

So the question's are:

1. How are you going to remove the past relationships between stories and an irrelevant word that's getting removed? (looks like we've answered this).

2. How are you going to make sure past stories gain the relationship to newly added irrelevant words?

3. How are you going to handle title changes.

After you get handle on these then what you really need to do provide an interface, from within the apps admin tools, to trigger the noted functionality. This way the business logic is in the app and it's modifying the data.

reply

2 points by hjek 4 days ago | link

> Is this irrelevant-word example a real feature you're building into the app or a contrived example to understand Racket DataLog DB use?

It's a simplified and slightly contrived illustration of an issue in this pre-alpha code I haven't published yet, perhaps just because I haven't thought of a name for the project yet. But yes, let's assume it's a real feature for now.

> 3. How are you going to handle title changes.

I like Hickey's idea of accretion of data - with a timestamp! - and not forgetting previous facts. I think he's talking about it in The Database as a Value[0]. So, a story could have a few different titles in the database, and the newest one is the one you get to see.

The thing is, it's easy to add a timestamp to facts as a way of not considering old facts without forgetting them, but not to relations. For example in Racket Datalog[1]:

    (! (voted "i4cu" 'up 134))
can easily be get a timestamp:

    (! (voted "i4cu" 'up 134 1542151773))
but that would not really make sense in a relation like

    (! (root A B) (ancestor A B) (parent null A))
So, I don't feel there's a need for ever retracting facts, because timestamps solve that. (Even when deleting something, you could just add the fact that is has been deleted.) But with relations (a.k.a. business logic) I think I will need to retract things, which is tricky because this logic is not only present in the code but also in the database. This was the problem I was asking about.

> 2. How are you going to make sure past stories gain the relationship to newly added irrelevant words?

Datalog queries reflect the current set of facts and relations, so the possible irrelevance of a story would not be stored anywhere, so it wouldn't need to be updated.

> 1. How are you going to remove the past relationships between stories and an irrelevant word that's getting removed?

Some as above. For example, in a place oriented program a story object could have a boolean attribute `irrelevant?`, whereas I'm sending a query every time this value is needed, so no stale `irrelevance` attributes are stored anywhere.

> After you get handle on these then what you really need to do provide an interface, from within the apps admin tools, to trigger the noted functionality. This way the business logic is in the app and it's modifying the data.

Yes, that is kind of there already, as in having functionality for changing titles. The `irrelevant` functionality is not there now.

Ok, I think I just need to work on getting this code publishable, because it might be easier to discuss tangible examples.

[0]: https://www.infoq.com/presentations/Datomic-Database-Value

[1]: https://docs.racket-lang.org/datalog/interop.html

reply

2 points by i4cu 4 days ago | link

Yeah, Datomic doesn't expect the relationships to be stored in the DB. It stores a bunch of indexes for you and it has a great query language, but that's it.

So where will you're data be? In a local data structure? I read your racket Datalog link, but it doesn't show any details for the database side (i.e. durability etc.) even though it's labelled a database.

Also, I'm curious what made you choose a graph db. It seems like you're inheriting a lot of complexity and I'm wondering what the benefit is over a more traditional sql or nosql db.

reply

2 points by hjek 4 days ago | link

> So where will you're data be? In a local data structure?

Yes, I think. The database just stored in memory but it can be serialized and saved to the disk using `write-theory`[0] and loaded `read-theory`. That is what I'm doing for now, and it's a very naive and inefficient to do a full database dump rather than just appending new data, and I presume it's particularly in this area where Datomic is way more optimised and well thought out.

> Also, I'm curious what made you choose a graph db. It seems like you're inheriting a lot of complexity and I'm wondering what the benefit is over a more traditional sql or nosql db.

Well, I did the initial work on the web app: creating user accounts, adding posts and replies, and then I got to data storage. Initially I did a News-style flat-file database, just saving data as lists in files, that are then loaded into memory when the program starts. It mostly worked but also felt a bit complicated, and I thought that perhaps I should just use a proper database?

What I like about news.arc is that you can just launch it without any configuration, so MySQL and PostGreSQL were out of the question, and I started reading a bit about SQLite. But I've also had this fascination with logic programming, from what people are posting here[1][2], and from reading a bit of The Reasoned Schemer, and I watched some of those Rich Hickey talks again, where he talks about Datalog, which happens to be available for Racket.

There are just some things that are incredibly simple in declarative/logic programming. For example, if you have facts about stories being `parent` of their replies, then it's simple to just define the `ancestor` relation, and when you have the `ancestor` relation, you automagically get `descendants` without having to write any code, because it's just the inverse of `ancestor`:

    (! (:- (ancestor A B)
           (parent A B)))
    (! (:- (ancestor A B)
           (parent A C)
           (ancestor C B)))
But, I've also bumped into some questions - more practical than theorical - and that is why it's interested hearing about your experience with Datomic, and why I'm asking here.

So, SQLite is still on the table. I'm not too familiar with NoSQL, but my impression is that they are all about speed and scalability of data storage. I haven't used MongoDB but isn't it essentially just like storing JSON in a file, except faster? It would be interesting if any of those could be used in conjunction with Datalog though, if don't add too complexity for the sake of increased speed.

[0]: https://docs.racket-lang.org/datalog/interop.html#%28def._%2...

[1]: http://arclanguage.org/item?id=20650

[2]: http://arclanguage.org/item?id=20519

reply

2 points by i4cu 4 days ago | link

So a few things I wanted to point out:

Datomic vs. DataLog

Datomic uses DataLog as part of its query language, but that's pretty much where the comparison should end. Things like "treating the database as a value", and features such as data accretion that Rich talks about have nothing to do with DataLog. They're features of Datomic. So for example when you mention never retracting data, well your data size is going to continuously grow unless you write your own data management layer on top. Datomic, on the other hand, does this for you. When you want to query the database over time, then you're going to need to store time intervals for all of your data and incorporate that into each query. Where as in Datomic (which has a time log) you can pass in the DB itself as a value (with an associated time interval) and Datomic will make sure your queries are working against the dataset that accounts for the time interval.

I'm pointing this out because it seems to me that you're doing (or are going to be doing) a lot of work that may not be worth it for what you're trying to accomplish.

Nosql

> I'm not too familiar with NoSQL, but my impression is that they are all about speed and scalability of data storage.

Yes and No. Often speed can be a feature Nosql dbs advertise, but really, for me anyway, it's about flexibility and ease of use. Traditional RDBMS, for example, require creating schemas. Many Nosql databases don't require a schema at all which makes it easier to use and more flexible to change. Nosql's are often a key-value store so it can be really easy to take a hash-map or table of data from your code and just dump it into an nosql datastore and be able to query it.

My personal favourite is Redis and it might be worth considering for your app.

You can:

- store a value under a key [1]

- store table data [2]

- store values in a set [3] (which allows intersection/difference queries)

- store values in a sorted-set [4] (which allows you query by some numerical value like timestamp)

- use it to manage relationships [5]

The reasons I mention Redis is that the HN app is very well suited to it. HN only keeps 'x' amount of data in memory. And in Redis the data lives in memory. Also Redis allows you to set expiry times on data for auto eviction [6]. And Redis also supports ordered lists [7] which can make it useful for lisp based languages.

However it's not embedded. And if that's a requirement I'd almost suggest you move away from Racket and adopt a language that has more options for embeddable databases. I guess if you're willing to roll your own (and it looks like you may be) then that's awesome too.

But in case you decide otherwise... The library I use is Redis Carmine [8], but there are Racket clients [9].

1. https://redis.io/commands/set

2. https://redis.io/commands/mset

2a. https://redis.io/commands/mget

3. https://redis.io/commands/sadd

4. https://redis.io/commands/zadd

5. search: "Representing and querying graphs using an hexastore" https://redis.io/topics/indexes

6. https://redis.io/commands/expire

7. https://redis.io/commands/lset

8. https://github.com/ptaoussanis/carmine

9. https://redis.io/clients#racket

reply

2 points by hjek 3 days ago | link

> Datomic uses DataLog as part of its query language, but that's pretty much where the comparison should end. Things like "treating the database as a value", and features such as data accretion that Rich talks about have nothing to do with DataLog.

I'm not sure I totally agree with this. I think that apart from talking about the design of Datomic, he also has a more general point against what he calls PLOP (PLace Oriented Programming), which Datalog does address.

For example in plain Racket a value is lost if something else is put in its place:

    > (define foo 'bar)
    > (define foo 'baz)
    > foo
    'baz
In Datalog you just accrete facts:

    > (! (is foo bar))
    > (! (is foo baz))
    > (? (is foo X))
    is(foo, bar).
    is(foo, baz).
Hickey is also mentioning how git doesn't do PLOP in that it doesn't throw out your commit history (without you asking it to do so).

> The reasons I mention Redis is that the HN app is very well suited to it. HN only keeps 'x' amount of data in memory. And in Redis the data lives in memory. Also Redis allows you to set expiry times on data for auto eviction [6].

Interesting. Just checked news.arc, and yes `initload*` is set to 15000. Interesting idea from Redis with expiry times. I'll check it out. I hadn't considered the scenario of storing text enough to max out on memory, because it would probably be premature optimisation, but good to keep in mind. I'd like to give Redis/Rackdis a try; thanks for the suggestion. I've been hosting an Etherpad Lite instance, and Redis was painless to setup.

> I'm pointing this out because it seems to me that you're doing (or are going to be doing) a lot of work that may not be worth it for what you're trying to accomplish.

Yes, my priorities here are definitely to make the code as brief and simple as possible, and to not have to do to much work. With plain Datalog it's very little work to timestamp a fact, and it's also kind of necessary, e.g. to figure out which fact is most recent, when previous facts are not removed. I'm just trying to get the gist of Hickey's ideas here.

reply

2 points by i4cu 3 days ago | link

> PLOP (PLace Oriented Programming), which Datalog does address.

Yeah, I was thinking more along the lines that Datomic has built-in functionality to address the caching, cache eviction, and indexing that goes along with all that data accumulation. But you're correct, DataLog does accumulate facts.

> Interesting. Just checked news.arc, and yes `initload*` is set to 15000.

I did the same thing, about 6 or 7 years ago, that you're doing now. I ported HN to Clojure (which is actually how I learned Clojure). If memory serves me correctly when I was doing the work I realized I needed a real DB if I wanted to support load balancing. i.e. I needed to centralize the data for the authentication and fnid session info. I think Arc calls them fnids... You probably know better than I do now, but Arc has all this code to expire these session fnids and so, for me, Redis was just a good fit for that task.

Anyways, I'll be sure to take a look at the final result of your work.

Cheers.

reply

2 points by hjek 3 days ago | link

> I did the same thing, about 6 or 7 years ago, that you're doing now. I ported HN to Clojure (which is actually how I learned Clojure).

Cool!

> I needed to centralize the data for the authentication and fnid session info. I think Arc calls them fnids... You probably know better than I do now, but Arc has all this code to expire these session fnids and so, for me, Redis was just a good fit for that task.

The Racket web server is quite "batteries included" and comes with these different managers[0] for dealing with expiration of sessions/continuations, such as the LRU manager:

> The memory limit is set to `memory-threshold` bytes. Continuations start with 24 life points. Life points are deducted at the rate of one every 10 minutes, or one every 5 seconds when the memory limit is exceeded. Hence the maximum life time for a continuation is 4 hours, and the minimum is 2 minutes.

> If the load on the server spikes—as indicated by memory usage—the server will quickly expire continuations, until the memory is back under control. If the load stays low, it will still efficiently expire old continuations.

[0]: https://docs.racket-lang.org/web-server/servlet.html?q=respo...

reply

2 points by i4cu 2 days ago | link

> If the load on the server spikes...

When I was referring to load balancing and centralizing the data I was referring to many web servers sharing a centralized/external source for auth/session data.

I'm unfamiliar with racket's web server 'servlets'. The docs are little unclear (at least to me). Can these servlets live on a separate server so that the data can be shared between web servers? I'm guessing that was/is not a requirement for you, but I'm just interested in knowing if that's how it can work.

Uh oh, you're getting me interested in Racket now. I can't have that... I have too many projects :)

edit: I guess at the end of the day these servlets are web-servers right, so you can, even if you have to do it over http and build an api.

reply

2 points by hjek 2 days ago | link

> Can these servlets live on a separate server so that the data can be shared between web servers?

Probably. I assume that serializable continuations[0] from stateless servlets can just be stored wherever, like in Redis or something, instead of in the memory of one server.

> I ported HN to Clojure

If that is something you have published, it'd be fun to see, whether it's finished or not.

> Uh oh, you're getting me interested in Racket now.

My impression is that Clojure is faster, less verbose partly due to clever syntax and provides more immutable data structures than Racket. But when it comes to documentation and error messages, I find Racket more coherent and comprehensible.

Say, if I wanted to connect to a SQL databse, with Racket I'd use the DB module[1], end of discussion. But with Clojure there's Korma, ClojureQL, Persist, HoneySQL, Yesql, a JDBC wrapper from Clojure contrib, SQLingvo, oj, Suricatta, aggregate, Hyperion, HugSQL, and probably a few more[2][3]. That multitude of libraries with similar purpose may be useful in some cases, sure, but also potentially a bit overwhelming for beginners, so I guess that's why I found it easier to get started with Racket.

[0]: https://docs.racket-lang.org/web-server/stateless.html#%28pa...

[1]: https://docs.racket-lang.org/db/

[2]: https://stackoverflow.com/questions/294802/use-a-database-wi...

[3]: https://adambard.com/blog/clojure-sql-libs-compared/

reply

2 points by i4cu 2 days ago | link

> If that is something you have published, it'd be fun to see, whether it's finished or not.

I actually tried to look it out the other day during this conv, but it's buried somewhere unavailable right now. If I find/get to it I'll post.

> That multitude of libraries with similar purpose may be useful in some cases, sure, but also potentially a bit overwhelming for beginners, so I guess that's why I found it easier to get started with Racket.

Agreed. Navigating the volume libraries and the options available is a real pain in the beginning, but once you get past that, then it's not bad at all. At the same time, take a look at the quality of Clojure's Redis Carmine Library vs. Racket's Redis Libraries. Miles apart.

To each their own, right :)

reply

2 points by akkartik 9 days ago | link | parent | on: Knark - rewrite in plain Racket?

Can you elaborate on what extra features Clojure's tables provide?

reply

4 points by i4cu 9 days ago | link

Where to start...

1. The ability to read them. Especially on output. It may seem small, but when you have a larger deeply nested map it really matters.

  {:user "akkartik"}
2. Keywords are ifns that work on hash-maps.

  (map :user list-of-maps)
3. All the built in library fns that work on them. Basics + Assoc-in, Get-in, merge, deep-merge, contains?..

Here's a good function to separate a collection of maps by any fn/ifn.

  (def separate (juxt filter remove))
 
  (separate :user data)

4. Iteration just works, ie they automatically become association lists for pretty much every composing fn (reduce, loop, ect)

5. Destructuring bind. Eg:

  (defn myfn [{:keys [user] :as record}]
    (str user " rocks"))
6. Nil values are supported.

It goes on, but I think that's enough... Besides my thumbs are getting sore typing via phone.

reply

3 points by akkartik 9 days ago | link

You'll need to elaborate more, apologies in advance to your thumbs. Some of the features you describe I don't understand (ability to read, maybe others). Others seem to already be in Arc (library support, iteration, destructuring bind; [_ 'user] doesn't seem much worse than :user). Nil values feels like the only definitely missing feature, and it feels more like a fork in the road with different trade-offs rather than one alternative being always superior to the other.

reply

7 points by i4cu 8 days ago | link

Ok so let's use this data

Clojure:

  > (def players
       {:joe {:class "Ranger" :score 100}
        :jane {:class "Knight" :weapon "Greatsword" :score 140}
        :ryan {:class "Wizard" :weapon "Mystic Staff" :score 150}})
response:

{:joe {:class "Ranger" :score 100} :jane {:class "Knight" :weapon "Greatsword" :score 140} :ryan {:class "Wizard" :weapon "Mystic Staff" :score 150}})

Arc:

  (= players
    (obj 'joe (obj 'class "Ranger" 'score 100)
         'jane  (obj 'class "Knight" 'weapon "Greatsword" 'score 140)
         'ryan  (obj 'class "Wizard" 'weapon "Mystic Staff" 'score 150))) 
response:

#hash(((quote jane) . #hash(((quote weapon) . "Greatsword") ((quote class) . "Knight") ((quote score) . 140))) ((quote ryan) . #hash(((quote weapon) . "Mystic Staff") ((quote class) . "Wizard") ((quote score) . 150))) ((quote joe) . #hash(((quote class) . "Ranger") ((quote score) . 100))))

-- Readability --

Reading that output is beyond a headache. It should be against the law! But more importantly, when writing code/troubleshooting you can't copy the evaluated output make a modification and then call the function again. I know you can bind it and apply functions to modify it, but often that's a real hassle compared to copy/paste.

-- Destructuring bind --

See links [a,b]

I think simple cases don't show the difference.

  > (def book {:name "SICP" :details {:pages 657 :isbn-10 "0262011530"}})
  
  > (let [{name :name {pages :pages isbn-10 :isbn-10} :details} book]
      (println "name:" name "pages:" pages "isbn-10:" isbn-10))
	  
response:

    name: SICP pages: 657 isbn-10: 0262011530
	
-- Library functions --

Clojure has DOZEN's of helper functions (and important traversal functions) that don't exist in Arc. Yeah I could write them, but I still have to. I was highlighting the ease with 'separate', but here's even 'deep-merge'....

  (defn deep-merge
    "Recursively merges maps. If keys are not maps, the last value wins."
    [& vals]
     (if (every? map? vals)
         (apply merge-with deep-merge vals)
         (last vals)))
		 
  > (deep-merge players {:joe {:weapon "LongBow"} :jane {:weapon "Lance"}})
  
It's not even these sample cases that expose the issues. It's about working with REAL data. I have many files containing hash-maps that are 15 levels deep. If I were to attempt pretty much anything, but the most simple cases, in Arc I would hit problem after problem after problem. Which I did. So fine... really I should use a DB, right? ok let me just grab that DB library in Arc....

-- Nil values --

I'd even be fine with any falsey values like boolean false. But Arc will not even store an #f value. To suggest I can't pass in REAL data with falsey values is really limiting. I can't pass in an options table argument. For every operation I would have to add more data or nuanced data or nuanced operations to support "this value was set, but it's set to 'NO' I don't want that option.".

[a] https://clojure.org/guides/destructuring#_associative_destru...

[b] https://clojure.org/guides/destructuring#_keyword_arguments

reply

3 points by akkartik 8 days ago | link

Thanks! That's very helpful.

reply

4 points by akkartik 21 days ago | link | parent | on: Tell Arc: Arc 3.2

Thanks so much, Scott!

reply

4 points by akkartik 45 days ago | link | parent | on: Recursive anonymous functions?

Is that the Y combinator? I don't think I've seen it ever used "for real". It's not in either Arc 3.1 or Anarki.

You aren't using bracket notation as you originally asked. Might as well just use afn. It can call itself recursively as self. http://arclanguage.github.io/ref/anaphoric.html

reply

3 points by waterhouse 41 days ago | link

The Y combinator itself is more cumbersome, having an extra currying step or two. I prefer the form hjek is using—which is a function that expects to take "itself" as an extra parameter, like this:

  (fn (f i)
    (aif i!parent
         (+ 1 (f f (item it)))
         0))
So the recursive call, "(<self> (item it))", is implemented as "(f f (item it))". And then usage is very simple: actually give it itself as an extra argument.

The Y combinator works with a different function signature:

  (fn (f)
    (fn (i)
      (aif i!parent
           (+ 1 (f (item it)))
           0)))
That is, the function takes "something that's not quite itself" as an argument, and returns a function which does one step of computation and may do a "recursive" call using the thing that was passed into it. The implementation would therefore like to be:

  (def fix (f) ;aka Y
    (fn (i)
      ((f (fix f)) i)))
But, if we're doing the entire thing with anonymous recursion, we can (laboriously) implement fix like this:

  (= fix ;aka Y
     (fn (f)
       ((fn (g) (g g))
        (fn (g)
          (fn (i)
            ((f (g g)) i))))))
Every recursion step involves creating multiple lambdas. Eek. (It's even worse if you use the general, n-argument Y combinator, in which case you must use "apply" and create lists.) Whereas with hjek's non-curried approach, only a constant number of lambdas have to be created at runtime. (Optimizing compilers might be able to cut it down to 0.)

If you want to create a macro like afn or rfn, and want the user to be able to act like the function is named F and accepts just the parameter i, you can put a wrapper into the macroexpansion, like this:

  (rfn F (i)
    (aif i!parent
         (+ 1 (F (item it)))
         0))
  ->
  (fn (i)
    ((fn (f)
       (f f i))
     (fn (f i)
       (let F (fn (i) (f f i))
         (aif i!parent
              (+ 1 (F (item it)))
              0)))))
And in this case, while the code does call for creating an F-lambda on every recursive call, I think it's easier for the compiler to eliminate it—I don't remember whether I'd gotten Racket to do it. (I think it probably did eliminate it when working with Racket code, but Arc, which generates all the ar-funcall expressions, might not have allowed that.)

The actual code for rfn will create a variable and then modify it, creating a lexical environment with a cycle in it. That's certainly a more straightforward approach. I figure the above is useful only if you're working in a context where you really want to avoid mutation or true cycles. (For example, I am considering a system that detects macros whose expansion is completely side-effect-free. It might be easier to use the above approach to defining iteration than to teach the system that rfn is "close enough" to being side-effect-free.)

reply

2 points by hjek 42 days ago | link

I've been using `aif` and `awhen` a lot but didn't know about `afn`. Thanks!

Is it the Y combinator? I didn't get that far in The Little Schemer yet, but I'll have to check.

reply

2 points by akkartik 42 days ago | link

Doesn't look quite like it, but close.

reply

2 points by akkartik 47 days ago | link | parent | on: Writing a sane list macro

That seems pretty much unhacky :)

reply


Check out John Shutt's thesis on Kernel: https://web.cs.wpi.edu/~jshutt/kernel.html. He takes your observation that quasiquotation is pointless to the limit, dropping it entirely and constructing his macros out of cons and friends.

reply

2 points by waterhouse 41 days ago | link

Thanks for the pointer. He has an entire chapter on hygiene... And then there is this:

"There is no need to provide quotation here because, having failed to enforce the prohibition against embedding combiners in a macro expansion, we don’t need to embed their unevaluated names in the expansion."

It's nice that his primitive $vau grabs the current lexenv, as this enables another kind of macro-like behavior I've thought might be useful: taking subexpressions, fully macroexpanding them, and doing something with the result (e.g. determining whether a variable is ever used, and omitting a computation if not). I don't know how that would mesh with the interpreter's semantics, though...

I'll probably have to read and brood on this further.

reply

3 points by akkartik 56 days ago | link | parent | on: About lobste.rs

I'll invite you.

reply

3 points by zck 56 days ago | link

Thanks! I'll spend some time checking it out. :)

reply

2 points by akkartik 58 days ago | link | parent | on: Self-hosting the Anarki community

Hmm, I wonder if it stays banned until they restart the server. I'd ping hn@ycombinator.com.

reply

2 points by hjek 58 days ago | link

Looks like banned IPs are written to the disk even:

    (def set-ip-ban (user ip yesno (o info))
      (= (banned-ips* ip) (and yesno (list user (seconds) info)))
      (todisk banned-ips*))

reply

4 points by akkartik 66 days ago | link | parent | on: With and withs

I actually tend to the opposite: use with everywhere unless I need withs. The reason isn't performance. It tends to make code more elegant to not rely on the order in which things are defined. And when I'm reading code, with gives me the warm fuzzies that the code is going to be cleaner. When I see withs I slow down to look at the dependencies between the bindings.

-----

3 points by zck 66 days ago | link

Similarly to this, when I'm writing Java, I use `final`^1 everywhere I can. It's nice to be able to know that anywhere later where the variable declared final is in scope, it will have the same value as at the point it's set. I don't need to look through any code to see if it's rebound; I know it hasn't been.

[1] "final" is kind of like "const", if I understand `const` right. `final int x = 3;` means that it is an error to later have the line of code `x = 4;`.

-----

3 points by prestonbriggs 66 days ago | link

OK, I get it, thanks. In scheme, I would use letrec for this situation; my intuition for Arc isn't very well developed.

-----

3 points by akkartik 66 days ago | link | parent | on: With and withs

Try running without your code.

    arc> (help get)
    [fn]  (get i)
    Returns a function to pass 'i' to its input.
    Useful in higher-order functions, or to index into lists, strings, tables, etc.
    Examples:
      arc> (get.2 '(1 2 3 4))
      3
      arc> (get!b (obj a 10 b 20))
      20
      arc> (get.9 sqrt)
      3
      arc> (map get.2
                '((a b c)
                  (1 2 3)
                  (p q r)))
      (c 3 r)
    nil

    arc> (help set)
    [mac] (set . args)
    Sets each place in 'args' to t.
    nil
These are the functions you end up calling because your dispatch can't see the earlier get and set bindings.

-----

More