Update: As of mid-2022, Clojurescript has fixed the problems described in this post.
Why having clojurescript in the classpath may lead to unexpected behaviour
The clojurescript maven artifact lists compile dependencies which include: data.json, tools.reader and transit-clj and transit-java.
However the clojurescript jar itself is something like an uberjar: It includes compiled data.json, tools.reader and transit-clj and transit-java namespaces inside itself. That means that although it declares dependencies on those libraries, when you use Clojurescript yourself, those libraries' artifacts are not used at all (despite being on your disk and in the classpath).
The output of this command shows the dependencies I am referring to:
clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.11.4" } }}' -Stree
The effect is that if you want to use a different version of one of those libraries compared to the one Clojurescript was compiled with, you can't. Well, you can't with the regular clojurescript artifact, you can use the non-aot version: org.clojure/clojurescript$slim {:mvn/version “1.11.4”}.
This was not an issue for a long time because those libraries didn't change. Now e.g. clojure.data.json has changed, hence why I hit the problem.
A handy technique I usually use to answer the question 'where on the classpath is namespace x.y.z getting loaded from?' is to call io/resource on it. Doing that with data.json and Clojurescript in the classpath gives result as follows:
(clojure.java.io/resource "clojure/data/json.clj")
=> ".../.m2/repository/org/clojure/data.json/2.4.0/data.json-2.4.0.jar!/clojure/data/json.clj"]
which is the wrong answer! I still don't understand why io/resource shows the file there, whereas calling require on that ns returns the one embedded in Clojurescript.
One might ask why I would be using Clojurescript and clojure.data.json together in the same jvm. Well, in my case, in development I tend to have my server and client dependencies combined, so I run cljs compile and server side stuff in one vm. When deploying and testing, I separate them (meaning Clojurescript jar is only on the classpath when cljs source is being compiled). It is possible to run separate server and cljs jvm's locally of course, but that then means I can't have a single .nrepl.edn file for example. There could be other reasons for using these 2 together though, writing data-reader functions that use json possibly.
I raised this on clojure slack and now Clojurescript's maintainers are aware, so hopefully this gets fixed.
A fix is likely to involve shading. This is where a library wants to use a fixed version of another library, so it copies the sources of that library into itself, but changes the namespaces/packages of the source library to be something different, and specific to itself.
My thanks go to Alex Miller for explaining this.
Discuss this post here.
Published: 2022-02-09
Tagged: clojure
Combining Java's promises with Clojure's laziness yields some interesting behaviour
Consider:
(doseq [x (range 1000000)])
Since range returns a lazy sequence and doseq does not retain the head of the sequence, there will only be one element of the sequence realized at every step of the doseq.
Now let's split the creation and consumption of the lazy sequence over chained promises. I am using the promesa library, which on the jvm is a thin wrapper over java.util.concurrent#CompletableFuture.
(-> (p/resolved (range 1000000))
(p/then (fn [xs]
(doseq [x xs]))))
It looks like it should be just as lazy. Nowhere is the code above retaining a reference to the head of the sequence - and yet it is!
The then promise internally has a reference to the preceding promise and that promise has a reference to its result - the head of the sequence. When the first promise returns, the sequence is unrealized, but as the subsequent then promise consumes the sequence it is realized and the head retained by the preceding promise!
What happens if there is a longer chain of promises? A promise executing in a chain only has reference to the preceding one. The preceding one has lost its reference to the next one upstream of itself, so in a chain just the current and immediately preceding one are not gc-able.
So? Well imagine you are streaming results out of a db for example - that might be modelled as a lazy seq, which is consumed through e.g. doseq and written out to a stream. Sounds like a nice memory-friendly solution, but if the db request results in a promise it might seem natural to keep chaining that result on.
Does this apply to
I haven't investigated.
Related to this topic is Stuart Sierra's Lazy Effects post, in which he says never mix laziness and side-effects. Good advice I would say.
Discuss this post here.
Published: 2022-02-03
Tagged: clojure
Tick provides a powerful, cross-platform date-time API way beyond what java.time offers. It is implemented on top of cljc.java-time which again is cross-platform as has exactly the same API as java.time.
For years now, the API has been alpha, by which we mean "Ready to use with the caveat that the API might still undergo minor changes". With the current release, the API of tick has been split into
tick.core namespace, which will have no breaking changes in future releasestick.alpha.interval namespace, which contains just the functions pertaining to Allen's intervalcalculus. As its name suggests, this is still alpha. Apologies if the title of this post is misleading, I didn't want to stuff it with caveats.There are plans to revisit the interval functions and documentation and some changes to the API may arise from that work.
If you are upgrading from an earlier (0.4* version), here is what needs to change:
tick.alpha.api requires with tick.core+ or - functions, note that these now only work where the arguments are allPeriods or all Durations. To move a time by an amount, use >> or << instead, keeping the arguments the same. This change was made to make +/- analagous to clojure.core's +/-, where the arguments are commutative andassociative. It also means there is now just one way to 'move' a time in Tick.parse function has now been removed from the API. Instead, choose the appropriate function for theformat of the string you need to parse, e.g. (t/date "2020-02-02"). The parse function was slow by definition, as it tried to find an appropriate entity matching the string it was given. It also meant that iffor some reason the string you passed it was not in the format you expected, it might be parsed intoa different date-time entity than you expected, which is never going to be good.tick.alpha.interval will address those.That's it.
Making breaking changes may be frowned on in the Clojure community, but they seem entirely reasonable to me in this case because:
alpha warning.end-user kind of library: I doubt any other libsdepend on it, so dependency hell is not an issue.One last thing, the current version is RC, release candidate. IOW please kick the tyres and let us know of any problems. We have already been using it in production for a while, with no issues. After a period of a couple of months, we'll remove the RC label.
Discuss this post here.
Published: 2021-09-28
Tagged: clojure