In this post I am going to look at using the importmap feature (supported by all modern browsers), as an alternative way for Clojurescript apps to access npm dependencies.
When a Clojurescript app depends on a regular JS library, such as React for example, then it is typical to:
There are different ways this can happen, for example:
:target :bundle with a bundler such as webpackI am a fan of shadow-cljs and so would typically use the second option. What this actually does is :simple optimizations on 3rd-party code, which means Google Closure code is going to read 3rd-party libs when the app is being built. Sometimes though, Closure cannot understand the 3rd party code, for example because it doesnt have support for Class fields. In this really interesting talk from Alex Davis, he says he is seeing more and more popular JS libraries that Closure can't handle. I've only had the issue once myself and thankfully was able to configure shadow to use a different file than the problem one.
So, what to do?
Sticking with shadow but using it with a different provider (e.g. webpack) is an option, but for browser apps there is another interesting option to consider: get pre-processed 3rd party libraries directly in the browser via a script tag (e.g. from a CDN such as unpkg).
I used a version of this approach for my experiments that called the deja-fu library's rationale into question. There, I just had a couple of script tags for 3rd party libs, followed by a script tag getting the application code. This meant script tags were order-sensitive and would not scale well because transitive dependencies would not be fetched automatically. Still, for a simple app it worked fine.
Recently though, all browsers have got support for importmap, which is best explained by example:
<script type="importmap">
{
"imports": {
"react": "https://esm.sh/react@18.2.0",
"react-dom/client": "https://esm.sh/react-dom@18.2.0",
"@tanstack/react-router": "https://esm.sh/@tanstack/react-router",
"my-demo-app": "/cljs-importmap-demo/cljs-out/main.js"
}
}
</script>
<script type="module">
import start from "my-demo-app";
start();
</script>
The imports map is a bit like package.json dependencies - it says what libraries are needed and details of how to get them - all must arrive as ES6 modules. Transitive dependencies are also retrieved. After the importmap, the script tag with type=module says to interpret the code within as an ES6 module. Here, it just imports the application code and starts it.
The module my-demo-app just contains the application code, not any 3rd party libraries. To generate the module from clojuresript code is just a matter of using Shadow-cljs documented options for that, for example:
{:target :esm
:js-options {:js-provider :import}
:modules {:main {:exports {'default 'com.widdindustries.demo-app.app/init}}}}
Here is an example app demonstrating this technique and here is the source code for that.
This is the first time I've tried using importmap. I failed to google any experience reports of anyone using it from Clojurescript, hence this post. Here are some pros and cons I am aware of so far:
Using importmap, 3rd party libraries can be cached. The application code will also be cached, but is likely to change at a faster rate than library versions get changed, so will be downloaded more frequently, but will be smaller than the tradition bundled version.
The importmap is specified in the html file, but will also need to be specified again for a page that loads tests for example. Also, it may be required to use a dev-time version of a library locally, but deploy with the optimized one. For example, React performance profiling tools only work with the dev-time React version. It is possible to conditionally create the importmap, for example, if on localhost, create one map, if deployed a different one.
The 3rd party libraries are being retrieved wholesale - ie no dead code elimination could happen here. Is significant dead code elimination a thing in JS-land these days though? I've heard of Rollup, but I haven't tried out how it would help trim down React and the like.
importmap is a relatively recent addition to browsers - so might not be suitable for some potential users.
Loading speed vs bundled apps, aka time-to-interactive (TTI)? I haven't measured anything yet. Please comment if you have experience of this.
Any more you'd add? Please use the link below to discuss.
Published: 2023-11-08
Tagged: clojure
In this post I am going to look at applying my regular dev setup to a project with a lot of code and tests that take a very long time to run.
Firstly, here are my Clojurescript dev setup requirements:
Anything you'd add? If so please see link at the bottom for how to discuss
I should also say that I'm generally developing in multi-person teams building SPA's with considerable business complexity. However, the list is still the same even for my own open source or hobby projects, but in some situations not everything above is a must-have. For example, if you only have a small number of fast-running tests then the items above about running subsets of tests will not be so important.
I recently started to do some work on a project with around 800 Clojurescript (browser) tests - which in itself may not sound massive, but there are a number of slower DOM-clicking tests, so total test-run time above 20 minutes. It goes without saying that one would not want to run all the tests in one go - but this fact meant that builds and tests had been split apart and this had led to quite a bit of complexity in the build setup: Local dev was done with figwheel+webpack, configured with multiple extra mains, and shadow+karma in the CI environment.
With the existing build setup, on saving a file you could be waiting 10+ seconds to see the incremental build finish - ouch! To run a single test required knowing which figwheel 'extra-main' the test would be compiled into and loading the auto-test browser page for that, and then doing some other steps I'm not even going to get into... all in all, not ideal.
So... what to do? My preferred Clojurescript build and test setup for the past couple of years has been shadow plus kaocha-cljs2. Everyone knows shadow of course, but kaocha-cljs2 seems weirdly unstarred (< 20) and un-discussed on the interwebs. The combination of these two gives me the above wishlist of course - that's why I chose it. But how well would it scale to the new megaproject? How easy would it be to set up?
Possibly one reason kaocha-cljs2 seems under-appreciated is that by design there are more moving parts compared to other Clojurescript test setups I have used - for example one needs an extra server (Funnel) for 2-way communication between jvm and js environments.
However, setup couldn't have been easier - and that's because I have a little ready-made shadow+kaocha_cljs2 template that I use in all my projects and libraries. I've called this template tiado-cljs2 and if you want to try it on your project, you'll have it up and running in minutes - see the README for instructions.
In the way I set up Shadow on this project, there are 4 'watches' going at once, one for the main app, one for tests and a couple of others for some miscellaneous apps. Shadow incrementally compiles just what is needed so if changing a test file, just the test build kicks in. So compared to the old build, incremental compile is often around 5-10x faster.
When it comes to the tests, I have used kaocha suites to split the tests based on ns-patterns - which in CI can be run concurrently. Kaocha-cljs2 doesn't support 'ns-patterns' out of the box yet (as kaocha does for clojure tests) - but luckily kaocha supports user-defined hooks so adding it was not difficult.
With these mega test-suites, the default timeout was not enough. User-defined timeouts are not currently respected by kaocha-cljs2 - so a little monkey patching was needed.
As well as running individual suites, the holy grail of clojurescript testing is surely having an IDE hotkey to run the test under the caret. This is achieved with a simple macro invoking (kaocha.repl/run xxx) - a macro was required (rather than a function) so that it can be invoked when either in a cljs or clj repl.
I could have had multiple test watches instead of one - each watching a subset of tests (this is a shadow feature). However the single one works well enough and makes life easier for developers as there is just a single place for tests to run.
So, whilst compilation and test-running times still seem a bit bigger compared to what I'm used to, the whole thing now feels far more manageable. There may be scope for modularization of the app - I don't know yet, but I'm much happier to investigate that and experiment with a solid, speedier build+test setup.
The fact that all this works as well as it does is thanks to the shadow and kaocha maintainers of course. Clojurescript would not be in such a good place without them!
Published: 2023-08-17
Tagged: clojure
This post looks at the meaning of the #inst reader literal from Extensible Data Notation (hereafter referred to as 'edn'), how it behaves by default in Clojure(script) and when it might not be sufficient for representing date/time information.
The majority of the content of this post comes from the Rationale section of time-literals, a Clojure(Script) library which provides tagged literals for java.time objects.
Support for edn and its Reader Literals were a headline addition in Clojure 1.4 and with that came built-in support for the #inst tag. The #inst tag is a part of the edn spec, where it is defined as representing an instant in time, which means a point in time relative to UTC that is given to (at least) millisecond precision. The format of #inst is RFC3339, which is like ISO8601 but slightly wider.
In Clojure(script), #inst is read as a legacy platform Date object by default, but as is made clear by the edn spec and by this talk from Rich Hickey the default implementation is just that: #inst may be read to whatever internal representation is useful to a consuming program. For example a program running on the jvm could read #inst tags to java.time.Instant (or java.time.OffsetDateTime if wanting to preserve the UTC offset information). It seems to me unfortunate that Clojure(script) provided defaults for #inst because users may not realise it is 'just a default', but that's just my opinion. My guess is that Clojure is trying to be both simple and easy in this case.
Although edn readme doesn't say this explicitly, to avoid 'reinventing the wheel', when conveying data using edn format, built-in elements seem to me to be preferable to user defined elements. For example, if one wants to convey a map, {:a 1 :b 2} is preferred to #foo/map "[[:a 1] [:b 2]]" - unless of course one wanted to convey something additional about the map, ordering perhaps. Similarly, if conveying an instant in time use #inst.
There are two situations where reader literals are useful:
edn data between processesAlthough they have many similarities and overlap, Clojure allows these cases to be considered separately and for good reason, as explained below:
There are many kinds of things relating to date and time that are not an instant in time, so #inst would not be an appropriate way to tag them. For example the month of a particular year such as 'January 1990' or a calendar date such as 'the first of June, 3030'. There are no built-in edn tags for these but tags can be provided in the user space, as they are by the the time-literals library.
Note that the default Clojure reader behaviour is to accept partially specified instants, such as #inst "2020" (and read that to a Date with millisecond precision) - but this is specific to the Clojure implementation and not valid edn (ie not RFC3339).
Clojure provides two mechanisms for printing objects - abstract and concrete as this code printing the same object shows:
(let [h (java.util.HashMap.)]
{:abstract (pr-str h)
:concrete (binding [*print-dup* true]
(pr-str h))})
=> {:abstract "{}", :concrete "#=(java.util.HashMap. {})"}
The concrete representation is sometimes useful to know and also the string output can be passed back to the reader to recreate the same internal representation again, which is known as round-tripping.
The default readers and printers of platform date objects don't allow round-tripping, the reason for which is unknown.
This is relevant to the two java.time types which logically correspond to #inst (java.time.Instant and java.time.OffsetDateTime). The the time-literals library contains specific readers and printers for those objects so that they do round-trip.
When conveying these objects in edn format, they should be tagged as #inst (as per above argument about preferring built-in elements). To do that with time-literals, simply provide your own implementation of clojure.core/print-method for Instant and/or OffsetDateTime. With *print-dup* true, the concrete type will still be printed.
Consider this code from a Clojure namespace:
(ns foo.bar)
(def one-day #time/period "P1D")
(defn one-more-day [period]
(-> period (.plusDays 1)))
Now answer:
(one-more-day one-day) work?Go back and have a look if required, I will reveal the answer in the next sentence
The answer to 1. is maybe, ie only if *data-readers* contains a mapping for time/period AND the reader function is already loaded in the process. Just having a mapping in data_readers.cljc is not enough. Add a side-effecting require for that reader function you say? No thanks.
The answer to 2. is again maybe. If the mapping for time/period is set up AND the reader function returned a java.time.Period then it will work.
So tl;dr reader literals in code can be made to work but is not good practice IMHO. That goes for user-defined literals but also #inst and #uuid. Typing a few extra characters to call the actual constructor function directly is not so hard.
I don't mean I never have files with literals in them, but not in code I expect anybody else (incl. myself at a later date) to just be able to 'pick up and run'. If I'm flowing around in my own space then it's fine. If I get to 'crystalising stuff out' - e.g. to tests for CI, then I replace any literals.
Btw if you want to do a find/replace for #inst in your source files then clojure.instant/read-instant-date or cljs.reader/parse-timestamp are probably the functions you need ;-)
Published: 2023-06-14