Nullable pattern: Experience report

For lesser mortals, such as myself, writing tests is a necessary
evil. No matter how much coffee nectar of the gods I
drink, I still, occassionaly, in weak moments… make
mistakes. This is our secret, so don’t tell anyone.
At work we have been experimenting with nullables for testing, and after half a year I thought it prudent to write a bit about it.
TLDR: That’s really long!
Yep, it’s a long article. I still recommend you read it. It has a lot of really valuable insights.
Core idea: Write your code in such a way that you can test your code even against infrastructure code that will be running in prod.
Setup
We have a couple of part of our system that would fit into the shell part of FCIS.
datomic
We don’t really have to deal with datomic, as we can run the in-memory database for our tests and treat it as a nullable.
postgres
We currently use postgres as a document database. We are not yet there in our rewrite that we have had a need to touch this. A nullable would probably end up as an atom, since it’s basically a one to one match with a document database.
HTTP calls
Next one on the chopping block. Haven’t thought much about how to write this one.
The browser
We went nuclear on the browser. It’s by far our biggest piece of infrastructure we have, and the one that is the most painful to test. At one point or another, you will still have to run your code in the browser, so that will never completely disappear, but you can realistically reduce the need with about hand wavy number%.
The code
This is our js-env file, where we have protocols for our browser
environment.
(defprotocol IOiikuJsEnv
:extend-via-metadata true
;; AG Grid
(create-ag-grid [js-env node grid-options interop-options]
"Create an AG Grid instance and register it in the datagrids registry.
grid-options must contain :datagrid-id, under which the instance will be
registered.")
(get-ag-grid-theme [js-env])
(get-ag-grid [js-env datagrid-id])
(focus-and-move-down [js-env datagrid-id]
"Move focus down in the given AG Grid instance")
(focus-and-move-up [js-env datagrid-id]
"Move focus up in the given AG Grid instance")
;; DOM API
(document [js-env]
"Get the Document object, i.e js/document")
(custom-event [js-env event-type detail] [_ event-type detail bubbles?])
(alert [js-env msg] [js-env msg data]
"Call js/alert with the message and, optionally, data to be printed")
(set-timeout [js-env f timeout-ms])
(clear-timeout [js-env timeout-id]))
(defprotocol INodeSelector
"Methods on the NodeSelector interface.
https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Selection_and_traversal_on_the_DOM_tree"
:extend-via-metadata true
(query-selector [node-selector selectors]
"document.querySelector
https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelector")
(query-selector! [node-selector selectors]
"Like query-selector, but throws when no Element is found"))
(defprotocol IEvent
:extend-via-metadata true
(event-detail [js-event])
(event-target-value [js-event])
(event-target-boolean [js-event])
(event-stop-propagation [js-event] "Stop the event from propagating")
(event-prevent-default [js-event] "Prevent the default action")
(event-target-input? [js-event] "Is the event target some sort of event input (text, email, tel, textarea, etc)")
(event-get-modifiers [js-event] "Get the modifier keys of the event target")
(event-get-key [js-event] "Get key of the event target")
(event-get-code [js-event] "Get code (physical mapping of the key) of the event target"))
(defprotocol IEventTarget
"Methods on the EventTarget interface.
https://developer.mozilla.org/en-US/docs/Web/API/EventTarget"
:extend-via-metadata true
(add-event-listener
[target event-type f]
[target event-type f options])
(remove-event-listener
[target event-type f]
[target event-type f options])
(dispatch-event
[target event]))
(defprotocol IHtmlElement
:extend-via-metadata true
(closest [element selectors]
"https://developer.mozilla.org/en-US/docs/Web/API/Element/closest")
(focus [element]
"https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/focus")
(select [element]
;; While this method is defined for HTMLInputElement in the spec, we can get
;; HTMLElement nodes that implement the .select method from Web Awesome.
"https://developer.mozilla.org/en-US/docs/Web/API/HTMLInputElement/select")
(get-attribute [element attribute]
"https://developer.mozilla.org/en-US/docs/Web/API/Element/getAttribute")
(set-attribute [element attribute value]
"https://developer.mozilla.org/en-US/docs/Web/API/Element/setAttribute"))
(defprotocol IJsObject
"Methods on JavaScript objects"
:extend-via-metadata true
(object-set! [obj k v]
"Sets the k property on obj to v")
(object-get [obj k]
"Gets the k property on obj"))
(defprotocol IWebAwesome
"Methods on WebAwesome objects"
:extend-via-metadata true
(wa-hide [obj]
"Runs .hide on the obj"))
(defn register-datagrid!
"Register the given AG Grid in the datagrid registry"
[registry ag-grid]
(swap! registry assoc (ag-grid/get-grid-id ag-grid) ag-grid))
(defn unregister-datagrid!
[registry ag-grid]
(swap! registry dissoc (ag-grid/get-grid-id ag-grid)))
(defn get-ag-grid!
[registry datagrid-id]
(if-let [datagrid (get @registry datagrid-id)]
datagrid
(throw
(ex-info (str "No such datagrid in registry: '" datagrid-id "'") {}))))
(defn focus-and-move-down*
"Doesn't actually move down, but just to the first row. Moving down from the
previous position is TBD."
[registry datagrid-id]
(let [ag-grid (get-ag-grid! registry datagrid-id)]
(ag-grid/set-focused-cell ag-grid 0 ag-grid/SELECTION-COLUMN-ID)))
(defn focus-and-move-up*
"Doesn't actually move up, but just to the last row. Moving up from the
previous position is TBD."
[registry datagrid-id]
(let [ag-grid (get-ag-grid! registry datagrid-id)
last-idx (dec (ag-grid/get-row-count ag-grid))]
(when (<= 0 last-idx)
(ag-grid/set-focused-cell ag-grid last-idx ag-grid/SELECTION-COLUMN-ID))))
#?(:cljs
(do
(defn create-oiiku-js-env
[]
(let [datagrid-registry (atom {})]
(reify IOiikuJsEnv
(create-ag-grid [_ node grid-options interop-options]
;; When using AG Grid, we sometimes need to know about AG Grid from
;; the outside. This causes a circular dependency, where js-env
;; needs to know about the ag grids that have been created, and
;; AG Grid needs to be able to tell js-env that it no longer exists
;; when destroyed. We have two options that are realistic:
;; 1. Let the datagrid widget be responsible for the cleanup, which
;; means that ag grid doesn't need to know about js env
;; 2. Let the AG Grid be responsible for the cleanup via a callback,
;; which means that AG Grid now need to know about js env in some
;; capacity, but the datagrid widget is now unaware of the coupling
;; We've gone with option 2, as option 1 complects things even
;; further by introducing a third component into the mix: the widget
;; With option 2 we keep it as an implementation detail that doesn't
;; leak to the rest of the app
(let [on-destroy (partial unregister-datagrid! datagrid-registry)
ag-grid (ag-grid/create-ag-grid
node
grid-options
(assoc interop-options :on-destroy on-destroy))]
(register-datagrid! datagrid-registry ag-grid)
ag-grid))
(get-ag-grid-theme [_]
(ag-grid/get-theme))
(get-ag-grid [_ datagrid-id]
(get @datagrid-registry datagrid-id))
(focus-and-move-down [_ datagrid-id]
(focus-and-move-down* datagrid-registry datagrid-id))
(focus-and-move-up [_ datagrid-id]
(focus-and-move-up* datagrid-registry datagrid-id))
(document [_] js/document)
(custom-event
[_ event-type detail]
(js/CustomEvent. (name event-type) #js {:detail detail}))
(custom-event
[_ event-type detail bubble?]
(js/CustomEvent. (name event-type) #js {:detail detail
:bubbles bubble?}))
(set-timeout [_ f timeout]
(js/setTimeout f timeout))
(clear-timeout [_ timeout-id]
(js/clearTimeout timeout-id))
(alert [_ msg]
(js/alert msg))
(alert [_ msg data]
(js/alert (str msg "\nData:\n" (pr-str data)))))))
(extend-type js/Document
INodeSelector
(query-selector [^js document selectors]
(.querySelector document selectors))
(query-selector! [^js document selectors]
(let [el (.querySelector document selectors)]
(assert el (str "No element matches '" selectors "'"))
el)))
(extend-type js/EventTarget
IEventTarget
(add-event-listener
([^js dom-node event-type f]
(.addEventListener dom-node event-type f))
([^js dom-node event-type f options]
(.addEventListener dom-node event-type f (clj->js options))))
(dispatch-event [^js dom-node event]
(.dispatchEvent dom-node event)))
(extend-type js/HTMLElement
IHtmlElement
(closest [^js element selectors]
(.closest element selectors))
(focus [^js element]
(.focus element))
(select [^js element]
(.select element))
(get-attribute [^js element attribute]
(.getAttribute element attribute))
(set-attribute [^js element attribute value]
(.setAttribute element attribute value))
INodeSelector
(query-selector [^js el selectors]
(.querySelector el selectors))
(query-selector! [^js el selectors]
(let [el (.querySelector el selectors)]
(assert el (str "No element matches '" selectors "'"))
el))
IWebAwesome
(wa-hide [^js obj]
(.hide obj)))
(extend-type js/Event
IEvent
(event-detail [^js event]
(.-detail event))
(event-target-value [^js event]
(some-> event .-target .-value))
(event-target-boolean [^js event]
(boolean (some-> event .-target .-checked)))
(event-stop-propagation [^js event]
(.stopPropagation event))
(event-prevent-default [^js event]
(.preventDefault event))
(event-target-input? [^js event]
(let [target (.-target event)]
(or (#{"INPUT" "TEXTAREA" "WA-INPUT" "WA-TEXTAREA"}
(.-tagName target))
(.-isContentEditable target))))
(event-get-modifiers [^js event]
(->> [(when (-> event .-shiftKey)
:shift)
(when (-> event .-altKey)
:alt)
(when (-> event .-ctrlKey)
:ctrl)
(when (-> event .-metaKey)
:meta)] (remove nil?) (set)))
(event-get-key [^js event]
(some-> event .-key))
(event-get-code [^js event]
(some-> event .-code)))
(extend-type object
IJsObject
(object-set! [^js obj k v]
(object/set obj (name k) v))
(object-get [^js obj k]
(object/get obj k)))))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Nullables
#?(:clj
(do
(import '(clojure.lang IDeref))
(defn merge-with-meta [& maps]
(with-meta
(apply merge maps)
(apply merge (map meta maps))))
(defn ->null-event-target
"Returns a map that implements IEventTarget via metadata.
Output Tracking:
::events - events dispatched
::event-listeners - event listeners registered and removed"
[initial]
(with-meta
{::event-listeners (atom (::event-listeners initial {}))
::events (atom (::events initial []))}
{`add-event-listener
(fn
([this event-type f]
(swap! (::event-listeners this) assoc [event-type f] f)
nil)
([this event-type f options]
(swap! (::event-listeners this) assoc [event-type f options] f)
nil))
`remove-event-listener
(fn
([this event-type f]
(swap! (::event-listeners this) dissoc [event-type f]))
([this event-type f options]
(swap! (::event-listeners this) dissoc [event-type f options])))
`dispatch-event
(fn [this event]
(swap! (::events this) conj event))}))
(defn ->null-node-selector
"Returns a map that implements INodeSelector via metadata.
Configurable Responses:
::elements - a map from selectors (String) to elements"
[initial]
(with-meta
{::elements (::elements initial)}
{`query-selector
(fn [this selectors]
(get (::elements this) selectors))
`query-selector!
(fn [this selectors]
(let [element (query-selector this selectors)]
(assert element (str "No element matches '" selectors "'"))
element))}))
(defn ->null-js-object
"Returns a map that implements IJsObject via metadata.
Output Tracking/Configurable Responses:
::properties - a map with properties"
[initial]
(with-meta
{::properties (atom (::properties initial))}
{`object-set!
(fn [this k v]
(swap! (::properties this) assoc k v))
`object-get
(fn [this k]
(get @(::properties this) k))}))
(defn ->null-document
[& [initial]]
(merge-with-meta
(->null-node-selector initial)
(->null-event-target initial)))
(defn ->null-js-env [& [initial]]
(with-meta
{:oiiku/js-env true
::document (::document initial (->null-document initial))
::timeouts (atom (::timeouts initial {}))
::datagrid-registry (atom (::datagrid-registry initial {}))}
{`create-ag-grid
(fn [js-env node grid-options interop-options]
(let [ag-grid (ag-grid/->null-ag-grid
node grid-options interop-options)]
(register-datagrid! (::datagrid-registry js-env) ag-grid)
ag-grid))
`get-ag-grid
(fn [js-env datagrid-id]
(get @(::datagrid-registry js-env) datagrid-id))
`get-ag-grid-theme
(fn [_]
::the-theme)
`focus-and-move-down
(fn [js-env datagrid-id]
(focus-and-move-down* (::datagrid-registry js-env) datagrid-id))
`focus-and-move-up
(fn [js-env datagrid-id]
(focus-and-move-up* (::datagrid-registry js-env) datagrid-id))
`document
(fn [js-env] (::document js-env))
`custom-event
(fn
([_ event-type detail]
{:event-type (name event-type)
:detail detail})
([_ event-type detail bubbles?]
{:event-type (name event-type)
:detail detail
:bubbles bubbles?}))
`set-timeout
(fn [js-env f timeout-ms]
#_:oiiku/todo #_"Make timeout-id configurable. It could just be a constant number, we don't need fancy"
(let [timeout-id (rand-int Integer/MAX_VALUE)]
(swap! (::timeouts js-env) assoc timeout-id [f timeout-ms])
(when (fn? f)
(f))
timeout-id))
`clear-timeout
(fn [js-env timeout-id]
(swap! (::timeouts js-env) dissoc timeout-id)
nil)}))
(defn create-oiiku-js-env
[]
(->null-js-env))
(defn ->null-node
[& [initial]]
(merge-with-meta
(->null-event-target initial)
(->null-node-selector initial)))
(defn ->null-html-element
[& [initial]]
(merge-with-meta
(->null-node-selector initial)
(->null-event-target initial)
(->null-js-object initial)
;; IHtmlElement
(with-meta
{;; Output Tracking
::attributes (atom (::attributes initial))
::focus (atom nil)
::select (atom nil)}
{`closest
(fn [this selectors]
(get-in this [::elements selectors]))
`focus
(fn [this] (reset! (::focus this) true))
;; HTMLInputElement
`select
(fn [this] (reset! (::select this) true))
`get-attribute
(fn [this attribute]
(get @(::attributes this) attribute))
`set-attribute
(fn [this attribute value]
(swap! (::attributes this) assoc attribute value))})
;; IWebAwesome
(with-meta
{::wa-hide (atom nil)}
{`wa-hide
(fn [this] (reset! (::wa-hide this) true))})))
(defn ->null-event
[event]
(let [store (atom event)]
(reify
IEvent
(event-detail [_] (:detail @store))
(event-target-value [_] (get-in @store [:target :value]))
(event-target-boolean [_] (get-in @store [:target :checked]))
(event-stop-propagation [_]
(swap! store assoc :propagation-stopped? true))
(event-prevent-default [_]
(swap! store assoc :default-prevented? true))
(event-target-input? [_] (get-in @store [:target :input?]))
(event-get-modifiers [_] (get-in @store [:target :modifiers]))
(event-get-key [_] (get-in @store [:target :key]))
(event-get-code [_] (get-in @store [:target :code]))
IDeref
(deref [_] (deref store)))))))We capture all frontend js infrastructure/mutating code in this
namespace. By doing so, and by using js-env in our frontend code, we
can write tests that test the entire chain of events, including the
parts that are tied to the browser. Since we also write all our code
as .cljc, we can run these tests in the backend with all the normal
testing frameworks that exist for Clojure.
Results
The tests run significantly faster. On the new monolith(ish) service that we are migrating to we are up to 455 tests. On my machine, those tests runs in less than a second. In comparison, the largest of the old microservices, runs trough all its 167 tests in 155.23s on my machine.
That’s a whooping 23 811% increase where we have more test coverage, more thorough testing and testing of the system that was previously not covered by automated tests.

What the future holds
We are still experimenting, but one PoC we wrote is very promising. Since we use the FCIS pattern and our chosen stack is replicant + nexus for the frontend and pure functions + datomic for the backend, AND… we use the nullable pattern, it’s possible to write tests that test end to end, without having the entire system up and running. All the logic and the complect points tested in the backend, in milliseconds.
What about the infrastructure?
Can we forgoe it completely? I don’t think so, but we have reduced the surface we need to test with actual infrastructure to a fraction of what it used to be, and the time to testing all the code touching the infrastructure to a fraction of a fraction.
For the parts of our infrastructure we cannot handle via automated tests we have manual scripts that we follow. This, combined with fallback plans if something were to go wrong, gives us a high degree of confidence when doing upgrades. As a bonus the manual scripts can be used for manual testing and/or webdriver tests.
Final thoughts
The compounding effect of so many tests are starting to pay rather large dividends. Since they are so cheap to run, and cheap (ish) to write, my prediction is
