by Clément Delafargue on March 4, 2024
The blog post was a pun on King Gizzard & the Lizard Wizard last record’s name, but they released yet another one while I was writing this post, and it seems they are already working on another one. I guess that says a lot both about me, and about them.
My journey with weird keyboards started many years ago, in 2009, with the acquisition of a typematrix 2030 and switching to bépo, an ergonomic keyboard layout designed for the French language.
I’ve used it for many years. I loved its orthogonal layout, and the fact that return and backspace were put under pointer fingers instead of straining my right pinky. At some point I stopped working with an external monitor and only worked on laptops, so I stopped using my typematrix. There were also issues with its skin (a kind of rubber layer that protects the keyboard), that would become kind of dirty, while still allowing they keyboard to get dirty as well.
Fast forward a couple years. I finally land a nice job working for a company based in Paris, and I have a proper office room at home. I wanted to try mechanical keyboards for some time, and I was particularly intrigued by ergodox keyboards: they are split in the middle, with two independent halves. This allows keeping the hands at shoulder width, tilting and tenting halves independently, etc. It also features thumb clusters: instead of using thumbs only for pressing space, they are now used for space, backspace, enter, esc, …. I look a bit at how to build one and get overwhelmed, so I decide to get a pre-built one from ergodox ez. I’m super happy with it, even though i don’t use thumb clusters to their full extent (they have 6 keys each, even though I only use 2), and it just takes a lot of place. Another issue I have with it is that it has unusual keycaps, so it’s hard to find alternative keycap sets.
Fast forward a couple years (again). I stumble upon the iris keyboard. I like a few things about it: way smaller thumb clusters with a better repartition, and it only uses 1u keycaps, so ortholinar keysets work with it. It’s also quite smaller than the ergodox, and works with usb c cables. I get an iris rev7 kit, which comes with hotswap sockets, no need to solder switches. I just solder a single rotary encoder on the top left.
Fast forward a couple months, I love my iris so much I wish I could bring it at work, but sadly it is still a bit too bulky. Then I remember those people talking about their fancy minimalist keyboards, amongst them, the ferris sweep. At first I’m not too interested because it has only 34 keys and I can’t even fit all letters in the bépo layout (namely m,z and w which are on the 6th column of the right hand). The idea still makes its way, because laptop keyboards start to make my wrists hurt. The ferris sweep is both small and low-profile thanks to the kailh choc switches, which makes it perfect for carrying it with me.
I finally buy a kit:
Assembling the kit was fun, soldering the switch sockets was ok. Soldering the controller sockets and the controllers themselves was a bit harder. Soldering the reset buttons was more challenging since they are very small and I was not experienced with SMD soldering.
I will detail further the layout I ended up with on the ferris. It’s not too different from my iris layout or my laptop keyboard layouts, which allows me to switch keyboards with minimal pain.
I used the ferris as my primary keyboard for a couple weeks after assembling it, not only because I wanted to break it in, but also because I hurt my wrists and the high-profile of the iris was less comfortable than the ultra-thin ferris. I have since gone back to using the iris as my primary keyboard because it is still more convenient and doesn’t require as much combos.
Typematrix will always have a special place in my heart as my first non-standard keyboard, but it is too narrow for me, is fully orthogonal and not just ortholinear, the skins get dirty over time, it does not use mech switches. I have kept mine as a souvenir, but I don’t plan on using it anymore.
The ergodox-ez was a great way to get started with split keyboards, but it is too big for my taste (and my hands), especially the thumb clusters. The specific keycaps also make it harder to customize. It has old micro controllers and is somewhat cumbersome to flash, even if the web interface makes editing layout easy.
The iris is wonderful keyboard. It is just the right size for me, especially the thumb keys. The rotary encoder for volume control is pure joy. Finally, it can be directly configured over USB via a web app. I love it.
Finally the ferris sweep is an uncompromising keyboard. I think fully adopting it allows to be the most efficient, but I did not want to give up on all my muscle memory, so I’m not using it to its full extent. That being said, it’s a great travel keyboard and I’m happy to use it when I’m not at home. I cannot directly configure it over USB, but I use qmk configurator to generate config files, compile them and then copy the firmware over USB mass storage. It’s a bit more convenient than what I had to do with the ergodox, where I needed to press a reset switch with a paperclip and then run a flashing utility.
You can stop reading here if you’re not interested in bépo layout minutia.
Once I had a working keyboard, it was time for tweaking the layout. Ferris offers only two 3x5 matrices, and 2x2 thumb keys. The rest is accesible through layers. While this works well for the qwerty layer where every letter is located on the 3x5 matrices, bépo makes full use of a regular keyboard, and moves some letters outside of it, in order to make space for commonly used diacritics like “é”.
I started from the default ferris layout, with home row mods. I kept the home row mod idea, but departed significantly from the rest, since my goal was to keep using other keyboards, I wanted to minimize the amount of changes to a regular keyboard.
Here is the layout file, even though it does not make much sense to use it directly.
My goal was to make sure every letter was on the main layer. That means moving m,z,w around, by removing other characters. My choice went to à,è,^. Since m is the most used of the three, I decided to keep it on the same hand, so it replaced !. Then z went to the è, and w to à.
Then, I used a dedicated layer for the top row symbols and numbers. On a regular keyboard, symbols are directly accessible and numbers are accessed via shift. What I did instead was to put symbols on the home row and numbers on the top row. This allowed me to keep muscle memory. Finally, I put extra symbols on the bottom row (à,$,ç,è,^,=,%).
In order to stay close to my other keyboards, I put meta and alt-gr on the main layer, on thumb keys. I use those two so much it made sense to give them direct access.
I kept layer 7 almost as is, with return on the home row, left pointer, esc on the top row, left pointer, tab on the left thumb. I kept the reset button on the bottom row, right pinky since I ended up flashing my keyboard very often. The only addition was volume and playback control on the left half.
Apart from that, not a lot of changes, I kept the navigation layer as is (it provides arrows). I don’t use mouse control on my keyboard, so I left the mouse layer unchanged.
by Clément Delafargue on July 17, 2023
Tagged as: .
Twitter has been an important part of my life, but I have decided to close my account. I remain active on mastodon: @clementd@framapiaf.org. I will close my account in a couple weeks, so that I can establish other communication means over DMs.
I have joined twitter more than 15 years ago. It would be hard to overstate how the people I have met there have helped me grow. I am mostly proud of the person I am now, and twitter has played an important role in that.
A few months ago, I have almost stopped going there and switched my social media presence to mastodon. After several attempts, the Musk acquisition helped me move my primary presence. Still, I kept my account open, mostly to stay in touch with people via DMs, but also to avoid having my handle stolen. Since my account was still active, I opened it from time to time, just to keep track of what was happening.
A few weeks ago, the police murdered Nahel Merzouk and I decided to use whatever remained of my twitter platform to amplify people protesting about police brutality. I still think it was an important thing to do, but it put and end to a multi-month twitter break and made me realize I had to actually close my account.
The next part is about French twitter, so I’ll switch to French. tl;dr: for the others: there are extremely unpleasant people in the french tech twitter bubble and I don’t want to interact with them more than required.
La semaine dernière, encore du harcèlement, encore par la même bande. Pas possible de me détacher de l’affaire cette fois-ci. Encore une fois, aucune conséquence pour les harceleurs, pas d’excuses. En bonus, des hypocrites se prédendant neutres qui tentent de faire taire toute critique pas assez policée. Je suis forcé de côtoyer ces gens pour le boulot, je n’ai aucune envie de m’infliger plus que ce qui est strictement nécessaire.
by Clément Delafargue on January 15, 2023
I started using vim full-time in 2009. Back then full-featured editors like emacs were tempting, but once hooked on modal editing, it was too late to switch.
When neovim came out, it wasn’t packaged for exherbo (the distro I was using at this time), so I never got to really play with it, even though it was a massive improvement.
At some point around 2017 or 2018, I tried kakoune.
Like vim, kakoune is a modal editor. Unlike vim, it strives to provide
feedback. The core tenet of kakoune’s model is to provide a selection
before acting on it (contrary to vim, where you provide the action, then the
selection). So instead of typing dw
to delete a word, you type wd
to select
a word, and then delete it. This seemingly simple change unlocks a powerful
property: the selection is highlighted as you define it, so if you want,
you can check it’s correct or perhaps amend it before acting on it. Of course
for small actions like wd
, you usually type it directly without thinking,
but when combined with incremental search and multi cursors, it becomes a
more powerful and less burdening way to use a modal editor. You still get
all the benefits like doing a lot of things without thinking or twisting your
fingers with 10-fingers chords, but you always have the option to get in a more
conscious feedback loop when you want.
Anyway long story short, I’ve used kakoune for 4 years or so, I’m sold on
its editing model. By choice, it does not provide a fully featured scripting
language like vim, so I ended up hacking a couple features on top of it with
rofi, installed kak-lsp
to get LSP
support, and run a patched kakoune to display trailing whitespace. It’s good,
but rofi does not work over SSH (I sometimes SSH over to my main rig from my
aging laptop), plugins are fragile and my patch never got merged upstream.
And then a couple months ago, I hear about helix, dubbed a post-modern editor. I’m super happy to see it’s inspired by kakoune, because I think the kakoune model is superior. I give it a try from time to time. The built- in fuzzy finder solves my main pain point with kakoune, and the rest is close enough to kakoune for me to not be too lost. I find it interesting and tell myself I should try it more seriously at some point. One remarkable thing: at this point, I have not tweaked its configuration at all. It just works okay out of the box.
And then, a week ago, I decide to try it more seriously, starting by reading the doc and seeing how I can make it better for me.
The first thing I did was to display whitespace (yeah, I’m this kind of guy), enable auto-format where I could, display rulers at 80 and 120 chars to gently nudge me away from writing long lines.
Then, making the cursor a bar in insert mode was a nice little improvement I found over the good old block. I try not to spend too much time in insert mode anyway.
Then, making the file picker display hidden files because I edit them just as much as normal files.
And finally, biting the bullet on having space
as the leader key (on kakoune,
I had ,
which is right next to my left pointer finger on a bépo layout) to
get quick access to :write
, :quit
and “go back to the last buffer”.
That’s it, the config file is just shy of 15 lines and I feel at home.
With that, I was able to make helix
my main driver, about a week ago. It
took a few tweaks over the course of two days. It has been my fastest
transition period for such a core tool. Switching to vim and then kakoune
full-time took me weeks.
# the default theme is nice, but is not contrasted enough for me
theme = "zenburn"
[editor]
rulers = [80,120]
auto-format = true
# no support yet for `trailing`, but there is an open PR
whitespace.render = "all"
[editor.cursor-shape]
insert = "bar"
[editor.file-picker]
hidden = false
[keys.normal.space]
"e" = ":write"
"q" = ":quit"
"space" = "goto_last_accessed_file"
With the basics set, it was time to look at bit more at other things that make helix helix.
Rust LSP features just work, as soon as you install rust-analyzer
(helix
helpfully tells you that when you run hx --health rust
).
So in any rust project, I can do all the cool LSP stuff:
<space>s
)<space>r
)gd
)<space>a
)Same for haskell, as long as you have haskell-language-server-wrapper
in
$PATH
.
Another strength of helix
is native integration of tree-sitter grammars.
Instead on relying on a pile of regexes for syntax highlighting and custom
plugins for syntax-aware features, helix
relies on actual grammars. Tree-
sitter provides support for incremental parsing, so parsing is super fast
during editing.
Once the document is parsed, the AST is used for coloring, but also for
movements (eg. Alt-o
selects the parent node in the tree, mif
selects the
current function body). Manipulating the syntax tree directly was something
that always impressed me when watching experienced lispers write code. Now I
can have that in any language.
Another benefit is that the cost of adding support for a language is amortized. The hard part is writing the tree-sitter grammar. But it has to be done only once (other editors like nvim support tree-sitter grammars trough plug-ins). In addition to that, there is tooling available that can transform an EBNF grammar into a tree-sitter grammar. So the hard part does not have to be hard.
helix
provides shortcuts for moving in a document (in the style of vim-
unimpaired). They allow to quickly move between LSP diagnostics, uncommitted
changes, functions.
Sadly, the default mapping is on [
and ]
, which are hidden under AltGr
on
a bépo layout. So I moved them on (
and )
, which are available directly.
[keys.normal."("]
"d" = "goto_prev_diag"
"D" = "goto_first_diag"
"g" = "goto_prev_change"
"G" = "goto_first_change"
"f" = "goto_prev_function"
"t" = "goto_prev_class"
"a" = "goto_prev_parameter"
"c" = "goto_prev_comment"
"T" = "goto_prev_test"
"p" = "goto_prev_paragraph"
"space" = "add_newline_above"
[keys.normal.")"]
"d" = "goto_next_diag"
"D" = "goto_last_diag"
"g" = "goto_next_change"
"G" = "goto_last_change"
"f" = "goto_next_function"
"t" = "goto_next_class"
"a" = "goto_next_parameter"
"c" = "goto_next_comment"
"T" = "goto_next_test"
"p" = "goto_next_paragraph"
"space" = "add_newline_below"
Like kakoune, helix has contextual menus that display available commands. So
when I type )
, the list of available commands is displayed. This works for
default and custom bindings. Neat!
So far, that is the only remapping I’ve done to accomodate bépo. Back on vim,
I had hjkl
bindings remapped to ctsr
in order to keep movement on the home
row, but the ripple effects and the added cognitive overhead were not worth it.
One big missing feature of helix was support for dhall. Fortunately, there was a tree-sitter grammar available. So it was just a matter of writing queries for syntax highlighting and text objects. Easy peasy. (I didn’t do indent queries for now, don’t hesitate to add them if you know how to do that).
The PR was reviewed and merged in under 3 hours. Neat!
Quite notably, all the features I’m waiting for in helix
are already in PRs
waiting for review.
Helix has a nice codebase, but since every change I’m looking for is already waiting in a PR, I don’t have any reason to hack on it. Maintainers on github and matrix have been nice and helpful, so I’m sticking around.
by Clément Delafargue on November 7, 2022
Finding creative ways to express yourself is not always easy when you’re all thumbs and you can’t keep the beat. Since I’m not entirely bad at programming, I gave generative art a try. I still get rubbish results, but I am having fun doing so. That’s success, as far as I am concerned.
I’ve been spamming my poor followers with ugly shapes and inscrutable code for a few days, so I tought it would be a good time to explain what I was doing.
tl;dr: I generate SVG through an esoteric stack-based language inspired by befunge. You can try it out online: https://befunsvge.cleverapps.io, but you might want read a bit more to be able to actually use it.
A link to the same program on a live playground
Pen plotters are tons of fun: instead of printing bitmaps, they move an arm with a little pen over a piece of paper. Parametric curves are a great fit for plotters, as they provide a super crisp output. Programatically generating SVG shapes is also relatively easy, so that’s an easy way to get started. Well I bought a shitty $100 plotter over ali express and never managed to get it to work, but I had fun nonetheless. I spent a fair bit of time drawing tesselated penrose triangles and I’m happy with the result, even though it’s still on a screen and not on a piece of paper.
After a bit, it became boring though, because while writing haskell is something I tremendously enjoy, it is also something I’m quite comfortable with now, so I lacked friction to guide my visual experimentations (don’t worry, Haskell is very good for abstract experimentations and provides me with ample head-scratching in this area, but that’s something different).
Befunge is an esoteric language created in 1993. It is stack-based,
but the fun part is that the execution model is based on a 2d
grid: the source code itself. Each character in the grid is
a command, and the execution pointer moves in a direction,
executing commands in sequence. Control flow is handled by
commands changing the pointer direction. Some are unconditional,
like <^v>
(they change the pointer direction), and some are
conditional, like |
or _
(they change the pointer direction,
based on the contents of the stack). Basic arithmetic is provided
(+-*/
), comparisons with `
. The whole list of
instructions fits
on a page, so it’s relatively easy to understand and memorize it.
Writing actual programs, on the other hand, that’s a bit more
challenging.
Here are a couple examples:
A hello world
"!dlroW ,olleH">:v
|,<
@
A factorial
&>:1-:v v *_$.@
^ _$>\:^
The original befunge (befunge-93
) was later expanded into a family
of languages (funge-98
),
which generalizes befunge over the number of dimensions, and adds
new features such as concurrency).
I played a bit with generative art first with Processing, then in pure JS and haskell, but sometimes the lack of constraints is a big blocker in itself. So I wondered if it would be fun to generate SVG from a purposefully constraining language, and befunge struck me as a good candidate. I don’t need a lot of abstraction to draw SVG, just math operations, and stack-based languages are fun.
So I wrote a befunge-93 implementation, enriched with a couple generative-art-oriented extensions. It’s available as a command-line tool, as well as an online playground. You can share links to the playground, program and configuration are stored in the URL.
0 >:"d"`#v_:5*0M:"d"\-5*0\Lv
1+^ >π@ >
The important part here is the loop:
0 >:"d"`#v_ v
1+^ > @ >
:"d"`
compares the counter to 100
(d
’s ascii code is
100
). _
conditionally jumps to the end of the loop when the
counter reaches 100
(_
moves to the left, then v
routes to
the line below, and @
ends the program). Before that, _
moves
to the right, to v
, then to >
then to 1+
(by wrapping around),
and then the flow goes back to the comparison bit. The jump #
command allows to jump over the v
when coming from the left. This
makes the code a bit more compact.
Then, :5*0M:"d"\-5*0\L
generates M 0 5*i L 5*(100- i) 0
, with
i
the current loop counter (remember, :
duplicates the top of the
stack, and \
swaps the first two elements). We need to do a bit of
stack magic to not forget the current i
value, and be able to compute
5*i
and (100 - i) * 5
. M
and L
pop values from the stack and
generate the corresponding SVG path commands.
The goal of befunsvge is to generate files amenable to pen plotting, so it only support drawing shapes, without support for color or filling.
Since it’s easy to type greek letters with my keyboard layout, I’ve decided to use greek letters for SVG-level commands:
ρ
(rho) pops coordinates for a rectangle and outputs a
<rect>
tag;ε
(epsilon) pops coordinates for an ellipse and outputs a
<ellipse>
tag;π
(pi) flushes the current path and outputs a <path>
tag.Path is the most versatile tool in befunsvge (and possibly in
SVG). It works with a d
parameter that contains a description of the
path. It works with a curser that can move to specific (absolute or
relative) positions, and draw lines, arcs or Bézier curves between
points. Befunsvge has special support for it: a dedicated buffer
for a path description, and operators that push commands to the
buffer. Finally, π
flushes the buffer by generating a <path>
tag with the buffer contents.
M
(resp. m
) moves the cursor to an absolute (resp. relative)
x,y
position without drawing anything;L
(resp. l
) draws a line to the absolute (resp. relative)
x,y
position;H
(resp. h
) draws a horizontal line to the absolute
(resp. relative) x
position;W
(resp. w
) draws a vertical line to the absolute
(resp. relative) y
position (in SVG it’s V
and v
, but v
is already part of befunge);A
(resp. a
) draws an ellipse arc to the provided absolute
(resp. relative) x,y
position, and the provided
rx,ry,angle,large-arc-flag,sweep-flag
parameters;CcSsQqTt
draw various Bézier curves.The MDN post on
d
gives you more detail about each commands. Befunsvge pops the
parameters in the same order (but from the top of the stack, so 01M
outputs M 1 0
).
Generative art relies a lot on randomness, since it’s a good way to generate unique output based on a set of otherwise deterministic rules. The most common (I think) source of randomness is Perlin noise. This provides randomness with nice locality characteristics (pure random noise is not very insteresting after all). A common use is to prodecurally generate terrain.
It might be possible to implement Perlin noise in befunge-93, but
that would be a bit of a hassle, so befunsvge directly implements
it as an input function i
: it pops x,y,z
off the stack, and
pushes back the Perlin noise value for these coordinates. External
configuration allows configuring the Perlin parameters themselves
(octaves, persistence, amplification).
Playing with befunsvge has been a lot of fun, but I still have a lot of ideas for further improvement.
One avenue I would like to explore is reading input from an external
source, typically images, so that values for individual pixels
can be used. The input function i
was build with this in mind:
external configuration. Currently it only supports Perlin noise, but
it could be extended to other kinds of input. The issue is that it
would then be impossible to mix several input sources.
Befunge-93 provides support for basic arithmetic (+-*/%
), but there
is nothing available for trigonometric functions. Such functions
are fundamental when drawing circular shapes. Befunsvge support for
SVG arcs and Bézier curves allows me to cheat a little, but that is
not enough. The issue here is that these functions operate on real
numbers, not on integers, while Befunge (and Befunsvge) only provide
integers. Extending befunsvge to support floating-point numbers is a
can of worms I would very much like not opening. Another solution
would be to work on degrees and take an amplification parameter.
Writing simple befunsvge programs can become easy after some time, but the development experience is cleary painful. The web UI doesn’t display anything, and the command-line runner only displays the stack contents after execution has stopped. A nice addition would be a step debugger, allowing to step through program execution, inspecting the stack, and maybe adding breakpoints. I am not sure how to do it yet, and whether to have it embedded in the browser. Depending on what I want to do, I might have to rewrite the whole engine.
Was I able to use a pen plotter to draw nice things and put them
on my wall?
No.
But was I able to generate pieces that could be plotted and put
on a wall?
Also no.
Anyway I have fun writing abtruse programs in an obscure language, generating pictures that don’t really make sense. Sue me.
by Clément Delafargue on June 9, 2022
A few days ago, I thought “haha, haskle, that sounds like wordle, but for haskell”.
Fast forward two days, and I’m releasing haskle. It’s a small
wordle-like game, where the goal is to guess a function from prelude given its (obfuscated)
type. The type elements are gradually revealed as you try names. It’s really fun for
functions with descriptive types (eg 🤷 🤷 => 🤷 🤷 => (🤷 ->🤷 🤷) -> 🤷 🤷 -> 🤷(🤷 🤷)
),
not so much when you’re in Floating
land. Still, I’ve learned about a lot of numeric
functions that I had never used until now.
It’s built with elm, as I
wanted something refreshing after having used purescript extensively at $PREVIOUS_JOB
.
Elm is fun, albeit a bit frustrating at times when you’re used to haskell and purescript,
but it has be quite enjoyable overall.
by Clement Delafargue on May 2, 2022
Tagged as: haskell.
This post was originally published on the technical blog of my former employer, which is now offline.
Macaroons are bearer tokens. As for every bearer token, the question of revocation is important: how can we stop a token from being accepted by services?
Macaroons are bearer tokens, introduced in Macaroons: Cookies with Contextual Caveats for Decentralized Authorization in the Cloud. There is a reference implementation that pins down the details.
As for every bearer token, the question of revocation is important: how can we stop a token from being accepted by services? The usual challenge with revocation is to distribute the list of revoked tokens to every service. In the case of macaroons, there is an extra challenge: uniquely identifying a token. This is what I’ll talk about in this article. Distributing revocation lists is a really interesting subject, but is highly context-dependent, and abundantly covered elsewhere.
Here is what we’ll cover:
Bearer tokens don’t need shared state: they are supposed to be verifiable without accessing a central authority. It is usually done through the use of cryptographic tools, such as HMAC, or digital signatures.
The benefit of not having shared state is that each service can independently trust a bearer token without performing extra network requests. This removes a single point of failure and makes decentralized systems easier, but the drawback is…
not having shared state.
With session tokens, everything is immediate, because the session is the single source of truth. Bearer tokens trade immediate updates for simpler semantics and more robustness.
In particular, revoking access is simple with session tokens: deleting the session automatically revokes all linked tokens, with no latency.
Revoking a bearer token is harder: since each service can verify them independently, you need to push down a list of revoked tokens to each service. This means that revoking access can take much longer, depending on the precise mechanism used to distribute the revocation list.
Of course, reducing revocation latency is extremely important. If there’s a leak, we want to plug it as fast as possible, reducing the vulnerability window as much as possible. In trying to do so, we still must be careful about not negating the benefits of bearer tokens. For instance, it might be tempting to contact a revocation server every time a token is verified. That reduces latency to zero, but this sneakily reintroduces a network dependency and a single point of failure, with much more complexity.
Once a token is revoked, you need to reject it until the heat death of the universe (or more realistically until your rewrite everything to use shiny new tech). This means that the revocation list can only grow. Not an issue at first but it can become an issue after a couple years (especially if you resist the urge to rewrite everything with shiny new tech).
One common way to mitigate this is to add TTLs to all your tokens, this way you can remove them from the revocation list once they are expired.
In the case of macaroons, we have an extra challenge: how do we uniquely identify a macaroon? Just listing the macaroon is both too broad and not broad enough, as you will see.
The main benefit of macaroons is that they allow offline attenuation: given a token, its holder can craft a new token, with more restrictions, without interacting with the token emitter.
A macaroon is a token made of several parts:
The key identifier and the caveats are freeform bytestrings. They usually contain text, but they can contain anything.
Usually it is enough to know that the key identifier is the only part that you can trust as coming from the authority and that the caveats can be freely added to a given token, but not removed. In our case, we will have to dig a bit more into the cryptography part to understand how to identify a macaroon.
Verifying a macaroon involves two operations:
Let’s clarify what discharging a caveat means and how it’s done in practice. A caveat is a restriction carried by the token: the token is only valid if the verifier can prove the property it ensures is satisfied. A common caveat is time < 2022-04-30T00:00:00Z
. It means “this token is only valid as long as the current datetime is before April 30, 2022, midnight UTC. For a macaroon to be valid, all of its caveats must be discharged. To discharge caveats, the verifying party can supply verifiers. A verifier is a function taking a caveat (a bytestring) and returning a boolean (true
if the verifier understands the caveat and can discharge it, false
if it cannot understand it, or if it understands it and can’t discharge it). Here is how it looks like in haskell:
timeVerifier :: UTCTime -> ByteString -> Bool
= fromMaybe False $ do
timeVerifier now <- BS.toString <$> BS.stripPrefix "time < "
dateTimeStr <- Time.iso8601ParseM dateTimeStr
validBefore pure $ now < validBefore
verifyMacaroon :: Macaroon -> IO Bool
= do
verifyMacaroon macaroon <- Time.getCurrentTime
now pure $ verify secret macaroon [timeVerifier now]
Here, verify
performs both verifications: it checks the signature, and it tries to discharge every caveat.
The caveats I have described above are called first-party caveats. It means they can be discharged directly by the service who verifies the macaroon. Macaroons also describe another kind of caveat (and it may be their most important innovation), called third-party caveat. It is a caveat that can only be discharged by a third-party service. In practice, it means that the macaroon holder has to contact the third-party service to get a proof that the caveat can be discharged, in the form of (you guessed it) another macaroon (which, crucially, can contain caveats of its own). The client has then to send both macaroons (serializing multiple macaroons is not spec’d in the paper, nor provided by the reference implementation) to the service that will then be able to discharge the third-party caveats with the attached discharge macaroons.
This is a very powerful mechanism, as it allows verification to be split across multiple services, without requiring the services to talk to each other. The only requirement is a shared secret between services. This is very useful when coordinating services across multiple independent entities (for example restricting access of a Dropbox link to Facebook friends).
Within services of a single entity, however, it tends to be overkill, as strong de-coupling of services is not as useful. Sadly, I have heard of a couple macaroons deployments going this way (because it is tempting, fun and interesting), and then suffering from complexity. Interestingly enough, I have not heard of cross-company macaroons deployments.
Now, for the signature part. Macaroons are based on HMACs: this allows to sign a payload with a shared secret: the verifying party computes the expected result and checks if it is the same as the provided signature.
To sign a macaroon, the emitter proceeds as follows:
With this, you should be able to convince yourself that holding a token allows you to add a caveat without knowing the shared secret, and that it’s impossible to remove or reorder caveats without knowing the shared secret.
To verify a macaroon, the verifying party computes the expected signature and checks if it matches the one carried by the macaroon.
Okay, so now your services maintain a revocation list. Good. What are you going to put in it?
You could store the whole macaroon but that is not super convenient, and you would need to be careful with the revocation list, since macaroons are sensitive tokens after all.
The macaroons paper does not cover revocation explicitly, but hints at it in an example of how third-party caveats could be used: when created, a macaroon can be attenuated with a third-party caveat that mandates a proof that the macaroon has not been revoked. In practice, it means that the client has to call out to a revocation server, to obtain a proof that the current token has not been revoked (how?), and bundle this proof with the original macaroon before finally sending the request to the service.
For. Every. Single. Request.
That sounds… cumbersome. Of course the client can cache validity proofs, but that still makes the client code way more cumbersome. The discharging of the third-party caveat can also be shifted to the service, but at that point, why use a third-party caveat at all? The service can just call the revocation server itself.
All in all, while this way of doing things is explicitly advised in the paper, I really don’t think it’s a convenient way to do things. Worse, the complexity it brings might lead you to doing unfortunate things like calling the revocation service at every request (thus negating all benefits of using bearer tokens), without really realizing it.
You could reject all macaroons with a given key identifier. In some cases it is a practical solution: if the key identifier is a session id, you end up in the session token case: you nuke the session and force a re-auth.
Some macaroon deployments will work well with this setup, but for pure bearer macaroons it will be too broad: if the key identifier only identifies a user, then the user would be banned forever. Not satisfying.
Another solution would be to store the macaroon signature: it’s computed from the key id and the caveats, so it allows you to identify a given macaroon.
All good then? Sadly, no. Such a system would be trivial to circumvent.
Macaroons are made to be restricted with caveats. So given a macaroon, you can mint a new one with an arbitrary caveat (say, a TTL caveat that expires in a thousand years), and get a new macaroon, with a new signature.
Thankfully, checking signatures can still be useful: the signature carried by a macaroon can be changed by adding a caveat, but during the process of verifying a macaroon signature, you compute all intermediate signatures. So it is possible to check each intermediate signature against a revocation list. This solution is great because it means that if you mint a restricted macaroon from a base macaroon, and want to revoke only this restricted macaroon, you can do it without revoking the base one. Conversely, if you revoke a macaroon, all of its derived macaroons are automatically revoked.
This sounds like a great solution, but there are two issues with it:
Macaroon generation is deterministic: there is no randomness involved. With the same key id and caveats, you’ll get the same signature. So when you revoke a macaroon based on its signature (or any intermediate signature), you don’t revoke just this macaroon, but all macaroons generated with the same key id and caveats.
Now, if you generate macaroons with TTLs, you get closer to unicity: revoking a macaroon will only revoke macaroons with the same key id, caveats (and this means that have been generated at the exact same moment, depending on the precision of the timestamp carried by the TTL caveat).
It is still not completely satisfying, since it is rather accidental, absolutely not random, and because some macaroons may not have TTLs (API-to-API macaroons for instance).
The best solution to make a macaroon unique is to include a random id upon generation. A simple way to do it is to add a dedicated caveat. Once you have that, every macaroon is unique when generated. You can do the same for derivation: if you want to be able to uniquely revoke a derived macaroon, you need to include a new unique id along with the caveats you’re adding.
not_revoked = <random_blob>
But now, if you’re adding unique ids in caveats, you don’t need access to intermediate signatures any more: the verifier for these caveats can check if they are not revoked. This is coarser grained than checking signatures (you can only target ids, not single out caveats, and if you have a macaroon with no id, then you can’t do anything), but that can be easily implemented without needing any library change.
Here is how it could look like in haskell:
revocationValidator :: (ByteString -> RevocationResult)
-> ByteString
-> Bool
=
revocationValidator isRevoked caveat let rid = BS.stripPrefix "not_revoked = " caveat
in case isRevoked <$> rid of
Nothing -> False
-- ^ this is not a revocation caveat
Just True -> False
-- ^ this id has been revoked, the caveat is not discharged
Just False -> True
-- ^ this id has not been revoked, the caveat can be discharged
Just remember that, due to how macaroons work, if you don’t have a revocation caveat, then the revocation check will not run. Embedding a revocation id in newly created macaroons is not enough, you’ll need to add a revocation check to all existing long-lived macaroons, and, after some time start manually checking for the presence of revocation ids as part of the verification pipeline.
hasRevocationId :: Macaroon -> Bool
=
hasRevocationId macaroon let isRevocationCaveat bs = BS.startsWith bs "not_revoked = "
in any isRevocationCaveat $ caveats macaroon
verifyMacaroon :: (ByteString -> RevocationResult)
-> [ByteString -> Bool]
-> Macaroon
-> IO Bool
= do
verifyMacaroon isRevoked verifiers macaroon <- Time.getCurrentTime
now let allVerifiers = timeVerifier now
: revocationVerifier isRevoked
: verifiers
pure $ hasRevocationId macaroon
&& verify secret macaroon allVerifiers
Having to embed this kind of manual check to macaroons validation is one of the reasons why you really should build an auth layer on top of macaroons and not just use them directly.
The first question you should ask yourself is “why don’t I just use session tokens?”. Using session tokens instead of bearer tokens makes a lot of things easier.
But if you’re set on bearer tokens, particularly macaroons, then read on.
Make sure you use TTLs. For tokens automatically delivered, bake it in the delivery process. Short-lived tokens can be coupled with a refresh token update mechanism. When using long-lived tokens, adding a TTL right before sending them on the wire mitigates the consequences of a leak.
TTLs won’t make revocation go away, but it will definitely make it easier and restrict the scope of security issues.
Even if you don’t end up implementing revocation capabilities, being able to identify tokens will help your ops team at some point. Once you’ve done that, not only you will be able to trace the use of tokens and know where they come from, but you’ll also be able to roll out revocation in the future without having to renew every token.
Adding revocation ids to new tokens is not enough. For short-lived tokens, you can wait a bit and now every token will have a revocation id, but for long-lived tokens you will have to go and replace them with fresh tokens with revocation ids.
The next step will be to make sure verified tokens have revocation ids. You can start by logging a warning when you see an unrevocable token. After a while, when logs have become quiet, you can flip the switch and start refusing unrevocable tokens altogether.
Start simple. If you don’t have a lot of services, a static revocation list read at startup (from a file, or an environment variable) can be more than enough. Then you can think about out-of-band polling, or even push notifications. Try to keep things simple, you don’t want too many moving parts. Just keep in mind that if you end up calling a central revocation service every time you’re checking a token, you might be better off with a centralized session.
Biscuits have been designed with macaroons strengths (and weaknesses) in mind, so they tend to provide out of the box all the mitigations required by macaroons. For instance they provide unique revocation ids as part of the spec and as such as part of the core API.
by Clement Delafargue on May 5, 2021
Tagged as: haskell.
J’ai eu la chance de participer à IFTTD pour parler de haskell (https://ifttd.io/de-limportance-dun-typage-qui-fonctionne/). C’était très sympa mais évidemment dès la fin du podcast je me suis aperçu que je n’avais pas mentionné tout ce que je voulais. Je vous mets tout ça pêle-mêle ici, si l’épisode vous a intéressé, peut-être que ces quelques notes feront un bon complément. Si vous avez des questions, n’hésitez pas à m’en parler sur twitter ou sur mastodon, je serai ravi d’y répondre.
Ça a longtemps été un point noir (car on s’en passe très bien, contrairement à java), mais ça reste sympa. Regardez hls dans VSCode, ça marche bien et ça s’améliore constamment.
Si vous voulez un truc simple qui marche, regardez du côté de ghcid
, qui lance la compil en boucle
et qui est très rapide.
Je n’ai même pas pensé à le mentionner, mais c’est un outil fondamental pour l’apprentissage de haskell. Oubliez les linters JS qui se plaignent sur le formattage du code. Là c’est des suggestions utiles, et qui vous apprendront des trucs sur haskell.
hlint est un formidable outil pédagogique.
Si vous utilisez ghcid
, lancez ghcid -l
pour lancer hlint
après la compilation.
Fun fact, le podcast a été enregistré deux fois, car le premier enregistrement a été perdu à cause d’un problème technique. C’est dommage, car on y a eu une discussion très intéressante sur l’abstraction et l’indirection.
Au delà de l’évaluation paresseuse, l’autre innovation de haskell, ce sont les type classes. C’est un mécanisme de modularité qui a depuis été repris d’une manière ou d’une autre par scala, rust, et même go avec go generics.
En gros, une typeclass c’est un peu comme une interface en java (ça définit des méthodes, certaines à implémenter, d’autres déjà implémentées en fonction des autres), mais ça diffère de java en deux points fondamentaux :
Pour la première propriété, ça évite d’avoir à tout wrapper, et ça permet aussi de s’intégrer proprement dans une lib ou un framework web par exemple, sans avoir une séparation nette entre “c’est prévu par le framework” et “c’est pas prévu par le framework”.
Pour la deuxième propriété, c’est un peu plus subtil. Quand on passe par du sous-typage, on utilise le supertype à la place du type réel. Du coup si on a deux types qui implémentent la même interface,
on se retrouve un peu bloqués. Il y a des solutions pour utiliser le sous-typage, tout en conservant
le type d’origine, mais c’est un peu alambiqué (cf f-bounded polymorphism). C’est parfois utilisé en java (cf Comparable<T>
) mais souvent on va plutôt se retrouver avec du Object
pour ne pas s’embêter.
Étant donné que l’OO à la java nous a pourri le cerveau en mélangeant héritage et sous-typage, et qu’en plus, les abstractions introduites sont souvent pas très bien conçues, “abstraction” ça fait un peu peur. Mais c’est souvent parcequ’on confond abstraction avec indirection.
L’indirection, c’est mettre des couches intermédiaires au dessus de notre code pour le rendre plus flexible. On s’éloigne du modèle concret, mais sans gagner en clarté.
L’abstraction est là pour exprimer avec clarté des concepts transverses, en “oubliant” les détails non nécessaires. Quand c’est bien appliqué, c’est une force simplificatrice : en utilisant des abstractions dans le code, on contraint les comportements à suivre des propriétés précises.
Une bonne abstraction, c’est un compromis entre deux propriétés : son applicabilité, et sa puissance.
En haskell, l’applicabilité, c’est le nombre de types qui implémentent une typeclass donnée. Par exemple Functor
est très applicable. Sa puissance, c’est ce qu’on peut faire avec. En haskell, c’est les types des méthodes de la classe. Par exemple, la classe Monad
permet de faire plus de
choses que Functor
(par exemple chaîner des opérations), mais est implémentée par moins de types.
On peut voir l’abstraction comme gênante, quand elle nous force à oublier les types concrets. D’une part on peut bénéficier de l’abstraction sans forcément oublier le concret, et d’autre part, ce n’est pas forcément une mauvaise chose.
On peut se servir localement d’une abstraction, sans oublier les types : par exemple quand je fais
("Toto" <> "Toto") :: Text
, j’utilise une abstraction (la notion de semigroup (demi groupe en bon français), tout en conservant des valeurs de type Text
. Je n’ai rien perdu sur le plan concret, mais j’ai un code qui m’indique clairement (pour peu que je connaisse les propriétés d’un semigroup) les propriétés qu’il respecte.
On peut aussi se forcer à n’utiliser que ce qui est nécessaire, en utilisant des types paramétrés et en ne listant que les typeclasses dont on a besoin. Là on perd le côté concret, mais on y gagne beaucoup en échange : on a la garantie en lisant les types que le code va se comporter uniquement en respectant les propriétés des typeclasses concernées.
Par exemple, Monoid a => [a] -> a
me dit que les éléments de la liste d’entrée vont être combinés, uniquement suivant la structure de la liste, sans faire de comparaisons entre éléments (pas de dédoublonnage par exemple), ni d’opération sur les éléments (si j’appelle cette fonction sur une liste de Text
, je sais par exemple que la casse ne va pas être modifiée, que des espaces ne vont pas être intercalées, …). Si je vois [Text] -> Text
, je n’ai aucune idée de ce qui peut se passer en pratique.
Ces deux approches sont complémentaires, parfois on a un contexte bien précis et c’est plus intéressant de réfléchir en termes de types concrets, parfois c’est plus pertinent de réfléchir en termes de propriétés plus génériques. Et on peut mélanger les approches : par exemple Monoid a => [a] -> a
,
c’est un mélange entre un type concret ([]
) et un type paramétré (a
). J’aurais pu aussi avoir
Monoid a, Foldable t => t a -> a
, qui restreint encore un peu les possibles.
https://www.youtube.com/watch?v=GqmsQeSzMdw
Encore une fois, haskell nous permet de choisir, ici. Là c’est des exemples simples, mais c’est un choix classique d’architecture des programmes en haskell, pour la représentation des effets, on peut soit choisir des types explicites (IO
, ou ReaderT Env IO
), soit utiliser des types contraints par les opérations que l’on veut réaliser (MonadReader Env m => …
), tout en pouvant mélanger les deux (MonadReader Env m => ExceptT Error m …
) dans certains cas. On a des outils à notre disposition, après c’est la responsabilité de la personne qui écrit le code de faire des choix
éditoriaux sur les propriétés intéressantes à mettre en avant.
Pour les débutant·e·s et intermédiaires (mais même des plus expérimenté·e·s peuvent y apprendre des trucs).
Deux bouquins pour mettre en pratique
Pour améliorer son style (à suivre au début, pour éviter de se poser trop de questions)
Super utile, plein de patterns très répandus mais pas forcément toujours documentés. Là c’est concentré en un seul endroit et bien illustré.
Une fois que vous avez les bases, je recommande chaudement d’utiliser relude
qui améliore énormément
l’expérience de dev.
Une ressource plus complète qui couvre beaucoup de terrain. Un bon départ pour découvrir un sujet précis (c’est bien à jour)
by Clement Delafargue on September 23, 2020
Tagged as: haskell.
This post was originally published on the technical blog of my former employer, which is now offline.
Edit 2022-08-01: My excellent former colleague Hécate found out (after leaving Fretlink) that those internal helpers were actually useful and reimplemented them in an open-source library: pg-entity.
Photo by Katie Barkman
“Should I use an ORM?” - the greatest thread in the history of forums, locked by a moderator after 12,239 pages of heated debate,
@dril (à peu près)
First things first, this is not an ORMs suck post. This is more about writing down tradeoffs I make time after time when it comes to DB layers.
I find myself making similar choices in different settings, and some of them may not be common practice. Everything here is about tradeoffs informed with my personal experience, so they may not align with your experience and might leave you horrified. That’s okay, it does not mean we have to fight about it. Just try to keep an open mind.
Even though this is published on my employer’s blog, it’s mostly personal opinions. I know some are shared by some colleagues, but not all of them.
I’ll start by stating my preferences about DB layers:
now()
if I can and prefer generating dates in codeThere is commonly found advice about DB layers that I don’t value that much. I understand where it comes from and why people advocate it, but my personal experience lead me to rank them lower than what’s above. Not that they’re bad per se, but that their benefits are not always balancing their drawbacks, as far as I’m concerned.
Based on all of these concerns, I most often end up using very thin DB layers: in Scala I used anorm, and in Haskell I use postgresql-simple. I like them because they don’t make assumptions for me and let me do what I want. Which means that used unwisely, they give rise to unmaintainable heaps of code (look at me, saying that I’m happy using sharp tools and saying you just have to use them correctly).
For those who don’t know what using postgresql-simple looks like:
-- here we're using pg-transact, a wrapper around postgres-simple that
-- makes transactions easier to work with: `DBT m` represents part of
-- a transaction: composite `DBT m` values are guaranteed to be run
-- inside a single transaction. This gives explicit control about
-- transaction semantics.
getBlogPostById :: MonadIO m
=> BlogPostId -> DBT m (Maybe BlogPost)
=
getBlogPostById postid let q = "select blogpost_id, author_id, title, content from blogposts where blogpost_id = ?"
in listToMaybe <$> query q (Only postid)
Basically, you get:
In Scala, I end up building anorm-pg-entities. When I changed jobs and switched to Haskell full-time, I more or less ported it.
Before talking libraries, the first thing I do when building a DB access layer is to represent the DB schema in code.
My first step is to declare one record type per entity table (maybe also association tables, if they have parameters). Pure association tables can very well be represented with tuples. This kind of things is often called a DAO, for Database Access Object.
The most important thing here is to make these types as obvious as possible:
plain records, with no external dependencies (unless super stable core types). Even though I mostly use UUIDs as primary keys, I still newtype every primary key type. The key is to have those types be a faithful description of the DB, in an easy-to-check way. So small types, all defined in dedicated modules, with limited external dependencies. As with types modelling JSON structures, these types have to be handled with care, as they turn untyped properties into types.
This might feel like boilerplate, but consider that you’re mapping something that exists outside the compiler-guaranteed realm of consistency. In addition to that, here you’re not mapping the DB schema exactly, but rather common projections.
Be always extremely wary of solutions that promise to remove boilerplate across untyped boundaries, if they’re not based on sound principles. In the case of SQL, one such solution would be to derive the actual schema from the types.
And since we want proper control over the actual schema, the types would need to be an obvious 1:1 description of the schema (unlike what’s usually done by ORMs).
Once you have that, you should always strive to talk to the DB through them. They provide a compiler-checked, easy-to-reason-about model.
With generic derivation, you can get parsers and serializers for free. Reducing boilerplate is a great way to reduce serialization issues. It’s not enough though, so I usually test those instances with a roundtrip test (more on that later).
newtype BlogPostId = BlogPostId { getBlogPostId :: UUID }
deriving newtype (Eq, Show, FromField, ToField)
newtype AuthorId = AuthorId { getAuthorId :: UUID }
deriving newtype (Eq, Show, FromField, ToField)
data BlogPost
= BlogPost
blogPostId :: BlogPostId
{ authorId :: AuthorId
, title :: Text
, content :: Text
, createdAt :: UTCTime
,
}deriving (Eq, Show)
If you choose to have a dedicated model layer, going through these types first will let you type check the model -> DB transformations. Projecting model types directly to DB queries when you don’t have a one-to-one mapping can be tricky: you end up having surprising code in parsers and serialization (IMO, you don’t want that, as you’re already in untyped territory).
Most of the time, my model layer is just aggregated entity types, as I need to carry the ids around anyway: a usual sequence of operations is:
In this case, I need to be able to do a DAO -> model transformation, and then a model -> DAO transformation back for persistence. If my model throws away DB-related information, then I have to carry this information around in another channel to be able to go back to DAOs. This usually ends up either compromising type safety, introducing magic in the serialization layer, or giving rise to model types that are isomorphic to DAOs, with extra ceremony.
Those types declare what a row looks like in each table. That’s not enough: we want to know about table names, field names.
That’s the first part of my libs: a typeclass
-- This is a simplified version. The real version has optional type annotations
-- for fields, and has a dedicated class for primary keys
-- This requires the `AllowAmbiguousTypes` extension to work: notice that the `e`
-- type does _not_ appear in the method types. It can't be inferred and has to be
-- provided explicitly through visible type applications.
-- Another possibility is to use `Proxy e` as arguments in the methods: it removes
-- the ambiguity (at the cost of increased verbosity).
class Entity e where
tableName :: String
primaryKey :: String
fields :: [String]
instance Entity BlogPost where
= "blogposts"
tableName = "blogpost_id"
primaryKey -- ^ this could be derived automatically from a `Typeable` constraint, but
-- handling pluralisation automatically is not really something i want to spend
-- time on
= [ "blogpost_id"
fields "author_id"
, "title"
, "content"
, "created_at"
, ]
Once you have all that, you can automate basic CRUD operations: you have all the information you need to generate basic INSERT
, SELECT
, UPDATE
and DELETE
queries.
-- simplified version of the actual helper, but conceptually it's the same
-- This relies on the `TypeApplications` and `ScopedTypedVariables` extension,
-- to tell the compiler which `Entity` instance we need.
getById :: forall a b m
. (Entity a, FromRow a, ToField b, MonadIO m)
=> b -> DBT m a
=
getById pk let q = selectQuery @a <> " where \"" <> primaryKey @a <> "\" = ?"
in listToMaybe <$> query q (Only pk)
-- you can do that only if all values are generated before talking to the DB.
-- else, you'd need a separate type representing an _entity creation request_.
insertEntity :: forall a m
. (Entity a, ToRow a, MonadIO m)
=> a -> DBT m ()
=
insertEntity entity let q = insertQuery @a
in void $ execute q entity
Generating ids and dates ahead of insertion means you don’t have to maintain a specific type for insertion. This nice property may go away at some time, but it’s quite convenient when starting a project.
This is the right moment to write basic tests to make sure the basics work, using tmp-postgres to spin up a temporary DB for tests.
testRoundtrip :: BlogPost -> Assertion
= do
testRoundtrip blogPost
insertEntity blogPost<- getById (blogPostId blogPost)
result Just blogPost) assertEqual result (
At this point you may think I said all that just to showcase a shitty untyped ORM ersatz. I’d argue that’s a neat improvement over more naïve uses of postgresql-simple, but let’s be honest, simple single-table queries are not why you’re using PostgreSQL in the first place.
The interesting part is when you want more complex queries. That’s where typed layers and ORMs start getting annoying. SQL is a rather fine language to describe queries, spending time to find the right incantations to have a layer generate the SQL you already have in your head gets tedious quickly.
In my experience, the issue with manually written SQL was when the schema got updated. I had to go over every query and make sure fields were correctly listed. That was the number one cause of regressions in the SQL layer.
The interesting part is the structure of the query: which tables you are joining, and how. Listing fields in the query is tedious, and combining parsers is usually trivial. A nice property of having one type per table is that joins are made obvious in the types. So when you retrieve an object from the DB, you can have a good idea of the query that was sent.
Since we have mapped the field names and types, we can mix static SQL (the aforementionned structure of the query) with a dynamically generated list of fields.
Does this prove your SQL statements are well-formed in 100% of the
cases? No. However it succeeds at sidestepping many issues with manual SQL queries, while keeping you in control.
Since I introduced this layer at work, my colleagues have started to use and extend it. The base features presented in this post are my work (except generic derivation of Entity
, which I have only implemented in Scala, while @ptit_fred handled it in Haskell).
Even though it has proven effective to avoid common issues, we’re still generating strings at runtime. This makes static analysis of queries hard and we have to rely on tests for a lot of things.
Even though the projection part is handled and provides a modicum of safety, we are still completely on our own on where
clauses.
Finally, the whole mapping makes entity-level operations easy, but for partial updates and partial projections you’re on your own. Designing everything around entities instead of actual use cases feels backwards, especially since I’ve been exposed to Domain Driven Design. Here it’s a purely pragmatic choice: a base language allowing to manipulate things at the entity level is surprisingly scalable and allows to handle a lot of use cases as they come.
I’m perfectly happy with these helpers used internally, in a rather controlled setting. I don’t think they are principled enough to be shared as an open-source library. The core tenet of this internal library is that it should not try to model SQL itself. In extremely simple cases, it is able to produce structured SQL, but at its core its only job is to keep track of field lists. This comes in tension with the natural desire to have it handle more things for us. However it sits on a very narrow sweet spot, where it’s powerful enough to be useful, but simple enough to stay out of the way. Other DB access libraries have stronger goals: ranging from staying out of the way completely (postgresql-simple) to completely taking over (persistent).
One library that stands out from the others is squeal, which faithfully embeds SQL semantics in Haskell. We use it at work in a few projects that benefit from fully typed queries. I like that it does not try to hide SQL semantics away. Unfortunately, for now, it does have a cost, in terms of compilation times (it’s being worked on in GHC), in terms of readability (that’s obviously subjective, but the type-level encoding warrants some extra syntax that makes “seeing” the original SQL rather hard), and in terms of what’s possible to automate (manipulating strings is easy, manipulating type-level constructs… not so much). This requires the developer to always be extremely explicit, even for simple queries working with common projections. Note these are not fundamental issues. This lib sits on the other side of the convenience tradeoff and would be my go-to choice for more complex domains. That’s talk for another day, but having this layer completely typed (and kept in sync with the actual schema) allows to completely avoid the “simple, stable DAOs” requirement, since everything can now be checked statically (ad-hoc projections, query parameters, …).
This is where I’m at right now, my core choices about SQL are quite stable, but the way I turn that into implementations will surely change (hopefully toward a more typed approach).
by Clement Delafargue on May 13, 2020
Tagged as: haskell.
This post was originally published on the technical blog of my former employer, which is now offline.
Ah, the do
-notation. Is it good, is it
bad;
who knows? It’s good for beginners,
it’s bad for beginners. It’s considered
harmful.
A lot of virtual ink has been spilled on this subject (and still is). Let me try to add a new perspective, informed by the use of Haskell in a heterogeneous team, in a professional setting. Then we’ll be able to gleefully conclude that it depends. But rather than fretting too much over the destination, let’s try to enjoy the journey.
do
?Let’s start by the beginning. do
blocks are a syntactic construct
in the language Haskell. It provides an alternative syntax over the
use of >>=
:
-- without do
main :: IO ()
=
main getLine >>= \name ->
let greeting = "Hello " <> name
in putStrLn greeting
-- with do
main :: IO ()
= do
main <- getLine
name let greeting = "Hello " <> name
putStrLn greeting
That’s the gist of do
-notation. There are a few additional features,
but we don’t need to talk about them for now. The example is formatted
in a way that makes the correspondence obvious. You might be tempted
to think that do
is a smart trick to remove syntactic noise when
writing monadic expressions. And you’d be right.
do
Julia Stiles as Kat Stratford, saying “But mostly, I hate the way I don’t hate you, not even close” in Ten Things I Hate About You
If you look at things this way, it’s easy to discard do
notation:
it doesn’t add any expressive power, and it makes things more complex
for no good reason. As Paul Hudack nicely put it,
Functions are in my comfort zone; syntax that hides them takes me out of my comfort zone.
Another issue is that it makes imperative code too nice-looking,
thus earning Haskell its famous finest imperative language badge
of shame. The familiarity provided by do
notation might encourage
imperative-minded programmers into putting too many things in IO
(or any other monadic context, for that matter) instead of composing
functions as ${DEITY:-or lack thereof}
intended.
do
The Beatles, crossing the street from right to left
All that is good and well, but let’s go back to our nice (and artificial) example:
main :: IO ()
=
main getLine >>= \name ->
let greeting = "Hello " <> name
in putStrLn greeting
What do you think happens when you insist on using function composition in a team of Haskell programmers?
Yeah, that’s right, getLine >>= putStrLn . ("Hello " <> )
. And in
real life, stuff like (runService =<< parseConfig) *> otherTask
will pop up, because composing functions is good, obviously. I do
love the smell of η-reduction in the morning, but hear me out:
what if (bear with me) η-reduction was not always good? Sometimes
you care about the pipeline of functions, sometimes you care about
the intermediate values.
Now for my main thesis: using do
over bind
(=<<
or >>=
)
is contextual. And the context can be very small. Ask yourself:
what part of your program is important enough to be obvious when you
read it? Are all those intermediate variables bringing meaning?
parseString :: String -> Handler Int
= …
parseString
getString :: Handler String
= …
getString
myDoHandler :: Handler Response
= do
myDoHandler <- getString
rawString <- parseString rawString
value log $ "Got value " <> show value
pure $ mkResponse value
myMixedHandler :: Handler Response
= do
myMixedHandler <- parseString =<< getString
value log $ "Got value " <> show value
pure $ mkResponse value
myPointfreeHandler :: Handler Response
=
myPointfreeHandler let logAnd = log . ("Got value" <>) . show >>= ($>)
in mkResponse <$> getString >>= parseString >>= logAnd
Which one do you prefer? My favourite is myMixedHandler
: I care about
value
, so that makes sense to give it a name. On the other hand,
the intermediate String
value is not that interesting, so I’m okay
with hiding it.
If your favourite is myPointfreeHandler
, I don’t know what to
say. Maybe you won’t agree with my conclusions.
In my opinion, do
and bind
are complementary: do
nudges
me into naming variables by providing a clean syntax. bind
makes
function pipelines more obvious by hiding intermediary values. Both
have their uses, even on the same line!
do
discourages applicative style!That’s right, do
works with monads (since it desugars to >>=
),
so that encourages a monadic style where applicative functors could
be enough. Two things:
do
blocks are
not your most pressing issuemyParser :: Parser (Int, Maybe String)
= do
myParser '@'
char <- int
intValue
many space<- optional string
stringValue pure (intValue, stringValue)
myApplicativeParser :: Parser (Int, Maybe String)
=
myApplicativeParser
liftA2 (,)'@' *> int <* many space)
(char (optional string)
Both parsers parse the same thing, but the second is obviously better because it truly showcases the applicative nature of what we’re doing.
Here, parser has both a Monad
and an Applicative
instance, so
that’s just style.
What about a parser that is not a Monad
?
Let’s put our environment variables parsers to work.
data MyConfig
= MyConfig
value2 :: String
{ value1 :: String
, value3 :: Int
,
}
configParser :: EnvParser MyConfig
= MyConfig
configParser <$> required (str "VALUE_1")
<*> required (str "VALUE_2")
<*> required (int "VALUE_3")
Ah that’s where applicative syntax shines! It’s clear
and easy to read. Oh by the way did you spot the surprising
behaviour? Yes? Congrats then. Something really close happened to us,
got past code review, went to production and caused an outage. Yes we
were using newtypes instead of plain strings. No it did not help us
(IsString
can be tricky). If you did not notice the issue, VALUE_1
maps to value2
, and VALUE_2
maps to value1
.
The issue here is that we rely on the order of arguments in the constructor. A nice rule of thumb when constructing records is to make fields explicit:
configParser :: EnvParser MyConfig
=
configPaser let mkConfig value1 value2 value3 =
MyConfig { value1 = value1
= value2
, value2 = value3
, value3
}in mkConfig <$> required (str "VALUE_1")
<*> required (str "VALUE_2")
<*> required (int "VALUE_3")
That’s a bit better, because everything is in the same function: we don’t have to look at the record definition to know what will happen. Still, the names are one level away from the parsers.
It turns out that we can have do
notation even without Monad
,
thanks to ApplicativeDo
(I’ve also thrown a RecordWildCards
in the mix since we’re playing with forbidden extensions.
{-# LANGUAGE ApplicativeDo #-}
{-# LaNgUaGe RecordWildCards #-}
configParser :: EnvParser Config
= do
configParser <- required (str "VALUE_1")
value1 <- required (str "VALUE_2")
value2 <- required (int "VALUE_3")
value3 pure Config{..}
Is it theoretically pure and syntactically minimal? No. Does it provide easier to read code, even in messy, real-worldy context? I do think so, yes.
ApplicativeDo
pitfallsSo like me, you think ApplicativeDo
is criminally
underused? Good! Now might be a good time to mention two things:
ApplicativeDo
changes the behaviour of all do
blocks. So if you
are using types with inconsistent Applicative
and Monad
instances,
you may curse a bit. Purescript solved it nicely by introducing ado
,
to make the behaviour explicit. Personally, I try to only enable
ApplicativeDo
in small modules where the only do
blocks are over
types with no Monad
instance.
Another pitfall is a somewhat surprising behaviour with let
bindings
in do
blocks:
-- this requires a `Monad` instance on `EnvParser`
myParser :: EnvParser Result
= do
myParser <- v1Parser
v1 <- v2Parser
v2 let v3 = f v1 v2
pure $ Result v1 v2 v3
The way do
blocks are de-sugared leads to the use of >>=
when
there are let
bindings.
You’ll run in a similar issue if you try to pattern-match on the
left-hand side of <-
:
-- this requires a `Monad` instance on `EnvParser`
myParser :: EnvParser Result
= do
myParser <- bothVsParser
(v1, v2) -- you can bypass the limitation with a lazy pattern
-- ~(v1, v2) <- bothVsParser
<- v3Parser
v3 pure $ Result v1 v2 v3
Thankfully there is a simple solution for both issues: using a let … in
expression as the last part (after pure
). This is always
possible for applicative composition.
myParser :: EnvParser Result
= do
myParser <- v1Parser
v1 <- v2Parser
v2 pure $ let v3 = f v1 v2
in Result v1 v2 v3
myOtherParser :: EnvParser Result
= do
myOtherParser <- bothVsParser
both <- v3Parser
v3 pure $ let (v1, v2) = both
in Result v1 v2 v3
I guess we learned to not
do
it again
There are multiple layers to the do
notation. At first you can
see it as a way to make Haskell look more imperative. It’s a good
beginner-friendly crutch, but you soon grow past it: function
composition is what Haskell is all about, anyway. That’s where I
was for a long time. But using Haskell in a professional setting
led me to a more nuanced opinion: the language provides different
tools, it’s up to me to use them in order to get something that best
expresses my intent. Of course nothing is really perfect and do
notation has its issues, but it can be of great help when it comes
to choosing how to structure my code.
Haskell is a mature language and, perhaps a bit counter-intuitively, not that opinionated: it provides developers with a lot of tools and alternative ways to do the same thing. Personally, I love it because I can craft code in a way that lets me outline what I want, depending on context. Of course a pragmatic compromise like this is not the best choice for everyone, some may prefer a more opinionated language that makes more decisions for you.
by Clement Delafargue on March 20, 2020
Tagged as: haskell, free applicative.
This post was originally published on the technical blog of my former employer, which is now offline.
Edit 2020-05-04: I have been pointed to envparse, which is touted as “optparse-applicative, but for enviroment variables” and is implemented with free applicative functors. Sounds familiar? It is more polished and has a better design than what’s shown in the article, but it is strikingly similar to what I came up with.
As cloud native™ citizens, we have been heavily inspired by the twelve factor app. One of its key points is application configuration through environment variables.
As the number of parameters grows, having a structured way to parse and validate them is paramount. In haskell, the base
library provides a really bare-bones way to get them, with getEnv
and lookupEnv
. This gets cumbersome quickly, if you want proper error reporting.
module Env where
-- either
import Data.Either.Validation
-- base
import Data.List.NonEmpty
import System.Environment
data Config
= Config
myValue :: String
{ myOtherValue :: String
,
}
-- this is the easiest way, but it will crash at the first missing
-- environment variable: `getEnv` returns an `IO String`, so it
-- throws if the variable is not defined. There is no way to
-- report all the errors.
-- This also does not parse the variable contents
readEnv :: IO Config
= Config
readEnv <$> getEnv "MY_VALUE"
<*> getEnv "MY_OTHER_VALUE"
-- Slightly better, returns all the missing environment variables:
-- `lookupEnv` returns an `IO (Maybe String)` so it does not throw
-- exceptions on missing values. Once we have run all the `IO` steps,
-- we can accumulate errors thanks to `Validation`
-- This still only handles `String`s, but `getValue` could handle
-- it as well, at the cost of a bit more complexity
betterReadEnv :: IO (Validation (NonEmpty String) Config)
= do
betterReadEnv let getValue name = maybe (Failure $ pure name) Success <$> lookupEnv name
<- getValue "MY_VALUE"
myValue <- getValue "MY_OTHER_VALUE"
myOtherValue pure $ Config <$> myValue <*> myOtherValue
There are a few libraries in haskell that allow to do that in a structured way (for instance envy), and you should definitely use them instead of rolling your own. Anyway my goal is to trick you into learning free applicatives, so bear with me.
In a way, we want something like the amazing optparse-applicative, but for environment variables. Note that typeclasses are not mentionned here: that’s on purpose. While they are common for serialization use cases, here we want to be as explicit as possible.
Given this constraints, we already know that the parsers will have to be
applicative and not monadic (error accumulation, parser inspection). This is in line with the analogy with optparse-applicative.
import MySuperLibrary
import Data.Text (Text)
data SubConfig
= SubConfig
subItem :: Text
{
}
data Config
= Config
myValue :: Text
{ someNumber :: Integer
, anOptionalThingy :: Maybe Text
, subConfig :: SubConfig
,
}
-- We want to be able to inspect the parser, so we need an
-- actual value, not just a function. This also makes it easier
-- to compose parsers
-- To be able to accumulate errors, we need something that is
-- applicative _only_
configParser :: EnvParser Config
=
configParser Config <$> required (textParser "MY_VALUE")
<*> required (intParser "SOME_NUMBER")
<*> optional (textParser "OPTIONAL_THINGY")
<*> prefixed "SUB_" (SubConfig <$> required (textParser "ITEM"))
-- same as with optparse-applicative, the lib can display the
-- error itself and give us a convenient helper
readConfig :: IO Config
= readFromEnv configParser
readConfig
-- we want to be able to generate documentation to describe all
-- the variables used by the application
configHelp :: Text
= renderHelp configParser configHelp
Now let’s add a final constraint: we don’t want to put too much work in it; the interesting thing to talk about is how to parse one environment variable: what’s its name, how to turn the string into another type, etc.
Composing multiple variables is not really new: we want to try every variable, collect and structure the results if everything is okay. If there are issues, collect errors and fail.
A complete implementation is available in a gist. It is a bit different from the examples of the blog post, which have been simplified.
So, what we want to do is to describe one effect, and then have composition for free. This sounds dangerously like a free something!
Free constructions lets you turn a base structure into a more powerful one: for instance, the free monoid over any type is the list. Free monads are another example: they let you get a monad out of any functor. Usually free constructs let you build a value with the desired property (introduction), and then it’s up to you to interpret them into what you want (elimination).
Here, we want to turn something describing how to parse a single variable into something that parses multiple variables, with applicative composition. Free applicatives provide us that. Funnily enough, one motivating example from the free applicatives paper is a parser for command-line interface arguments. A nice confirmation that “optparse-applicative, but for environment variables” was not too far off the mark.
The free
package provides us with a host of free constructions, including free applicatives.
I don’t feel confident explaining what free applicatives are and how they work, but here, having a look at what they can do will be enough.
-- module Control.Applicative.Free
-- a free applicative is a _data structure_
data Ap f a where
-- we don't need to talk about the contructors here
-- It gives us a functor instance for *any* `f`.
-- No constraints! Amazing!
fmap :: (a -> b) -> Ap f a -> Ap f b
-- It gives us an applicative instance for *any* `f`.
-- No constraints! Amazing!
pure :: a -> Ap f a
(<*>) :: Ap f (a -> b) -> Ap f a -> Ap f b
-- given a way to turn the initial `f` into an applicative `g`,
-- we can eliminate the `Ap` wrapper
runAp :: Applicative g => (forall x. f x -> g x) -> Ap f a -> g a
-- we can also eliminate the `Ap` to get a `Monoid`
runAp_ :: Monoid m => (forall a. f a -> m) -> Ap f b -> m
-- we can turn a `f` into an applicative
liftAp :: f a -> Ap f a
-- we can change the `f` into a `g` inside the `Ap`
hoistAp :: (forall a. f a -> g a) -> Ap f b -> Ap g b
The first step is the actual work: how to describe a parser for a single environment variable.
For simplicity, I won’t handle optional values, only required ones. We need the variable name and a parser function.
import Data.Either.Combinators (maybeToRight)
import Text.Read (readMaybe)
data EnvVarParser a
= EnvVarParser
parser :: String -> Either String a -- the value or a parsing error
{ name :: String
,deriving Functor
}
textParser :: String -> EnvVarParser Text
= EnvVarParser (pure . pack) name
textParser name
intParser :: String -> EnvVarParser Integer
=
intParser name let parser = maybeToRight "not an integer" . readMaybe
in EnvVarParser parser name
Once we have that, we can use [liftAp](https://hackage.haskell.org/package/free-5.1.3/docs/Control-Applicative-Free.html#v:liftAp)
to get an applicative. From there, we can create basic parsers thanks to its Functor
and Applicative
instances.
type EnvParser = Ap EnvVarParser
required :: EnvVarParser a -> EnvParser a
= liftAp
required
data SubConfig
= SubConfig { subItem :: Text }
data Config
= Config
textValue :: Text
{ intValue :: Integer
, subConfig :: SubConfig
,
}
configParser :: EnvParser Config
= Config
configParser <$> required (textParser "TEXT_VALUE")
<*> required (intParser "INT_VALUE")
<*> (SubConfig <$> required (textParser "SUB_ITEM"))
OK, now that we got that out of the way, on to more complex things:
Applying group modifiers on parsers can be handled by [hoistAp](https://hackage.haskell.org/package/free-5.1.3/docs/Control-Applicative-Free.html#v:hoistAp)
if you squint just right. Here we don’t want to turn EnvVarParser
into another type, we want to apply a modification on it. So hoistAp
’s type becomes (forall x. EnvVarParser x -> EnvVarParser x) -> (EnvParser a -> EnvParser a)
: it lifts a function modifying a parser for a single environment variable into a function modifying the whole tree of parsers. Don’t get intimidated by the forall
bit: since our function will be called on each variable parser, we can’t assume anything about the parsed types: we only care about modifying the metadata in a consistent way.
Here is how the prefix
modifier is implemented. It allows to add a prefix to all the variable names, to avoid repetition or repurpose an existing parser.
addPrefix :: String -> EnvVarParser a -> EnvVarParser a
= parser { name = prefix <> name parser }
addPrefix prefix parser
prefixed :: String -> EnvParser a -> EnvParser a
= hoistAp (addPrefix prefix) prefixed prefix
Side note if you’re familiar with servant: this is quite close to [hoistServer](https://hackage.haskell.org/package/servant-server-0.17/docs/Servant-Server.html#v:hoistServer)
: it lets us apply a transformation on all the handlers in an API tree. Look at the signatures, they’re really close. The name [hoist](https://hackage.haskell.org/package/mmorph-1.1.3/docs/Control-Monad-Morph.html#v:hoist)
comes from the mmorph
package (it stands for “Monad Morphisms”).
Now the interesting part: we want to get actual values. So we need to eliminate the Ap
.
runAp
seems interesting. We just have to find the suitable g
. Remember when I said we wanted to get either a structured value or all the errors? There’s a very good reason this sounds like the perfect job for Validation
: that’s because it is the perfect job for Validation
.
Now you’ll tell me “Clément, stop messing with me, we still need to perform IO
”. And you’d be right. But we know exactly what kind of IO
we need: read the environment variables. So if we have all the env vars we need in scope, we can write our EnvVarParser a -> Validation (NonEmpty Error) a
. One way to have all the env vars we need is to read all the env vars, but we’ll see in a bit that we can be a bit less heavy handed.
data EnvError
= Missing String
-- ^ The name of the missing variable
| ParsingError String Text
-- ^ The name of the variable and the parsing error
deriving Show
runEnvParser :: [(String, String)]
-- ^ We need to close over the environment
-> EnvVarParser a
-> Validation (NonEmpty EnvError) a
EnvVarParser parser name) =
runEnvParser env (case (lookup name env) of
Nothing -> Failure . pure $ Missing name
Just v -> either (Failure . pure . ParsingError name) Success $ parser name
readFromEnv :: IO (EnvParser a)
= do
readFromEnv <- getEnvironment
env case runAp (runEnvParser env) of
Success a -> pure a
Failure errors -> fail (show errors) -- this can be done better
Oh by the way, if you look at runEnvParser
, you will notice that it is completely decoupled from environment variables. So we can reuse not only parsers, but also core machinery, to read variables from other key-value stores, for instance hashicorp’s vault. Another win for interpreters!
One of our goals was to be able to inspect parsers without actually running them, for instance for generating documentation.
For example, we would like to list the needed variables names. That would let us read only what’s needed in the IO
phase.
For that, we could run [runAp](https://hackage.haskell.org/package/free-5.1.3/docs/Control-Applicative-Free.html#v:runAp)
and extract the variable names from the errors. That would be a convoluted way to do what [runAp_](https://hackage.haskell.org/package/free-5.1.3/docs/Control-Applicative-Free.html#v:runAp_)
does.
Here, we’re interested in a list of environment variable names, so the monoid we’re interested in is the good old list. So we are turning a free applicative into a free monoid. That’s cool!
What’s even better is that we want to go from an applicative to just its accumulated effects. It turns out there is an applicative functor that does just that: Const
. And that’s how runAp_
is implemented.
Anyway, back to listing environment variables names
getVars :: EnvParser a -> [String]
= runAp_ (pure . name) getVars
Yes, that’s it.
Once you have that, you can extract anything you want from your applicative: variable names, documentation. One nice thing it allowed us to do is to generate dhall types from the environment requirements. This allows to type-check the environment generation in the CI pipelines directly, instead of waiting for them to fail during deployment like cavepeople.
Yay types!
Earlier, I papered over optional
support. It’s not too complicated to implement, but it adds some complexity (and room for inconsistencies) in EnvVarParser
. Alongside the blog post is a full example, if you want to see how it works.
Let’s discuss instead two use cases that seem simple but are actually out of applicatives’ reach.
Applications running in production require more configuration than on local installations. We could make everything optional, but that’s not really satisfying. A common solution is to start by looking up ENVIRONMENT_NAME
and then decide what to do based on that.
The keen reader will notice that it looks like monadic behaviour. This can’t be done purely (pun intended) with applicative. A solution is to interleave monadic and applicative layers: start by reading what you need, then decide on the actual parser based on that. You keep applicative behaviour in the two layers, instead of making everything monadic and opaque.
That’s what we have done. Another fancy solution would be to use selective parsers instead. Selective parsers are quite recent middle ground between applicative functors and monads: you want some effect to depend on a previous one, but you can in advance enumerate the possible paths. It’s not fully arbitrary monadic power. The good news is that there are free selective functors. I still have no idea on how to use them to solve this, but that would be an interesting discussion to have, so please ping me if you want to discuss it.
The last one is trickier. Optional variables are good, but what about optional groups?
This looks easy: EnvParser a -> EnvParser (Maybe a)
. The common cases are not too complicated:
Just
;Nothing
;There is one last possibility that is harder, though:
In that case, we want to warn the user, instead of silently ignoring input. And that’s where this simple requirement introduces a constraint that’s not compatible with applicatives: variables are interdependent.
I’ve tried a few things there, the most promising is to use a free Alternative
(after all, it provides that out of the box optional :: Alternative f => f a -> f (Maybe a)
), but I was not satisfied with anything, so I left it at that. It’s always possible to emulate this behaviour with some boilerplate (declare all the variables as optional, and then add a post-processing pass after the actual parsing).
The goal of this article is to showcase the use of free applicatives on a concrete case. I’m not advocating the use of free applicatives everywhere, rather trying to show that they’re convenient, and not too hard to use. From a practical standpoint they’re great as long as you need purely applicative behaviour: extended use cases get cumbersome quickly. For instance, optparse-applicative uses its own type and does not use free applicatives. At some point I guess that it’s easier to extend things with your own type. From a pedagogical standpoint, I love how free applicatives showcase the essence of applicative functors, especially the runAp
/ runAp_
pair. There is no way to cheat or to accidentally introduce a behaviour that’s too powerful.
One last remark: there are several implementations of free applicatives. I have only tested the naive one, as I did not have any performance constraint. The three implementations (Control.Applicative.Free
, Control.Applicative.Free.Fast
and Control.Applicative.Free.Final
) all expose the same API, so it should be possible to try them all out with minimal fuss.