The Hyperpessimist

The grandest failure.

Finding the Optimal Parameter Order

When I first started programming I tought the order of function parameters was essentially arbitrary; just the order the author decided to structure it. But after dabbling with a number of functional programming languages I came to the conclusion that the order is in fact important.

Let me try to convince you.

When programming Clojure, you often end up chaining function calls, like (baz (bar (foo x))), since after all, we all work on data and that data is processed by functions. This has been totally normal, at least until the threading/thrush operator came along, -> (a really long time ago). So you can replace the code with (-> x foo bar baz) and it looks much cleaner and obvious. It’s fantastic!

This only works for functions with the arity of 1 (single argument functions) and by extension, for functions which take the element to be “threaded in” as their first argument. Therefore, Clojure also has also its sibling, ->> (called “thread-last”, analogous to -> being “thread-first”), which unsurprisingly threads the value in as last argument to the specified functions.

Working with code, we often have a seq that we want to operate on, so that’s what we usually thread. Unfortunately, the standard library is not very consistent about this, since common seq operations take the collection as first argument, like update, assoc, dissoc, conj. So we could use them with ->. But then when we want to use some combinators like map, filter, reduce, the collection has to be provided last, which would require ->> instead.

The reason why e.g. assoc has the collection first is that is a multi-arity function and can associate multiple values at once, so the order of arguments of (assoc coll :arg1 val1 :arg2 val2 :arg-n val-n) is logical. Generally, most clojure.core functions which take collections and an unspecified amount of arguments seem to be this way, which is understandable considering how & arguments are handled in Clojure.

To avoid the awkward mess of mixing code that uses -> and ->>, Clojure 1.5 introduced as->, which allows naming the argument to be threaded (I usually go naming the argument <>, aka “diamond”), so it can be put in the proper place to be resolved, but this feels very much like a clumsy (albeit effective) compromise to get around the argument order mess.

So, which “side” of the ->/->> split is right? Personally, I subscribe to the thread-last school of argument order. This means that I order the arguments in functions according to their specificity: from the most general to the least general. Consider (map f coll), which takes the function first (since it might work on any coll element) and then only the specific values to be applied on. Similarly reduce. Working this way also has the advantage that partial can be used to pre-populate some arguments with known values and then just operate on a function of lesser arity.

This approach is not without precedent. For languages with implicit currying like OCaml or Haskell this order is completely normal. Currying creates out of a function like

1
2
(defn foo [bar baz]
  (+ bar baz))

a function like

1
2
3
(defn foo [bar]
  (fn [baz]
    (+ bar baz)))

So when calling (foo bar) a function is returned which takes baz and returns the result. So basically it’s like using partial for every argument. This of course means that arguments can only be supplied left to right. The OCaml way of threading is then coll |> map inc so the argument is threaded in at the end, just like our friend ->> does. The actual reason for this is of course a bit different, since map f returns a single-arity function so it doesn’t really matter whether threading first or last element, since they are identical in that case.

A more accurate translation to Clojure would be

1
2
(-> coll
    ((partial map inc)))

Which is silly, since we can just use the less awkward ->> in this case:

1
2
(->> coll
     (map inc))

So, I definitely recommend preferring ->> as it leads to more reasonable argument order that can better be composed with other functions. Unfortunately, we can’t just be all happy using ->> as we’ll have to keep using -> for functions like assoc/dissoc. Maybe having them with multiple arity was not such a great idea to start with.

Destructuring Order in Or-clauses

Lately I was wondering how to set variables in a specific order in a :or clause in Clojure’s destructuring mini language. I needed to set a key b to the return value of a function f that is dependant on a key a.

My first thought was that the :or {} syntax wouldn’t obey the order (since it is a map), so I’ve used :or []. But that doesn’t work at all, it does not do anything, both get bound to nil, as if no :or was specified.

When using :or {} it magically worked, but how can you trust that the order is always correct, since maps are inherently unordered?

Turns out, the order in which keys are taken from the :or map depends on the :keys vector! Compare specifying a before b, which works as expected:

1
2
3
4
5
6
7
8
9
10
11
12
13
(destructure '[{:keys [a b] :or {a x b (f a)}} x])
;=>
[map__39123
 x
 map__39123
 (if
  (clojure.core/seq? map__39123)
  (clojure.lang.PersistentHashMap/create (clojure.core/seq map__39123))
  map__39123)
 a
 (clojure.core/get map__39123 :a x)
 b
 (clojure.core/get map__39123 :b (f a))]

with specifying b before a which fails since a is not yet known.

1
2
3
4
5
6
7
8
9
10
11
12
13
(destructure '[{:keys [b a] :or {a x b (f a)}} x])
;=>
[map__39117
 x
 map__39117
 (if
  (clojure.core/seq? map__39117)
  (clojure.lang.PersistentHashMap/create (clojure.core/seq map__39117))
  map__39117)
 b
 (clojure.core/get map__39117 :b (f a))
 a
 (clojure.core/get map__39117 :a x)]

This behaviour works well but it was rather unexpected. It kinda does explain why the :keys takes a vector as a value and not a set, because the order is important.

Thanks to Jan Stępień for getting to the bottom of this.

Disabling Mouse in Spacemacs

Those of you following me on Twitter already heard that I’ve been trying out Spacemacs lately — a GNU Emacs configuration to emulate Vim. The reason behind it is, I want to check out whether writing Clojure in Emacs works better than in Vim (which is pretty decent, actually).

Now there is one thing that bothered me the most, coming from console vim: every time I switch focus to Emacs by clicking in a random place on the window, the cursor moves to this place. Incredibly inconvenient, since I don’t want to move my cursor accidentally. Since I never use the mouse for anything in my editor anyway, I decided to disable it.

Turn’s out, it is not that easy, since it is not a global key binding, but one that is local to the Evil mode. Frustrating to figure out, so here’s how to do it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
(defun dotspacemacs/config ()
  ;; your configuration stuff here

  ;; dummy
  (defun silence ()
    (interactive))

  ;; don't jump the cursor around in the window on clicking
  (define-key evil-motion-state-map [down-mouse-1] 'silence)
  ;; also avoid any '<mouse-1> is undefined' when setting to 'undefined
  (define-key evil-motion-state-map [mouse-1] 'silence)

  ;; more of your config, maybe
)

Hope that helps and watch this space for more adventures/rants in Space(macs)!

Lenovo X1 Carbon 2015 on Linux

In 2015, Lenovo managed to create a new ThinkPad that was again usable, namely the X1 Carbon (aka 3rd gen). After the last iteration of the Carbon was pretty terrible they went back to a more proven formula, for the better.

I needed a laptop for work, so I decided I give this device a spin. I got a BTO model with an i7-5500U (Broadwell, 2.4 GHz), the 256 GB PCIe SSD and the WQHD screen (2560x1440) along with the US keyboard with Euro key.

The device

Generally, I am quite satisfied. Compared with my T41 and T42p it is much faster, compared to my T420 it is much thinner. The keyboard is, unlike on the T420 an island-style keyboard but works pretty well in general. The layout is pretty sensible. I never use the keyboard backlight, just like I never used the ThinkLight, so whatever. The screen is pretty good, the frame is much thinner than my other devices. The touchpad is quite nice, although it is sometimes tricky to right-click — but I’ve always been a TrackPoint kind of guy. This is where the device has the biggest flaw: the trackpoint is set slightly below the keyboard and requires surprisingly much force to use. I’ve ended up getting an external mouse.

The SSD is ridiculously fast, but my workflows are rarely disk-bound.

If there is one word to describe the hardware it would be “slick”.

Linux

Having read some pretty sobering experiences I can now offer a counterpoint: the hardware support in Fedora 22 (Kernel 4.0.4) is pretty solid. The installer of the Workstation edition works without issues, no problems with graphics whatsoever, did not encounter any problems with WiFi (using it since two weeks). Suspend works, the cursor works and sound is no problem.

I’ve also tried DisplayPort to HDMI which worked only with a 1080p resolution (which is supposedly the fault of the Dell 27” screen I’ve been using), but when using DisplayPort to DisplayPort natively, the external screen lights up with 2560x1440 just fine. I’ve also tried the VGA adapter, this also worked out of the box. I did not try Bluetooth (my mouse uses Logitech Unifying dongle) or mobile data (since I haven’t ordered any mobile data cards).

Battery time seems quite decent, six hours or so. I’ve never discharged the device that much, since most of the time I’m sitting at my desk with the charger plugged it. Unfortunately, they changed the charger again, so now I have a third generation of chargers for ThinkPads. Honestly, I liked the round plug of the T420 the most.

In normal use, the device is absolutely silent. Whee. When I start some more complicated processes, it gets a bit louder, but it still isn’t anywhere bad.

Overall: pretty good! Fedora also did a good job packaging it together.

The bad parts

This is something that I cannot blame on Lenovo. The HiDPI mode on Linux sucks. GTK+ only supports integer scaling factors, so I can choose between 2560x1440 or 1280x720. The latter is of course completely ridiculous on such a pretty device, the former is ok but quite small. Coupling a HiDPI device (X1 Carbon) with a non-HiDPI device (external screen) isn’t supported, so either you use both HiDPI or both unscaled. A 27” screen in 1280x720 is just hilariously terrible. Maybe Wayland will solve this issue, since i don’t think this is an overly exotic problem.

Firefox fortunately does support scaling by any factor, which is nice. So when working I have my terminals on my external screen and Firefox on the laptop. I’ve set the scaling manually to something like 1.6x, which does not make everything look huge but is still legible. While this sounds awesome, I of course ran into another bug, where context menus are displayed on the wrong screen when using a scaling factor that is not 1 or 2. So, everytime I open the context menu, it just opens on my other screen :-(

tl;dr

Hardware works really well with Linux, Linux software has issues with HiDPI, maybe solvable through Wayland in the next couple of decades. Would recommend the device.

Go 1.4.2 for Raspberry Pi

I was notified by multiple mails that my package signing key expired. Thanks. I’ve changed the expiration date. So in case you are affected, run the first line (wget …) below and you can access the packages again. Took me some time because I moved and didn’t plug the Raspberry Pi back in.

While I was updating, Debian jessie came out and the Debian release freeze ended, so I decided that I might as well update to the newest Go release available from Debian, 1.4.2.

So if you just stumbled onto this blog, looking for Go packages for your Raspberry Pi (I’ve tried with the Raspberry Pi Model B), here are the instructions that you should run to get a working Go compiler on your mini-computer.

1
2
3
4
5
wget https://xivilization.net/~marek/raspbian/xivilization-raspbian.gpg.key -O - | sudo apt-key add -
sudo wget https://xivilization.net/~marek/raspbian/xivilization-raspbian.list -O /etc/apt/sources.list.d/xivilization-raspbian.list

sudo aptitude update
sudo aptitude install golang

Cycling in Munich

Inspired by a similar post by Julius, let’s contrast cycling in London with cycling in Munich. Especially as now the weather has been pretty good, so I started cycling to work almost every day.

Munich calls itself the “Radlhauptstadt”, which means the cycling capital. As Julius points out, there are not many capitals in Bavaria, so the competition for the title is rather easy. Am I hinting on what is coming? You bet.

Let’s start with the good things: Munich is contrary to the fact that it is reasonably close to the Alps, pretty flat, which makes overall pretty easy cycling. The second thing that is good in Munich is that pedestrians usually say clear from cycleways (if you see people there it’s 95% chance they are Asian, who don’t care about cycleways, something I’ve experienced in Japan as well).

But cycling to work in Munich is an exercise in frustration and the city doesn’t take this seriously at all. Let me illustrate this with a couple of points.

Cycleways end abruptly

Munich has a number of roads reserved for cyclists. This is all well and good, but unfortunately, you can’t get anywhere using them. They mostly start at some random place and end at some other random place and in between you have to switch to the street.

I can get to the main station in Munich on cycleways but then they completely drop the ball and you have to either take a detour or take the road. The road shared with light trams, huge long distance busses and rush hour traffic.

After surviving the main station, there is some cycleway until you get to the old city, which is also horrible to traverse by bike, due to mixing cyclist traffic with pedestrians, light trains (and especially their tracks, which are deadly to road bike wheels) and paving.

Crappy cycleways

Even if there are cycleways, they are often of bad quality: very bumpy, narrow, badly maintained. In this regard I’ll disagree with Julius, the cycleways on the streets are above and beyond much better quality-wise than the cycleways. Because roads for cars are built with quality in mind so that space to the right reserved for bikes is fine. Usually, they are also wide enough and in Germany, cars tend to leave you alone there. But these are rare, on the 8km I cycle to work, its two stretches, 1km tops.

Mandatory cycleways

There is a certain sign, pretty commonly used in Germany which forces cyclist to take the cycleway. It is incredibly infuriating when you see a really nice, wide street and you’re forced on the sidewalk with some white line drawn on it. You’re losing so much time and comfort, by being, effectively, sidelined.

It gets even better, since the cycleways tend to be directional, so you can’t just return the way you came, you’d have to cross the road. Which on bigger streets would mean that you have to drive around the block to the next crossing. All the downsides of cars, with none of the upsides.

Cars

Even if you are on a cycleway and are going in the right direction (congratulations!), cars like to park on the side and block at least part of it. Also, while german drivers tend to be pretty good, I often get shouted at by some know-it-alls for not obeying some fictional traffic rules. The atmosphere is sometimes quite hostile.

Part of this is surely caused by cyclist breezing through red lights, but some lights are adjusted horribly, cyclists would need to stop every 20m for the next red light. No wonder they often breeze through shortly after it switches to red.

Locking, oh gosh, the locks

So, there are bike locking stands on many places, great. You lock your bike, you come back after a day and what is left is your front wheel.

German locking stands have (for the most part) stagnated since 1927 with the invention of the Quick release skewer. It allows taking of your wheels with no tools required, and conversely also taking of your bike from a bike locking stand which only locks your front wheel to the stand.

I used to think, that these kinds of locking stands were standard, but I have seen exacly none of them visiting London. All locking stands had a way to lock your frame to it, which makes sense if you like to keep your bike. So it is pure ignorance that locking is so bad.

I usually lock my bike to handrails of stairs or similar because proper locking stands are still rare. At least, this is getting better.

Concluding

I don’t see any serious effort of the city to make cycling more appealing. The current traffic rules for bikes are out of touch with reality and to create better infrastructure for cyclists would require rethinking how bicycles, cars and pedestrians interact in a modern city. Maybe cities like Amsterdam or Copenhagen can lead us the way.

Go 1.3.3 for Raspberry Pi

As I had some time, I decided to update the Go(lang) package to the newest version available in Debian testing/unstable. Go 1.4.1 is only in experimental yet, so I’ve been holding off for now.

These packages were built on an Raspberry Pi Model B, and should be able to run on every Raspberry Pi (also Raspberry Pi 2) with Raspbian. They include support for all kinds of crosscompilers for Linux on x86 and x86_64 as well as FreeBSD, NetBSD, OS X and Windows.

If you have installed my backport already, then Go 1.3.3 is just a aptitude upgrade away. If you haven’t, here’s the steps:

1
2
3
4
5
wget https://xivilization.net/~marek/raspbian/xivilization-raspbian.gpg.key -O - | sudo apt-key add -
sudo wget https://xivilization.net/~marek/raspbian/xivilization-raspbian.list -O /etc/apt/sources.list.d/xivilization-raspbian.list

sudo aptitude update
sudo aptitude install golang

Avoiding Action at a Distance Is the Fast Track to Functional Programming

That’s an odd title, for sure. What is “action at a distance”? Well, how about an example? Let’s take Python 3.x, but it would work very much the same with JavaScript. Or Lua. Or most other languages.

1
2
3
4
lat = [1, 2, 3]
for e in [4, 5, 6]:
    lat.append(e)
print(lat)

A perfectly fine program, right? It adds some elements to a list, nothing unusual (yes, I know extend exists, but extend is only a shortcut for exactly this functionality), you might see that in every Python program ever.

Now, how about we wrap that in a function?

1
2
3
4
5
6
lat = [1, 2, 3]
def extend():
    for e in [4, 5, 6]:
        lat.append(e)
extend()
print(lat)

That’s something most semi-knowledgeable Pythoneers would object, since you can call that function anywhere and the value of lat magically changes. This is what I was calling “action at a distance”. You can call that also “nonlocal modification/assignment”, and Python 3 even has a nonlocal keyword to allow such nonlocal assignments.

So, looking at the function, it takes no arguments and returns None. Python-programmers will say: ok, then, make lat an argument and return the new list instead.

1
2
3
4
5
6
7
lat = [1, 2, 3]
def extend(lst):
    for e in [4, 5, 6]:
        lst.append(e)
    return lst
lat = extend(lat)
print(lat)

Great, now the function is not able to magically change lat any time it wants. When we look into this code, we have basically run into a circle: the for loop “takes” no input data and “returns” no output data, it just magically changes lst. This is a small toy function, so it is easy to see, but conceptually we can think about it again as a function that does action on a distance.

The usual way is to do so is to write functions that take arguments and return some new values. Unfortunately, append does not return a value, it, again, does action at a distance in some way, “magically” adding an element to the list. That’s easy to fix, let’s make our own append operator:

1
2
def append(lst, e):
    return lst + [e]

With this in order, we would need to call append(lst, 4) and then append(that_result, 5) and append(previous_result, 6) on it, thus constructing append(append(append(lst, 4), 5, 6). Fortunately, Python ships with a function that does exactly that, in 2.x it is called reduce and in 3.x it is still called reduce but moved into the functools module.

1
2
def extend(lst):
    return functools.reduce(append, [4, 5, 6], lst)

The whole code looks like this now:

1
2
3
4
5
6
7
8
import functools
lat = [1, 2, 3]
def append(lst, e):
    return lst + [e]
def extend(lst):
    return functools.reduce(append, [4, 5, 6], lst)
lat = extend(lat)
print(lat)

This time we eliminated all action-at-a-distance operations. Every operation takes an input and returns a result, there is no more things changing in the background without an assignment. Interestingly enough, we have just invented functional programming, a paradigm that avoids action-at-a-distance as much as possible, which makes reasoning about code easier. This is, by the way called “referential transparency”, a fancy-pants name for “nothing happens at a distance”.

Would you think, that there are people who made programming languages which only do functional programming?

Secure Secure Shell on Debian Wheezy

If you have passingly paid attention to the internet at all lately, you probably ran across the Secure Secure Shell article, which advises on how to better secure your SSH sessions. This is nice and all, but it doesn’t apply to Debian 7, Wheezy, too much: “Warning: You will need a recent OpenSSH version. It should work with 6.5 but I have only tested 6.7 […]”.

Yeah, I’ve tested it on the version shipped with Debian Wheezy, 6.0 and guess what, many of the secure algorithms are unavailable. You could try installing backports of newer OpenSSH versions, but then you have to hope that these get maintained as well as the regular Debian security updates. And if you have many machines installing the backport might be tedious. So now that the next version of Debian, Jessie/8 is in freeze state which will ship with OpenSSH 6.7, I don’t feel like updating.

So what exactly is missing? As I saw everything using Curve25519 is not yet implemented in OpenSSH 6.0, so forget about the key exchange and Ed25519 public keys, but the Encrypt-then-MAC modes aren’t here either. ChaCha20? Nope, sorry.

Stripping out the algorithms that are missing, left me with these settings in /etc/ssh/sshd_config:

1
2
3
KexAlgorithms diffie-hellman-group-exchange-sha256
Ciphers aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512,hmac-sha2-256,hmac-ripemd160

On the positive side: you can upgrade your RSA keys to 4096 bits (if they aren’t already) and you can delete your DSA and ECDSA keys. You can edit the module to only have larger values, as described in the article.

Overall, I’d say it is decent. I haven’t seen any incompatibility so far with the clients I used (OpenSSH and JuiceSSH, the latter only recently implemented diffie-hellman-group-exchange-sha256 and support for higher DH moduli), so that’s good.

Reinventing the Semicolon Operator

Lately I was writing some semi-imperative code in OCaml and thinking to myself “man, these ; everywhere sure look ugly”. As I am quite a fan of the |> operator, which threads the execution of multiple functions into a chain: every function gets the result of the application of the previous function, so a |> b |> c corresponds to c (b a).

What I wanted was an operator like |> which throws away the result of the previous function and calls the next function in a normal way. So I named it |-, which coult be used like fun1 |- fun2, which would evaluate to the value of fun2. We could even enforce that the value that fun1 is evaluated to is to be () (unit), so we don’t throw away any important data.

So, let’s implement it:

1
2
let (|-) a b =
  a; b

Then it dawned on me: The operator I was looking for existed all along, it is just ;, I can replace all occurences of |- by ;!

This might look like a trivial epiphany, but it has indeed deepened my understanding of the semicolon operator in the language. What I have previously thought of as an purely syntactic element of the language turned out to be an operator in disguise1 and rather unlike how for example Python uses ; (to concatenate multiple expressions into one expression).


  1. Actually, this is not quite true, since ; is indeed syntax, but can be thought as just syntactic sugar for an operator that drops the result of the evaluation of its first argument.