There is a Linux distribution called Gentoo, named after a type of penguin (of course it’s named after a penguin), where installing an app doesn’t mean that you download a working app. Instead, when you say “install this app”, it downloads the source code for that app and then compiles it on your computer. This apparently gives you the freedom to make changes to exactly how that app is built, even as it requires you to have a full set of build tools and compilers and linkers just to get a calculator. I think it’s clear that the world at large has decided that this is not the way to do things, as evidenced by how almost no other OSes take this approach — you download a compiled binary of an app and run it, no compiling involved — but it’s nice that it exists, so that the few people who really want to take this approach can choose to do so.

This sort of thing gets a lot of sneering from people who think that all Linux OSes are like that, that people who run Linux think that it’s about compiling your own kernels and using the Terminal all the time. Why would you want to do that sort of thing, you neckbeard, is the underlying message, and I largely agree with it; to me (and most people) it seems complicated and harder work for the end user, and mostly a waste of time — the small amount of power I get from being able to tweak how a thing is built is vastly outweighed by the annoyance of having to build it if I want it. Now, a Gentoo user doesn’t actually have to know anything about compilation and build tools, of course; it’s all handled quietly and seamlessly by the install command, and the compilers and linkers and build tools are run for you without you needing to understand. But it’s still a bunch of things that my computer has to do that I’m just not interested in it doing, and I imagine you feel the same.

So I find it disappointing that this is how half the web industry have decided to make websites these days.

We don’t give people a website any more: something that already works, just HTML and CSS and JavaScript ready to show them what they want. Instead, we give them the bits from which a website is made and then have them compile it.

Instead of an HTML page, you get some templates and some JSON data and some build tools, and then that compiler runs in your browser and assembles a website out of the component parts. That’s what a “framework” does… it builds the website, in real time, from separate machine-readable pieces, on the user’s computer, every time they visit the website. Just like Gentoo does when you install an app. Sure, you could make the case that the browser is always assembling a website from parts — HTML, CSS, some JS— but this is another step beyond that; we ship a bunch of stuff in a made-up framework and a set of build tools, the build tools assemble HTML and CSS and JavaScript, and then the browser still has to do its bit to build that into a website. Things that should be a long way from the user are now being done much closer to them. And why? “We’re layering optimizations upon optimizations in order to get the SPA-like pattern to fit every use case, and I’m not sure that it is, well, worth it.” says Tom MacWright.

Old joke: someone walks into a cheap-looking hotel and asks for a room. You’ll have to make your own bed, says the receptionist. The visitor agrees, and is told: you’ll find a hammer and nails behind the door.

What about the About page? That is just static text, surely that’s quicker? No, it is worse. (Of course, all the JS/CSS would be cached, but so to on mine, and you still don’t have to wait for JS to construct your HTML if you just give it HTML.) pic.twitter.com/Rxyz322ngt

— Matthew Somerville (@dracos) April 19, 2020

Almost all of us don’t want this for our native apps, and think it would be ridiculous; why have we decided that our users have to have it on their websites? Web developers: maybe stop insisting that your users compile your apps for you? Or admit that you’ll put them through an experience that you certainly don’t tolerate on your own desktops, where you expect to download an app, not to be forced to compile it every time you run it? You’re not neckbeards… you just demand that your users have to be. You’re neckbeard creators. You want to browse this website? Here’s a hammer and nails.

Unless you run Gentoo already, of course! In which case… compile away.

Sometimes you’ll find yourself doing a job where you need to make alterations to a web page that already exists, and where you can’t change the HTML, so your job is to write some bits of JavaScript to poke at the page, add some attributes and some event handlers, maybe move some things around. This sort of thing comes up a lot with accessibility remediations, but maybe you’re working with an ancient CMS where changing the templates is a no-no, or you’re plugging in some after-the-fact support into a site that can’t be changed without a big approval process but adding a script element is allowed. So you write a script, no worries. How do you test it?

Well, one way is to actually do it: we assume that the way your work will eventually be deployed is that you’ll give the owners a script file, they’ll upload it somehow to the site and add a script element that loads it. That’s likely to be a very slow and cumbersome process, though (if it wasn’t, then you wouldn’t need to be fixing the site by poking it with JS, would you? you’d just fix the HTML as God intended web developers to do) and so there ought to be a better way. A potential better way is to have them add a script element that points at your script on some other server, so you can iterate on that and then eventually send over the finished version when done. But that’s still pretty annoying, and it means putting that on the live server (“a ‘staging’ server? no, I don’t think we’ve got one of those”) and then having something in your script which only runs it if it’s you testing. Alternatively, you might download the HTML for the page with Save Page As and grab all the dependencies. But that never works quite right, does it?

The way I do this is with Greasemonkey. Greasemonkey, or its Chrome-ish cousin Tampermonkey, has been around forever, and it lets you write custom scripts which it then takes care of loading for you when you visit a specified URL. Great stuff: write your thing as a Greasemonkey script to test it and then when you’re happy, send the script file to the client and you’re done.

There is a little nuance here, though. A Greasemonkey script isn’t exactly the same as a script in the page. This is partially because of browser security restrictions, and partially because GM scripts have certain magic privileged access that scripts in the page don’t have. What this means is that the Greasemonkey script environment is quite sandboxed away; it doesn’t have direct access to stuff in the page, and stuff in the page doesn’t have direct access to it (in the early days, there were security problems where in-page script walked its way back up the object tree until it got hold of one of the magic Greasemonkey objects and then used that to do all sorts of naughty privileged things that it shouldn’t have been able to, and so it all got rigorously sandboxed away to prevent that). So, if the page loads jQuery, say, and you want to use that, then you can’t, because your script is in its own little world with a peephole to the page, and getting hold of in-page objects is awkward. Obviously, your remediation script can’t be relying on any of these magic GM privileges (because it won’t have them when it’s deployed for real), so you don’t intend to use them, but because GM doesn’t know that, it still isolates your script away. Fortunately, there’s a neat little trick to have the best of both worlds; to create the script in GM to make it easy to test and iterate, but have the script run in the context of the page so it gets the environment it expects.

What you do is, put all your code in a function, stringify it, and then push that string into an in-page script. Like this:

// ==UserScript==// @name     Stuart's GM remediation script// @version  1// @grant    none// ==/UserScript==functionmain(){/* All your code goes below here... *//* ...and above here. */}letscript=document.createElement("script");script.textContent="("+main.toString()+")();";document.body.appendChild(script);

That’s it. Your code is defined in Greasemonkey, but it’s actually executed as though it were a script element in the page. You should basically pretend that that code doesn’t exist and just write whatever you planned to inside the main() function. You can define other functions, add event handlers, whatever you fancy. This is a neat trick; I’m not sure if I invented it or picked it up from somewhere else years ago (and if someone knows, tell me and I’ll happily link to whoever invented it), but it’s really useful; you build the remediation script, doing whatever you want it to do, and then when you’re happy with it, copy whatever’s inside the main() function to a new file called whatever.js and send that to the client, and tell them: upload this to your creaky old CMS and then link to it with a script element. Job done. Easier for you, easier for them!

An interesting writeup by Brian Kardell on web engine diversity and ecosystem health, in which he puts forward a thesis that we currently have the most healthy and open web ecosystem ever, because we’ve got three major rendering engines (WebKit, Blink, and Gecko), they’re all cross-platform, and they’re all open source. This is, I think, true. Brian’s argument is that this paints a better picture of the web than a lot of the doom-saying we get about how there are only a few large companies in control of the web. This is… well, I think there’s truth to both sides of that. Brian’s right, and what he says is often overlooked. But I don’t think it’s the whole story.

You see, diversity of rendering engines isn’t actually in itself the point. What’s really important is diversity of influence: who has the ability to make decisions which shape the web in particular ways, and do they make those decisions for good reasons or not so good? Historically, when each company had one browser, and each browser had its own rendering engine, these three layers were good proxies for one another: if one company’s browser achieved a lot of dominance, then that automatically meant dominance for that browser’s rendering engine, and also for that browser’s creator. Each was isolated; a separate codebase with separate developers and separate strategic priorities. Now, though, as Brian says, that’s not the case. Basically every device that can see the web and isn’t a desktop computer and isn’t explicitly running Chrome is a WebKit browser; it’s not just “iOS Safari’s engine”. A whole bunch of long-tail browsers are essentially a rethemed Chrome and thus Blink: Brave and Edge are high up among them.

However, engines being open source doesn’t change who can influence the direction; it just allows others to contribute to the implementation. Pick something uncontroversial which seems like a good idea: say, AVIF image format support, which at time of writing (May 2020) has no support in browsers yet. (Firefox has an in-progress implementation.) I don’t think anyone particularly objects to this format; it’s just not at the top of anyone’s list yet. So, if you were mad keen on AVIF support being in browsers everywhere, then you’re in a really good position to make that happen right now, and this is exactly the benefit of having an open ecosystem. You could build that support for Gecko, WebKit, and Blink, contribute it upstream, and (assuming you didn’t do anything weird), it’d get accepted. If you can’t build that yourself then you ring up a firm, such as Igalia, whose raison d’etre is doing exactly this sort of thing and they write it for you in exchange for payment of some kind. Hooray! We’ve basically never been in this position before: currently, for the first time in the history of the web, a dedicated outsider can contribute to essentially every browser available. How good is that? Very good, is how good it is.

Obviously, this only applies to things that everyone agrees on. If you show up with a patchset that provides support for the <stuart> element, you will be told: go away and get this standardised first. And that’s absolutely correct.

But it doesn’t let you influence the strategic direction, and this is where the notions of diversity in rendering engines and diversity in influence begins to break down. If you show up to the Blink repository with a patchset that wires an adblocker directly into the rendering engine, it is, frankly, not gonna show up in Chrome. If you go to WebKit with a complete implementation of service worker support, or web payments, it’s not gonna show up in iOS Safari. The companies who make the browsers maintain private forks of the open codebase, into which they add proprietary things and from which they remove open source things they don’t want. It’s not actually clear to me whether such changes would even be accepted into the open source codebases or whether they’d be blocked by the companies who are the primary sponsors of those open source codebases, but leave that to one side. The key point here is that the open ecosystem is only actually open to non-controversial change. The ability to make, or to refuse, controversial changes is reserved to the major browser vendors alone: they can make changes and don’t have to ask your permission, and you’re not in the same position. And sure, that’s how the world works, and there’s an awful lot of ingratitude out there from people who demand that large companies dedicate billions of pounds to a project and then have limited say over what it’s spent on, which is pretty galling from time to time.

Brian references Jeremy Keith’s Unity in which Jeremy says: “But then I think of situations where complete unity isn’t necessarily a good thing. Take political systems, for example. If you have hundreds of different political parties, that’s not ideal. But if you only have one political party, that’s very bad indeed!” This is true, but again the nuance is different, because what this is about is influence. If one party wins a large majority, then it doesn’t matter whether they’re opposed by one other party or fifty, because they don’t have to listen to the opposition. (And Jeremy makes this point.) This was the problem with Internet Explorer: it was dominant enough that MS didn’t have to give a damn what anyone else thought, and so they didn’t. Now, this problem does eventually correct itself in both browsers and political systems, but it takes an awfully long time; a dominant thing has a lot of inertia, and explaining to a peasant in 250AD that the Roman Empire will go away eventually is about as useful as explaining to a web developer in 2000AD that CSS is coming soon, i.e., cold comfort at best and double-plus-frustrating at worst.

So, a qualified hooray, I suppose. I concur with Brian that “things are better and healthier because we continue to find better ways to work together. And when we do, everyone does better.” There is a bunch of stuff that is uncontroversial, and does make the web better, and it is wonderful that we’re not limited to begging browser vendors to care about it to get it. But I think that definition excludes a bunch of “things” that we’re not allowed, for reasons we can only speculate about.

I’m not outside.

Right now, a mass of people are in Centenary Square in Birmingham.

They’ll currently be chanting. Then there’s music and speeches and poetry and a lie-down. I’m not there. I wish I was there.

This is part of the Black Lives Matter protests going on around the world, because again a black man was murdered by police. His name was George Floyd. That was in Minneapolis; a couple of months ago Breonna Taylor, a black woman, was shot eight times by police in Louisville. Here in the UKblack and minority ethnicity people die in police custody twice as much as others.

It’s 31 years to the day since the Tiananmen Square protests in China in which a man stood in front of a tank, and then he disappeared. Nobody even knows his name, or what happened to him.

The protests in Birmingham today won’t miss one individual voice, mine. And the world doesn’t need the opinion of one more white guy on what should be done about all this, about the world crashing down around our ears; better that I listen and support. I can’t go outside, because I’m immunocompromised. The government seems to flip-flop on whether it’s OK for shielding people to go out or not, but in a world where there are more UK deaths from the virus than the rest of the EU put together, where as of today nearly forty thousand people have died in this country — not been inconvenienced, not caught the flu and recovered, died, a count over half that of UK civilian deaths in World War 2 except this happened in half a year — in that world, I’m frightened of being in a large crowd, masks and social distancing or not. But the crowd are right. The city is right. When some Birmingham council worker painted over an I Can’t Breathe emblem, causing the council to claim there was no political motive behind that (tip for you: I’m sure there’s no council policy to do it, and they’ve unreservedly apologised, but whichever worker did it sure as hell had a political motive), that emblem was back in 24 hours, and in three other places around the city too. Good one, Birmingham.

There are apparently two thousand of you. I can hear the noise from the window, and it’s wonderful. Shout for me too. Wish I could be there with you.

My city would be shit, insipid and soulless, if it wasn’t for the beautiful fusion of cultures and heritages from all over the world - each contributing so much (arts, music, food and much more) that has enriched our lives immeasurably ✊🏾✊🏽✊🏿✊🏼https://t.co/A4E1CDfK4opic.twitter.com/FVbwOJ9X2e

— Cold War Steve (@Coldwar_Steve) June 3, 2020

One of the projects I’m working on involves creating a little device which you talk to from your phone. So, I thought, I’ll do this properly. No “cloud service” that you don’t need; no native app that you don’t need; you’ll just send data from your phone to it, locally, and if the owners go bust it won’t brick all your devices. I think a lot of people want their devices to live on beyond the company that sold them, and they want their devices to be under their own control, and they want to be able to do all this from any device of their choosing; their phone, their laptop, whatever. An awful lot of devices don’t do some or all of that, and perhaps we can do better. That is, here’s the summary of that as a sort of guiding principle, which we’re going to try to do:

You should be able to communicate a few hundred KB of data to the device locally, without needing a cloud service by using a web app rather than a native app from an Android phone.

Here’s why that doesn’t work. Android and Chrome, I am very disappointed in you.

Bluetooth LE

The first reaction here is to use Bluetooth LE. This is what it’s for; it’s easy to use, phones support it, Chrome on Android has Web Bluetooth, everything’s gravy, right?

No, sadly. Because of the “a few hundred KB of data” requirement. This is, honestly, not a lot of data; a few hundred kilobytes at most. However… that’s too much for poor old Bluetooth LE. An excellent article from AIM Consulting goes into this in a little detail and there’s a much more detailed article from Novelbits, but transferring tens or hundreds of KB of data over BLE just isn’t practical. Maybe you can get speeds of a few hundred kilo bits per second in theory, but in practice it’s nothing like that; I was getting speeds of twentybytes per second, which is utterly unhelpful. Sure, maybe it can be more efficient than that, but it’s just never going to be fast enough: nobody’s going to want to send a 40KB image and wait three minutes for it to do so. BLE’s good for small amounts of data; not for even medium amounts.

WiFi to your local AP

The next idea, therefore, is to connect the device to the wifi router in your house. This is how most IoT devices work; you teach them about your wifi network and they connect to it. But… how do you teach them that? Normally, you put them in some sort of “setup” mode and the device creates its own wifi network, and then you connect your phone to that, teach it about your wifi network, and then it stops its own AP and connects to yours instead. This is maybe OK if the device never moves from your house and it only has one wifi network to connect to; it’s terrible if it’s something that moves around to different places. But you still need to connect to its private AP first to do that setup, and so let’s talk about that.

WiFi to the device

The device creates its own WiFi network; it becomes a wifi router. You then connect your phone to it, and then you can talk to it. The device can even be a web server, so you can load the controlling web app from the device itself. This is ideal; exactly what I planned.

Except it doesn’t work, and as far as I can tell it’s Android’s fault. Bah humbug.

You see, the device’s wifi network obviously doesn’t have a route to the internet. So, when you connect your phone to it, Android says “hey! there’s no route to the internet here! this wifi network sucks and clearly you don’t want to be connected to it!” and, after ten seconds or so, disconnects you. Boom. You have no chance to use the web app on the device to configure the device, because Android (10, at least) disconnects you from the device’s wifi network before you can do so.

Now, there is the concept of a “captive portal”. This is the thing you get in hotels and airports and so on, where you have to fill in some details or pay some money or something to be able to use the wifi; what happens is that all web accesses get redirected to the captive portal page where you do or pay whatever’s necessary and then the network suddenly becomes able to access the internet. Android will helpfully detect these networks and show you that captive portal login page so you can sign in. Can we have our device be a captive portal?

No. Well, we can, but it doesn’t help.

You see, Android shows you the captive portal login page in a special cut-down “browser”. This captive portal browser (Apple calls it a CNA, for Captive Network Assistant, so I shall too… but we’re not talking about iOS here, which is an entirely different kettle of fish for a different article), this CNA isn’t really a browser. Obviously, our IoT device can’t provide a route to the internet; it’s not that it has one but won’t let you see it, like a hotel; it doesn’t have one at all. So you can’t fill anything into the CNA that will make that happen. If you try to switch back to the real browser in order to access the website being served from the device, Android says “aha, you closed the CNA and there’s still no route to the internet!” and disconnects you from the device wifi. That doesn’t work.

You can’t open a page in the real browser from the CNA, either. You used to be able to do some shenanigans with a link pointing to an intent://URL but that doesn’t work any more.

Maybe we can run the whole web app inside the CNA? I mean, it’s a web browser, right? Not an ideal user experience, but it might be OK.

Nope. The CNA is a browser, but half of the features are turned off. There are a bunch of JavaScript APIs you don’t have access to, but the key thing for our purposes is that <input type="file"> elements don’t work; you can’t open a file picker to allow someone to choose a file to upload to the device. So that’s a non-starter too.

So, what do we do?

Unfortunately, it seems that the plan:

communicate a few hundred KB of data to the device locally, without needing a cloud service by using a web app rather than a native app from an Android phone

isn’t possible. It could be, but it isn’t; there are roadblocks in the way. So building the sort of IoT device which ought to exist isn’t actually possible, thanks very much Android. Thandroid. We have to compromise on one of the key points.

If you’re only communicating small amounts of data, then you can use Bluetooth LE for this. Sadly, this is not something you can really choose to compromise on; if your device plan only needs small volumes, great, but if it needs more then likely you can’t say “we just won’t send that data”. So that’s a no-go.

You can use a cloud service. That is: you teach the device about the local wifi network and then it talks to your cloud servers, and so does your phone; all data is round-tripped through those cloud servers. This is stupid: if the cloud servers go away, the device is a brick. Yes, lots of companies do this, but part of the reason they do it is that they want to be able to control whether you can access a device you’ve bought by running the connection via their own servers, so they can charge you subscription money for it. If you’re not doing that, then the servers are a constant ongoing cost and you can’t ever shut them down. And it’s a poor model, and aggressively consumer-hostile, to require someone to continue paying you to use a thing they purchased. Not doing that. Communication should be local; the device is in my house, I’m in my house, why the hell should talking to it require going via a server on the other side of the world?

You can use a native app. Native apps can avoid the whole “this wifi network has no internet access so I will disconnect you from it for your own good” approach by calling various native APIs in the connectivity manager. A web app can’t do this. So you’re somewhat forced into using a native app even though you really shouldn’t have to.

Or you can use something other than Android; iOS, it seems, has a workaround although it’s a bit dodgy.

None of these are good answers. Currently I’m looking at building native apps, which I really don’t think I should have to do; this is exactly the sort of thing that the web should be good at, and is available on every platform and to everyone, and I can’t use the web for it because a bunch of decisions have been taken to prevent that. There are good reasons for those decisions, certainly; I want my phone to be helpful when I’m on some stupid hotel wifi with a signin. But it’s also breaking a perfectly legitimate use case and forcing me to use native apps rather than the web.

Unless I’m wrong? If I am… this is where you tell me how to do it. Something with a pleasant user experience, that non-technical people can easily do. If it doesn’t match that, I ain’t doin’ it, just to warn you. But if you know how this can be done to meet my list of criteria, I’m happy to listen.

In the spring of 2020, the GNOME project ran their Community Engagement Challenge in which teams proposed ideas that would “engage beginning coders with the free and open-source software community [and] connect the next generation of coders to the FOSS community and keep them involved for years to come.” I have a few thoughts on this topic, and so does Alan Pope, and so we got chatting and put together a proposal for a programming environment for making simple apps in a way that new developers could easily grasp. We were quite pleased with it as a concept, but: it didn’t get selected for further development. Oh well, never mind. But the ideas still seem good to us, so I think it’s worth publishing the proposal anyway so that someone else has the chance to be inspired by it, or decide they want it to happen. Here:

Cabin: Creating simple apps for Linux, by Stuart Langridge and Alan Pope

I’d be interested in your thoughts.

My dad’s got a Brother DCP-7055W printer/scanner, and he wanted to be able to set it up as a network scanner to his Ubuntu machine. This was more fiddly than it should be, and involved a bunch of annoying terminal work, so I’m documenting it here so I don’t lose track of how to do it should I have to do it again. It would be nice if Brother made this easier, but I suppose that it working at all under Ubuntu is an improvement on nothing.

Anyway. First, go off to the Brother website and download the scanner software. At time of writing, https://www.brother.co.uk/support/dcp7055/downloads has the software, but if that’s not there when you read this, search the Brother site for DCP-7055 and choose Downloads, then Linux and Linux (deb), and get the Driver Installer Tool. That’ll get you a shell script; run it. This should give you two new commands in the Terminal: brsaneconfig4 and brscan-skey.

Next, teach the computer about the scanner. This is what brsaneconfig4 is for, and is all done in the Terminal. You need to know the scanner’s IP address; you can find this out from the scanner itself, or you can use avahi-resolve -v -a -r to search your network for it. This will dump out a whole load of stuff, some of which should look like this:

=  wlan0 IPv4 Brother DCP-7055W                             UNIX Printer         local   hostname = [BRN008092CCEE10.local]   address = [192.168.1.21]   port = [515]   txt = ["TBCP=F" "Transparent=T" "Binary=T" "PaperCustom=T" "Duplex=F" "Copies=T" "Color=F" "usb_MDL=DCP-7055W" "usb_MFG=Brother" "priority=75" "adminurl=http://BRN008092CCEE10.local./" "product=(Brother DCP-7055W)" "ty=Brother DCP-7055W" "rp=duerqxesz5090" "pdl=application/vnd.brother-hbp" "qtotal=1" "txtvers=1"]

That’s your Brother scanner. The thing you want from that is address, which in this case is 192.168.1.21.

Run brsaneconfig4 -a name="My7055WScanner" model="DCP-7055" ip=192.168.1.21. This should teach the computer about the scanner. You can test this with brsaneconfig4 -p which will ping the scanner, and brsaneconfig4 -q which will list all the scanner types it knows about and then list your added scanner at the end under Devices on network. (If your Brother scanner isn’t a DCP-7055W, you can find the other codenames for types it knows about with brsaneconfig4 -q and then use one of those as the model with brsaneconfig4 -a.)

You only need to add the scanner once, but you also need to have brscan-skey running always, because that’s what listens for network scan requests from the scanner itself. The easiest way to do this is to run it as a Startup Application; open Startup Applications from your launcher by searching from it, and add a new application which runs the command brscan-skey, and restart the machine so that it’s running.

If you don’t have the GIMP¹ installed, you’ll need to install it.

On the scanner, you should now be able to press the Scan button and choose Scan to PC and then Scan Image, and it should work. What will happen is that your machine will pop up the GIMP with the image, which you will then need to export to a format of your choice.

This is quite annoying if you need to scan more than one thing, though, so there’s an optional extra step, which is to change things so that it doesn’t pop up the GIMP and instead just saves the scanned photo which is much nicer. To do this, first install imagemagick, and then edit the file /opt/brother/scanner/brscan-skey/script/scantoimage-0.2.4-1.sh with sudo. Change the last line from

echo gimp -n $output_file 2>/dev/null \;rm -f $output_file | sh &

to

echo convert $output_file $output_file.jpg 2>/dev/null \;rm -f $output_file | sh &

Now, when you hit the Scan button on the scanner, it will quietly create a file named something like brscan.Hd83Kd.ppm.jpg in the brscan folder in your home folder and not show anything on screen, and this means that it’s a lot easier to scan a bunch of photos one after the other.

1. I hate this name. It makes us look like sniggering schoolboys. GNU Imp, maybe, or the new Glimpse fork, but the upstream developers don’t want to change it ↩

So, I was awarded a medal.

OpenUK, who are a non-profit organisation supporting open source software, hardware, and data, and are run by Amanda Brock, have published the honours list for 2021 of what they call “100 top influencers across the UK’s open technology communities”. One of them is me, which is rather nice. One’s not supposed to blow one’s own trumpet at a time like this, but to borrow a line from Edmund Blackadder it’s nice to let people know that you have a trumpet.

There are a bunch of names on this list that I suspect anyone in a position to read this might recognise. Andrew Wafaa at ARM, Neil McGovern of GNOME, Ben Everard the journalist and Chris Lamb the DPL and Jonathan Riddell at KDE. Jeni Tennison and Jimmy Wales and Simon Wardley. There are people I’ve worked with or spoken alongside or had a pint with or all of those things — Mark Shuttleworth, Rob McQueen, Simon Phipps, Michael Meeks. And those I know as friends, which makes them doubly worthy: Alan Pope, Laura Czajkowski, Dave Walker, Joe Ressington, Martin Wimpress. And down near the bottom of the alphabetical list, there’s me, slotted in between Terence Eden and Sir Tim Berners-Lee. I’ll take that position and those neighbours, thank you very much, that’s lovely.

I like working on open source things. It’s been a strange quarter-of-a-century, and my views have changed a lot in that time, but I’m typing this right now on an open source desktop and you’re probably viewing it in an open source web rendering engine. Earlier this very week Alan Pope suggested an app idea to me and two days later we’d made Hushboard. It’s a trivial app, but the process of having made it is sorta emblematic in my head — I really like that we can go from idea to published Ubuntu app in a couple of days, and it’s all open-source while I’m doing it. I like that I got to go and have a curry with Colin Watson a little while ago, the bloke who introduced me to and inspired me with free software all those years ago, and he’s still doing it and inspiring me and I’m still doing it too. I crossed over some sort of Rubicon relatively recently where I’ve been doing open source for more of my life than I haven’t been doing it. I like that as well.

There are a lot of problems with the open source community. I spoke about divisiveness over “distros” in Linux a while back. It’s still not clear how to make open source software financially sustainable for developers of it. The open source development community is distinctly unwelcoming at best and actively harassing and toxic at worst to a lot of people who don’t look like me, because they don’t look like me. There’s way too much of a culture of opposing popularity because it is popularity and we don’t know how to not be underdogs who reflexively bite at the cool kids. Startups take venture capital and make a billion dollars when the bottom 90% of their stack is open source that they didn’t write, and then give none of it back. Products built with open source, especially on the web, assume (to use Bruce Lawson’s excellent phrasing) that you’re on the Wealthy Western Web. The list goes on and on and on and these are only the first few things on it. To the extent that I have any influence as one of the one hundred top influencers in open source in the UK, those are the sort of things I’d like to see change. I don’t know whether having a medal helps with that, but last year, 2020, was an extremely tough year for almost everyone. 2021 has started even worse: we’ve still got a pandemic, the fascism has gone from ten to eleven, and none of the problems I mentioned are close to being fixed. But I’m on a list with Tim Berners-Lee, so I feel a little bit warmer than I did. Thank you for that, OpenUK. I’ll try to share the warmth with others.

Today I was vaccinated for Covid.

It occurred to me that people might have a question or two about the process, what it’s like, and what happens, and I think that’s reasonable.

Hang on, why did you get vaccinated? You’re not 70, you’re, what, thirty-six or something?

[bad Southern damsel accent] “Why, Mr Vorce in my Haird, I do declare, I’ve come over all a-flutter!” [bats eyelashes].

Nah, it’s ‘cos I’m considered clinically extremely vulnerable.

What’s vulnerable?

According to Google if you’re strong and vulnerable then it’s recommended to open with a no-Trump bid. That sounds like a good thing to me.

What’s it like?

It’s like the flu jab.

Like what?

Fair enough. If you’re reading this to learn about vaccination around the time that I’m posting it, January 2021, then maybe you’re someone like me who has had a bunch of injections and vaccinations, such as the flu jab every year. But if you’re reading it later, you may not be, and the idea of being vaccinated against something for the first time since school might be vaguely worrying to you because it’s an unknown sort of experience. This is fair. So, here’s the short form of what happens: you walk into a room and roll up your sleeve, they give you an injection into your arm that you hardly even feel, you sit in the foyer for 15 minutes, and then you go home. It’s like buying a kebab, except with fewer sheep eyebrows.

That’s a pretty short form. Some more detail than that would be nice.

True. OK, well, the first tip I can give you is: don’t put your big heavy coat and a scarf on in the mistaken belief that it must be cold because it’s January and then walk for 45 minutes to get to the hospital, because you’ll be boiling hot when you get there and aggravated.

Just the coat was the problem?

Well, it also turns out that if you stay indoors “shielding” for a year and then go on a long walk, you may discover that your formerly-Olympic-level of fitness has decayed somewhat. Might have to do a sit-up or two.

Or two hundred and two maybe, fatso.

Shut up, Imaginary Voice in my Head.

What next?

I was told to go to the hospital for the vaccination. Other people may be told to go to their GP’s office instead; it seems to vary quite a lot depending on where you live and what’s most accessible to you, and it’s possible that I am very lucky that I was sent to City Hospital, somewhere within walking distance. I’ve heard of others being sent to hospitals twenty miles away, which would have been a disaster for me because I don’t have a car. So, either Sandwell and West Birmingham NHS Trust are particularly good, or others are particularly bad, or I happened to roll double-sixes this time, not sure which.

How were you told?

I got a phone call, yesterday, from my GP. They asked when I was available, and suggested 12.55pm today, less than twenty-four hours later; I said I was available; that was it.

And at the hospital?

Finding the specific building was annoying. SWBH: put up some signs or something, will you? I mean, for goodness sake, I’ve been to the hospital about twenty times and I still had no idea where it was.

No more griping!

I haven’t got anything else to gripe about. It was a marvellously smooth process, once I found the building. This is what happened:

I walked up to the door at 12.45 for my appointment at 12.55. A masked woman asked for my name and appointment time; I gave them; she checked on a list and directed me inside, where I was met by a masked man. He gave me some hand sanitiser (I like hospital hand sanitiser. Seems better than the stuff I have) and directed me to a sort of booth thing, behind which was sat another man.

The booth seemed like a quite temporary thing; a rectangular box, like a ticket booth but probably made of thick cardboard, and with a transparent plastic screen dividing me from him; one of two or three of them in a line, I think. He asked me to confirm my details — name, appointment time, address — and then asked four or five very basic questions such as “do you have the symptoms of coronavirus?” Then he handed me two leaflets and directed me to two women, one of whom led me directly to an examination room in which were a man and a woman.

The man confirmed my details again, and asked if I’d been injected with anything else in the previous month; the woman asked me to roll up my sleeve (I ended up half taking my shirt off because it’s long-sleeved and rolling it all the way up to the shoulder is hard), and then gave me the injection in the upper part of my arm. Took about two seconds, and I hardly felt anything.

Wait, that’s it?

Yup. That whole process, from walking up to the door to being done, took maybe ten minutes maximum.

And then you left?

Not quite: they ask you to sit in the waiting room for fifteen minutes before leaving, just in case you have some sort of a reaction to the injection. People who have had the flu jab will recognise that they do the same thing there, too. This gave me a chance to read the two leaflets, both of which were fairly boring but important descriptions of what the vaccine is, what vaccines are in general, and any risks.

They also stuck a big label on my shirt showing when my fifteen minutes was up, which I think is a really good idea — they don’t do this for the flu jab, in my experience, but it’s a good idea for a vaccination where you have to put basically everybody through the process. I also got a little card which I’m meant to show at the second vaccination, which is now safely in my wallet and will probably still be there in twenty years unless they take it off me, along with the receipts for everything I’ve bought since about 2007.

So then you left?

Yes. Another of the staff confirmed I’d been there for long enough, asked if I was feeling OK (which I was), and asked if I had any questions. I didn’t, but I did ask if I had to go back to work now or if I could just hang out there for a while longer — I’m sure she’d heard variations on that eighty times already that day, but she laughed anyway, and I like trying to chat to hospital staff as human beings. God alone knows what a dreadful year they’ve had; they deserve courtesy and smiles from me at least.

Were they smiling?

Indeed they were. Even though you can’t tell because of the masks. Everyone I spoke to and everyone there was cheery, upbeat, courteous, and competent, without being dismissive or slick. I can’t speak to other NHS trusts, or other hospitals, or even other days in my hospital, but every time I’ve been there the staff have been helpful and nice and ready to share a joke or a chat or to answer a question, and this time was no exception.

What, you didn’t ask any questions at all? What about whether you’re being microchipped by Bill Gates? And the risks of 5G with the vaccine? And…

No, of course I didn’t. Vaccination is safe, and it’s one of the greatest inventions humanity has ever come up with. The NHS guidance on vaccinations is a nice summary here. If I’d walked in the door and someone in scrubs had said to me, “the best way to make web pages is to require 500KB of JavaScript to load before you put any text on screen”, or “Ubuntu has secret motives to undermine free software”, I would have said to them “no, you’re wrong about that” and argued. But me and the medical profession have an agreement: they don’t tell me how to build software, and I don’t tell them how to save millions of lives. This agreement is working out fine for both of us so far.

What’s that about risks and side-effects, though?

Apparently I may feel tired, or an ache in my arm, for the next couple of days. I’ll keep an eye out. They said that if it aches a bit, taking paracetamol is OK, but I am not a medical professional and you should ask the question yourself when you get vaccinated.

Which vaccine did you have?

Remember the two leaflets? One of them is a generic NHS leaflet about Covid vaccines; the other is specific to the one I had and is from BioNTech, which means it’s the Pfizer one. The little wallet card also says it’s Pfizer. I didn’t ask the staff because I could work it out myself and I’m not going to change anything based on their answer anyway; it’d be pure curiosity on my part. Also, see previous point about how I don’t tell them how to do their jobs. I assume that which vaccine I got was decided at some sort of higher-up area level rather than tailored for me specifically, but hey, maybe I got a specific one chosen for me. Dunno; that’s for medical people to know about and for me to follow.

What now?

The vaccine doesn’t properly kick in for a few weeks, plus you need the second injection to be properly vaccinated. That should be 9-12 weeks from now, I’m told, so I’ll be staying inside just as I was before. Might empty all the litter out of my wallet, too.

But I have more questions!

Well, I’m on twitter, if they’re sensible ones. Conspiracy stuff will just get you blocked and possibly reported with no interaction, so don’t do that. But this has been a delightfully simple process, made very easy by a bunch of people in the NHS who deserve more than having people clap a bit for them and then ignore the problems. So if I can help by answering a question or two to alleviate the load, I’m happy to do that. And thank you to them.

Are you really thirty-six?

Ha! As if. It is my birthday on Saturday, though.

You know that thing you said about bridge bids is nonsense, right?

Ah, a correction: also do not ask me questions about bridge. Please.

It’s my birthday!

This year, in the midst of a coronavirus lockdown, it’s been something of a quiet one. I got lots of nice best wishes from a bunch of people, which is terribly pleasing, and I had a nice conversation with the family over zoom. Plus, a really good Chinese takeaway delivered as a surprise from my mum and dad, and I suspect that if there were a video of them signing up for a Deliveroo account to do so it would probably be in the running for the Best Comedy BAFTA award.

Also I spent some time afternoon doing the present from my daughter, which is the Enigmagram, an envelope of puzzles which unlock a secret message (which is how I discovered it was from my daughter). I like this sort of thing a lot; I’ve bought a couple of the Mysterious Package Company‘s experiences as presents and they’re great too. Must be a fun job to make these things; it’s like an ARG or something, which I’d also love to run at some point if I had loads of time.

I’ve just looked back at last year’s birthday post, and I should note that Gaby has excelled herself again with birthday card envelope drawing this year, but nothing will ever, ever exceed the amazing genius that is the bookshelf portal that her and Andy got me for Christmas. It is amazing. Go and watch the video immediately.

Time for bed. I have an electric blanket now, which I was mocked for, but hey; I’m allowed. It’s really cosy and warm. Shut up.

Last week I was part of a meeting with the UK’s Competition and Markets Authority, the regulator, to talk about Apple devices and the browser choice (or lack of it) on them. They’re doing a big study into Apple’s conduct in relation to the distribution of apps on iOS and iPadOS devices in the UK, in particular, the terms and conditions governing app developers’ access to Apple’s App Store, and part of that involves looking at browsers on iOS, and part of that involves talking to people who work on the web. So myself and Bruce Lawson and another UK developer of iOS and web apps put together some thoughts and had a useful long meeting with the CMA on the topic.

They asked that we keep confidential the exact details of what was discussed and asked, which I think is reasonable, but I did put together a slide deck to summarise my thoughts which I presented to them, and you can certainly see that. It’s at kryogenix.org/code/cma-apple and shows everything that I presented to the CMA along with my detailed notes on what it all means.

Bruce had a similar slide deck, and you can read his slides on iOS’s browser monopoly and progressive web apps. Bruce has also summarised our other colleague’s presentation, which is what we led off with. The discussion that we then went into was really interesting; they asked some very sensible questions, and showed every sign of properly understanding the problem already and wanting to understand it better. This was good: honestly, I was a bit worried that we might be trying to explain the difference between a browser and a rendering engine to a bunch of retired colonel types who find technology to be baffling and perhaps a little unmanly, and this was emphatically not the case; I found the committee engaging and knowledgeable, and this is encouraging.

In the last few weeks we’ve seen quite a few different governments and regulatory authorities begin to take a stand against tech companies generally and Apple’s control over your devices more specifically. These are baby steps — video and music apps are now permitted to add a link to their own website, saints preserve us, after the Japan Fair Trade Commission’s investigation; developers are now allowed to send emails to their own users which mention payments, which is being hailed as “flexibility” although it doesn’t allow app devs to tell their users about other payment options in the app itself, and there are still court cases and regulatory investigations going on all around the world. Still, the tide may be changing here.

What I would like is that I can give users the best experience on the web, on the best mobile hardware. That best mobile hardware is Apple’s, but at the moment if I want to choose Apple hardware I have to choose a sub-par web experience. Nobody can fix this other than Apple, and there are a bunch of approaches that they could take — they could make Safari be a best-in-class experience for the web, or they could allow other people to collaborate on making the browser best-in-class, or they could stop blocking other browsers from their hardware. People have lots of opinions about which of these, or what else, could and should be done about this; I think pretty much everyone thinks that something should be done about it, though. Even if your goal is to slow the web down and to think that it shouldn’t compete with native apps, there’s no real reason why flexbox and grid and transforms should be worse in Safari, right? Anyway, go and read the talk for more detail on all that. And I’m interested in what you think. Do please hit me up on Twitter about this, or anything else; what do you think should be done, and how?

Last week I sat down with the UK Competition and Markets Authority to talk about browser choice on Apple devices, and whether the claims that limiting choice is good for security and privacy actually hold up. Here's the presentation I gave.https://t.co/7i35fwdnrM

— Stuart Langridge (@sil) September 6, 2021

A couple of weeks ago, my grandma died.

This was not wholly unexpected, and at the same time it was completely unexpected. I should probably explain that.

She was ninety, which is a fair age for anyone, but her mother (my great-grandmother) lived to be even older. What’s different is that nobody knew she was ninety, other than us. Most of her friends were in their mid-seventies. Now, you might be thinking, LOL, old, but this is like you in your mid-forties hanging out with someone who’s 30, or you in your late twenties hanging out with someone who’s 13, or you at eighteen hanging out with someone who’s still learning what letters are. Gaps get narrower as we get older, but the thing that most surprised me was that all her friends were themselves surprised at the age she was. She can’t have been that age, they said when we told them, and buried in there is a subtle compliment: she was like us, and we’re so much younger, and when we’re that much older we won’t be like her.

No. No, you won’t be like my grandma.

I don’t want to talk much about the last few weeks. We, my mum and me, we flew to Ireland in the middle of the night, we sorted out her house and her garden and her affairs and her funeral and her friends and her family, and we came home. All I want to say about it is that, and all I want to say about her is probably best said in the eulogy I wrote and spoke for her death, and I don’t want to say it again.

But (and this is where people in my family should tune out) I thought I’d talk about the website I made for her. Because of course I made a website. You know how some people throw themselves into work to dull the pain when something terrible happens to the people they love? I’m assuming that if you were a metalworker in 1950 and you wanted to handle your grief that a bunch of people got a bunch of metal stuff that you wouldn’t ordinarily have made. Well, I am no metalworker; I build the open web, and I perform, on conference stages or for public perception. So I made a website for my grandma; something that will maybe live on beyond her and maybe say what we thought about her.

Firstly I should say: it’s at kryogenix.org/nell because her name was Nell and I made it. But neither of those things are really true. Her name was Ellen, and what I did was write down what we all said and what we all did to say goodbye. I wanted to capture the words we said while they were still fresh in my memory, but more while how I felt was fresh in my memory. Because in time the cuts will become barely noticeable scars and I’ll be able to think of her not being here without stopping myself crying, and I don’t want to forget. I don’t want to lose it amongst memories of laughter and houses and lights. So I wrote down what we all said right now while I can still feel the hurt of it like fingernails, so maybe I won’t let it fade away.

I want to write some things about the web, but that’s not for this post. This post is to say: goodbye, Grandma.

Goodbye, Grandma. I tried to make a thing that would make people think of you when they looked at it. I wanted people to think of memories of you when they read it. So I made a thing of memories of you, and I spoke about memories of you, and maybe people who knew you will remember you and people who didn’t know you will learn about you from what we all said.

Goodbye, Grandma.

kryogenix.org/nell

The Adirondack Forty-Sixers are an organization of hikers who have climbed all forty-six of the traditionally recognized High Peaks of the Adirondack Mountains. They are often referred to just as 46ers.

Hi, I’m Stuart. I’m a 0er. But I am forty six, so that’s nearly the same.

The number 46 is boring, if I’m honest, but quite frankly I could do with a boring year. This last one has been troublesome — not just for me, for everybody, of course. Wikipedia suggests that 46 is a centred triangular number and the sum of the totient function for the first twelve integers, which is such a yawnarama that I think Ramanujan’s thing about the first dull number might actually be about 46. (The people’s republic of encyclopedia does then drop the bomb that 46 is the molar mass of ethanol, mind. Someone’s been paying attention.)

This is the twentieth year I’ve been wishing myself happy birthday on the web, which is nice (2003 up until last year and in between). The last few years have shown a worrying trend of, well, worrying about stuff, and this year is no different. Things kinda got worse generally, didn’t they?

I spent today talking to the family, which was good. It wasn’t quite the weekend we were all hoping for, but it was still good. I did really hope I’d get to see my dad today. Plus I have cool new shirts and a marzipan thing which I’m going to eat! And I got to have drinks with friends before going away for the weekend and that was also cool and they are excellent, thank you for that, especially when cakes showed up. Soon I shall get back to playing with the nRF52840 PCA10059 dongle single-board computer that Bill bought me for our Secret Project, or possibly just my new iPhone 13 mini. All quite exciting stuff. There’s D&D to plan, escape rooms to visit, holidays to sort out, restaurants to go to with my parents and my daughter. Pizza to eat, doubtless. Things will be back to normal soon. Now I have to do a bit of writing. But happy birthday to me.

The CMA, the UK’s regulator of business competition and markets, what the USA calls “antitrust”, is conducting a study into mobile platforms and the mobile ecosystem. You may recall that I and others presented to the CMA in September 2021 about Apple’s browser ban. They have invited public comments, and they honestly are eager to hear from people: not solely big players with big legal submissions, but real web developers. But the time is nigh: they need to hear from you by 5pm UK time today, Monday 7th February 2022.

Bruce Lawson, who I presented with, has summarised the CMA’s interim report. What’s important for our perspectives today is how they feel about mobile browsers. In particular, they call out how on Apple’s iOS devices, there is only one browser: Safari. While other browser names do exist — Chrome, Firefox, and the like — they are all Safari dressed up in different clothes. It’s been surprising how many developers didn’t realise this: check out the Twitter hashtag #AppleBrowserBan for more on that. So the CMA are looking for feedback and comments from anyone in the UK or who does any business in the UK, on how you feel about the mobile ecosystem of apps and browsers in general, and how you feel about the browser landscape specifically. Did you decide to use the web, or not use the web, on mobile devices in a way that felt like you had no choice? Do you feel like web apps are a match for native apps or not?

If you’re a web developer, you may have already been part of, or encountered, some parts of this discussion on social media already. And that may mean that you’re a bit tired of it, because it can be quite bad-tempered in places, and because there’s an awful lot of disingenuous argument. People on or closely allied with browser vendors have a very bad habit of wholly ignoring problems in their own camp and lauding problems in others: this is not of course specific to browser developers (everybody does this in arguments!) but it’s pretty annoying. Chrome defenders generally divert the conversation away from privacy, data collection, and Google’s desire to inhale all your data from everywhere: I’ve asked in the past about Chrome integrating with the platform it’s on and have been unembarrassedly told that “Chrome is the platform”, which I’m sure sounds great if you’re Google and maybe not for everyone else. And Safari developers tend to pretend to be deaf when asked questions they don’t like, and then complain that they get no engagement after haughtily refusing any that isn’t on their own terms. Yes, this is all very irritating, I agree with you. But here’s the thing: you do not have to take a side. This does not have to be tribal. The CMA want your opinions by the end of today, and you don’t have to feel like you’re striking a blow in the browser wars or a blow for privacy or a blow against native by doing so. The thing that every survey and every poll always flags is that they only hear from people with an axe to grind. You don’t have to feel like you’re defending one side or another to have an opinion on how you think that mobile platforms treat the web, and how you think they could do that differently. Be on the side of the web: that’s the right side to be on, because there’s no other side. (Nobody claims to be against the web — nobody with any sense or knowledge of history, anyway — so you know it’s a good place to be.)

Bruce has very excellently provided some sample responses to the CMA along with guidance about points you may want to cover or include, and you should definitely read that. But get your skates on: responses have to be in today.

Send email to mobileecosystems@cma.gov.uk. And do post your response publicly if you can, so others can see it, learn from it, and see that you’ve done it.

I invented a solitaire card game. I was thinking about solo roleplaying, and the Carta SRD stuff I did for Borealis, and I was thinking about the idea of the cards forming the board you’re playing on and also being the randomness tool. Then I came up with the central mechanic of the branching tree, and the whole game sorta fell into place, and now I’ve been experimenting with it for a day so I want to write it down.

I’m not sure about the framing device; originally it was a dungeon crawl, but since it’s basically “have a big series of battles” I’m a bit uncomfortable with that story since it’s very murder-y. So maybe it’s a heist where you’re defeating many traps to steal the big diamond from a rich guy’s booby-trapped hoard? Not sure, suggestions welcomed.

The game

Required: a pack of cards. Regular 52-card deck. You don’t need jokers.

Setup

Remove the Ace of Spades from the deck and place it in front of you, face up, on the table. This is where you start; the entrance to the dungeon. Deal four cards face down vertically above it in a line. Above that, place the King of Diamonds, face up, so you have six cards in a vertical line, K♦ at the top, 4 face-down cards, A♠ at the bottom. The K♦ is the target, the thing you’re trying to get to (it is treasure, because it is the king, do you see, of diamonds! get it? it’s the biggest diamond). Now return the four face-down cards to the deck and shuffle the deck; this is to ensure that there are exactly four card lengths between A♠ and K♦. Deal one card, face up, over to the left. This is your SKILL score. Its suit is not relevant; its value is the important number. Ace is 1, JQK are 11, 12, 13. Higher is better.

The play: basic

In each round, you confront a monster. Deal one card face up, to the right, which represents the monster, and the monster’s SKILL score (A=1, 2=2, K=13).

Deal three cards face down: these are the monster’s attacks. Deal yourself three cards. Now turn the monster’s cards over. You now pair your cards against the monster’s, in whichever order you please, so there are three pairs. Your score in each pair is your SKILL plus your dealt card; the monster’s score in each pair is its SKILL plus its dealt card.

For each pair where your score is higher than the monster’s, you get a point; for each where the monster wins, you lose a point; a tie scores 0. This means that you will have a score between 3 and -3 for the round.

An example: imagine that you have a SKILL of 7, and you deal the 9♦ as the monster. This means this monster has a SKILL of 9. You then deal three cards for monster attacks; 4♥, J♥, 6♣. You deal yourself three cards: 2♣, 7♦, Q♣. So you elect to pair them as follows:

So that’s an overall score of +2 this round: you won the round and defeated the monster!

If your score for the round is positive, then next round when you deal the card for the monster, you can deal this many extra cards and choose the monster you want from them. (So since we won with +2, next round we deal the next three cards out and choose the one we want to be the monster. The other two cards are returned to the pack, which is shuffled.) If your score is negative, then you have to remove that many cards from the dungeon (which will be explained shortly). (The Ace of Spades always stays in place.)

Return the monster attack and your attack cards to the deck and shuffle it ready for the next round. If your round score was negative or zero, or if the monster was a King, then put the monster card in the discard pile. If your score was positive, then add the monster card to the dungeon.

Adding to the dungeon

To add a card to the dungeon, it must be touching the last card that was added to the dungeon (or the Ace of Spades, if no card has yet been added). Rotate the card so that its orientation is the same as the card’s value, on a clock face. So a Queen (value 12) is placed vertically. A 3 is placed horizontally. An Ace is placed pointing diagonally up and to the right. This should be at 30 degrees, but you can eyeball it; don’t get the protractor out. Remember, it must be touching or overlapping the last card that was added. In this way, the path through the dungeon grows. The goal is to have the path reach the King of Diamonds; if there is a continuous path from A♠ to K♦ then you have won!

Optional rules, ramblings, and clarifications

That’s the game. Build the dungeon turn by turn, see if you can obtain the treasure, the king of diamonds. Here are some extra thoughts, possible extra rules, and questions that I have which I haven’t yet worked out answers for.

“Special” cards: armour and weapons

It would be nice to add a bit more skill and planning to the game; there’s not a huge amount of agency. So here’s the first optional rule, about armour and weapons. A monster card which is a Club is potentially a weapon. If you deal a monster card that’s a Club, then you can elect to treat it as a weapon instead and put it in your stash. When you have weapons in your stash, you can choose to add the stored weapon to any one attack, and it adds its rank to your score for that attack. (So if you have SKILL of 7, and you play a 3 for an attack, and you have a 4♣ stored as a weapon, then you can play that 4 as well for a total score of 14, not 10.) Similarly, a monster card that’s a Spade can be treated as armour. If you have armour, the next attack you lose will become a tie. So if your score for a round would be -2 (you lost two attacks and tied one) but you have an armour card then you would discard your armour card and that score becomes -1 (one loss, two ties). Clubs and Spades used this way go into the discard pile, not back into the pack.

This is an optional rule becuase I’m not sure about the balancing of it. In particular, when do you get to add a weapon card? Should you have to add a weapon before the monster attacks are turned over, so it’s a bit of a gamble? Or can you add it when you know whether it’ll win or not? (If yes, then everyone holds weapons until they know they’ll make the difference between a win and a loss, which doesn’t require any skill or judgement to do.)

The length of the dungeon

The distance of 4 cards is based on some rough simulations I ran which suggest that with a 4 card distance a player should win about 5% of the time, which feels about right for a difficult solitaire game; you want to not win that often, but not so infrequently that you doubt that winning is possible. But changing the distance to 3 cards may make a big difference there (it should give a win about one time in 10, in the simulation).

Removing cards

Question: should it be allowed to delete cards in the middle of the path if you lose, thus leaving a gap in the path. You shouldn’t be able to win by reaching the king of diamonds if there’s a gap, of course, but having gaps mid-game seems ok. However, then you have to be able to add cards to fill the gap up, which seems very difficult. This is because we have to require that newly added cards are added to the end of the path, otherwise everyone makes all “negative” cards simply build by touch the Ace of Spades and so we never actually go backwards.

Angle of cards

Cards are reversible. So an 8, which should be a negative card, is actually the same as a 2, which is positive. What’s the best way to enforce this? When considering the “clock face” for orientation, does the centre of the clock face have to be in the centre of the most recent card?

Also, kings not being able to add to the path seems a bit arbitrary. Problem is that there aren’t 13 hours on a clock. This can obviously be justified in-universe (maybe kings are boss monsters or something?) but it feels a bit of a wart.

And that’s it

That’s it. Game idea written down, which should hopefully get it out of my head. If anyone else plays it, or has thoughts on the rules, on improvements, or on a theme and setting, I’d love to hear them; @sil on Twitter is probably the easiest way.

As has been mentioned here before the UK regulator, the Competition and Markets Authority, are conducting an investigation into mobile phone software ecosystems, and they recently published the results of that investigation in the mobile ecosystems market study. They’re also focusing in on two particular areas of concern: competition among mobile browsers, and in cloud gaming services. This is from their consultation document:

Mobile browsers are a key gateway for users and online content providers to access and distribute content and services over the internet. Both Apple and Google have very high shares of supply in mobile browsers, and their positions in mobile browser engines are even stronger. Our market study found the competitive constraints faced by Apple and Google from other mobile browsers and browser engines, as well as from desktop browsers and native apps, to be weak, and that there are significant barriers to competition. One of the key barriers to competition in mobile browser engines appears to be Apple’s requirement that other browsers on its iOS operating system use Apple’s WebKit browser engine. In addition, web compatibility limits browser engine competition on devices that use the Android operating system (where Google allows browser engine choice). These barriers also constitute a barrier to competition in mobile browsers, as they limit the extent of differentiation between browsers (given the importance of browser engines to browser functionality).

They go on to suggest things they could potentially do about it:

A non-exhaustive list of potential remedies that a market investigation could consider includes:

• removing Apple’s restrictions on competing browser engines on iOS devices;
• mandating access to certain functionality for browsers (including supporting web apps);
• requiring Apple and Google to provide equal access to functionality through APIs for rival browsers;
• requirements that make it more straightforward for users to change the default browser within their device settings;
• choice screens to overcome the distortive effects of pre-installation; and
• requiring Apple to remove its App Store restrictions on cloud gaming services.

But, importantly, they want to know what you think. I’ve now been part of direct and detailed discussions with the CMA a couple of times as part of OWA, and I’m pretty impressed with them as a group; they’re engaged and interested in the issues here, and knowledgeable. We’re not having to educate them in what the web is. The UK’s potential digital future is not all good (and some of the UK’s digital future looks like it could be rather bad indeed!) but the CMA’s work is a bright spot, and it’s important that we support the smart people in tech government, lest we get the other sort.

So, please, take a little time to write down what you think about all this. The CMA are governmental: they have plenty of access to windy bloviations about the philosophy of tech, or speculation about what might happen from “influencers”. What’s important, what they need, is real comments from real people actually affected by all this stuff in some way, either positively or negatively. Tell they whether you think they’ve got it right or wrong; what you think the remedies should be; which problems you’ve run into and how they affected your projects or your business. Earlier in this process we put out calls for people to send in their thoughts and many of you responded, and that was really helpful! We can do more this time, when it’s about browsers and the Web directly, I hope.

If you feel as I do then you may find OWA’s response to the CMA’s interim report to be useful reading, and also the whole OWA twitter thread on this, but the most important thing is that you send in your thoughts in your own words. Maybe what you think is that everything is great as it is! It’s still worth speaking up. It is only a good thing if the CMA have more views from actual people on this, regardless of what those views are. These actions that the CMA could take here could make a big difference to how competition on the Web proceeds, and I imagine everyone who builds for the web has thoughts on what they want to happen there. Also there will be thoughts on what the web should be from quite a few people who use the web, which is to say: everybody. And everybody should put their thoughts in.

So here’s the quick guide:

1. You only have until July 22nd
2. Read Mobile browsers and cloud gaming from the CMA
3. Decide for yourself:
   • How these issues have personally affected you or your business
   • How you think changes could affect the industry and consumers
   • What interventions you think are necessary
4. Email your response to browsersandcloud@cma.gov.uk

Go to it. You have a month. It’s a nice sunny day in the UK… why not read the report over lunchtime and then have a think?

So, I made a game. It’s called Farmbound. It’s a puzzle; you get a sequence of farm things — seeds, crops, knives, water — and they combine to make better items and to give you points. Knives next to crops and fields continually harvest them for points; seeds combine to make crops which combine to make fields; water and manure grow a seed into a crop and a crop into a field. Think of it like a cross between a match-3 game and Little Alchemy. The wrinkle is that the sequence of items you get is the same for the whole day: if you play again, you’ll get the same things in the same order, so you can learn and refine your strategy. It’s rather fun: give it a try.

It’s a web app. Works for everyone. And I thought it would be useful to explain why it is, why I think that’s the way to do things, and some of the interesting parts of building an app for everyone to play which is delivered over the web rather than via app stores and downloads.

Why’s it a web app and not a platform-specific native app?

Well, there are a bunch of practical reasons. You get completely immediate play with a web app; someone taps on a share link, and they’re playing. No installation, no platform detection, it Just Works (to coin a phrase which nobody has ever used before about apps ever in the history of technology). And for something like this, an app with platform-specific code isn’t needed: sure, if you’re talking to some hardware devices, or doing low-level device fiddling or operating system integration, you might need to build and deliver something separately to each platform. But Farmbound is not that. There is nothing that Farmbound needs that requires a native app (well, nearly nothing, and see later). So it isn’t one.

There are some benefits for me as the developer, too. Such things are less important; the people playing are the important ones. But if I can make things nicer for myself without making them worse for players, then I’m going to do it. Obviously there’s only one codebase. (For platform-specific apps that can be alleviated a little with cross-platform frameworks, some of which are OK these days.) One still needs to test across platforms, though, so that’s not a huge benefit. On the other hand, I don’t have to pay extra to distribute it (beyond it being on my website, which I’d be paying for anyway), and importantly I don’t have to keep paying in order to keep my game available for ever. There’s no annual tithe required. There’s no review process. I also get support for minority platforms by publishing on the web… and I’m not really talking about something in use by a half-dozen people here. I’m talking about desktop computers. How many people building a native app, even a relatively simple puzzle game like this, make a build for iOS and Android and Windows and Mac and Linux? Not many. The web gets me all that for minimal extra work, and if someone on FreeBSD or KaiOS wants to play, they can, as long as they’ve got a modern browser. (People saying “what about those without modern browsers”… see below.)

But from a less practical and more philosophical point of view… I shouldn’t need to build a platform-specific native app to make a game like this. We want a world where anyone can build and publish an app without having to ask permission, right? I shouldn’t need to go through a review process or be beholden to someone else deciding whether to publish my game. The web works. Would Wordle have become so popular if you had to download a Windows app or wait for review before an update happened? I doubt it. I used to say that if you’re building something complex like Photoshop then maybe go native, but in a world with Figma in it, that maybe doesn’t apply any more, and so Adobe listened to that and now Photoshop is on the web. Give people a thing which doesn’t need installation, gets them playing straight away, and works everywhere? Sounds good to me. Farmbound’s a web app.

Why’s it not got its own domain, then, if it’s on the web?

Farmbound shouldn’t need its own domain, I don’t think. If people find out about it, it’ll likely be by shared links showing off how someone else did, which means they click the link. If it’s popular then it’ll be top hit for its own name (if it isn’t, the Google people need to have a serious talk with themselves), and if it isn’t popular then it doesn’t matter. And, like native app building, I don’t really want to be on the hook forever for paying for a domain; sure, it’s not much money, but it’s still annoying that I’m paying for a couple of ideas that I had a decade ago and which nobody cares about any more. I can’t drop them, because of course cool URIs don’t change, and I didn’t want to be thinking a decade from now, do I still need to pay for this?

In slightly more ego-driven terms, it being on my website means I get the credit, too. Plus, I quite like seeing things that are part of an existing site. This is what drove the (admittedly hipster-ish) rise of “tilde sites” again a few years ago; a bit of nostalgia for a long time ago. Fortunately, I’ve also got Cloudflare in front of my site, which alleviates worries I might have had about it dying under load, although check back with me again if that happens to see if it turns out to be true or not. (Also, I’m considering alternatives to Cloudflare at the moment too.)

So what was annoying and a problem when building an app on the web?

Architecture

Firstly, I separated the front and back ends and deployed them in different places. I’m not all that confident that my hosted site can cope with being hammered, if I’m honest. This is alleviated somewhat by cloud caching, and hopefully quite a bit more by having a service worker in place which caches almost everything (although see below about that), but a lot of this decision was driven by not wanting to incur a server hit for every visitor every time, as much as possible. This drove at least some of the architectural decisions. The front end is on my site and is plain HTML, CSS, and JavaScript. The back end is not touched when starting the game; it’s only touched when you finish a game, in order to submit your score and get back the best score that day to see if you beat that. That back end is written in Deno, and is hosted on fly.io, who seem pretty cool. (I did look at Deno Deploy, but they don’t do permanent storage.)

Part of the reason the back end is a bit of extra work is that it verifies your submitted game to check you aren’t cheating and lying about your score. This required me to completely reimplement the game code in Deno. Now, you may be saying “what? the front end game code is in JavaScript and so is the back end? why don’t they share a library?” and the answer is, because I didn’t think of it. So I wrote the front end first and didn’t separate out the core game management from all the “animate this stuff with CSS” bits, because it was a fun weekend project done as a proof of concept. Once I got a bit further into it and realised that I should have done that… I didn’t wanna, because that would have sucked all the fun out of the project like a vampire and meant that I’d have never done it. So, take this as a lesson: think about whether you want a thing to be popular up front. Not that you’ll listen to this advice, because I never do either.

Similarly, this means that there’s less in the way of analytics, so I don’t get information about users, or real-time monitoring of popularity. This is because I did not want to add Google Analytics or similar things. No personal data about you ever leaves your device. You’ll have noticed that there’s no awful pop-up cookie consent dialogue; this is because I don’t need one, because I don’t collect any analytics data about players at all! Guess what, people who find those dialogues annoying (i.e., everyone?) You can tell companies to stop collecting data about you and then they won’t need an annoying dialogue! And when they say no… well, then you’ll have learned something about how they view you as customers, perhaps. Similarly, when scores are submitted, there’s no personal information that goes with them. I don’t even know whether two scores were submitted by the same person; there’s no unique ID per person or per device or anything. (Technically, the IP is submitted to the server, of course, but I don’t record it or use it; you’ll have to take my word for that.)

This architecture split also partially explains why the game’s JavaScript-dependent. I know, right? Me, the bloke who wrote “Everyone has JavaScript, right?“, building a thing which requires JS to run? What am I doing? Well, honestly, I don’t want to incur repeated server hits is the thing. For a real project, something which was critical, then I absolutely would do that; I have the server game simulation, and I could relatively easily have the server pass back a game state along with the HTML which was then submitted. The page is set up to work this way: the board is a <form>, the things you click on are <button>s, and so on. But I’m frightened of it getting really popular and then me getting a large bill for cloud hosting. In this particular situation and this particular project, I’d rather the thing die than do that. That’s not how I’d build something more critical, but… Farmbound’s a puzzle game. I’m OK with it not working, and if I turn out to be wrong about that, I can change that implementation relatively quickly without it being a big problem. It’s not architected in a JS-dependent way; it’s just progressively enhanced that way.

iOS browser

I had a certain amount of hassle from iOS Safari. Some of this is pretty common — how do I stop a double-tap zooming in? How do I stop the page overscrolling? — but most of the “fixes” are a combination of experimentation, cargo culting ideas off Stack Overflow, and something akin to wishing on a star. That’s all pretty irritating, although Safari is hardly alone in this. But there is a separate thing which is iOS Safari specific, which is this: I can’t sensibly present an “add this to your home screen” hint in iOS browsers other than Safari itself. In iOS Safari, I can show a little hint to help people know that they can add Farmbound to their home screen (which of course is delayed until a second game is begun and then goes away for a month if you dismiss it, because hassling your own players is a foolish thing to do). But in non Safari iOS browsers (which, lest we forget, are still Safari under the covers; see Open Web Advocacy if this is a surprise to you or if you don’t like it), I can’t sensibly present that hint. Because those non-Safari iOS browsers aren’t allowed to add web apps to your home screen at all. I can’t even give people a convenient tap to open Farmbound in iOS Safari where they can add the app to their home screen, because there’s no way of doing that. So, apologies, Chrome iOS or Firefox iOS users and others: you’ll have to open Farmbound in Safari itself if you want an easy way to come back every day. At least for now.

Service workers

And finally, and honestly most annoyingly, the service worker.

Building and debugging and testing a service worker is still so hard. Working out why this page is cached, or why it isn’t cached, or why it isn’t loading, is incredibly baffling and infuriating still, and I just don’t get it. I tried using “workbox”, but that doesn’t actually explain how to use it properly. In particular, for this use case, a completely static unchanging site, what I want is “cache this actual page and all its dependencies forever, unless there’s a change”. However, all the docs assume that I’m building an “app shell” which then uses fetch() to get data off the server repeatedly, and so won’t shut up about “network first” and “cache first, falling back” and so on rather than the “just cache it all because it’s static, and then shut up” methodology. And getting insight into why a thing loaded or didn’t is really hard! Sure, also having Cloudflare caching stuff and my browser caching stuff as well really doesn’t help here. But I am not even slightly convinced that I’ve done all this correctly, and I don’t really know how to be better. It’s too hard, still.

Conclusion

So that’s why Farmbound is the way it is. It’s been interesting to create, and I am very grateful to the Elite Farmbound Testing Team for a great deal of feedback and helping me refine the idea and the play: lots of love to popey, Roger, Simon, Martin, and Mark, as well as Handy Matt and my mum!

There are still some things I might do in the future (achievements? maybe), and I might change the design (I’m not great at visual design, as you can tell), and I really wish that I could have done all the animations with Shared Element Transitions because it would have been 312 times easier than the way I did it (a bunch of them add generated content and then web-animations-api move the ::before around, which I thought was quite neat but is also daft by comparison with SET). But I’m pleased with the implementation, and most importantly it’s actually fun to play. Getting over a thousand points is really good (although sometimes impossible, on some days), and I don’t really think the best strategies have been worked out yet. Is it better to make fields and tractors, or not go that far? Is water a boon or an annoyance? I’d be interested in your thoughts. Go play Farmbound, and share your results with me on Twitter.

Hear this talk performed (with appropriate background music):

Friends and enemies, attendees of Tech Mids 2022.

Don’t read off the screen.

If I could offer you only one piece of advice for why and how you should speak in public, don’t read off the screen would be it. Reading your slides out is guaranteed to make your talk boring, whereas the rest of my advice has no basis in fact other than my own experience, and the million great people who gave me thoughts on Twitter.

I shall dispense this advice… now.

Every meetup in every town is crying out for speakers, and your voice is valuable. Tell people your story. The way you see things is unique, just like everybody else.

Everybody gets nervous about speaking sometimes. Anybody claiming that they don’t is either lying, or trying to sell you something. If you’re nervous, consider that its a mark of wanting to do a good job.

Don’t start by planning what you want to say. Plan what you want people to hear. Then work backwards from there to find out what to say to make that happen.

You can do this. The audience are on your side.

Find your own style. Take bits and pieces from others and make them something of your own.

Slow down. Breathe. You’re going faster than it feels like you are.

Pee beforehand. If you have a trouser fly, check it.

If someone tells you why you should speak, take their words with a pinch of salt, me included. If they tell you how to speak, take two pinches. But small tips are born of someone else’s bad experience. When they say to use a lapel mic, or drink water, or to have a backup, then listen; they had their bad day so that you didn’t have to.

Don’t put up with rambling opinions from questioners. If they have a comment rather than a question, then they should have applied to do a talk themselves. You were asked to be here. Be proud of that.

Practice. And then practice again, and again. If you think you’ve rehearsed enough, you haven’t.

Speak inclusively, so that none of your audience feels that the talk wasn’t for them.

Making things look unrehearsed takes a lot of rehearsal.

Some people script their talks, some people don’t. Whether you prefer bullet points or a soliloquy is up to you. Whichever you choose, remember: don’t just read out your notes. Your talk is a performance, not a recital.

Nobody knows if you make a mistake. Carry on, and correct it when you can. But keep things simple. Someone drowning in information finds it hard to listen.

Live demos anger the gods of speaking. If you can avoid a live demo, do so. Record it in advance, or prep it so that it looks live. Nobody minds at all.

Don’t do a talk only once.

Acting can be useful, if that’s the style you like. Improv classes, stage presence, how you stand and what you do with your hands, all of this can be taught. But put your shoulders back and you’ve got about half of it.

Carry your own HDMI adapter and have a backup copy of your talk. Your technology will betray you if it gets a chance.

Record your practices and watch yourself back. It can be a humbling experience, but you are your own best teacher, if you’re willing to listen.

Try to have a star moment: something that people will remember about what you said and the way you said it. Whether that’s a surprising truth or an excellent joke or a weird gimmick, your goal is to have people walk away remembering what you said. Help them to do that.

Now, go do talks. I’m Stuart Langridge, and you aren’t. So do your talk, your way.

But trust me: don’t read off the screen.

Some posts are written so there’s an audience. Some are written to be informative, or amusing. And some are literally just documentation for me which nobody else will care about. This is one of those.

I’ve moved phone network. I’ve been with Three for years, but they came up with an extremely annoying new tactic, and so they must be punished. You see, I had an account with 4GB of data usage per month for about £15pm, and occasionally I’d go over that; a couple of times a year at most. That’s OK: I don’t mind buying some sort of “data booster” thing to give me an extra gig for the last few days before the next bill; seems reasonable.

But Three changed things. Now, you see, you can’t buy a booster to give yourself a bit of data until the end of the month. No, you have to buy a booster which gives you extra data every month, and then three days later when you’re in the new month, cancel it. There’s no way to just get the data for now.¹

This is aggressively customer-hostile. There’s literally no reason to do this other than to screw people who forget to cancel it. Sure, have an option to buy a “permanent top-up”, no arguments with that. But there should also be an option to buy a temporary top-up, just once! There used to be!

I was vaguely annoyed with Three for general reasons anyway — they got rid of free EU roaming, they are unhelpful when you ask questions, etc — and so I was vaguely considering moving away regardless. But this was the straw that broke the camel’s back.² So… time to look around.

I asked the Mastodon gang for suggestions, and I got lots, which is useful. Thank you for that, all.

The main three I got were Smarty, iD, and Giffgaff. Smarty are Three themselves in a posh frock, so that’s no good; the whole point of bailing is to leave Three. Giffgaff are great, and I’ve been hearing about their greatness for years, not least from popey, but they don’t do WiFi Calling, so they’re a no-no.³ And iD mobile looked pretty good. (All these new “MVNO” types of thing seem quite a lot cheaper than “traditional” phone operators. Don’t know why. Hooray, though.)

So off I went to iD, and signed up for a 30-day rolling SIM-only deal⁴. £7 per month. 12GB of data. I mean, crikey, that’s quite a lot better than before.

I need to keep my phone number, though, so I had to transfer it between networks. To do this, you need a “PAC” code from your old network, and you supply it to the new one. All my experience of dealing with phone operators is from the Old Days, and back then you had to grovel to get a PAC and your current phone network would do their best to talk you out of it. Fortunately, the iron hand of government regulation has put a stop to these sorts of shenanigans now (the UK has a good tech regulator, the Competition and Markets Authority⁵) and you can get a PAC, no questions asked, by sending an SMS with content “PAC” to 65075. Hooray. So, iD mobile sent me a new SIM in the post, and I got the PAC from Three, and then I told iD about the PAC (on the website: no person required), and they said (on the website), ok, we’ll do the switch in a couple of working days.

However, the SIM has some temporary number on it. Today, my Three account stopped working (indicating that Three had received and handled their end of the deal by turning off my account), and so I dutifully popped out the Three SIM from my phone⁶ and put in the new one.

But! Alas! My phone thought that it had the temporary number!

I think this is because Three process their (departing) end, there’s an interregnum, and then iD process their (arriving) end, and I was in the interregnum. I do not know what would have happened if someone rang my actual phone number during this period. Hopefully nobody did. I waited a few hours — the data connection worked fine on my phone, but it had the wrong number — and then I turned the phone off and left it off for ten minutes or so. Then… I turned it back on, and… pow! My proper number is mine again! Hooray!

That ought to have been the end of it. However, I have an Apple phone. So, in Settings > Phone > My Number, it was still reading the temporary number. Similarly, in Settings > Messages > iMessage > Send and Receive, it was also still reading the temporary number.

How inconvenient.

Some combination of the following fixed that. I’m not sure exactly what is required to fix it: I did all this, some more than once, in some random order, and now it seems OK: powering the phone off and on again; disabling iMessage and re-enabling it; disabling iMessage, waiting a few minutes, and then re-enabling it; disabling iMessage, powering off the phone, powering it back on again, and re-enabling it; editing the phone number in My Number (which didn’t seem to have any actual effect); doing a full network reset (Settings > General > Transfer or Reset Device > Reset > Reset Network Settings). Hopefully that’ll help you too.

Finally, there was voicemail. Some years ago, I set up an account with Sipgate, where I get a phone number and voicemail. The thing I like about this is that when I get voicemail on that number, it emails me an mp3 of the voicemail. This is wholly brilliant, and phone companies don’t do it; I’m not interested in ringing some number and then pressing buttons to navigate the horrible menu, and “visual voicemail” never took off and never became an open standard thing anyway. So my sipgate thing is brilliant. But… how do I tell my phone to forward calls to my sipgate number if I don’t answer? I did this once, about 10 years ago, and I couldn’t remember how. A judicious bit of web searching later, and I have the answer.

One uses a few Secret Network Codes to do this. It’s called “call diversion” or “call forwarding”, and you do it by typing a magic number into your phone dialler, as though you were ringing it as a number. So, let’s say your sipgate number is 0121 496 0000. Open up the phone dialler, and dial *61*01214960000# and press dial. That magic code, *61, sets your number to divert if you don’t answer it. Do it again with *62 to also divert calls when your phone is switched off. You can also do it again with *67 to divert calls when your phone is engaged, but I don’t do that; I want those to come through where the phone can let me switch calls.

And that’s how I moved phone networks. Stuart, ten years from now when you read this again, now you know how to do it. You’re welcome.

1. Well, there is: you can spend twenty actual pounds to get unlimited data until the end of the month. But that’s loads of money. ↩
2. is there an actual camel whose back was broken in some fable somewhere? this is a terribly unfortunate metaphor! ↩
3. Coverage in my flat — for any network — is rubbish. So wifi calling is a hard requirement. I don’t know why this is — I’m in the city! Maybe the walls all have copper wiring in them like the room that Elodin was imprisoned in, I don’t know ↩
4. I always buy my own phones, so I only ever need a SIM, and I don’t like being locked in ↩
5. we, meaning Open Web Advocacy, have worked extensively with the CMA (and other regulators around the world) about assisting and requiring mobile phone manufacturers to provide browser diversity — that is, to make it so you can use web apps on your phone that are as capable as platform-specific apps, and the CMA are great ↩
6. removing a SIM tray is harder than it looks when you don’t wear earrings. I had to search everywhere to find one of those little SIM tools ↩

Twitter currently has problems. Well, one specific problem, which is the bloke who bought it. My solution to this problem has been to move to Mastodon (@sil@mastodon.social if you want to do the same), but I’ve invested fifteen years of my life providing twitter.com with free content so I don’t really want it to go away. Since there’s a chance that the whole site might vanish, or that it continues on its current journey until I don’t even want my name associated with it any more, it makes sense to have a backup. And obviously, I don’t want all that lovely writing to disappear from the web (how would you all cope without me complaining about some random pub’s music in 2011?!), so I wanted to have that backup published somewhere I control… by which I mean my own website.

So, it would be nice to be able to download a list of all my tweets, and then turn that into some sort of website so it’s all still available and published by me.

Fortunately, Zach Leatherman came to save us by building a tool, Tweetback, which does a lot of the heavy lifting on this. Nice one, that man. Here I’ll describe how I used Tweetback to set up my own personal Twitter archive. This is unavoidably a bit of a developer-ish process, involving the Terminal and so on; if you’re not at least a little comfortable with doing that, this might not be for you.

Step 1: get a backup from Twitter

This part is mandatory. Twitter graciously permit you to download a big list of all the tweets you’ve given them over the years, and you’ll need it for this. As they describe in their help page, go to your Twitter account settings and choose Your account > Download an archive of your data. You’ll have to confirm your identity and then say Request data. They then go away and start constructing an archive of all your Twitter stuff. This can take a couple of days; they send you an email when it’s done, and you can follow the link in that email to download a zip file. This is your Twitter backup; it contains all your tweets (and some other stuff). Stash it somewhere; you’ll need a file from it shortly.

Step 2: get the Tweetback code

You’ll need both node.js and git installed to do this. If you don’t have node.js, go to nodejs.org and follow their instructions for how to download and install it for your computer. (This process can be fiddly; sorry about that. I suspect that most people reading this will already have node installed, but if you don’t, hopefully you can manage it.) You’ll also need git installed: Github have some instructions on how to install git or Github Desktop, which should explain how to do this stuff if you don’t already have it set up.

Now, you need to clone the Tweetback repository from Github. On the command line, this looks like git clone https://github.com/tweetback/tweetback.git; if you’re using Github Desktop, follow their instructions to clone a repository. This should give you the Tweetback code, in a folder on your computer. Make a note of where that folder is.

Step 3: install the Tweetback code

Open a Terminal on your machine and cd into the Tweetback folder, wherever you put it. Now, run npm install to install all of Tweetback’s dependencies. Since you have node.js installed from above, this ought to just work. If it doesn’t… you get to debug a bit. Sorry about that. This should end up looking something like this:

$ npm install
npm WARN deprecated @npmcli/move-file@1.1.2: This functionality has been moved to @npmcli/fs

added 347 packages, and audited 348 packages in 30s

52 packages are looking for funding
  run `npm fund`for details

found 0 vulnerabilities

Step 4: configure Tweetback with your tweet archive

From here, you’re following Tweetback’s own README instructions: they’re online at https://github.com/tweetback/tweetback#usage and also are in the README file in your current directory.

Open up the zip file you downloaded from Twitter, and get the data/tweets.js file from it. Put that in the database folder in your Tweetback folder, then edit that file to change window.YTD.tweet.part0 on the first line to module.exports, as the README says. This means that your database/tweets.js file will now have the first couple of lines look like this:

module.exports=[{"tweet":{

Now, run npm run import. This will go through your tweets.js file and load it all into a database, so it can be more easily read later on. You only need to do this step once. This will print a bunch of lines that look like { existingRecordsFound: 0, missingTweets: 122 }, and then a bunch of lines that look like Finished count { count: 116 }, and then it’ll finish. This should be relatively quick, but if you’ve got a lot of tweets (I have 68,000!) then it might take a little while. Get yourself a cup of tea and a couple of biscuits and it’ll be done when you’ve poured it.

Step 5: Configure a subdirectory (optional)

If you’re setting up your own (sub)domain for your twitter archive, so it’ll be at the root of the website (so, https://twitter.example.com or whatever) then you can skip this step. However, if you’re going to put your archive in its own directory, so it’s not at the root (which I did, for example, at kryogenix.org/twitter), then you need to tell your setup about that.

To do this, edit the file eleventy.config.js, and at the end, before the closing }, add a new line, so the end of the file looks like this:

eleventyConfig.addPlugin(EleventyHtmlBasePlugin);return{pathPrefix:"/twitter/"}};

The string "/twitter/" should be whatever you want the path to the root of your Twitter archive to be, so if you’re going to put it at mywebsite.example.com/my-twitter-archive, set pathPrefix to be "/my-twitter-archive". This is only a path, not a full URL; you do not need to fill in the domain where you’ll be hosting this here.

Step 6: add metadata

As the Tweetback README describes, edit the file _data/metadata.js. You’ll want to change three values in here: username, homeLabel, and homeURL.

username is your Twitter username. Mine is sil: yours isn’t. Don’t include the @ at the beginning.

homeLabel is the thing that appears in the top corner of your Twitter archive once generated; it will be a link to your own homepage. (Note: not the homepage of this Twitter archive! This will be the text of a link which takes you out of the Twitter archive and to your own home.)

homeURL is the full URL to your homepage. (This is “https://kryogenix.org/” for me, for example. It is the URL that homeLabel links to.)

Step 7: (drum roll, please!) Build the site

OK. Now you’ve done all the setup. This step actually takes all of that and builds a website from all your tweets.

Run npm run build.

If you’ve got a lot of tweets, this can take a long time. It took me a couple of hours, I think, the first time I ran it. Subsequent runs take a lot less time (a couple of minutes for me, maybe even shorter for you if you’re less mouthy on Twitter), but the first run takes ages because it has to fetch all the images for all the tweets you’ve ever written. You’ll want a second cup of tea here, and perhaps dinner.

It should look something like this:

$ npm run build

> tweetback@1.0.0 build
> npx @11ty/eleventy --quiet

[11ty] Copied 1868 files / Wrote 68158 files in 248.58 seconds (3.6ms each, v2.0.0-canary.18)

You may get errors in here about being unable to fetch URLs (Image request error Bad response for https://pbs.twimg.com/media/C1VJJUVXEAE3VGE.jpg (404): Not Found and the like); this is because some Tweets link to images that aren’t there any more. There’s not a lot you can do about this, but it doesn’t stop the rest of the site building.

Once this is all done, you should have a directory called _site, which is a website containing your Twitter archive! Hooray! Now you publish that directory, however you choose: copy it up to your website, push it to github pages or Netlify or whatever. You only need the contents of the _site directory; that’s your whole Twitter archive website, completely self-contained; all the other stuff is only used for generating the archive website, not for running it once it’s generated.

Step 8: updating the site with newer tweets (optional)

If you’re still using Twitter, you may post more Tweets after your downloadable archive was generated. If so, it’d be nice to update the archive with the contents of those tweets without having to request a full archive from Twitter and wait two days. Fortunately, this is possible. Unfortunately, you gotta do some hoop-jumping to get it.

You see, to do this, you need access to the Twitter API. In the old days, people built websites with an API because they wanted to encourage others to interact with that website programmatically as well as in a browser: you built an ecosystem, right? But Twitter are not like that; they don’t really want you to interact with their stuff unless they like what you’re doing. So you have to apply for permission to be a Twitter developer in order to use the API.

To do this, as the Tweetback readme says, you will need a Twitter bearer token. To get one of those, you need to be a Twitter developer, and to be that, you have to fill in a bunch of forms and ask for permission and be manually reviewed. Twitter’s documentation explains about bearer tokens, and explains that you need to sign up for a Twitter developer account to get them. Go ahead and do that. This is an annoying process where they ask a bunch of questions about what you plan to do with the Twitter API, and then you wait until someone manually reviews your answers and decides whether to grant you access or not, and possibly makes you clarify your answers to questions. I have no good suggestions here; go through the process and wait. Sorry.

Once you are a Twitter developer, create an app, and then get its bearer token. You only get this once, so be sure to make a note of it. In a clear allusion to the delight that this whole process brings to users, it probably will begin by screaming AAAAAAAAAAAAAAA and then look like a bunch of incomprehensible gibberish.

Now to pull in new data, run:

TWITTER_BEARER_TOKEN=AAAAAAAAAAAAAAAAAAq3874nh93q npm run fetch-new-data

(substituting in the value of your token, of course, which will be longer.)

This will fetch any tweets that aren’t in the database because you made them since! And then run npm run build again to rebuild the _site directory, and re-publish it all.

I personally run these steps (fetch-new-data, then build, then publish) daily in a cron job, which runs a script with contents (approximately):

#!/bin/bashcd"$(dirname "$0")"echo Begin publish at $(date)echo Updating Twitter archive
echo========================TWITTER_BEARER_TOKEN=AAAAAAAAAAAAAA9mh8j9808jhey9w34cvj3g3 npm run fetch-new-data 2>&1echo Updating site from archive
echo==========================
npm run build 2>&1echo Publishing site
echo===============
rsync -e "ssh" -az _site/ kryogenix.org:public_html/twitter 2>&1echo Finish publish at $(date)

but how you publish and rebuild, and how often you do that, is of course up to you.

Step 9: improve the archive (optional, but good)

What Tweetback actually does is use your twitter backup to build an 11ty static website. (This is not all that surprising, since 11ty is also Zach’s static site builder.) This means that if you’re into 11ty you could make the archive better and more comprehensive by adding stuff. There are already some neat graphs of most popular tweets, most recent tweets, the emoji you use a lot (sigh) and so on; if you find things that you wish that your Twitter archive contained, file an issue with Tweetback, or better still write the change and submit it back so everyone gets it!

Step 10: add yourself to the list of people using the archive (optional, but you know you wanna)

Go to tweetback/tweetback-canonical and add yourself to the mapping.js file. What’s neat about this is that that file is used by tweetback itself. This means that if someone else with a Tweetback archive has a tweet which links to one of your Tweets, now their archive will link to your archive directly instead! It’s not just a bunch of separate sites, it’s a bunch of sites all of which are connected! Lots of connections between sites without any central authority! We could call this a collection of connections. Or a pile of connections. Or… a web!

That’s a good idea. Someone should do something with that concept.

Step 11: big hugs for Zach

You may, or may not, want to get off Twitter. Maybe you’re looking to get as far away as possible; maybe you just don’t want to lose the years of investment you’ve put in. But it’s never a bad thing to have your data under your control when you can. Tweetback helps make that happen. Cheers to Zach and the other contributors for creating it, so the rest of us didn’t have to. Tell them thank you.