How to tame Mithril’s router and write clear apps

I moved to Mithril and I seri­ous­ly love it! Sad­ly, the router alone has been enough of a pain point to make me rethink how to use Mithril. I will use pro­fan­i­ty on the sub­ject as a cop­ing mech­a­nism while I write, but don’t let that fool you into think­ing that I blame any­one but myself for any bad code I write, or that I reserve any­thing oth­er than respect for Mithril’s patient and knowl­edge­able main­tain­ers.

If you don’t know what a router is or does, don’t wor­ry. I wrote a sec­tion to get you up to speed.

For those of you new to Mithril, I want to flat­ten your learn­ing curve for the v1.1.x Mithril router and then show you how tam­ing the router tames your Mithril apps. Every­thing I will show you can already be found in the doc­u­men­ta­tion, but I read the docs and still made “gotcha” mis­takes that didn’t reg­is­ter until they hap­pened. My hope is that by para­phras­ing the docs in terms of the router’s quirks, oth­ers won’t have as frus­trat­ing of an expe­ri­ence. For that rea­son, if you are already famil­iar with sin­gle-page appli­ca­tion (SPA) routers this arti­cle will still help you avoid paint­ing your­self into some cor­ners.

Once we’re squared away on the router I will show you how to clean­ly sep­a­rate rout­ing pol­i­cy to make your app eas­i­er to change. Final­ly, I’ll show you one demo Mithril appli­ca­tion orga­nized in this way.

I assume you are either curi­ous about get­ting start­ed with Mithril or have used it just enough to won­der how to improve your use of it. I also assume that you know how to write Mithril com­po­nents.

Preamble for routing newbies

If you’ve used a SPA router before, skip this sec­tion.

A SPA router takes a path you type into the address bar and nav­i­gates the end-user to a spe­cif­ic spot in your app. Unlike how nav­i­ga­tion tra­di­tion­al­ly works, SPA routers do not refresh the brows­er when the user moves about. This makes web appli­ca­tions feel more cohe­sive and flu­id. A route might look like /pages/home or #!/shop/checkout, depend­ing on what part of the web address the router uses to track the user’s loca­tion.

Typ­i­cal­ly routers want you to set them up by telling them all the places the user can go, along with what hap­pens to the user at each of those places. This can be done in sev­er­al ways. Some­times you give a router a big object (some­times called a rout­ing table) with the routes as keys and some imple­men­ta­tion-spe­cif­ic type as val­ues. Rout­ing tables typ­i­cal­ly show clear intent.

    '/': HomePage,
    '/about': CorporateInfo,
    '/shop': Storefront,
    '/support': HelpDesk,

Oth­er routers might give you a dec­o­ra­tor func­tion that mark oth­er func­tions to han­dle spe­cif­ic routes. The premise is the same.

function HomePage() {
  // ...

function CorporateInfo() {
  // ...

You use routers to map out your appli­ca­tion from a navigator’s per­spec­tive. Routers tend to have oth­er fea­tures, such as pars­ing val­ues from routes as func­tion argu­ments, or his­to­ry manip­u­la­tion. The Mithril router is just one of many vari­ants out there.

Mithril’s Router

You sum­mon Mithril’s router using the m.route() func­tion, a rout­ing table, a DOM node that will hold ren­dered con­tent, and a default route.

See the Pen Sim­ple Mithril router exam­ple by Sage Ger­ard (@zyrolasting) on Code­Pen.0

Each route can be asso­ci­at­ed with either a Mithril com­po­nent or a RouteRe­solver object (dis­cussed lat­er) that com­mands more refined con­trol. In the above exam­ple, I use a Mithril com­po­nent that will print Hello, Sage! when you vis­it the /hello/sage route. You can see that the name was parsed out of the route itself by Mithril.

In your code, you can tell the router to nav­i­gate some­where using m.route.set(). You can ask for the last resolved route using m.route.get(). You can parse our vari­ables in your route using m.route.param().

For these sim­ple cas­es, that’s all you need to know. But I want to jump right into the gotchas lest you end up with a false sense of secu­ri­ty.

Mithril resolves routes on its own schedule

The Mithril router resolves routes asyn­chro­nous­ly. Say if you are on a /cart page and you want to go to your pro­file page at /profile. You might use m.route.set() to nav­i­gate. But if your code then asks Mithril’s router for the cur­rent loca­tion with m.route.get(), your state is going to be wrong.

See the Pen Mithril router async res­o­lu­tion by Sage Ger­ard (@zyrolasting) on Code­Pen.0

The above exam­ple shows a view that cor­rect­ly prints the active route, but the console.log shown in the exam­ple says the route is undefined right after m.route.set() kicks off nav­i­ga­tion. That’s because m.route.get() only tells you the last resolved route, which can lead to con­fus­ing bugs if you use m.route.get() to update a view mod­el at a bad time.

To expand on this point, did you ever make a view that looks like it has data from one page “ago”? If so, then you used m.route.get() or m.route.param() to change state out of sync with the Mithril router. In some cas­es, you want the pend­ing route, that is, the route that the router is cur­rent­ly resolv­ing when you are try­ing to sync up your app state to router behav­ior. The com­po­nent in the above exam­ple doesn’t have this prob­lem because by the time it ren­ders the Mithril router has already deemed the route “resolved” so that m.route.get() and m.route.param() deliv­er the lat­est data.

Mithril has no gimme-the-pend­ing-route API, and you can’t use window.location with­out track­ing the router’s pre­fix (like #!)—which also lacks a get­ter. You can only set the pre­fix using m.route.prefix(). Mithril is basi­cal­ly clam­ming up so that you have to solve your rout­ing needs with your state. The only way to reli­ably access and use the pend­ing route for track­ing pur­pos­es is by writ­ing RouteRe­solvers.

RouteResolvers: To block, or not to block?

When Mithril match­es a route, it will start ren­der­ing once you give it a com­po­nent. In this exam­ple, a Mithril com­po­nent ren­ders out­right and redraws implic­it­ly as per the default behav­ior of m.request().

See the Pen Mithril Rout­ing — Load data with­out block­ing by Sage Ger­ard (@zyrolasting) on Code­Pen.0

This approach is easy enough, but if the net­work fetch fin­ish­es quick­ly, the load­ing indi­ca­tor may look like an unpleas­ant flick­er. To fix this, the docs sug­gest block­ing ren­der­ing using a RouteRe­solver, which expos­es the pend­ing route as dis­cussed ear­li­er. But I won’t show that here because I instead want you to under­stand how RouteRe­solvers change the flow of route res­o­lu­tion. All you need to know is that your pend­ing route info is in the argu­ments to the onmatch() func­tion (which fires when, guess what, the asso­ci­at­ed route match­es).

See the Pen Mithril Rout­ing — Load data with block­ing by Sage Ger­ard (@zyrolasting) on Code­Pen.0

This exam­ple removes the flick­er by remov­ing the load­ing indi­ca­tor entire­ly. The doc­u­men­ta­tion acts like the scorched-earth approach is a sell­ing point, but tell that to users with bad con­nec­tions. So long as you return a Promise, that onmatch() can be used to delay ren­der­ing until data is avail­able for a com­po­nent. Reject­ing the promise makes Mithril fall back to the default route. A more robust approach would always allow for an indi­ca­tor to account for slow­er con­nec­tions while pre­vent­ing flick­er. You can also use RouteRe­solvers to opti­mize lay­out diff­ing using an option­al render() func­tion. For now both tech­niques are left as an exer­cise to the read­er, but if you ask me, nev­er block ren­der­ing and be smart about load­ing indi­ca­tors.

We now know enough to make a mess.

Introducing policy

Assume you are work­ing on an admin dash­board for “ten­ants” that make wid­gets. We need:

  • A wid­get list­ing page for one ten­ant. Authen­ti­cat­ed users only.
  • A wid­get detail page. Authen­ti­cat­ed users only.
  • An error page to dis­play prob­lems load­ing data, and
  • A login page.

I call the deci­sions on how to nav­i­gate and han­dle relat­ed prob­lems the rout­ing pol­i­cy because they are nav­i­ga­tion­al require­ments no mat­ter what. Here’s a Mithril router con­fig­u­ra­tion that meets these require­ments.

m.route(root, '/login', {
  '/login': Login,
  '/error': ErrorReport,
  '/:tenant/widgets': {
    oninit() {
      if (!app.user) {
      } else {
        app.tenants[m.route.param('tenant')].loadWidgets().catch((err) => {
          m.route.set('/error', null, {err});
    view() {
      const {widgets} = app.tenants[m.route.param('tenant')];

      return (widgets)
        ? m(WidgetsOverview, widgets)
        : m('p', 'Loading...');
  '/:tenant/widgets/:id': {
    onmatch({tenant, id}) {
      if (!app.user) {
        return Promise.reject();

      return app.tenants[tenant].loadWidgetDetail(id)
      .then((detail) => {
        return {view: () => m(WidgetDetail, detail)};
      .catch((err) => {
        m.route.set('/error', null, {err});

The pol­i­cy says that guests are redi­rect­ed to /login if they request a pri­vate route. To do this, the route han­dlers do dif­fer­ent things under the assump­tion the app state is in scope.

  • The com­po­nent attached to /:tenant/widgets redi­rects the user in oninit()
  • The RouteRe­solver attached to /:tenant/widgets/id returns a reject­ed Promise.

The lat­ter case caus­es Mithril to fall back to the default route, which is /login in this case. This implic­it­ly pro­grams the same behav­ior. You could still call m.route.set() in the RouteRe­solver and return undefined assum­ing there isn’t a ren­der pass that would be sur­prised by that.


I inten­tion­al­ly left out error han­dling and the ori­gin of app for brevi­ty, but in the shoes of a junior who has nev­er seen a router or Mithril before, does this code make any god­damn sense?

Look at how much stuff we have to know just to write code that block guests. If one flow blocks ren­der­ing and the oth­er doesn’t, our pol­i­cy shouldn’t have to change shape to accom­mo­date in this case.

You might (cor­rect­ly) argue that this case is easy to refac­tor into a more con­sis­tent form. Incon­sis­ten­cy in how we resolve routes is not Mithril’s fault. How­ev­er, Mithril does oblige us to write our rout­ing pol­i­cy in its router’s terms. Once rout­ing tables get large and you have juniors on your team, keep­ing order will become a chore. For exam­ple, to bring back a load­ing indi­ca­tor in /:tenant/widgets/:id (Prod­uct will ask for it; don’t act like they won’t), you have to either:

  1. Replace the RouteRe­solver with a com­po­nent to ren­der that indi­ca­tor imme­di­ate­ly; or
  2. Make onmatch return a com­po­nent imme­di­ate­ly, out of sync with the loadWidgetDetail call.

That’s not obvi­ous to every­one. And if you are from the “pass state down as attrs” school of thought, you can get fucked because chang­ing around Mithril’s rout­ing code means you have to rewire how state cir­cu­lates to your com­po­nents (No, I will not import my app state in every JS mod­ule I write. That’s a dif­fer­ent arti­cle). It’s easy for a team to end up in an awk­ward posi­tion where sim­ple changes lead to non-triv­ial refac­tor jobs.

Let’s make this more inter­est­ing: What do you do if you want sev­er­al load­ing indi­ca­tors start­ing with a big spin­ner, then a dash­board break­down with a bunch of lit­tle baby spin­ners on ana­lyt­ics wid­gets crunch­ing num­bers? The answer is not worth think­ing about, but I can tell you my first attempt involved an unholy union of dec­o­rat­ed RouteRe­solvers.

Organizing code around policy

Again, the strug­gles with Mithril’s router is not a state­ment on its qual­i­ty. How­ev­er, some­thing is back­wards. Our rout­ing pol­i­cy should not depend on Mithril’s router. The oppo­site must be true to clean up our code. From the Bob Mar­tin school of thought, it is the rules that make an app work that should sit at the cen­ter of your soft­ware. While we can­not phys­i­cal­ly force Mithril to depend on our code, we can reor­ga­nize to say that the rules of our app dic­tate every­thing from rout­ing, to state, to views, and so on.

After writ­ing and rewrit­ing my rout­ing code sev­er­al times to do what should be sim­ple things, I asked myself what I want­ed my code to look like, with this in mind.

For afore­men­tioned rea­sons I knew that the code I want­ed would have to meet the fol­low­ing require­ments:

  • The code says what it does and does what it says.
  • You should be able to change rout­ing pol­i­cy with­out need­ing 6 can­dles, a virgin’s sharp­ened femur, and a sheep.
  • Appli­ca­tion state is more author­i­ta­tive than the router’s state.
  • Nev­er block ren­der­ing. Views ren­der the app at any moment. If the views look wrong, I either changed state wrong or redrew at a bad time.
  • The Mithril com­po­nents are pure (attrs return con­tent deter­min­is­ti­cal­ly).

I came up with this:

dispatch(document.getElementById('root'), '/error', app, {
    '/': () => Login,
    '/login': () => Login,
    '/error': () => ErrorReport,
    '/:tenant/widgets': loggedin((app, task) => {
        const {tenant} = app.spaRequest;

        task(() =>
               .then(() => app.loadWidgetListing()))

        return WidgetsOverview;
    '/:tenant/widgets/:id': loggedin((app, task) => {
        const {id, tenant} = app.spaRequest;

        task(() =>
               .then(() => app.loadWidget(id)));

        return WidgetDetail;

dispatch() also sets up the Mithril router, but you have to inter­pret the set­up dif­fer­ent­ly. It assumes that all state comes from one place, à la Redux. That’s where you see app passed in. It can be a plain old emp­ty object if you want. It also assumes that the pend­ing route and all relat­ed argu­ments are part of appli­ca­tion state, and makes it avail­able as part of app.spaRequest.

Sec­ond, the /error route spec­i­fied is not just a default route. It is con­sid­ered the route you vis­it in error con­di­tions, and I treat it that way for the same rea­son I would write try..catch in an entry point.

Final­ly, the rout­ing table itself is a table of routes to func­tions. The func­tions return com­po­nents to ren­der syn­chro­nous­ly but may change state asyn­chro­nous­ly in terms of the pend­ing route using a task() func­tion. The task func­tion takes a func­tion that returns a Promise and redi­rects to the error route in the event of an unhan­dled fail­ure. Oth­er­wise, it starts a new ren­der pass when the task fin­ish­es.

That loggedin() func­tion you see is just an app-spe­cif­ic dec­o­ra­tor that redi­rects users to /login if an authen­ti­cat­ed user is not in app state. You will see it in the below Code­Pen. I want you to see that as an exam­ple of why it should be easy to express rout­ing pol­i­cy. Pre­vi­ous­ly pol­i­cy had to be parsed from Mithril seman­tics, but in this case I could remove Mithril from my project entire­ly and not vio­late the rules as gov­erned by this approach. If you want to guard a route from unau­then­ti­cat­ed users, slap loggedin on it. If you want to add a colony of nest­ed load­ing indi­ca­tors, do that in your com­po­nent, then run task calls that iter­a­tive­ly update state with data in stages.

Some might also want to fac­tor out the “authen­ti­cat­ed ten­ant selec­tion” pat­tern in the han­dlers for clean­li­ness. That is now an easy change, and you can clear­ly see what guar­an­tees are made about state with func­tion dec­o­ra­tors, even if the flow is high­ly sub­jec­tive.

function tenantRoute(fn) {
  return loggedin((app, task) => {
    const tenantTask = (next) => task(() => 

    return fn(app, tenantTask);

dispatch(document.getElementById('root'), '/error', app, {
    '/': () => Login,
    '/login': () => Login,
    '/error': () => ErrorReport,
    '/:tenant/widgets': tenantRoute((app, tenantTask) => {
        tenantTask(() => app.loadWidgetListing()))

        return WidgetsOverview;
    '/:tenant/widgets/:id': tenantRoute((app, tenantTask) => {
        tenantTask(() => app.loadWidget(;

        return WidgetDetail;

This approach is not with­out caveats. For one, you are oblig­ed to write LBYL-style com­po­nents that check for incom­plete state to ren­der one of many pos­si­ble vari­a­tions of a view. Sec­ond, you have to be extra care­ful to syn­chro­nize state between redraws. This should go with­out say­ing, but the approach in dispatch() makes it you to spawn con­cur­rent Promise chains that all mutate state. But those of a Redux mind­set would see an oppor­tu­ni­ty to extend task() with the trans­ac­tion­al behav­ior expect­ed from reduc­ers.

Demo, plus other considerations

If you want to see an exam­ple of dispatch() in action, play with this SPA.

See the Pen Con­trived dis­patch() demo by Sage Ger­ard (@zyrolasting) on Code­Pen.0

The Code­Pen con­tains the tran­spiled demo from the mithril-dis­patch GitHub project, which uses a rout­ing pol­i­cy based on a real-world SaaS shop despite the sim­plis­tic con­tent. I encour­age you to start read­ing here to see the code orga­ni­za­tion ben­e­fits brought by dispatch(). As a bonus, the demo also shows a high­ly-opin­ion­at­ed use of lay­out diff­ing opti­miza­tion for users famil­iar with RouteResolver.render(). You can also see that the com­po­nents end up pure despite com­plex, slow oper­a­tions that would involve heavy use of life­cy­cle meth­ods oth­er­wise.


In this arti­cle I have intro­duced you to SPA routers. From there, we learned about Mithril’s router and how it han­dles dif­fer­ent pol­i­cy deci­sions when rout­ing. We learned that chang­ing the rout­ing pol­i­cy to meet sim­ple require­ments is hard­er than just express­ing the pol­i­cy in plain lan­guage. By adding a dispatch() func­tion to sep­a­rate pol­i­cy from Mithril seman­tics, the Mithril com­po­nents we write only ren­der what they are giv­en, state con­cerns itself with the data, and route res­o­lu­tion mere­ly con­nects the two togeth­er.

I also empha­sized that more than any­thing, the rules of your app are king. Just because I wrote dispatch() in the man­ner shown does not mean I will use that exact imple­men­ta­tion lat­er. Just be sure that you do not make your rules depend on Mithril’s router, or even Mithril in gen­er­al. Frame­works are meant to enable your pro­duc­tiv­i­ty, not tell you how to code.

I love cod­ing and have done so pro­fes­sion­al­ly for over a decade. I use what lit­tle free time I have to exper­i­ment on ideas of var­i­ous qual­i­ty and to help oth­ers write qual­i­ty soft­ware. If this post was help­ful to you, please leave a tip and share this arti­cle with any­one that need to tame the Mithril router.

Mind map freeze-dried web searches with Eyedact

I seri­ous­ly love mindmaps. If you’ve nev­er used them before, a mind map dia­grams a knowl­edge domain. They’re awe­some because you can make one in no time at all, and if they are well-made you can infer what you have to under­stand about any top­ic.

Mind maps are easy to write on paper, and there are many apps out there for mak­ing them fan­cy for pre­sen­ta­tions. Yet, if you must use mindmap­ping soft­ware for per­son­al use, you only need a text edi­tor and Google. You can write a mindmap as an indent­ed text file. Since I typ­i­cal­ly draw mind maps by hand, writ­ing a text file is the most I would do when mak­ing one on a com­put­er.

American Football

This exam­ple seems like it is offen­sive­ly incom­plete, but it works because we still see a knowl­edge domain’s shape. For most pur­pos­es this is enough. You prob­a­bly don’t need mind map apps that allow detailed notes or file attach­ments due to their pro­pri­etary file for­mats and lock-in. Mind maps are not for hold­ing small details any­way; they are for note-tak­ing. Fur­ther­more, we already can search the web any­thing, so mind maps should facil­i­tate access to exist­ing knowl­edge.

Using a mind map to freeze-dry web searches

Eye­dact helps mind map­pers search the web. Much like with Aloe, it’s com­plete­ly non-inva­sive. Unin­stall it, and you can still do what it does, just a lit­tle slow­er.

Obvi­ous­ly you could skip Eye­dact and Google what you want to know direct­ly, but there’s rea­son to think about doing bet­ter: Those new to a sub­ject might deal with vague tech­ni­cal terms. How many def­i­n­i­tions of “com­po­nent”, “remove” and “link” do you think are out there? What if you need to search for that word and you don’t remem­ber where it was rel­e­vant to you?  Eye­dact lets you write search­es that are too vague for Google and still get spe­cif­ic results using pre­served con­text. Hence “freeze-dried” web search­es.

Mind maps shape upcom­ing research for note-tak­ers, and pro­vide implic­it doc­u­men­ta­tion for main­tain­ers. Let’s talk more about that bold­ed point. If I’m writ­ing a frame­work with a big stack, I could write a para­graph explain­ing what we use and why, or I can just splat con­tent in front of peo­ple and tell them to read. Assume I have an NPM pack­age with the fol­low­ing con­tent in what-you-should-know.

    best practices
    best practices
    best practices
    best practices

In package.json, I can do some­thing like this:

    "scripts": {
        "doc": "eyedact what-you-should-know,"
        "doc:beginner": "npm run doc tutorial -al",
        "doc:journeyman": "npm run doc practices -al",
        "doc:advanced": "npm run doc caching async plugins -a",
        "doc:local": "less",

in this hypo­thet­i­cal, if you fin­ish review­ing the con­tent all the way down the list, you are ready to work on the project.


We briefly cov­ered the text rep­re­sen­ta­tion a of mind map and how you can use one to give con­text to many web search­es. Doing this guides peo­ple in search­ing with­in a new knowl­edge domain, and auto­mat­i­cal­ly gives peo­ple direc­tion in learn­ing lots of rel­e­vant mate­r­i­al from a sin­gle com­mand.

Moral cheating, part 2: GUI automation on Khan Academy

GUI automa­tion means writ­ing a pro­gram that pre­tends to be a human using a mouse and key­board. Assum­ing your tar­get doesn’t look for bots, you have enough to exploit the sys­tem.

In seri­ous games, bots must become indis­tin­guish­able from legit­i­mate play­ers. Naïve bots give them­selves away imme­di­ate­ly; they click the exact same pix­el on a frame to per­form the same action, they chain actions togeth­er with remark­ably per­sis­tent tim­ing, and they don’t respond when addressed in con­ver­sa­tion. Humans fid­get in antic­i­pa­tion of their next task, wig­gle the game cam­era, click to inspect enti­ties, chat with friends, and self-cor­rect. The best cheat­ing bot needs to exhib­it these flaws to the degree and style of the human they replace, which involves train­ing. This of course makes the bots woe­ful­ly slow to con­fig­ure and run, but if you are aim­ing to get past bot-detec­tion, these are nec­es­sary evils.

We will start with a sim­ple bot tar­get­ing Khan Acad­e­my, the lat­ter being an edu­ca­tion ser­vice that rewards users with ener­gy points and badges akin to gold stars. Here I write a bot that could earn all ener­gy points pos­si­ble for videos; This breaks the com­pul­sion loop for users retained by KA rewards.

Of course I am not going to vio­late KA’s ToS. I want to pay spe­cial atten­tion to sec­tion 8.14, mean­ing I will not run my bot to its extreme con­clu­sion and risk harm­ing “any user’s enjoy­ment of [the Web­site],” what­ev­er that means to legal demons. I think call­ing the badges/points “valu­able” is debat­able, but I won’t risk cheap­en­ing the val­ue oth­er users find in earn­ing badges and ener­gy points. We will only emu­late a human user’s activ­i­ty that would put no more load on their servers than nor­mal, and I will per­form a small demo for edu­ca­tion­al pur­pos­es. If you try using this bot or some­thing like it for your­self, then I am not respon­si­ble for your deci­sions.

The job entails open­ing a video in a brows­er, watch­ing it, and then mov­ing to the next one. KA’s embed­ded play­er moves auto­mat­i­cal­ly to the next relat­ed bit of con­tent, but it might redi­rect the user to some­thing oth­er than a video. There­fore we need to con­trol the nav­i­ga­tion our­selves.

GUI automa­tion requires many assump­tions to work, so just reuse the con­cepts if KA changes and breaks the code here.

If you want to max out your account, you could use curl and pup to scrape togeth­er a text file with every video link. For CYA rea­sons it would not be a good idea to dis­trib­ute the playlist should you build one. I am only show­ing you what you could do with a playlist if you had it for the sake of this dis­cus­sion on com­pul­sion loops, and you can test this just by copy­ing some links from your address bar man­u­al­ly in a text file.

Let’s look at a Python con­troller for Sikuli imple­ment­ing the fol­low­ing behav­ior giv­en a playlist text file:

  1. Find the next video to play.
  2. Open the video link, assum­ing a KA ses­sion is active.
  3. Start Sikuli, block­ing until it ends.
  4. Go to step 1.
  5. (On ter­mi­na­tion) Mark the line of the cur­rent video being watched.

Sikuli allowed me to pro­gram the bot using images, so I didn’t have to fuss with hard-cod­ed coor­di­nates. Since KA videos cur­rent­ly auto­play on load, the Sikuli script only needs to wait until the video ends and then close the brows­er.

You can study the source code for the above bot on GitHub.

What was the point of this GUI automation talk?

We just cov­ered an exam­ple of GUI automa­tion to break a sim­ple com­pul­sion loop in a real-world sys­tem. The ques­tion I want peo­ple to ask is how many users would KA have if every­one used this bot? If the bot killed reten­tion, then sites like KA can’t get by on con­tent alone. I think for marketing’s sake, that’s impor­tant to know. I hypoth­e­size is that few would return to KA if their rewards were use­less. If so, do we have free will to improve our­selves with­out a com­pul­sion loop to pro­gram us?

The bot shown here rais­es every red flag on cheat-detec­tion soft­ware. Bots cheat­ing com­pet­i­tive games must bet­ter emu­late human behav­ior.

But bet­ter bots to cheat at games with detec­tion soft­ware is not the direc­tion we are going to go, because as I stat­ed in part 1, the moral good from cheat­ing ends in com­pet­i­tive envi­ron­ments. Instead, we will next look at break­ing com­pul­sion loops by replac­ing them with bet­ter ones. Those who wish to reassess their pri­or­i­ties and reclaim con­trol of their health and atten­tion would rather sub­sti­tute, for exam­ple, gam­i­fied exer­cise in place of their World of War­craft account.

Terminal recording for GIF screencasts using toughtty

Ter­mi­nal record­ing soft­ware isn’t too friend­ly to con­tent pro­duc­ers, I’ve found. But there’s a Node.js imple­men­ta­tion of a ter­mi­nal recorder called ttys­tu­dio that showed promise even if it had not been touched for two years. I didn’t have to deal with weird sys­tem depen­den­cies and con­fig­u­ra­tion depend­ing on my dis­tro. I just ran npm i and went to town.

ttystudio was real­ly cool but I wasn’t real­ly agree­ing with the whole idea of record­ing the ter­mi­nal in real-time. I want­ed to have time to think about what I was typ­ing, and then have the out­put GIF not show any disk-space-hog­ging paus­es relat­ed to my doubt, inde­ci­sion or inter­mit­tent typ­ing accu­ra­cy.

After fork­ing and tak­ing sev­er­al shame­less­ly sub­jec­tive cre­ative lib­er­ties I end­ed up with this com­par­i­son. The first GIF is by ttystudio, and the sec­ond is by my ver­sion, toughtty*

ttystudio’s GIF weighs in at 84 frames and 3805 bytes, against toughtty’s 38 frames and 2980 bytes for the same ses­sion. Also, if I sat and spent 15 min­utes typ­ing that echo com­mand, the met­rics for toughtty’s GIF would not change, where­as ttystudio’s GIF would have grown beyond a rea­son­able size for use on the web. The only dif­fer­ence is that ttystudio cap­tures frames over time, where­as toughtty cap­tures frames in response to changes.

Throw­ing out record­ing in real-time does come with a caveat; You can’t add a pause in your GIF by just sit­ting back and wait­ing a moment. Want paus­es added auto­mat­i­cal­ly with­out real-time record­ing? Well, toughtty. I added a --padding option that lets the user add padding frames one, two or ten at a time to allow the view­er to review key con­tent.

I eat my own dog­food, so you can see GIFs pro­duced by toughtty on my Aloe arti­cle. If you want to record your ter­mi­nal ses­sions and save them as GIFs, check out the README in the toughtty source for instal­la­tion and usage instruc­tions.

*(I can’t believe that name wasn’t tak­en!)

Aloe enables rapid scaffolding for SCSS projects

aloe is a Sass-author­ing tool writ­ten in Python 2.7 that I use to write styles. Like oth­er style-cen­tric tools it is opin­ion­at­ed and does not do any­thing spe­cial. But, it is friend­ly to ter­mi­nal junkies. Run pip install aloe-pattern and fol­low along. I think you’ll like it.

Structuring projects with an aloe plant analogy

To enable a few fea­tures I apply a niche fla­vor of the Com­pos­ite pat­tern:

Any stylesheet direc­to­ry can be con­vert­ed to a new project at will, or vise-ver­sa.

This assumes fea­tures do not cut across direc­to­ries. Like with many oth­er pat­terns, the ben­e­fits to aloe’s pat­tern come from con­ve­nient­ly-avail­able con­text.

Sass, LESS or CSS on a laptop screen

Aloe works with a few goals in mind:

  1. Code shar­ing. One change may affect as many projects as I choose with­out involv­ing a remote resource.
  2. Rapid scal­able scaf­fold­ing. We won’t type @import  any­more, we will gen­er­ate con­tex­tu­al­ized rule­sets and we will have a cus­tom SMACSS project with BEM-ready wid­gets and themes for Christ­mas and Hal­loween ready yes­ter­day, with­out using tem­plates!
  3. Non-inva­sive­ness. I typ­i­cal­ly dis­like soft­ware that makes me feel like I need it. Aloe has no con­fig­u­ra­tion file and does not do any­thing weird to your code. You can use aloe and then imme­di­ate­ly unin­stall it with­out los­ing any of its ben­e­fits.

Tech Demo

Managing local dependencies

It’s eas­i­er to show than tell, so let’s start that SMACSS exam­ple I was talk­ing about. I’ll run aloe + base/normalize layout/{grid,sticky-footer} module/clock state/{form,notifications} theme/{xmas,halloween} to spin up a filesys­tem.

Using aloe to spin up a custom SMACSS project

This builds out of the box. The aloe plant anal­o­gy comes in with the __.scss par­tials. I call them rinds as part of the anal­o­gy, and all they do is import oth­er par­tials for me and pro­vide the frac­tal struc­ture for Sass com­pi­la­tions. The fol­low­ing rules apply:

  1. Every direc­to­ry con­tains exact­ly one rind named __.scss.
  2. Rinds con­tain only @import direc­tives, and are also the only files to con­tain @import direc­tives. They import par­tials inside their own direc­to­ries before oth­er rinds.
  3. Rinds may only import oth­er rinds in imme­di­ate sub­di­rec­to­ries. So ~/scss/__.scss may import ~/scss/foo/__.scss, but not ~/scss/bar/baz/_.scss.

These rules con­sol­i­date depen­den­cy man­age­ment to one file at a time, and you can use rinds as a sin­gle point to include or exclude entire swaths of func­tion­al­i­ty from a com­pi­la­tion, as we will see lat­er.

Get­ting back to our SMACSS exam­ple, if I need to reorder or remove any depen­den­cies, it’s just as easy. If I run aloe - state for exam­ple, state styles will be exclud­ed if I build any styles from the root of the project.

To elab­o­rate, let’s look at a fresh exam­ple. Here I run aloe + a b c d e. You will see that my root rind imports the depen­den­cies in the order I declare. If I clear out this file (>__.scss) and then link up the par­tials in reverse order (aloe + e d c b a), the effect is as expect­ed.

Writing partial contents

This seems like its all well and good, but what about set­ting the con­tents for these par­tials? One thing I do as a next step for scaf­fold­ing is use the aloe plot com­mand that lets me write a sum­ma­ry of rule­sets. If I run some­thing like aloe plot "clock(hourhand,minhand,sechand)", I get SCSS rule­sets. There’s a switch I throw to enable BEM-style writ­ing à la Stu­art Rob­son, so aloe plot -b "button(--big,__text(--upper,--lower))" gives me a BEM-y par­tial.

Don’t wor­ry about leav­ing these rule­sets emp­ty, because Sass will not include any emp­ty rule­sets in com­piled CSS.

On that note, let’s build some CSS. Here I use aloe build to out­put CSS using rinds. I can spec­i­fy mul­ti­ple tar­gets to build and con­cate­nate in order so I can deploy tar­get styles depend­ing on what I need. This nice­ly mesh­es with per-request style deploy­ment.

aloe fol­lows sym­links, so I can main­tain a link to any (S)CSS stylesheet to ben­e­fit from its fea­tures as if it were a par­tial.

In this post we cov­ered a non-inva­sive Sass author­ing tool used for rapid scaf­fold­ing and cus­tom deploy­ment of styles using a frac­tal orga­ni­za­tion pat­tern.

Moral cheating, part 1: Breaking the Compulsion Loop

Devel­op­ers of video games or gam­i­fied sys­tems use com­pul­sion loops to oblige con­tin­ued use of their prod­uct. A com­pul­sion loop is a iter­a­tive process that instills in you a new habit. Con­ve­nient­ly, your new habit nor­mal­ly makes mon­ey for some­one else.

Com­pul­sion loops cre­ate the illu­sion of val­ue, as South Park stat­ed bril­liant­ly in S18E6Freemi­um isn’t Free.

If you must exploit human psy­chol­o­gy to keep users, then your prod­uct lacks intrin­sic val­ue. If it weren’t for com­pul­sion loops, some­one should still find rea­son to stay with you.

But we live with com­pul­sion loops, and those prone to addic­tion arguably suf­fer the most. Addicts end up stuck in com­pul­sion loops on a down­ward spi­ral for the next dopamine hit. Take Runescape, a MMORPG by Jagex Ltd. designed to keep you play­ing regard­less of your mood or health. I don’t even need to men­tion World of War­craft. I’ve played both games and had fun, but not because the games were fun in them­selves. Because the games had no intrin­sic val­ue, I had to rely on friends and the antic­i­pa­tion com­pul­sion loops offer to tol­er­ate the grind.

This is not to say com­pul­sion loops are bad, just that they are pow­er­ful. We can only hope to get stuck to com­pul­sion loops that make us want to diet, exer­cise and keep the house clean. Unfor­tu­nate­ly busi­ness­es are excel­lent at pro­gram­ming your habits to suit them before you set your­self up for suc­cess. Neale Mar­tin, author of Habit: The 95% of Behav­ior Mar­keters Ignore, helps busi­ness­es shape their own cus­tomers in this way.

I am a staunch oppo­nent of bad com­pul­sion loops. This is not to be tak­en as an active and hyp­o­crit­i­cal push to make peo­ple behave dif­fer­ent­ly. Rather, I would hope to guide peo­ple out of com­pul­sion loops them­selves if they want out but are strug­gling to leave.

If you know­ing­ly choose to stay in a loop and main­tain the cost, more pow­er to you. My favorite self-pro­claimed alco­holic come­di­an Doug Stan­hope has more to say on that sub­ject.

Stan­hope says (like­ly in jest) that addic­tion doesn’t exist. Even if that were true, the pur­pose of this arti­cle series is to break an addict’s spi­ral and com­pul­sion loop so they can find more con­trol and enjoy­ment that ben­e­fits them more.

For con­ve­nience I will refer to both gam­i­fied sys­tems and video games as “activ­i­ties,” because we should talk about when it’s okay to cheat them both.

But why cheat? It’s not the only way to fight addic­tion. You have cold turkey and habit sub­sti­tu­tion, so why pick this more con­tro­ver­sial approach?

We need the rewards from cheat­ing to cheap­en the val­ue of our own work. If you were banned from a game for cheat­ing or sim­ply went cold turkey, you still have antic­i­pa­tion for reward, which is one of the dri­vers keep­ing you in a com­pul­sion loop.

In my expe­ri­ence hav­ing spent more time and mon­ey than I want to admit addict­ed to games, cheat­ing breaks a com­pul­sion loop so emphat­i­cal­ly that you feel no antic­i­pa­tion, pur­pose or dri­ve to con­tin­ue. When you run iddqd in DOOM or max out all of your stats after run­ning a bot, you start your last joyride. With noth­ing left to antic­i­pate, cheat­ing means ruin­ing the game for your­self. You won’t play it again for a long time after the nov­el­ty of god-teir game­play runs off.

The Compulsion Loop has its own darkness

This is why cheat­ing ruins games for the play­er on an indi­vid­ual lev­el. Now, if pulling the lever in the Skin­ner box for a food pel­let takes more away from you than the pel­let can ever give back, then cheat­ing is an under­rat­ed detox pro­gram that breaks a down­ward spi­ral. Cheat­ing becomes the right thing to do. You WANT to ruin the activ­i­ty for your­self before the activ­i­ty ruins you.

Even if you enjoy some addict­ing activ­i­ties and wish to keep doing them, you should learn how to cheat as a way to con­trol your habits and the influ­ence that com­pul­sion loops have on you. If some­thing changes in your life and the habit starts to take its toll, you need that out.

I can’t cure addic­tion, but when pos­si­ble I can teach addicts to cheat as a means to regain lost con­trol. Next in the series we will dis­cuss GUI automa­tion with Sikuli to cheat our way to fake points on Khan Acad­e­my, a gamei­fied edu­ca­tion plat­form. Our goal will be to break the com­pul­sion loop in Khan Acad­e­my so that only the peo­ple who val­ue edu­ca­tion for it’s own sake will con­sume its con­tent.

Addendum: What about the other players?

Cheat­ing in games that involve mon­ey and com­pe­ti­tion rais­es one sim­ple ques­tion: Is it ever right to cheat if one play­er can gain an advan­tage over oth­ers that play legit­i­mate­ly?

It’s nev­er right. That’s an easy answer because this ques­tion assumes a cheater is com­pet­ing. There’s a dif­fer­ence between using a mem­o­ry edi­tor to win addic­tive sin­gle-play­er games and using a bot to win a Rock­et League sea­son.

Cheat­ing in com­pet­i­tive envi­ron­ments must stop imme­di­ate­ly because the reward for the obvi­ous cheat­ing harms oth­ers. Oth­er­wise, cheat­ing sab­o­tages addic­tive activ­i­ties, which is a good thing.

Morals become grey for me when cheat­ing hurts only the activity’s own­ing busi­ness. Mobile freemi­um games or MMOs might cre­ate a per­fect storm where busi­ness­es wins at the expense of addicts.

Design­ers should allow play­ers to cheat so long as the cheats don’t affect com­pe­ti­tion. Revert­ing a char­ac­ter to a point before they start­ed cheat­ing is a good approach. This gives seri­ous play­ers an incen­tive to grind and train legit­i­mate­ly, and gives addicts an out.

Games like Runescape won’t allow this because real-world cur­ren­cy is tied an in-game mar­ket. Any cheat­ing would vapor­ize their busi­ness and reduce the game to blind rage and lia­bil­i­ty issues. If Jagex could cap­i­tal­ize only on the com­pet­i­tive nature of the game, maybe they wouldn’t be in this mess.

Despite the harm it would do to Bliz­zard, Jagex, etc., cheat­ing their games frees addicts. Addicts can­not make lucid or informed con­sent. Busi­ness­es should not take mon­ey from addicts because those trans­ac­tions aren’t legit­i­mate.

Giv­ing addicts an out so that they can invest more in them­selves is the moti­va­tion behind this series. That, and encour­ag­ing for intrin­sic val­ue in prod­ucts once again.

Creative writing with git-flow and the Snowflake Method

git-flow’s val­ue depends on the nature of a project. Take cre­ative writ­ing: Randy Ingermanson’s Snowflake Method makes you start from a crude—yet complete—one-sentence sto­ry and iter­ate until you are left with a good sto­ry. Require­ments imposed by The Snowflake Method are anal­o­gous to git-flow’s role for the master branch. Giv­en a LaTeX project man­aged with a com­bi­na­tion of git-flow and the Snowflake Method (“Snowflow”), we get some inter­est­ing prop­er­ties.

Assume this file sys­tem:          # Compile PDF(s) in dist using story/
dist/             # .pdf files
concepts/         # whatever
story/            # .tex files
    aggregate.tex # \document{book} 

At min­i­mum runs some­thing like pdflatex -halt-on-error -output-directory ./dist ./story/aggregate.tex to make a PDF of your sto­ry. The concepts/ direc­to­ry con­tains assets describ­ing char­ac­ters, set­tings, con­flicts and plot deci­sions. One rule for this project is that the concepts/ direc­to­ry be checked in, but nev­er be processed by code. This allows free-form cre­ativ­i­ty in asset pro­duc­tion that a pre­cise process would oth­er­wise cur­tail.

A woman writing on a Mac, hopefully with help from the Snowflake Method

Snowflow branch­es behave anal­o­gous­ly to their git-flow coun­ter­parts, with some added expec­ta­tions.

  • The master branch holds a project that com­piles to PDF and tells a com­plete sto­ry.
  • An elaborate (develop) branch adds detail to the sto­ry.
  • concept (feature) branch intro­duces at least one unique, self-con­tained con­cept in concept/ lat­er used to elaborate.
  • review (release) branch holds a com­plete sto­ry for review. If the sto­ry is of suf­fi­cient edi­to­r­i­al qual­i­ty it may merge to master.
  • reconcile (hotfix) branch fix­es press­ing log­i­cal errors such as plot­holes.
  • seed (support) branch address­es cir­cum­stan­tial con­cerns such as dif­fer­ences in sto­ry arcs or fic­tion­al uni­vers­es.

To enable the Snowflake Method, master should a com­plete sto­ry, but that sto­ry does not have to be pub­lish­able. On that note an ear­ly iter­a­tion for Therac-25’s firmware shouldn’t have seen the light of day. It may seem insen­si­tive to com­pare death by radi­a­tion over­dose to bad writ­ing, but only if you’ve nev­er read any­thing by Dan Sacharow.

A Snowflow project will face a “soft end” on a com­mit to master rep­re­sent­ing a pub­lish­able sto­ry. Unless you come up with dif­fer­ent uni­vers­es, sto­ry arcs or deriv­a­tive prod­ucts there may be no need to mea­sure pro­gres­sion or com­pat­i­bil­i­ty with ver­sion num­bers or tags.

In exper­i­ment­ing with this work­flow my favorite dis­cov­ery is that concepts/ even­tu­al­ly takes the shape of “behind the scenes” con­tent for read­ers, which may be sep­a­rate­ly pack­aged and sold. So long as com­mits are dis­ci­plined, the com­mit his­to­ry helps you build sev­er­al prod­ucts at once, where the main sto­ry you intend to pub­lish implic­it­ly pro­motes con­tent you could pro­duce using infor­ma­tion in concepts/.

The concepts/ direc­to­ry also serves as a sand­box for inspired writ­ing ses­sions. Writ­ing is pret­ty moody. Some­times I feel dis­ci­plined and can see how to pack­age my com­mits, and oth­er times I just want to write with no dis­trac­tions. So if you want to ham­mer out a few thou­sand words in a text file in concepts/, go nuts. You can always wor­ry about struc­tur­ing the con­tent with Snowflow when you are ready.

Elaboration process

I must elab­o­rate on the elaborate branch. elaborate may either expand on the sto­ry or per­form tech­ni­cal tasks using LaTeX. In the for­mer sce­nario, I use foot­notes con­tain­ing at least one ques­tion or instruc­tion to iden­ti­fy oppor­tu­ni­ties to build on the sto­ry.

You don’t have to use foot­notes, but keep in mind that some­one who reviews your work should not have to be a devel­op­er. I like hav­ing some­thing vis­i­ble in the prod­uct for an author and edi­tor to dis­cuss.

For exam­ple:

Bob jumped over Alice. \footnote{But WHY did Bob jump over Alice?}

To make the elab­o­ra­tion process sen­si­ble, you should write con­tent that address­es a foot­note either in the vicin­i­ty of where the foot­note appeared, or in a loca­tion that bet­ter estab­lish­es con­text. When you resolve the mat­ter raised by a foot­note, remove that foot­note.

When you com­mit to an elaborate branch you may add at least zero foot­notes, but you must remove at least one of the foot­notes found when you start­ed. By the time you start a review branch there should not exist any foot­notes.

Review process

  1. Elab­o­rate on all rel­e­vant foot­notes.
  2. git flow release start ...
  3. Com­pile a PDF to release to trust­ed read­ers for feed­back.
  4. From the feed­back, insert a list of foot­notes (or oth­er anno­ta­tions) where applic­a­ble accord­ing to your own best judge­ment.
  5. Address all foot­notes.
  6. Repeat steps 3–6 until there exist no foot­notes.
  7. git flow release finish


Writ­ing can’t be con­strained by too many rules, but I did note these guide­lines and obser­va­tions down as I worked.

  • Do not adjust the story/ to the concepts/, adjust the concepts/ to the story/.
  • Do not mod­i­fy story/ when on a concept branch.
  • Your com­fort and focus on writ­ing comes before the process. Don’t be afraid of relax­ing with pen and paper to decide what to do. Lay down on the floor and sketch on a bunch of Post-Its next to a cup of tea before typ­ing any­thing.

Licensing Snowflake Method content

If you decide to write using this process, stay mind­ful of where you pub­lish your work­ing code. If your prod­uct is a book, license it like a book. But more than any­thing, con­sult some­one qual­i­fied to talk about licens­ing. Of course some books like You Don’t Know JS are open source, but if you are con­cerned about dis­tri­b­u­tion, do your research and choose a license.

Generate tailored résumés with Spin

Recruiters might ask me to change my résumé, and by “might” I mean “always.” Some­times it’s reorder­ing sec­tions. Oth­er times it’s giv­ing them a Word doc­u­ment instead of a PDF. This hap­pened enough times to jus­ti­fy at least some lev­el of automa­tion, so I wrote Spin, a CLI that gen­er­ates résumés tai­lored to dif­fer­ent audi­ences. Spin con­verts a super­set of Mark­down Extra to HTML5. So if I made fur­ni­ture on the side, I can gen­er­ate two résumés with $ spin ./woodworker ./programmer.

The woodworker and programmer pro­files are writ­ten in Mark­down Extra mixed with @ direc­tives. The direc­tives define sec­tions and expe­ri­ences with infor­ma­tion like start and end dates. Check out the exam­ples in Spin’s source.

Mak­ing résumés com­piled prod­ucts yield some ben­e­fits:

  • Code reuse. Two pro­files may share let­ter­heads, con­tact info and sec­tions. Corol­lary to this, chang­ing a résumé to entice Agile Cube Farm LLC doesn’t have to change the résumé I lat­er send to Hour­Long Standup Co.
  • Con­sol­i­da­tion. I can write every­thing I ever did in my life in one place. I may now jour­nal my qual­i­fi­ca­tions freely in world’s most obses­sive­ly struc­tured diary.
  • It’s free. Spin is not friend­ly to peo­ple who aren’t used to cod­ing. But all of the résumé man­age­ment tools I’ve seen so far cost mon­ey, are down, don’t sup­port pro­files, or require more labor than I want to put in. I got every­thing I want with Spin for free after the ini­tial time invest­ment.

Implementation notes

I picked PHP to churn this out quick­ly and gen­er­ate doc­u­ments in HTML5 to allow an inter­me­di­ary step for styling and script­ing for those who care enough. I didn’t want to use PHP­Word because my use of it would be too opin­ion­at­ed and would force con­sumers to fuss with PHP exten­sions. Pan­doc works just fine thank youverymuch.

One can use Spin to main­tain a few pro­files and con­vert every out­put doc­u­ment to DOCX and PDF using Pan­doc. My qual­i­fi­ca­tions serve as a real-world exam­ple of this. The base HTML doc­u­ments them­selves may also have extra fea­tures that make them more pleas­ant for view­ing on a brows­er.

If Spin helps you, please con­sid­er buy­ing me a beer!

Iterating Git commit history using Qi

In the Code­less Code, a cre­ative blog on soft­ware engi­neer­ing, there’s this scribe named Qi and he is a total badass. Qi is source con­trol per­son­i­fied; a prag­mat­ic sto­ry­teller that rea­sons about sys­tems as the sum total of all that came before.

I wrote the qi CLI tool (GitHub, PyPi) as a hat-tip to the author—and with his gra­cious per­mis­sion for use of the name. The idea is to bring devel­op­ers lever­age in dis­cus­sions with man­age­ment through a notice­ably less vio­lent vari­ant of Case 89qi is only a Python wrap­per to git that prints revi­sion hash­es at the end of fixed time inter­vals. This enables run­ning an analy­sis at, say, the end of every week over the past two quar­ters to check if a team deci­sion in Jan­u­ary is hav­ing an intend­ed effect in June.

This helps devel­op­ers and man­agers accept the same facts under pres­sure. I once worked in a high­ly active JS shop with over 30 devel­op­ers and over one hun­dred com­mits every day. We had Bam­boo run­ning builds and auto­mat­ed tests every so often to catch prob­lem­at­ic changes, but it was hard to know if our changes this sprint fol­lowed a spe­cif­ic “direc­tion” for sys­tem qual­i­ty, what­ev­er that meant.  After I wrote and ran qi, I dis­cov­ered that the team was, on aver­age, cre­at­ing sev­en pro­duc­tion code files for every one test suite. To make mat­ters worse, rough­ly 20% of our com­bined JS and CSS went com­plete­ly unused.

We all knew that these things hap­pened, but once I had the num­bers to show that mat­ters were get­ting worse due to man­age­ment pres­sure, we re-pri­or­i­tized to allow more time to increase code cov­er­age for crit­i­cal tests. Qi knew that com­mit his­to­ry could not just be read, but lever­aged.

qi is use­ful when one of the fol­low­ing con­di­tions apply to you:

  • You want to gath­er facts and cor­re­la­tions to study the code’s his­to­ry, review recent changes, or to prove a point.
  • You have no way to mon­i­tor the state of a code­base in terms of met­rics or cor­re­la­tions that are impor­tant to you or man­age­ment.

qi is not use­ful under these con­di­tions:

  • Your project(s) are small or neglect­ed because there is rarely a big pic­ture to derive, let alone a desire to find one.
  • Your team has a strong grasp on the sys­tem being main­tained, or already per­forms suf­fi­cient analy­sis.

What’s happening?

Con­sid­er the com­mand qi --every month --past "3 years".

The com­mand does what it says. It will print revi­sion hash­es that mark the end of every month for the past three years, with some caveats:

  • qi starts from HEAD. The query against his­to­ry will search 3 years pri­or to the com­mit date from HEAD, not from the time you ran the com­mand.
  • If HEAD is younger than 3 years, then the out­put will end at the com­mit mark­ing the end of the first month of devel­op­ment.
  • If HEAD is not even a month old, there will be no out­put.

Print­ing hash­es is one thing, but we want to do some­thing with this infor­ma­tion. Since qi is basi­cal­ly a time iter­a­tor, we can tell it to run a com­mand at each com­mit. Note that qi’s short options use the same let­ter that starts their long options.

qi -e month -p "3 years" -c "./analyze"

When you spec­i­fy a com­mand for qi to run, qi will check out each sig­nif­i­cant com­mit (detach­ing HEAD) and run the com­mand against the code­base at that time.
If your analy­sis fin­ish­es or crash­es, qi will attempt to put HEAD back where it start­ed. But if qi crash­es, HEAD will be left detached and you will need to run git checkout to put it back your­self.

qi -c lets you mod­el infor­ma­tion­al met­rics over time, such as the pro­por­tion of dead code in the project and the cur­rent code cov­er­age by tests. You decide what met­rics mat­ter, because qi is not a report­ing tool, it is one kind of iter­a­tor a report­ing tool might use.

Let’s look at a cou­ple of fun exam­ples.

For triv­ia, if you run qi -e year -p "12 years" -c "du -sh ." on Torvald’s Lin­ux source, you will see the disk usage growth of the project over it’s time on GitHub.

[sage linux]$ qi -e year -p "12 years" -c "du -sh ."
2.8G . # 2017 (to date)
2.7G . # 2016
2.6G . # 2015
2.6G . # 2014
2.5G . # 2013
2.5G . # 2012
2.4G . # 2011
2.4G . # 2010
2.4G . # 2009
2.3G . # 2008
2.2G . # 2007
2.2G . # 2006
2.2G . # 2005

Anoth­er exam­ple is if you want React’s com­plex­i­ty report at the end of every year of its devel­op­ment. In this case you could use this set­up:

$ git clone && cd react
$ qi --every year --past "3 years" --command "./"

TIME=$(git show -s --format=%ct HEAD)

# Using
cr -e -f json ./src > $TIME.json


Hiding annoying diffs on BitBucket pull request pages

Some shops man­date check­ing in long, auto­gen­er­at­ed files into source con­trol that clut­ter up Bit­Buck­et pull request pages with large diffs, leav­ing you to scroll epic dis­tances or depend on the top nav­i­ga­tion to vis­it files key to a code review.

Despite at least three dif­fer­ent tick­ets dat­ing back to 2014 ask­ing Atlass­ian to sup­port exclud­ing spe­cif­ic file diffs from pull request pages, a John Gar­cia explained why they would not imple­ment this fea­ture:

This response is rea­son­able for the ref­er­enced case, yet unhelp­ful to users as a whole. Even if the peo­ple ref­er­enced in the con­ver­sa­tion were using source con­trol inap­pro­pri­ate­ly by track­ing files that they should not track, oth­ers review­ing code might not be in a posi­tion to change that. Then there are big (as in, often has diffs that take up at least one screen) auto­gen­er­at­ed files that fre­quent­ly change, aren’t nor­mal­ly worth read­ing and should be in source con­trol, such as yarn.lock. Cit­ing a Stack­Over­flow answer address­ing one case with 11 votes at the time of this writ­ing does not make the fea­ture request unjus­ti­fied.

I had the same prob­lem as the peo­ple sup­port­ing the tick­et (with many yarn.lock files, no less) so I wrote a user­script cre­ative­ly named “Hide Annoy­ing Bit­Buck­et Diffs (HABD).” It’s set to exe­cute when you view a pull-request page pow­ered by Bit­Buck­et. The script does not sup­port bina­ry files. Instal­la­tion instruc­tions are at the bot­tom of this arti­cle.

In order for this script to func­tion, you will need to con­fig­ure a cou­ple of things. Thank­ful­ly it’s quick!

You can con­trol what diffs to remove by ref­er­enc­ing a plain text resource list­ing one reg­u­lar expres­sion per line (Point­ing to a raw Paste­bin helps here). A diff is removed from the pull request page if the file­name cor­re­spond­ing to that diff match­es ANY of the expres­sions in the resource. 

For exam­ple, this resource hides file diff blocks for package.json and yarn.lock, both of which are com­mon in Node projects and are checked into source con­trol.


If we’re clear up to this point and you want to use the script, let’s get you set up. If you encounter any prob­lems, please report an issue.

Script installation for TamperMonkey

  1. While using Chrome, add Tam­per­Mon­key.
  2. Go to HABD’s page on and click “Install.” You will see the source code and a con­fir­ma­tion. Con­firm the instal­la­tion.
  3. Click the Tam­per­mon­key icon in Chrome and click “Dash­board.” You will see a list of installed scripts.
  4. At the far right of the row where you see HABD, you will see a lit­tle notepad icon. Click that to edit the script.
  5. Change the line start­ing with “@resource” to point to a raw Paste­Bin or some oth­er clear text resource with one reg­u­lar expres­sion per line. The link already in the script points to an exam­ple. Remem­ber that the reg­u­lar expres­sions match against file­names. If ANY of the expres­sions match, the diff is removed from the page.
  6. (Option­al) Change the line start­ing with “@match” so that it match­es only the pages in which you review Bit­Buck­et pull requests.
  7. (Option­al) If you want to hide diffs on both pull request and com­mit pages, use the @include rule to add anoth­er URL match­ing pat­tern.

Script installation for GreaseMonkey

  1. While using Fire­fox (or some oth­er sup­port­ed brows­er), add Grease­Mon­key.
  2. Go to HABD’s page on and click “Install.” You will see the source code and a con­fir­ma­tion. Con­firm the instal­la­tion.
  3. Click the arrow next to the Grease­Mon­key icon in Fire­fox and click “Man­age User Scripts.” You will see a list of installed scripts.
  4. On the row for “Hide Annoy­ing Bit­buck­et Diffs”, click “Pref­er­ences,” then “Edit This User Script.”
  5. Change the line start­ing with “@resource” to point to a raw Paste­Bin or some oth­er clear text resource with one reg­u­lar expres­sion per line. The link already in the script points to an exam­ple. Remem­ber that the reg­u­lar expres­sions match against file­names. If ANY of the expres­sions match, the diff is removed from the page.
  6. (Option­al) Change the line start­ing with “@match” so that it match­es only the pages in which you review Bit­Buck­et pull requests.
  7. (Option­al) If you want to hide diffs on both pull request and com­mit pages, use the @include rule to add anoth­er URL match­ing pat­tern.
Do NOT follow this link or you will be banned from the site!