Terminal recording for GIF screencasts using toughtty

Ter­mi­nal record­ing soft­ware isn’t too friend­ly to con­tent pro­duc­ers, I’ve found. But there’s a Node.js imple­men­ta­tion of a ter­mi­nal recorder called ttys­tu­dio that showed promise even if it had not been touched for two years. I didn’t have to deal with weird sys­tem depen­den­cies and con­fig­u­ra­tion depend­ing on my dis­tro. I just ran npm i and went to town.

ttystudio was real­ly cool but I wasn’t real­ly agree­ing with the whole idea of record­ing the ter­mi­nal in real-time. I want­ed to have time to think about what I was typ­ing, and then have the out­put GIF not show any disk-space-hog­ging paus­es relat­ed to my doubt, inde­ci­sion or inter­mit­tent typ­ing accu­ra­cy.

After fork­ing and tak­ing sev­er­al shame­less­ly sub­jec­tive cre­ative lib­er­ties I end­ed up with this com­par­i­son. The first GIF is by ttystudio, and the sec­ond is by my ver­sion, toughtty*

ttystudio’s GIF weighs in at 84 frames and 3805 bytes, against toughtty’s 38 frames and 2980 bytes for the same ses­sion. Also, if I sat and spent 15 min­utes typ­ing that echo com­mand, the met­rics for toughtty’s GIF would not change, where­as ttystudio’s GIF would have grown beyond a rea­son­able size for use on the web. The only dif­fer­ence is that ttystudio cap­tures frames over time, where­as toughtty cap­tures frames in response to changes.

Throw­ing out record­ing in real-time does come with a caveat; You can’t add a pause in your GIF by just sit­ting back and wait­ing a moment. Want paus­es added auto­mat­i­cal­ly with­out real-time record­ing? Well, toughtty. I added a --padding option that lets the user add padding frames one, two or ten at a time to allow the view­er to review key con­tent.

I eat my own dog­food, so you can see GIFs pro­duced by toughtty on my Aloe arti­cle. If you want to record your ter­mi­nal ses­sions and save them as GIFs, check out the README in the toughtty source for instal­la­tion and usage instruc­tions.

*(I can’t believe that name wasn’t tak­en!)

Aloe enables rapid scaffolding for SCSS projects

aloe is a Sass-author­ing tool writ­ten in Python 2.7 that I use to write styles. Like oth­er style-cen­tric tools it is opin­ion­at­ed and does not do any­thing spe­cial. But, it is friend­ly to ter­mi­nal junkies. Run pip install aloe-pattern and fol­low along. I think you’ll like it.

Structuring projects with an aloe plant analogy

To enable a few fea­tures I apply a niche fla­vor of the Com­pos­ite pat­tern:

Any stylesheet direc­to­ry can be con­vert­ed to a new project at will, or vise-ver­sa.

This assumes fea­tures do not cut across direc­to­ries. Like with many oth­er pat­terns, the ben­e­fits to aloe’s pat­tern come from con­ve­nient­ly-avail­able con­text.

Sass, LESS or CSS on a laptop screen

Aloe works with a few goals in mind:

  1. Code shar­ing. One change may affect as many projects as I choose with­out involv­ing a remote resource.
  2. Rapid scal­able scaf­fold­ing. We won’t type @import  any­more, we will gen­er­ate con­tex­tu­al­ized rule­sets and we will have a cus­tom SMACSS project with BEM-ready wid­gets and themes for Christ­mas and Hal­loween ready yes­ter­day, with­out using tem­plates!
  3. Non-inva­sive­ness. I typ­i­cal­ly dis­like soft­ware that makes me feel like I need it. Aloe has no con­fig­u­ra­tion file and does not do any­thing weird to your code. You can use aloe and then imme­di­ate­ly unin­stall it with­out los­ing any of its ben­e­fits.

Tech Demo

Managing local dependencies

It’s eas­i­er to show than tell, so let’s start that SMACSS exam­ple I was talk­ing about. I’ll run aloe + base/normalize layout/{grid,sticky-footer} module/clock state/{form,notifications} theme/{xmas,halloween} to spin up a filesys­tem.

Using aloe to spin up a custom SMACSS project

This builds out of the box. The aloe plant anal­o­gy comes in with the __.scss par­tials. I call them rinds as part of the anal­o­gy, and all they do is import oth­er par­tials for me and pro­vide the frac­tal struc­ture for Sass com­pi­la­tions. The fol­low­ing rules apply:

  1. Every direc­to­ry con­tains exact­ly one rind named __.scss.
  2. Rinds con­tain only @import direc­tives, and are also the only files to con­tain @import direc­tives. They import par­tials inside their own direc­to­ries before oth­er rinds.
  3. Rinds may only import oth­er rinds in imme­di­ate sub­di­rec­to­ries. So ~/scss/__.scss may import ~/scss/foo/__.scss, but not ~/scss/bar/baz/_.scss.

These rules con­sol­i­date depen­den­cy man­age­ment to one file at a time, and you can use rinds as a sin­gle point to include or exclude entire swaths of func­tion­al­i­ty from a com­pi­la­tion, as we will see lat­er.

Get­ting back to our SMACSS exam­ple, if I need to reorder or remove any depen­den­cies, it’s just as easy. If I run aloe - state for exam­ple, state styles will be exclud­ed if I build any styles from the root of the project.

To elab­o­rate, let’s look at a fresh exam­ple. Here I run aloe + a b c d e. You will see that my root rind imports the depen­den­cies in the order I declare. If I clear out this file (>__.scss) and then link up the par­tials in reverse order (aloe + e d c b a), the effect is as expect­ed.

Writing partial contents

This seems like its all well and good, but what about set­ting the con­tents for these par­tials? One thing I do as a next step for scaf­fold­ing is use the aloe plot com­mand that lets me write a sum­ma­ry of rule­sets. If I run some­thing like aloe plot "clock(hourhand,minhand,sechand)", I get SCSS rule­sets. There’s a switch I throw to enable BEM-style writ­ing à la Stu­art Rob­son, so aloe plot -b "button(--big,__text(--upper,--lower))" gives me a BEM-y par­tial.

Don’t wor­ry about leav­ing these rule­sets emp­ty, because Sass will not include any emp­ty rule­sets in com­piled CSS.

On that note, let’s build some CSS. Here I use aloe build to out­put CSS using rinds. I can spec­i­fy mul­ti­ple tar­gets to build and con­cate­nate in order so I can deploy tar­get styles depend­ing on what I need. This nice­ly mesh­es with per-request style deploy­ment.

aloe fol­lows sym­links, so I can main­tain a link to any (S)CSS stylesheet to ben­e­fit from its fea­tures as if it were a par­tial.

In this post we cov­ered a non-inva­sive Sass author­ing tool used for rapid scaf­fold­ing and cus­tom deploy­ment of styles using a frac­tal orga­ni­za­tion pat­tern.

Moral cheating, part 1: Breaking the Compulsion Loop

Devel­op­ers of video games or gam­i­fied sys­tems use com­pul­sion loops to oblige con­tin­ued use of their prod­uct. A com­pul­sion loop is a iter­a­tive process that instills in you a new habit. Con­ve­nient­ly, your new habit nor­mal­ly makes mon­ey for some­one else.

Com­pul­sion loops cre­ate the illu­sion of val­ue, as South Park stat­ed bril­liant­ly in S18E6Freemi­um isn’t Free.

If you must exploit human psy­chol­o­gy to keep users, then your prod­uct lacks intrin­sic val­ue. If it weren’t for com­pul­sion loops, some­one should still find rea­son to stay with you.

But we live with com­pul­sion loops, and those prone to addic­tion arguably suf­fer the most. Addicts end up stuck in com­pul­sion loops on a down­ward spi­ral for the next dopamine hit. Take Runescape, a MMORPG by Jagex Ltd. designed to keep you play­ing regard­less of your mood or health. I don’t even need to men­tion World of War­craft. I’ve played both games and had fun, but not because the games were fun in them­selves. Because the games had no intrin­sic val­ue, I had to rely on friends and the antic­i­pa­tion com­pul­sion loops offer to tol­er­ate the grind.

This is not to say com­pul­sion loops are bad, just that they are pow­er­ful. We can only hope to get stuck to com­pul­sion loops that make us want to diet, exer­cise and keep the house clean. Unfor­tu­nate­ly busi­ness­es are excel­lent at pro­gram­ming your habits to suit them before you set your­self up for suc­cess. Neale Mar­tin, author of Habit: The 95% of Behav­ior Mar­keters Ignore, helps busi­ness­es shape their own cus­tomers in this way.

I am a staunch oppo­nent of bad com­pul­sion loops. This is not to be tak­en as an active and hyp­o­crit­i­cal push to make peo­ple behave dif­fer­ent­ly. Rather, I would hope to guide peo­ple out of com­pul­sion loops them­selves if they want out but are strug­gling to leave.

If you know­ing­ly choose to stay in a loop and main­tain the cost, more pow­er to you. My favorite self-pro­claimed alco­holic come­di­an Doug Stan­hope has more to say on that sub­ject.

Stan­hope says (like­ly in jest) that addic­tion doesn’t exist. Even if that were true, the pur­pose of this arti­cle series is to break an addict’s spi­ral and com­pul­sion loop so they can find more con­trol and enjoy­ment that ben­e­fits them more.

For con­ve­nience I will refer to both gam­i­fied sys­tems and video games as “activ­i­ties,” because we should talk about when it’s okay to cheat them both.

But why cheat? It’s not the only way to fight addic­tion. You have cold turkey and habit sub­sti­tu­tion, so why pick this more con­tro­ver­sial approach?

We need the rewards from cheat­ing to cheap­en the val­ue of our own work. If you were banned from a game for cheat­ing or sim­ply went cold turkey, you still have antic­i­pa­tion for reward, which is one of the dri­vers keep­ing you in a com­pul­sion loop.

In my expe­ri­ence hav­ing spent more time and mon­ey than I want to admit addict­ed to games, cheat­ing breaks a com­pul­sion loop so emphat­i­cal­ly that you feel no antic­i­pa­tion, pur­pose or dri­ve to con­tin­ue. When you run iddqd in DOOM or max out all of your stats after run­ning a bot, you start your last joyride. With noth­ing left to antic­i­pate, cheat­ing means ruin­ing the game for your­self. You won’t play it again for a long time after the nov­el­ty of god-teir game­play runs off.

The Compulsion Loop has its own darkness

This is why cheat­ing ruins games for the play­er on an indi­vid­ual lev­el. Now, if pulling the lever in the Skin­ner box for a food pel­let takes more away from you than the pel­let can ever give back, then cheat­ing is an under­rat­ed detox pro­gram that breaks a down­ward spi­ral. Cheat­ing becomes the right thing to do. You WANT to ruin the activ­i­ty for your­self before the activ­i­ty ruins you.

Even if you enjoy some addict­ing activ­i­ties and wish to keep doing them, you should learn how to cheat as a way to con­trol your habits and the influ­ence that com­pul­sion loops have on you. If some­thing changes in your life and the habit starts to take its toll, you need that out.

I can’t cure addic­tion, but when pos­si­ble I can teach addicts to cheat as a means to regain lost con­trol. Next in the series we will dis­cuss GUI automa­tion with Sikuli to cheat our way to fake points on Khan Acad­e­my, a gamei­fied edu­ca­tion plat­form. Our goal will be to break the com­pul­sion loop in Khan Acad­e­my so that only the peo­ple who val­ue edu­ca­tion for it’s own sake will con­sume its con­tent.

Addendum: What about the other players?

Cheat­ing in games that involve mon­ey and com­pe­ti­tion rais­es one sim­ple ques­tion: Is it ever right to cheat if one play­er can gain an advan­tage over oth­ers that play legit­i­mate­ly?

It’s nev­er right. That’s an easy answer because this ques­tion assumes a cheater is com­pet­ing. There’s a dif­fer­ence between using a mem­o­ry edi­tor to win addic­tive sin­gle-play­er games and using a bot to win a Rock­et League sea­son.

Cheat­ing in com­pet­i­tive envi­ron­ments must stop imme­di­ate­ly because the reward for the obvi­ous cheat­ing harms oth­ers. Oth­er­wise, cheat­ing sab­o­tages addic­tive activ­i­ties, which is a good thing.

Morals become grey for me when cheat­ing hurts only the activity’s own­ing busi­ness. Mobile freemi­um games or MMOs might cre­ate a per­fect storm where busi­ness­es wins at the expense of addicts.

Design­ers should allow play­ers to cheat so long as the cheats don’t affect com­pe­ti­tion. Revert­ing a char­ac­ter to a point before they start­ed cheat­ing is a good approach. This gives seri­ous play­ers an incen­tive to grind and train legit­i­mate­ly, and gives addicts an out.

Games like Runescape won’t allow this because real-world cur­ren­cy is tied an in-game mar­ket. Any cheat­ing would vapor­ize their busi­ness and reduce the game to blind rage and lia­bil­i­ty issues. If Jagex could cap­i­tal­ize only on the com­pet­i­tive nature of the game, maybe they wouldn’t be in this mess.

Despite the harm it would do to Bliz­zard, Jagex, etc., cheat­ing their games frees addicts. Addicts can­not make lucid or informed con­sent. Busi­ness­es should not take mon­ey from addicts because those trans­ac­tions aren’t legit­i­mate.

Giv­ing addicts an out so that they can invest more in them­selves is the moti­va­tion behind this series. That, and encour­ag­ing for intrin­sic val­ue in prod­ucts once again.

Creative writing with git-flow and the Snowflake Method

git-flow’s val­ue depends on the nature of a project. Take cre­ative writ­ing: Randy Ingermanson’s Snowflake Method makes you start from a crude—yet complete—one-sentence sto­ry and iter­ate until you are left with a good sto­ry. Require­ments imposed by The Snowflake Method are anal­o­gous to git-flow’s role for the master branch. Giv­en a LaTeX project man­aged with a com­bi­na­tion of git-flow and the Snowflake Method (“Snowflow”), we get some inter­est­ing prop­er­ties.

Assume this file sys­tem:

build.sh          # Compile PDF(s) in dist using story/
dist/             # .pdf files
concepts/         # whatever
story/            # .tex files
    aggregate.tex # \document{book} 

At min­i­mum build.sh runs some­thing like pdflatex -halt-on-error -output-directory ./dist ./story/aggregate.tex to make a PDF of your sto­ry. The concepts/ direc­to­ry con­tains assets describ­ing char­ac­ters, set­tings, con­flicts and plot deci­sions. One rule for this project is that the concepts/ direc­to­ry be checked in, but nev­er be processed by code. This allows free-form cre­ativ­i­ty in asset pro­duc­tion that a pre­cise process would oth­er­wise cur­tail.

A woman writing on a Mac, hopefully with help from the Snowflake Method

Snowflow branch­es behave anal­o­gous­ly to their git-flow coun­ter­parts, with some added expec­ta­tions.

  • The master branch holds a project that com­piles to PDF and tells a com­plete sto­ry.
  • An elaborate (develop) branch adds detail to the sto­ry.
  • concept (feature) branch intro­duces at least one unique, self-con­tained con­cept in concept/ lat­er used to elaborate.
  • review (release) branch holds a com­plete sto­ry for review. If the sto­ry is of suf­fi­cient edi­to­r­i­al qual­i­ty it may merge to master.
  • reconcile (hotfix) branch fix­es press­ing log­i­cal errors such as plot­holes.
  • seed (support) branch address­es cir­cum­stan­tial con­cerns such as dif­fer­ences in sto­ry arcs or fic­tion­al uni­vers­es.

To enable the Snowflake Method, master should a com­plete sto­ry, but that sto­ry does not have to be pub­lish­able. On that note an ear­ly iter­a­tion for Therac-25’s firmware shouldn’t have seen the light of day. It may seem insen­si­tive to com­pare death by radi­a­tion over­dose to bad writ­ing, but only if you’ve nev­er read any­thing by Dan Sacharow.

A Snowflow project will face a “soft end” on a com­mit to master rep­re­sent­ing a pub­lish­able sto­ry. Unless you come up with dif­fer­ent uni­vers­es, sto­ry arcs or deriv­a­tive prod­ucts there may be no need to mea­sure pro­gres­sion or com­pat­i­bil­i­ty with ver­sion num­bers or tags.

In exper­i­ment­ing with this work­flow my favorite dis­cov­ery is that concepts/ even­tu­al­ly takes the shape of “behind the scenes” con­tent for read­ers, which may be sep­a­rate­ly pack­aged and sold. So long as com­mits are dis­ci­plined, the com­mit his­to­ry helps you build sev­er­al prod­ucts at once, where the main sto­ry you intend to pub­lish implic­it­ly pro­motes con­tent you could pro­duce using infor­ma­tion in concepts/.

The concepts/ direc­to­ry also serves as a sand­box for inspired writ­ing ses­sions. Writ­ing is pret­ty moody. Some­times I feel dis­ci­plined and can see how to pack­age my com­mits, and oth­er times I just want to write with no dis­trac­tions. So if you want to ham­mer out a few thou­sand words in a text file in concepts/, go nuts. You can always wor­ry about struc­tur­ing the con­tent with Snowflow when you are ready.

Elaboration process

I must elab­o­rate on the elaborate branch. elaborate may either expand on the sto­ry or per­form tech­ni­cal tasks using LaTeX. In the for­mer sce­nario, I use foot­notes con­tain­ing at least one ques­tion or instruc­tion to iden­ti­fy oppor­tu­ni­ties to build on the sto­ry.

You don’t have to use foot­notes, but keep in mind that some­one who reviews your work should not have to be a devel­op­er. I like hav­ing some­thing vis­i­ble in the prod­uct for an author and edi­tor to dis­cuss.

For exam­ple:

Bob jumped over Alice. \footnote{But WHY did Bob jump over Alice?}

To make the elab­o­ra­tion process sen­si­ble, you should write con­tent that address­es a foot­note either in the vicin­i­ty of where the foot­note appeared, or in a loca­tion that bet­ter estab­lish­es con­text. When you resolve the mat­ter raised by a foot­note, remove that foot­note.

When you com­mit to an elaborate branch you may add at least zero foot­notes, but you must remove at least one of the foot­notes found when you start­ed. By the time you start a review branch there should not exist any foot­notes.

Review process

  1. Elab­o­rate on all rel­e­vant foot­notes.
  2. git flow release start ...
  3. Com­pile a PDF to release to trust­ed read­ers for feed­back.
  4. From the feed­back, insert a list of foot­notes (or oth­er anno­ta­tions) where applic­a­ble accord­ing to your own best judge­ment.
  5. Address all foot­notes.
  6. Repeat steps 3–6 until there exist no foot­notes.
  7. git flow release finish

Guidelines

Writ­ing can’t be con­strained by too many rules, but I did note these guide­lines and obser­va­tions down as I worked.

  • Do not adjust the story/ to the concepts/, adjust the concepts/ to the story/.
  • Do not mod­i­fy story/ when on a concept branch.
  • Your com­fort and focus on writ­ing comes before the process. Don’t be afraid of relax­ing with pen and paper to decide what to do. Lay down on the floor and sketch on a bunch of Post-Its next to a cup of tea before typ­ing any­thing.

Licensing Snowflake Method content

If you decide to write using this process, stay mind­ful of where you pub­lish your work­ing code. If your prod­uct is a book, license it like a book. But more than any­thing, con­sult some­one qual­i­fied to talk about licens­ing. Of course some books like You Don’t Know JS are open source, but if you are con­cerned about dis­tri­b­u­tion, do your research and choose a license.

Generate tailored résumés with Spin

Recruiters might ask me to change my résumé, and by “might” I mean “always.” Some­times it’s reorder­ing sec­tions. Oth­er times it’s giv­ing them a Word doc­u­ment instead of a PDF. This hap­pened enough times to jus­ti­fy at least some lev­el of automa­tion, so I wrote Spin, a CLI that gen­er­ates résumés tai­lored to dif­fer­ent audi­ences. Spin con­verts a super­set of Mark­down Extra to HTML5. So if I made fur­ni­ture on the side, I can gen­er­ate two résumés with $ spin ./woodworker ./programmer.

The woodworker and programmer pro­files are writ­ten in Mark­down Extra mixed with @ direc­tives. The direc­tives define sec­tions and expe­ri­ences with infor­ma­tion like start and end dates. Check out the exam­ples in Spin’s source.

Mak­ing résumés com­piled prod­ucts yield some ben­e­fits:

  • Code reuse. Two pro­files may share let­ter­heads, con­tact info and sec­tions. Corol­lary to this, chang­ing a résumé to entice Agile Cube Farm LLC doesn’t have to change the résumé I lat­er send to Hour­Long Standup Co.
  • Con­sol­i­da­tion. I can write every­thing I ever did in my life in one place. I may now jour­nal my qual­i­fi­ca­tions freely in world’s most obses­sive­ly struc­tured diary.
  • It’s free. Spin is not friend­ly to peo­ple who aren’t used to cod­ing. But all of the résumé man­age­ment tools I’ve seen so far cost mon­ey, are down, don’t sup­port pro­files, or require more labor than I want to put in. I got every­thing I want with Spin for free after the ini­tial time invest­ment.

Implementation notes

I picked PHP to churn this out quick­ly and gen­er­ate doc­u­ments in HTML5 to allow an inter­me­di­ary step for styling and script­ing for those who care enough. I didn’t want to use PHP­Word because my use of it would be too opin­ion­at­ed and would force con­sumers to fuss with PHP exten­sions. Pan­doc works just fine thank youverymuch.

One can use Spin to main­tain a few pro­files and con­vert every out­put doc­u­ment to DOCX and PDF using Pan­doc. My qual­i­fi­ca­tions serve as a real-world exam­ple of this. The base HTML doc­u­ments them­selves may also have extra fea­tures that make them more pleas­ant for view­ing on a brows­er.

If Spin helps you, please con­sid­er buy­ing me a beer!

Iterating Git commit history using Qi

In the Code­less Code, a cre­ative blog on soft­ware engi­neer­ing, there’s this scribe named Qi and he is a total badass. Qi is source con­trol per­son­i­fied; a prag­mat­ic sto­ry­teller that rea­sons about sys­tems as the sum total of all that came before.

I wrote the qi CLI tool (GitHub, PyPi) as a hat-tip to the author—and with his gra­cious per­mis­sion for use of the name. The idea is to bring devel­op­ers lever­age in dis­cus­sions with man­age­ment through a notice­ably less vio­lent vari­ant of Case 89qi is only a Python wrap­per to git that prints revi­sion hash­es at the end of fixed time inter­vals. This enables run­ning an analy­sis at, say, the end of every week over the past two quar­ters to check if a team deci­sion in Jan­u­ary is hav­ing an intend­ed effect in June.

This helps devel­op­ers and man­agers accept the same facts under pres­sure. I once worked in a high­ly active JS shop with over 30 devel­op­ers and over one hun­dred com­mits every day. We had Bam­boo run­ning builds and auto­mat­ed tests every so often to catch prob­lem­at­ic changes, but it was hard to know if our changes this sprint fol­lowed a spe­cif­ic “direc­tion” for sys­tem qual­i­ty, what­ev­er that meant.  After I wrote and ran qi, I dis­cov­ered that the team was, on aver­age, cre­at­ing sev­en pro­duc­tion code files for every one test suite. To make mat­ters worse, rough­ly 20% of our com­bined JS and CSS went com­plete­ly unused.

We all knew that these things hap­pened, but once I had the num­bers to show that mat­ters were get­ting worse due to man­age­ment pres­sure, we re-pri­or­i­tized to allow more time to increase code cov­er­age for crit­i­cal tests. Qi knew that com­mit his­to­ry could not just be read, but lever­aged.

qi is use­ful when one of the fol­low­ing con­di­tions apply to you:

  • You want to gath­er facts and cor­re­la­tions to study the code’s his­to­ry, review recent changes, or to prove a point.
  • You have no way to mon­i­tor the state of a code­base in terms of met­rics or cor­re­la­tions that are impor­tant to you or man­age­ment.

qi is not use­ful under these con­di­tions:

  • Your project(s) are small or neglect­ed because there is rarely a big pic­ture to derive, let alone a desire to find one.
  • Your team has a strong grasp on the sys­tem being main­tained, or already per­forms suf­fi­cient analy­sis.

What’s happening?

Con­sid­er the com­mand qi --every month --past "3 years".

The com­mand does what it says. It will print revi­sion hash­es that mark the end of every month for the past three years, with some caveats:

  • qi starts from HEAD. The query against his­to­ry will search 3 years pri­or to the com­mit date from HEAD, not from the time you ran the com­mand.
  • If HEAD is younger than 3 years, then the out­put will end at the com­mit mark­ing the end of the first month of devel­op­ment.
  • If HEAD is not even a month old, there will be no out­put.

Print­ing hash­es is one thing, but we want to do some­thing with this infor­ma­tion. Since qi is basi­cal­ly a time iter­a­tor, we can tell it to run a com­mand at each com­mit. Note that qi’s short options use the same let­ter that starts their long options.

qi -e month -p "3 years" -c "./analyze"

When you spec­i­fy a com­mand for qi to run, qi will check out each sig­nif­i­cant com­mit (detach­ing HEAD) and run the com­mand against the code­base at that time.
If your analy­sis fin­ish­es or crash­es, qi will attempt to put HEAD back where it start­ed. But if qi crash­es, HEAD will be left detached and you will need to run git checkout to put it back your­self.

qi -c lets you mod­el infor­ma­tion­al met­rics over time, such as the pro­por­tion of dead code in the project and the cur­rent code cov­er­age by tests. You decide what met­rics mat­ter, because qi is not a report­ing tool, it is one kind of iter­a­tor a report­ing tool might use.

Let’s look at a cou­ple of fun exam­ples.

For triv­ia, if you run qi -e year -p "12 years" -c "du -sh ." on Torvald’s Lin­ux source, you will see the disk usage growth of the project over it’s time on GitHub.

[sage linux]$ qi -e year -p "12 years" -c "du -sh ."
2.8G . # 2017 (to date)
2.7G . # 2016
2.6G . # 2015
2.6G . # 2014
2.5G . # 2013
2.5G . # 2012
2.4G . # 2011
2.4G . # 2010
2.4G . # 2009
2.3G . # 2008
2.2G . # 2007
2.2G . # 2006
2.2G . # 2005

Anoth­er exam­ple is if you want React’s com­plex­i­ty report at the end of every year of its devel­op­ment. In this case you could use this set­up:

$ git clone https://github.com/facebook/react.git && cd react
$ qi --every year --past "3 years" --command "./analyze.sh"

analyze.sh

#!/bin/bash
TIME=$(git show -s --format=%ct HEAD)

# Using https://www.npmjs.com/package/complexity-report
cr -e -f json ./src > $TIME.json

 

Hiding annoying diffs on BitBucket pull request pages

Some shops man­date check­ing in long, auto­gen­er­at­ed files into source con­trol that clut­ter up Bit­Buck­et pull request pages with large diffs, leav­ing you to scroll epic dis­tances or depend on the top nav­i­ga­tion to vis­it files key to a code review.

Despite at least three dif­fer­ent tick­ets dat­ing back to 2014 ask­ing Atlass­ian to sup­port exclud­ing spe­cif­ic file diffs from pull request pages, a John Gar­cia explained why they would not imple­ment this fea­ture:

This response is rea­son­able for the ref­er­enced case, yet unhelp­ful to users as a whole. Even if the peo­ple ref­er­enced in the con­ver­sa­tion were using source con­trol inap­pro­pri­ate­ly by track­ing files that they should not track, oth­ers review­ing code might not be in a posi­tion to change that. Then there are big (as in, often has diffs that take up at least one screen) auto­gen­er­at­ed files that fre­quent­ly change, aren’t nor­mal­ly worth read­ing and should be in source con­trol, such as yarn.lock. Cit­ing a Stack­Over­flow answer address­ing one case with 11 votes at the time of this writ­ing does not make the fea­ture request unjus­ti­fied.

I had the same prob­lem as the peo­ple sup­port­ing the tick­et (with many yarn.lock files, no less) so I wrote a user­script cre­ative­ly named “Hide Annoy­ing Bit­Buck­et Diffs (HABD).” It’s set to exe­cute when you view a pull-request page pow­ered by Bit­Buck­et. The script does not sup­port bina­ry files. Instal­la­tion instruc­tions are at the bot­tom of this arti­cle.

In order for this script to func­tion, you will need to con­fig­ure a cou­ple of things. Thank­ful­ly it’s quick!

You can con­trol what diffs to remove by ref­er­enc­ing a plain text resource list­ing one reg­u­lar expres­sion per line (Point­ing to a raw Paste­bin helps here). A diff is removed from the pull request page if the file­name cor­re­spond­ing to that diff match­es ANY of the expres­sions in the resource. 

For exam­ple, this resource hides file diff blocks for package.json and yarn.lock, both of which are com­mon in Node projects and are checked into source con­trol.

package.json$
yarn.lock$

If we’re clear up to this point and you want to use the script, let’s get you set up. If you encounter any prob­lems, please report an issue.

Script installation for TamperMonkey

  1. While using Chrome, add Tam­per­Mon­key.
  2. Go to HABD’s page on OpenUserJs.org and click “Install.” You will see the source code and a con­fir­ma­tion. Con­firm the instal­la­tion.
  3. Click the Tam­per­mon­key icon in Chrome and click “Dash­board.” You will see a list of installed scripts.
  4. At the far right of the row where you see HABD, you will see a lit­tle notepad icon. Click that to edit the script.
  5. Change the line start­ing with “@resource” to point to a raw Paste­Bin or some oth­er clear text resource with one reg­u­lar expres­sion per line. The link already in the script points to an exam­ple. Remem­ber that the reg­u­lar expres­sions match against file­names. If ANY of the expres­sions match, the diff is removed from the page.
  6. (Option­al) Change the line start­ing with “@match” so that it match­es only the pages in which you review Bit­Buck­et pull requests.
  7. (Option­al) If you want to hide diffs on both pull request and com­mit pages, use the @include rule to add anoth­er URL match­ing pat­tern.

Script installation for GreaseMonkey

  1. While using Fire­fox (or some oth­er sup­port­ed brows­er), add Grease­Mon­key.
  2. Go to HABD’s page on OpenUserJs.org and click “Install.” You will see the source code and a con­fir­ma­tion. Con­firm the instal­la­tion.
  3. Click the arrow next to the Grease­Mon­key icon in Fire­fox and click “Man­age User Scripts.” You will see a list of installed scripts.
  4. On the row for “Hide Annoy­ing Bit­buck­et Diffs”, click “Pref­er­ences,” then “Edit This User Script.”
  5. Change the line start­ing with “@resource” to point to a raw Paste­Bin or some oth­er clear text resource with one reg­u­lar expres­sion per line. The link already in the script points to an exam­ple. Remem­ber that the reg­u­lar expres­sions match against file­names. If ANY of the expres­sions match, the diff is removed from the page.
  6. (Option­al) Change the line start­ing with “@match” so that it match­es only the pages in which you review Bit­Buck­et pull requests.
  7. (Option­al) If you want to hide diffs on both pull request and com­mit pages, use the @include rule to add anoth­er URL match­ing pat­tern.
Do NOT follow this link or you will be banned from the site!