GUI automation means writing a program that pretends to be a human using a mouse and keyboard. Assuming your target doesn’t look for bots, you have enough to exploit the system.
In serious games, bots must become indistinguishable from legitimate players. Naïve bots give themselves away immediately; they click the exact same pixel on a frame to perform the same action, they chain actions together with remarkably persistent timing, and they don’t respond when addressed in conversation. Humans fidget in anticipation of their next task, wiggle the game camera, click to inspect entities, chat with friends, and self-correct. The best cheating bot needs to exhibit these flaws to the degree and style of the human they replace, which involves training. This of course makes the bots woefully slow to configure and run, but if you are aiming to get past bot-detection, these are necessary evils.
We will start with a simple bot targeting Khan Academy, the latter being an education service that rewards users with energy points and badges akin to gold stars. Here I write a bot that could earn all energy points possible for videos; This breaks the compulsion loop for users retained by KA rewards.
Of course I am not going to violate KA’s ToS. I want to pay special attention to section 8.14, meaning I will not run my bot to its extreme conclusion and risk harming “any user’s enjoyment of [the Website],” whatever that means to legal demons. I think calling the badges/points “valuable” is debatable, but I won’t risk cheapening the value other users find in earning badges and energy points. We will only emulate a human user’s activity that would put no more load on their servers than normal, and I will perform a small demo for educational purposes. If you try using this bot or something like it for yourself, then I am not responsible for your decisions.
The job entails opening a video in a browser, watching it, and then moving to the next one. KA’s embedded player moves automatically to the next related bit of content, but it might redirect the user to something other than a video. Therefore we need to control the navigation ourselves.
GUI automation requires many assumptions to work, so just reuse the concepts if KA changes and breaks the code here.
If you want to max out your account, you could use curl
and pup
to scrape together a text file with every video link. For CYA reasons it would not be a good idea to distribute the playlist should you build one. I am only showing you what you could do with a playlist if you had it for the sake of this discussion on compulsion loops, and you can test this just by copying some links from your address bar manually in a text file.
Let’s look at a Python controller for Sikuli implementing the following behavior given a playlist
text file:
- Find the next video to play.
- Open the video link, assuming a KA session is active.
- Start Sikuli, blocking until it ends.
- Go to step 1.
- (On termination) Mark the line of the current video being watched.
Sikuli allowed me to program the bot using images, so I didn’t have to fuss with hard-coded coordinates. Since KA videos currently autoplay on load, the Sikuli script only needs to wait until the video ends and then close the browser.
You can study the source code for the above bot on GitHub.
What was the point of this GUI automation talk?
We just covered an example of GUI automation to break a simple compulsion loop in a real-world system. The question I want people to ask is how many users would KA have if everyone used this bot? If the bot killed retention, then sites like KA can’t get by on content alone. I think for marketing’s sake, that’s important to know. I hypothesize is that few would return to KA if their rewards were useless. If so, do we have free will to improve ourselves without a compulsion loop to program us?
The bot shown here raises every red flag on cheat-detection software. Bots cheating competitive games must better emulate human behavior.
But better bots to cheat at games with detection software is not the direction we are going to go, because as I stated in part 1, the moral good from cheating ends in competitive environments. Instead, we will next look at breaking compulsion loops by replacing them with better ones. Those who wish to reassess their priorities and reclaim control of their health and attention would rather substitute, for example, gamified exercise in place of their World of Warcraft account.