Cookie Clicker

The code is available at https://github.com/fab5code/guirecognizer/tree/main/examples/cookieClicker and includes instructions to run the bot. The bot plays the game automatically.

Let’s explain how guirecognizer is used to handle different parts of the gameplay.

Buying buildings

At this point, the bot behaves like an auto-clicker, but it can do much more. To gain more cookies it’s important to buy upgrades and especially the buildings listed on the right.

List of the upgrades to produce more cookies, called *buildings* in this tutorial. More appear when some are unlocked.

Map the buildings on screen

Each building type is highlighted when it is possible to buy another one. Otherwise, it is greyed out. Let’s find where each building ui element is on the screen.

Create a Get Coordinates action: Manage Actions -> Add Action Get Coordinates that points at the top of the first building. Name this action buildingTop.

Select a point at the top of the first building ui element.

Create another Get Coordinates action that selects the height of a building ui element. Name this action buildingHeight.

Select the height of a building ui element.

Of course you could hardcode those values but this is one reason to use guirecognizerapp and a config file. Moreover, if the bot needs to be adapted to another screen, only the borders will need to change, along with some minor tweaks.

We can now compute the screen coordinates of all buildings.

from guirecognizer import Recognizer

recognizer = Recognizer('cookieClickerConfig.json')
nbBuildingsInScreen = 9
buildingCoords = []
topCoord = recognizer.executeCoordinates('buildingTop')
coord = recognizer.executeCoordinates('buildingHeight')
height = coord[3] - coord[1]
for i in range(nbBuildingsInScreen):
  coord = (topCoord[0], int(topCoord[1] + i * height))
  buildingCoords.append(coord)
print(buildingCoords)

[(1826, 362), (1826, 442), (1826, 522), (1826, 602), (1826, 682), (1826, 762), (1826, 842), (1826, 922), (1826, 1002)]

Assess whether a building can be bought

We could look at the pixel color of the screen coordinate of each building and compare it to a value to know whether the building is highlighted meaning it can be bought or not. In practice, the building texture contains color variations. So a more stable solution is to get the average color of a screen portion of the building ui element.

Create a Compare Pixel Color action: Manage Actions -> Add Action Compare Pixel Color and select a rectangle starting at the same point of the previous action buildingTop with 50 pixel width and 10 pixel height. Name it buildingAvailableDiff.

Select the area for the *Compare Pixel Color* action.

Create another action called buildingUnavailableDiff. For instance buy the first building until it’s unavailable, take a screenshot, then select the same area of buildingAvailableDiff. It’s possible to select the same area as another action in the controls of the screenshot tab.

Select the area for the *Compare Pixel Color* action to test when a building cannot be bought.

Let’s find out in the bot for each building if it can be bought.

from guirecognizer import ActionType, Recognizer

recognizer = Recognizer('cookieClickerConfig.json')
nbBuildingsInScreen = 9
buildingCoords = []
topCoord = recognizer.executeCoordinates('buildingTop')
coord = recognizer.executeCoordinates('buildingHeight')
height = coord[3] - coord[1]
for i in range(nbBuildingsInScreen):
  coord = (topCoord[0], int(topCoord[1] + i * height))
  buildingCoords.append(coord)

canBuy = []
for coord in buildingCoords:
  pixels = (coord[0], coord[1], coord[0] + 50, coord[1] + 10)
  color = recognizer.executePixelColor(ActionType.PIXEL_COLOR, coord=pixels)
  availableDiff = recognizer.executeComparePixelColor('buildingAvailableDiff', pixelColor=color)
  unavailableDiff = recognizer.executeComparePixelColor('buildingUnavailableDiff', pixelColor=color)
  canBuy.append(availableDiff < unavailableDiff)
print(canBuy)

Here is the result when this is the state of the game.

[True, True, False, False, False, False, False, False, False]

Then the bot can decide whether to buy any of the available ones but the logic is up to you.

Here is how to buy a building.

recognizer.executeClick(ActionType.CLICK, coord=buildingCoords[1])

Assembling the pieces together

The bot handles three independent tasks: clicking on the cookie, managing updates and finding golden cookies. Since at least clicking on the cookie and searching golden cookies are full time jobs, multiple processes are necessary. As an analogy, instead of one employee multitasking, we assign one employee to each task. This is why we are going to use the Python library multiprocessing.

We are going to spawn one process for each task then wait for them to finish. They are multiple processes but only one mouse. So we are going to use a multiprocessing.Lock to handle the mouse between the tasks.

Here is the full code example.

import multiprocessing as mp
import time
from multiprocessing.synchronize import Lock

def clickCookie(mouseLock: Lock) -> None:
  while True:
    mouseLock.acquire()
    print('click for 1s')
    time.sleep(1) # Simulate clicking continuously on the big cookie.
    mouseLock.release()

def manageUpdates(mouseLock: Lock) -> None:
  while True:
    time.sleep(5) # Simulate assessing if any update should be bought and waiting.
    mouseLock.acquire()
    print('manage updates') # Simulate clicking on the building to buy.
    mouseLock.release()

def findGolden(mouseLock: Lock) -> None:
  while True:
    time.sleep(12) # Simulate searching for golden cookies with action findGolden until one is find.
    mouseLock.acquire()
    print('find golden') # Simulate clicking on the found golden cookie.
    mouseLock.release()

def main() -> None:
  mouseLock = mp.Lock()
  processes = []
  args = (mouseLock,)
  processes.append(mp.Process(target=clickCookie, args=args))
  processes.append(mp.Process(target=manageUpdates, args=args))
  processes.append(mp.Process(target=findGolden, args=args))
  for process in processes:
    process.start()
  for process in processes:
    process.join()

if __name__ == '__main__':
  main()

For the sake of this tutorial each task is simulated with time.sleep.

On lines 30, 31, and 32, a new process is created for each task. Line 36 is essential to wait for each process to finish.

You can stop the bot with Ctrl+C or implement a cleaner way to stop it with multiprocessing.Event for instance.

Here is a part of the console output.

click for 1s
click for 1s
click for 1s
click for 1s
click for 1s
manage updates
click for 1s
click for 1s
click for 1s
click for 1s
click for 1s
manage updates
click for 1s
click for 1s
find golden
click for 1s

Now you can replace the content of the functions clickCookie, manageUpdates and findGolden and create a functional bot.

What’s next?

You can extend the bot by buying upgrades, supporting more than the first buildings, fine tune the golden cookie detection and more.

You can try running a functional bot at https://github.com/fab5code/guirecognizer/tree/main/examples/cookieClicker.

Cookie Clicker

What is Cookie Clicker?

Continuously click on the cookie

Create the bot configuration file

Python script

Buying buildings

Map the buildings on screen

Assess whether a building can be bought

The golden cookie

Simple approach

Improve performance

Assembling the pieces together

What’s next?