Having learned to play Pokémon, betta Grayson Hopper is precocious by fish standards. But the real brains behind Fish Plays Pokémon are two tech-savvy university students.
Catherine Moresco and Patrick Facheris built the program that allows Grayson to navigate a game of Pokémon depending on which tank quadrant he swims to. The program is written in Python using OpenCV (a Python library that gives computers the ability to detect vision), and the only hardware component is a webcam.
“Python was my first language,” said Moresco, who taught herself to use it in high school. “It’s a very friendly language to use. I still haven’t taken an official Python class.”
Moresco, a University of Chicago physics major, and Facheris, a Columbia University computer science major, met in this summer’s HackNY fellows program, and quickly became friends.
“HackNY has been a really great program not just for networking in the tech community, but also for building really strong personal relationships with like-minded hacker folks,” said Moresco.
Grayson is an even more recent acquaintance. The two picked him up at the pet store few weeks ago. He’s named for Moresco’s hero, iconic computer scientist and U.S. Navy rear admiral Grace Hopper.
See also: My Fish Just Sent Me A Text Message
Each year, the HackNY program caps off with a 24-hour hackathon for the fellows. Two days beforehand, the inspiration struck to use Grayson in their hackathon project.
“Cat texted me at 2 a.m. saying, ‘I know we talked about a fish game, but what about a fish plays pokémon game like Twitch Plays Pokémon,’” Facheris said, referencing this years’ social experiment in which thousands of people attempting to all play the same game at once.
All in all, the project took days and two sleepless nights to complete. Moresco wrote the Python program and integrated OpenCV, in about 200 lines of code. Meanwhile, Facheris put it online and set up the emulator. The biggest obstacle? Setting up the emulator using a Linux machine, which Facheris hadn’t been able to find documentation of anyone doing before.
“I chose Digital Ocean—and Optimal is very similar—because you could spin up a VM [Virtual Machine] of a certain size and power and size it up really easily,” Facheris said.
Fish Plays Pokémon went from 0 to millions of views in just a few days, but fortunately scalability isn’t an issue, since the Twitch website handles that, Facheris said.
“All we have to do is make sure we have the processing power to do the stream.”
As Grayson’s fame grows, so do the two’s aspirations for building on the project. The first improvement was a randomizer quadrant. When Grayson swims into this quadrant, the buttons all switch up. Since Grayson spends a lot of his time asleep on a leaf, this adds a much-needed element of chance to his movements.
For future projects, they’ve considered giving Grayson the ability to chat in the Twitch livestream, or allow viewers to vote on when Grayson’s buttons randomize.
“That’s all the control I’d want to give to the audience though,” Moresco said. “It’s not Twitch Plays Pokémon, it’s Fish Plays Pokémon.”
Moresco and Facheris’ project lives in a GitHub repository where anybody who wants to can submit pull requests, or clone their own version. “Favorited” on GitHub just five times, it’s clear that far fewer people than those watching the stream are checking out the back end.
“We link the repository to anyone who wants to see it, but we haven’t been asked much,” said Facheris. “We also still need to add the list of dependencies to the Readme file.”
Feel free to fork the repository, but keep in mind that you’ll have to figure out some of the details—like dependencies—on your own. After all, the program’s still in beta. (Get it?)
Screenshot via Fish Plays Pokémon stream. All other photos courtesy of Catherine Moresco and Patrick Facheris.