In 2014, I became aware of the social experiment that was TwitchPlaysPokemon. As an enthusiast of both games and novelty coding projects, I was fascinated by the idea and was eager to see it re-imagined in other genres.

Many games followed Twitch Plays Pokemon. Most of which were simple adaptations being a different game on a different emulator, using some Twitch chat parsing control mechanism. Many games were turn based RPGs (such as this Final Fantasy VI playthrough, as this format best compensated for the stream lag. Not all games were necessarily turn based though. One community came together to beat Super Mario Odyssey in a Twitch Plays format -- using the start menu and command chaining in order to compensate for input lag in this action platformer.

Eager to take this a step further, I thought about what had not been explored in this format yet. Originally, my thoughts turned to putting "player against player", or more accurately, chat against chat. I considered exploring playthroughs of linked games that could end in a versus battle -- such as running a simultaneous stream of Pokemon Red and Blue, and at the end of an arbitrary time period, putting the two into a link battle with one another -- connecting the streams to compete against one another.

The idea felt unimaginative, and given that TwitchPlaysPokemon already ran the venture of-- well, Twitch playing Pokemon -- it likely not something to draw a crowd worthy of the feat.

Choosing Zelda: Four Swords Adventures

Going back to the drawing board, I tried to find a game that matched both competition and cooperation. Eventually, I came across the idea of The Legend of Zelda: Four Swords Adventure. I ended up getting ahold of a couple of Gameboy Advances and a copy of the game and played through it with a friend. The game felt perfect as a cooperative for many reasons.

The game is a cooperative, story driven progression game.
You are cooperating with one another, but also competing. This advocates for both cooperative and competitive play
There is no real "game over" mechanic for a player dying. While you lose force gems (a currency of sorts), this is forgiving for sake that progression is not hindered by not managing realtime input.
Unlike a typical game/emulator + chat setup, this each "screen" in a sense is its own player. This creates an interesting addition to the dynamics

Original Design Philosophy

Once I had the idea in mind, I starting prototyping out what this would look like. Originally, I thought to have 4 live streams set up. Each stream would consist of the Dolphin emulator + corresponding player GBA screen (Green, Red, Blue, or Purple). There were a few reasons I didn't like this as I looked into it more.

Bandwidth. Pumping out 4 streams was redundant
It would split the community. Players would be separated from one another inherently. And, unless the stream got to the fanatic levels of TwitchPlaysPokemon, likely a small memberbase split four ways would not be sustainable
Hardware requirements. As it turns out, running 5 emulators all networked together is resource intensive.

In the end, I settled instead on the concept of running a single stream targeting all 5 emulators.

Setup

Gamecube Emulator

Surprisingly, the Gamecube emulator was the easiest setup and least configuration. Dolphin is a powerful Gamecube emulator that is perfect for this exercise. Dolphin supports Game Boy Advance controller input through a locally hosted server where GBA emulators can connect over the network.

After installing Dolphin, all that was required was setting the controllers to Game Boy Advances and setting sound to DSP LLE Recompiler (this allowed sound to still play from Dolphin whenever Game Boy Emulators connected).

Game Boy Advance Emulators

Game Boy Advance emulation actually ended up being the trickiest part of the whole project. Because all devices would have to run on the same machine, I would be running 4 instances of Game Boy Advance over one another.

As of today, the only emulator that appears to work with network play is VBA-M. So VBA-M would have to be configured to be run in parallel with itself in order to achieve four player, as Four Swords Adventures required a GBA per player.

Because I would also need to be able to provide input to all the emulators at any given time, background input support would also be required in order to run a simultaneous 4-player game on a single machine.

As it so happens, VBA-M supports this, but the behavior did not work on the latest stable. So this actually required leveraging a VBA-M nightly to get this behavior.

Lastly, in order to set unique control schemes for each, I had to set up each VBA-M emulator in its own directory with a custom config file. This config file had a select mapping that corresponded to each Link. To ensure each link had the expected control scheme, I also had to be precise in my order of operations when opening up the emulators

Chat-to-Input

The last element to figure out was capturing input from chat.

Discord to Twitch

Originally, Discord was favored to be used as it was figured it would reduce complexity of having to target a player per action. Instead, each player would have its own channel. However, this was found to be unintuitive for many and many would rather jump right into Twitch chat, even with the increased argument complexity.

Leveraging a control library

As Node is my preferred language of development, I sought out tools available that would be able to perform input actions on my computer. I had actually pursued a few projects like this in the past and came up with a library of robotjs. The library is no longer maintained today and instead succeeded by nutjs, but much of the keyboard functionality is the same.

I did not see options to perform extended key assignment with either RobotJS or NutJS. For the most part, I only had the regular keys on the keyboard available to me. As such, I came up with a very naive implementation of mapping most of my keyboard to the four game boy advance buttons

Overall, implementation was simple. My implementation scheme was a bit verbose as it was adapted from another TwitchPlays project that never saw the light of day due to complications with mouse input. That may be something to explore again with nutjs's Mouse and OCR abilities.

Streaming

Streaming was set up through OBS. This was fairly straightfoward for the most part, as OBS is built for streaming. The only gotcha in this was wrangling OBS to cooperate on there being four VBA-M instances running. VBA-M has a poor time differentiating one from the other, so you have to be incredibly sensitive when setting it up each time to re-select the proper instance of each.

Hosting

Once this was all said and done, it was time to host this. Unfortunately for me, the only machine available to me powerful enough to run all four emulators and streaming software was my daily driver. All other machines were a bit too dated to handle all the overhead. Virtual servers for this could also be tedious and expensive. Thankfully, I had an amazing co-worker who offered up one of his unused Windows machines for hosting of the project (Thanks Cullen!).

Checking it out

Website for the project can be found over at https://twitchplaysfsa.com
The livestream is hosted at Twitch at https://www.twitch.tv/twitchplaysfsa
The code repository and instructions are hosted on Github at https://github.com/mstan/TwitchPlaysFourSwords