Introducing Magpie, the newest Scrabble engine
Before our game at the 2022 Word Cup, Joshua Castellano told me about the work he'd been doing on computer Scrabble with César del Solar and Jesse Day. Together, they'd generated new "leave values" for Macondo, César's engine that powers the bots and analysis on Woogles.io. I was excited to hear that their values performed better than the ones I'd created for Quackle back in 2006. These values are strategic adjustments like +8 for an S or +25 for a blank, but for every combination of tiles a player could leave on their rack.
It had been years since I'd worked on computer Scrabble myself, and I was thrilled to find out that other Scrabble players were working on things like this. When I got home, I tested the new leave values against Quackle's over millions of simulated games. The results confirmed a small but statistically significant advantage for the new values.
A few months later, in 2023, Josh started the Magpie project, a rewrite of Macondo, but in the C programming language rather than in Go. I've always been excited to find new Scrabble software, but Magpie was an especially good fit for me because Josh shared my mission: to write the strongest computer player possible and to use it to learn new things about how to play the game well.
I was happy to contribute to his efforts, and both in the early days and still today, my main contribution has been helping to make it run simulations faster. Faster simulations let us search deeper and estimate the strength of candidate plays more accurately. Just how fast is Magpie? I recently compared equivalent simulations of 12 positions in Magpie and the current versions of Macondo and Quackle. Magpie is about 4 times faster than Macondo and 30 times faster than Quackle!
Josh and I have been the primary authors of the Magpie code, but we've had very significant contributions from others in the community. Many of our data formats are inherited from Andy Kurnia's Wolges Scrabble engine. Andy found many optimizations and bug fixes for us as the project was taking off. Macondo wasn't just a blueprint for Magpie; it has continued to evolve in its own right. César's later work on inference (reasoning about what tiles an opponent's last move might have kept), endgame, and preendgame (solving all possible endgames with tiles still remaining in the bag) all made their way into Magpie, through both his direct contributions and our use of his code as a model. More recently, Morris Greenberg has helped us rethink how Magpie runs simulations, encouraging us to consider a more diverse set of candidate plays. Instead of simply taking the top moves ranked by score and leave value, we're exploring ways to include plays that differ in length, placement, and risk.
We've been working hard on this for years behind the scenes without sharing much of the details, but we're proud now to announce how you can help our project and try Magpie for yourself. First, we have birdtest.org, patterned after Fishtest, the site used for testing versions of the chess engine Stockfish. Through Birdtest, you can volunteer your computer's processing time to help us test improvements to Magpie. When we make bug fixes or pure speedups, we're usually able to test those well enough by ourselves, but lately we've been experimenting with changes in how Magpie plays that have tradeoffs we evaluate by having the program play against another version of itself. Often, the differences are subtle enough that we need thousands of games to tell which version is stronger. Running thousands of games with the thinking time we'd allow in a real game could take weeks or months on a single computer, so we need your help!
Second, at magpie.birdtest.org, we have an early web interface where you can play against Magpie and analyze your games. I'd love for you to try it and let me know what features you'd like to see added.