What is Magic Sudoku?

With Magic Sudoku, the makers are bringing this classic game to Augmented Reality.

 Sudoku is a puzzle in which players insert the numbers one to nine into a grid consisting of nine squares subdivided into a further nine smaller squares in such a way that every number appears once in each horizontal line, vertical line, and square.

Why did you make Magic Sudoku?

Several reasons.

 First, I personally wanted to learn some new technologies. I’ve never made a native app before so amongst the things I learned from this project were: Swift, ARKit, SceneKit, and CoreML. I also wanted to learn about machine learning and this seemed like a great first project to dip my toe in the water.

 On the business side I think augmented reality and machine learning are going to be increasingly important for us at Hatchlings. We primarily make games so I wanted to take this opportunity to branch out into something else.

 And for the app specifically we wanted to make something that was truly improved by using AR. We didn’t just want to plop a 3d model on top of the real world. We wanted to make something that would transform reality into something new and better.

What difficulties arose when building it?

Starting from scratch was pretty difficult; at the onset, I didn’t even know if it would be possible to build what I envisioned. There’s a steep learning curve with each of the technologies we used individually… but that was greatly amplified trying to learn them all at once!

Who was involved?

I did all of the programming (including several backend tools to help us collect and crowdsource data for our machine learning algorithm). Towards the end, I brought in one of my designers at Hatchlings to create the branding, instructional UI, and visual design for the app.

I also owe a great deal of gratitude to the Hatchlings community for helping us label over a million images that we used to train the neural network to classify Sudoku squares.

What tools did you use?

Swift, ARKit, CoreML, and Keras.

Did you use any tutorials you can recommend?

I wrote a technical write-up about how we built the app in which I detailed some of the behind the scenes (including some links to some libraries and tutorials that were very helpful).

What’s coming next? What are your plans?

For Magic Sudoku, the near-term will bring bug fixes and improvements to the current feature-set (along with some eye candy like particle effects and improved visual styling). In the mid-term, we have been adding support to our neural network to recognize handwriting. Once that’s perfected it will allow us to add major features like checking completed puzzles and providing spoiler-free hints while you’re working on completing paper Sudoku puzzles.

From a broader perspective, we have a long list of unique augmented reality apps we’d like to make someday. I expect Magic Sudoku won’t be our last foray into AR!