What is the #wordup AR app?
Write anything in augmented reality, then take a photo or video and share with your friends.
What made you want to build this app?
I was instantly thinking about the rise of video, Instagram, Youtube, Snapchat and wondered how people could add some wow to their posts using a simple AR tool.
Adding tracked 3D text inside a video couldn’t be done in less than a minute before, you had to use post-prod softwares like After effects and spend a bit of time to make the title of your movie look like it’s in the scene you are filming as the camera moves.
ARKit enables to do that live.
What were the biggest challenges you faced while building it?
It’s been a very steep learning curve as I wasn’t familiar with SceneKit in the first place. So going from 2D to 3D gave me a few headaches!
Adding large text in the scene is quite expensive, which means that the frames wouldn’t refresh 60 times per seconds as the user was recording and that made videos look horrible.
I wanted both the text and the videos to look stunning so I learnt about tessellation, extrusion depth, chamfer radius and many other things that I had no idea existed before!
Also shadows play a very important role to create the illusion that objects are really there. At first I started to add basic shadows but I wasn’t happy with the result and spent a long time to get it right. When an object’s shadow is in the opposite direction than everything else in the shot, it just looks fake. So the moment a user opens the app, they need to make their screen face the main light source (sun, window, lamp..) until the iPhone vibrates after 3 seconds, that creates a virtual light 3 metres in front of it using the its orientation and position. it’s invisible but casts some light on the 3D text in the scene, who in turn will cast shadows on any flat surfaces detected by ARKit.
Who was involved?
What tools did you use?
100% made with Xcode 9 and swift 4
Did you use any tutorials you can recommend?
I watched the WWDC videos for ARKit and SceneKit, also a blog post from Mark Dawson.
Then just reading through code that I downloaded from Stackoverflow.
What’s coming next? What are your plans?
I’ll keep updating #wordup
from users’ feedback and add new features. We’re still on version 1 so there is lots to improve.
Good looking 3D emojis are definitely on the radar! And probably more gestures so people can interact with the 3D objects.
There are so many things to explore with AR, everything needs to be re-invented with a third dimension in mind.
UX is still clumsy for AR I think, we’ll see new apps coming up with new ways to interact with digital content for sure.