This project, Randy’s Diner, was made in an attempt to make my own kind of “movie magic”, like what you’d see in a Jim Henson movie, but in an entirely simulated medium. I created my own puppets in 3D, wrote a screenplay with heartwarming messages and a thrilling climax, cast charismatic and talented voice actors, and hired a composer to create appropriate western music.

While the cinematic production followed a process that would be expected from any film shoot or animation project (screenwriting, storyboarding, pre-visualization), the actual virtual production was far from conventional. For this project I devised a workflow that I believe is a very new and experimental process for DIY filmmaking. Stinky, the coyote puppet, was animated using a combination of VR driven motion capture, facial motion capture with a remapped blueprint of Apple AR Kit expression recognition software, as well as real time particle and physics simulations for Stinky’s fur (over 500,000 individually simulated strands of hair), all of which was rendered in real time using Ray Traced Global Illumination inside of Unreal Engine 4.26.

This project wouldn’t have been possible without the resources provided by Epic Games. I relied heavily on scan data from the Megascan’s library, particularly making use of its Utah Canyon collection, as well as the LiveLink capture app. As well as Chilton Webb’s Glycon3D motion capture app for Oculus Quest.

Everything else was designed on paper, and then in Blender, then off to Substance Painter, and straight into Unreal Engine.

There is much more to this story, and a lot of work left to do over the next couple months. Enjoy act 1.

A little coyote on the run stumbles upon something strange at the end of the desert. Rendered in real time with Unreal Engine 4.26.