Optimizing On-Set Capabilities and In-Camera Visual Effects
Research Team
Spencer Idenouye
Kevin Santos
James Rowan
Jason Hunter
Emerson Chan
Partners
Rotate Virtual Productions
NRC IRAP
CTO
Impact
- Genlock and synchronization.
- Optitrack motion capture with Unreal Engine.
- Optimized virtual production workflows.
A Technical Assessment and Prototype Integration for LED Video Plate and ICVFX Systems
This collaborative project between Rotate Virtual Productions and Sheridan’s SIRT Centre evaluated and optimized key workflows in a real-time virtual production (VP) environment. The goal was to identify and resolve limitations in LED video plate playback, motion capture integration, and in-camera visual effects (ICVFX). A comprehensive assessment revealed challenges such as ghosting due to mismatched shutter angles, unstable tracking setups, and hardware sync issues.
The research team addressed these by implementing OptiTrack motion capture systems, configuring genlock synchronization with Ambient Lockit and AJA devices, and adjusting LED wall configurations. Two working prototypes were developed, including a use-case testing environment, and two refined production processes were established. Recommendations were made to upgrade hardware, streamline user permissions, and configure network reliability. The project delivers a replicable model for VP calibration and workflow enhancement, positioning Rotate’s studio for future production demands and technological scalability.
Understanding Gaps in Virtual Production Capabilities
As virtual production (VP) becomes increasingly essential in film and media, Rotate Virtual Productions partnered with Sheridan’s SIRT Centre to assess and enhance its VP studio capabilities. The goal was to identify inefficiencies in their existing workflows, especially in relation to LED wall configurations, camera sync, video plate fidelity, and tracking systems. Challenges such as video ghosting, motion blur, and inconsistent hardware synchronization were limiting the reliability of real-time in-camera visual effects (ICVFX). The need for precise system calibration and robust tracking infrastructure was clear.
This project aimed to bridge those gaps, enabling a smoother, more professional-grade virtual production pipeline to meet the evolving standards of the creative industries.
A Calibrated System for Real-Time Virtual Production
The result of the research was a functioning prototype showcasing a fully integrated ICVFX environment. This included a synced, artifact-free driving plate playback, improved LED-camera shutter alignment, and an OptiTrack-powered motion capture setup validated for dynamic, real-world studio use. Two prototypes—one for workflow configuration, another for use-case testing—were created, alongside two significantly improved production processes. Through documentation and technical demonstrations, Rotate’s studio is now positioned for a more stable, scalable, and industry-ready virtual production pipeline.
These outcomes not only addressed current operational hurdles but also laid the groundwork for future expansion in VP technology and practices.