Filmmaking is inherently spatial. Directors block scenes in 3D, production designers build physical models, and cinematographers frame shots around objects that don't exist yet. But the tools used to plan all of this are still flat: scripts, storyboards, and 2D pre-visualization software. Working with Microsoft's Mixed Reality at Work team, we researched where HoloLens could bridge these gaps across the pre-production and production pipeline.
The industry context
Technology has reshaped filmmaking, from George Lucas pioneering digital cinema to Steven Spielberg using VR headsets to direct Ready Player One. But almost all of these innovations live in post-production: better VFX pipelines, faster rendering, more realistic CGI. The earlier phases, where creative decisions are actually made, remain largely analog.
We mapped the five-stage filmmaking pipeline: development, pre-production, production, post-production, and distribution, and confirmed that technology interventions cluster heavily in post. The question became: what would it mean to bring spatial computing into pre-production and production?
The challenge
Early expert conversations surfaced two recurring problems: filmmakers struggle to externalize spatial ideas using 2D tools, and collaboration breaks down when stakeholders can't share a common spatial reference. We framed the design challenge around these two axes.
Three research questions guided the investigation:
How does the director communicate envisioned ideas to different stakeholders?
What are the gaps in collaboration across the filmmaking process?
Would HoloLens be an appropriate design response given the context of use?
Research approach
We triangulated across three methods: a literature review of MR applications in film, semi-structured interviews with 16 participants (8 industry experts, 8 practitioners), and a competitive analysis of existing tools for spatial ideation and collaboration.
Over half of our expert interviews were remote video calls, as filmmakers are constantly moving between locations. We spoke with professionals across major motion pictures (Black Panther, Captain America, Thor: Ragnarok, Game of Thrones) and independent filmmakers to ensure diversity of perspective.
We also evaluated existing tools against eight design principles: flexibility, customization, familiarity, externalization, remote use, role support, feedback, and sharing, to map the competitive landscape.
Synthesis
We externalized all interview data onto sticky notes, clustered them into categories, and identified four larger themes through affinity diagramming. We also built process diagrams to map how information flows (and breaks down) across the production pipeline.
Key findings
From the synthesis of 16 interviews and extensive secondary research, we distilled our data into four themes. Each theme surfaced specific insights about where spatial computing could intervene in the filmmaking process.
Understanding space
How do filmmakers translate 3D spatial intent using 2D tools, and where does that translation break down?
Blocking relies on flat artifacts for a 3D problem
Blocking, deciding where actors stand, how they move relative to camera and lighting, is fundamentally spatial. Yet directors communicate it through storyboards and scripts that strip away the spatial dimension entirely.
"It's hard to verbally explain the geometrics, where's everyone positioned. Just explaining the ideas to the DP takes a lot of time and effort."
Physical models remain essential for set design
Production designers still rely on physical 3D models to plan sets, because 2D concept art doesn't communicate scale, cost, or spatial constraints. But physical models are expensive and slow to iterate on.
"The analog style of having a physical model that everyone can look at, around the table is immensely helpful."
Visualizing virtual objects
How do filmmakers see and work with things that don't physically exist on set, such as CG characters, environments, and effects?
Low fidelity enables faster spatial experimentation
Lower-fidelity representations, such as rough CG models and wireframe characters, are actually more effective for early-stage collaboration. They keep conversations focused on position, scale, and movement rather than surface detail.
Directors can't use current pre-visualization tools
Pre-visualization today runs on Maya, Unity, and Unreal, powerful tools with steep learning curves that directors can't operate themselves. The result: a slow feedback loop where directors describe shots to animators who interpret and render them.
"Communicating to previs animators what kind of shots they want to see can be a frustrating and a slow process. They wish for the control to let them create the shots themselves."
Actors can't see what they're acting with
On sets with CG characters and objects, actors and camera crews work blind. They can't see the position, scale, or movement of things that will be added in post. This creates repeated takes and spatial confusion.
"They're like 'look out to the castle' and you just think 'well, how far away is it? Is it right here?' It never looks right."
Vision alignment
How do different departments maintain a shared creative vision when they work asynchronously across production phases?
Vision misalignment compounds across production phases
Each department carries its own artistic vision. Without a shared spatial reference, stylistic decisions made in pre-production diverge from what's captured in production and composited in post. The result is visual incoherence that's expensive to fix.
Business viability
Who gets to use spatial tools today, and what prevents wider adoption across production scales?
Access to spatial tools correlates with budget
Large productions are experimenting with immersive technologies. Independent filmmakers, who could benefit most from faster spatial planning, lack the resources, knowledge, and infrastructure to adopt them.
Full research report ↗Next steps
These findings identified clear opportunity spaces for HoloLens in pre-production and on-set workflows. The next phase focused on translating these insights into design concepts, prototyping interventions for spatial blocking, virtual set dressing, and on-set CG visualization, in close collaboration with the Microsoft MRW team.