Leveraging Mixed Reality for Bike Repair

Since 2020, our team focusing on eXtended Reality (XR) at Deloitte Digital Belgium has been exploring the edges of immersive technology: experimenting, prototyping, and learning our way through a rapidly evolving landscape. With approximately 20% of our time dedicated to Research & Development, we’ve built a rich body of work that spans both client-driven projects and internal Proofs of Concept (PoC), reflecting not only technological progress but also the team’s growing intuition about what meaningful, human-centered XR can be.

This article is part of a chronological series that retraces the hidden side of that R&D journey: each article highlights one POC: the context in which it was created, the technology choices behind it, the challenges we faced, and the insights that shaped our next steps. Together, these stories reveal how experimentation fuels capability building.

Next projects to be revealed soon…

Picture a bike repair shop. A technician gets a bike, looks it over, figures out what needs fixing. Then walks to a computer to check inventory. Finds a missing part, walks back to the bike, spots another issue they missed, walks back to the computer again. These trips back and forth, sometimes just a few steps, but dozens of times a day, add up. With the rise of electric bikes, the complexity has dramatically increased, leading to a challenging situations for technicians.

In the early days of our spatial development work, we partnered with a bike reseller dealing with exactly this problem. We thought Mixed Reality could help.

What if a technician could wear a headset, scan the bike’s serial number, and see everything they needed right there, parts lists, inventory status, repair instructions—without leaving the workstation?

That’s what we set out to build.

We created a Mixed Reality diagnostic tool that used 3D object recognition and holographic overlays to walk technicians through the repair process. Here’s how it worked:

1. Scan the bike. Technician wears a HoloLens 2 headset and scans the barcode.
2. Get instant data. System retrieves the bike model and all associated parts information.
3. Check inventory in real time. Select which parts need replacement, see stock status, without leaving the workstation.
4. Order missing parts. If something’s out of stock, order it directly from the interface.
5. Follow step-by-step tutorials. Once parts arrive, 3D tutorials with visual highlights show exactly how to disassemble and reassemble components.
6. See the hidden. The system reveals hidden parts, like brake cables running through the frame, by making them transparent in the digital overlay.

For a team just starting to explore spatial development, this was foundational work. It brought together spatial UI design, 3D object recognition, real-time data integration, and hands-free interaction.

We built the application in Unity using Microsoft’s HoloLens 2 SDK, which came with pre-built building blocks we could assemble quickly. This let us focus on the application logic rather than reinventing spatial computing. Development took about six weeks, rather fast for a proof of concept. But the real value was in what we discovered along the way.

The first big insight: not all UI behaves the same

We learned something fundamental about spatial UI design: not all interface elements should behave the same way.

Some panels needed to follow the user as they moved around the bike. Others needed to stay anchored in 3D space so the technician could walk around them. This distinction, which elements follow, which stay put, became foundational for everything we built afterward.

We tested a wrist-based menu: a watch-like interface that would appear when you looked at your wrist. It failed immediately. Your hand gets in the way. It’s not comfortable. This taught us something about spatial design: intuitive in theory doesn’t always work in practice.

The second discovery: pragmatism over perfection

We wanted the system to automatically recognize bike models through computer vision. But image recognition was too unreliable. So we used visual markers, stickers on the bike frame, to enable reliable object detection.

It wasn’t the elegant solution we’d hoped for. But it worked. And it taught us something important: sometimes the best solution is the pragmatic one.

The client shaped the product

The bike reseller was deeply involved in development. For instance, it’s based on his feedback after some user testing sessions that we added floating window panels that could be repositioned. He also suggested video tutorials would help, so we integrated those. His feedback shaped what we built.

If we were to build this application today, it would look very different.

• Lighter hardware: AR glasses today are dramatically lighter than the HoloLens 2. The HoloLens 2 was too heavy to wear all day which was a constraint to a further adoption of the solution by the bike reseller. This would no longer be a blocker.
• True hands-free operation: speech-to-text technology has improved dramatically. We could enable voice commands without pinch gestures or button presses, critical for someone whose hands are occupied with repair work.
• Better object recognition: advanced AI and computer vision would enable reliable object recognition without visual markers. The stickers on the bike frame would no longer be necessary.
• Mature spatial UI frameworks: the tools available now would streamline development and improve the user experience.

But here’s what matters most: the most important evolution isn’t technological, it’s conceptual.

We now understand, from lived experience, what works in spatial environments and what doesn’t. We know how to design for hands-free workflows. We know how to balance technical ambition with hardware constraints. And we know that sometimes the most valuable outcome of a proof of concept isn’t a commercial product, but the expertise and intuition it builds within a team.