Home » R3CAP

R3CAP

Real-time 3D Reconstruction and Collaborative Authoring Platform

SLAM · Gaussian Splatting · WebXR | Python · Nerfstudio · Babylon.js · MongoDB

---

Overview

R3CAP is an open-source system for real-time 3D reconstruction and collaborative editing that enables users to create and interact with digital twins as they are being captured.

Unlike traditional pipelines that are offline, fragmented, and expert-driven, R3CAP provides a unified, interactive, and accessible workflow for non-expert users.

Key Contributions

• Real-time progressive 3D reconstruction with immediate visual feedback
• Collaborative multi-user editing interface for web and immersive (VR) environments
• Edge-server architecture for scalable processing and low-latency interaction
• User-centric design supporting non-expert users with commodity hardware
• Unified end-to-end system integrating capture, reconstruction, and interaction

---

System Overview

R3CAP consists of three core components:

1. Capture Client (Mobile)

• RGB-D data acquisition using consumer devices
• Streams data continuously to the server
• Provides live feedback during scanning

2. Edge Server

• Performs progressive reconstruction and scene updates
• Maintains a shared 3D representation
• Synchronizes updates across users

3. Editing & Exploration Client

• Web-based interface with optional VR support
• Enables scene editing, annotation, navigation and inspection

This enables parallel workflows:
capture → reconstruction → interaction (simultaneously, not sequentially)

---

Technical Stack

• Front-end: Babylon.js, WebXR, Swift (iOS capture client)
• Back-end: Python, SLAM, Gaussian Splatting, MongoDB

Key Applications

• Medical Rehabilitation: Virtual environments for therapy, monitoring, and assessment.
• Disaster Response: Rapid 3D mapping of affected areas for emergency planning.
• Architectural Design: Collaborative design and visualization of architectural projects.
• Industrial Training: Realistic, interactive training scenarios for complex procedures.

Presentations

• ACM CHI 2025 Interactivity: Showcased in Yokohama, Japan · April 26 – May 1, 2025
• ACM SIGGRAPH 2024 Labs: Showcased in Denver, USA · July 28 – August 1, 2024

Resources

• Project Website: R3CAP
• Code Repository: singaporetech-r3cap
• Research Lab: Centre for Immersification

---

Research Impact

R3CAP contributes to spatial computing by unifying real-time reconstruction, collaborative editing, and web-based deployment into a single framework. The system lowers the barrier to creating and interacting with digital twins, enabling both researchers and practitioners to adapt it across domains.

The platform is currently being piloted with Tan Tock Seng Hospital (TTSH), Singapore, for virtual rehabilitation workflows, highlighting its potential for translation from research to clinical practice.

---