A Tangible Interactive Projection Surface Experiment
Category: #UI/UX design and Development, #Web Development, #Physical Computing, #TUI (Tangible User Interface)
Date: 2016 - Present
Adviser: Matthew Swarts


TIPSE is a multi-faceted user interface for collaborative urban planning. It uses projection mapping to display simulation results onto custom built reconfigurable construction manipulatives which are used as geometric input. Project consists of real-time depth sensor data streaming between tangible table and virtual simulation. Proposed software is developing in node.js, C# and C++. Visualization of spatial analysis for user interface and interaction is in javascript client-side of web and projection on tangible table.

TIPSE Components:

  1. Physical model includes a table that constrains the placement of manipulatives representing buildings or urban blocks.

  2. A depth sensor for scanning the table and configuration of manipulatives.

  3. Server applications include software developed for different building performance analyses at the individual building and urban scales. This also includes the communication between servers and clientts.

  4. Web client applications includes communication between server applications and screen based visualization of simulation results.

  5. Projectors for 3D surface projection mapping augments the physical model with simulation results relevant to configuration of manipulatives.


Workflow in proposed system

Shadow Study - User Interaction with Virtual Simulation through Web and with Physical Prototype through Tangible Table

Solar Irradiance Analysis - Physical model and simulation result which is projected on TUI

Interactive web interface for users' interaction with application

Rationalization: functions, sub-functions, function requirements and performance criteria

Performance Criteria and Quantification Metrics