Completed
2015 - 2020
Senbay Kit
Platform embedding sensor data into video frames using animated 2D barcodes via real-time video processing. Supports iOS, Android, macOS, and web.
Video Processing
Sensor Data
Open Source
Overview
Synchronizing video and sensor data is essential for context-aware applications, but existing methods require complex infrastructure. Senbay Kit is a platform that embeds collected sensor data into video frames using animated two-dimensional barcodes via real-time video processing on smartphones.
Approach
- Sensor-federated video (MobileHCI 2018, UbiComp 2015): Embedding sensor data as animated QR codes directly into video frames during recording on smartphones, enabling easy sharing through popular video sharing services
- Multi-platform: Supporting iOS, Android, macOS, Unix-based OS, and web browsers
- Data integration: Seamlessly combining video and sensor data into a single video stream for context-aware applications, integrated data recording, and sensor-data-based video editing
Results
- Demonstrated real-time embedding and extraction of sensor data in video frames on consumer smartphones
- Enabled sharing of "sensor-federated" video through existing video sharing platforms without additional infrastructure
- Supported diverse use cases including context-aware video applications, synchronized data recording, and sensor-driven video editing
Significance
Senbay provides a lightweight, infrastructure-free solution for integrating sensor data with video. By leveraging animated QR codes and existing video sharing platforms, it enables researchers and developers to easily capture, share, and restream synchronized multi-sensor video data using only smartphones.
