Navigation for the Visually Impaired Using a Google Tango RGB-D Tablet
Starting in the spring of 2015, ss part of my work at Purdue’s Envision Center, I began creating a navigation system for people with visual disabilities, using the Google Project Tango tablet depth camera. In short, it builds a 3D representation of the user’s surroundings in real time and converts it to 3D audio for navigational cues.
Here is a video demonstrating the current system (as of May 2015). Please note that the audio is very quiet due to hasty recording.
More information can be found here.
STAR: System for Telementoring with Augmented Reality
Since May 2014, I’ve been working with an interdisciplinary team at Purdue University on a research project to use augmented reality to enhance surgical telementoring. With the System for Telementoring with Augmented Reality (STAR), we want to increase the mentor and trainee sense of co-presence through an augmented visual channel that will lead to measurable improvements in the trainee’s surgical performance.
Here is a video demonstrating the current system (as of April 2015):
You can find more information on our project page.
GPU-Accelerated 3D Ant Colony Simulation and Visualization
This is the final project I did during my first semester of graduate school, for Purdue University’s Fall 2014 CS535 (Interactive Computer Graphics) course.
Ant colony optimization algorithms are useful for pathfinding, such as in robotics and distributed networks. In this simulated scenario, a collection of ants can converge on common solutions to finding nearby food, despite each ant only having knowledge of its immediate surroundings. This particular quality makes ant colony simulation suitable for parallelization and GPGPU.
The basic setup of the simulation is that ants wander randomly from a central nest point, laying a pheromone trail behind them. They pick up food when they encounter it and return it to the nest. Other ants will follow existing trails, strengthening them over time.
Cupola VR Viewer for Oculus Rift Head-Tracking in WebGL Virtual Environments
In the app, I’ve provided links to a couple sample WebGL pages that support Cupola, that you can load in the Chrome app and get head-tracking working. You can also drag and drop the Oculus config files into the app to use your calibration data (still experimental, doesn’t persist on exit/restart).
– vr.js is an awesome NPAPI plugin for Chrome and Firefox, but unfortunately Chrome is retiring NPAPI support.
– oculus-bridge uses a standalone application that interacts with the Oculus SDK, and then provides a WebSocket stream of orientation data that a website can connect to. However, WebSockets are kind of slow, and give about a 10-millisecond delay that I find noticeable and disorienting.
If you’re interested in VR and the Rift when used with browser-based virtual environments, please check this out! I think that WebGL and three.js make it really easy to set up 3D environments and having a system like this will be really useful to the VR community.
Let me know (at firstname.lastname@example.org) if there are any questions, comments, feedback, bug reports, pull requests or anything like that. I really want to make something useful for all of you in the Rift community. Thanks!
In Spring 2011, I developed a streaming camera system for the University of Utah Robotics Club (RoboUtes) as part of the 2011 RASC-AL ROBO-OPS competition. This competition, funded by NASA, had over a dozen teams creating “Mars rovers” that would traverse a desert landscape in Texas, picking up specially-colored rocks, while being remotely controlled by team members located at the home campus (in our case, in Utah).
As part of the software team for this project, I was responsible for developing a camera system that would run on a low-powered “BeagleBoard” single-board computer, taking raw image data from a series of Logitech cameras and wirelessly streaming the data to a client application running at our “mission control” on campus. It also included pan/tilt controls that a user could use to remotely manipulate the camera.
Finally, I did some image processing with OpenCV to analyze the image data as it came in to highlight rocks of a particular color, to help the team more quickly seek out targets for collection.
Our rover came in second place in the competition.