The Whisper Deck

Overview

The Whisper Deck is a voice-controlled augmented reality data visualization tool that immerses users within a fluid information ecosystem of their own design.  The project is an experimental interface that explores new ways in which we can examine the vast amount of data being generated by the world on a daily basis.

Visualizing google trends via the Whisper Deck

Video

Description

Using an off the shelf Vuzix Cam-AR head mounted display, users can look around their local environment and examine the world through the integrated webcam unit on the front of the display.  Upon noticing a pre-defined symbol, a 3D world instantly appears.  As long as this symbol remains in view, this newly created augmented space will continue to persist and will allow the user to examine it from any direction by simply moving around it in real space.

Users can issue requests to the Whisper deck using a series of voice commands.  These commands will cause the world to reconfigure itself based upon your preferences.   For example, if you would like to have the world gather information about a topic you are interested in – say, Boston Terrier Puppies – simply say the command “search Boston terrier puppies” with the keyword “over” at the end of your sentence.  You can search almost any topic from ice cream to casino to cars or anything that interests you. The system will go out to the Internet and retrieve information relating to your request, including a spoken definition from Wikipedia as well as a set of images from various publicly accessible image search engines.

Image Search using the Whisper Deck

In addition, the Whisper Deck also allows visitors to compare the relative popularity of search term by interfacing with Google Trends.  Speaking the command “compare” will allow you to name any number of terms which will be visualized as a 3D bar chart that can be further inspected.

Technology

The Whisper Deck uses a number of different tools.  While most of the technologies described below are web-friendly, the voice controlled aspect of the system is handled via a desktop speech to text package.

• Flash ActionScript 3
  • FLARToolkit (marker detection)
  • Papervison 3D (3D rendering)

• Web Services
  • Yahoo!  Pipes (Flickr, Picasa & Google Images feed aggregation)
  • Perl + Python (Google Trends integration)

• Voice Recognition / Playback
  • Mac Speech Dictate
  • Perl + integrated Apple OS X text to speech engine