The Pixel Farm

The “Whisper Deck” is an experimental voice-controlled augmented reality data visualization interface (wow, that’s a mouthful!)  The system allows you to don a pair head-mounted goggles and speak commands into a small microphone.  These commands are then recognized by a speech-to-text parser and executed in augmented reality in near real-time.  Think of it like a simplistic, very awkward non-tangible holodeck

As of today the system can help to visualize search volume using a tie in to Google Trends.  Simply speak the command “compare” followed by a series of keywords and the Whisper Deck will do the rest! Here’s a quick video that shows the prototype in action.

Now that I have a working prototype I hope to expand the systems functionality to do a number of things, including:

1. Access to additional APIs (Wikipedia, Google Images, Facebook, etc.)
2. New commands that provide visual, 3D search capabilities
3. Export options for Twitter and Facebook (say “tweet this” and a snapshot of your page + a brief spoken caption are uploaded to a twitter feed)

For the last year I’ve been on the lookout for a decently priced head mounted display with an integrated camera, and last week I found a device that fits this bill.  Introducing the iWear Vuzix VR920 + AR unit, which offers two 640 x 480 LCD screens as well as a clip-on USB powered webcam.

Just call me Geordi LaForge

The Vuzix VR920 + CamAR webcam (snapped onto the front)

Upon delivery I immediately “goggled” into the metaverse and thew together a quick augmented reality demo.  The system is fairly responsive, and I found that installing the optional “light filter” insert (basically a piece of molded rubber that blocks out ambient light from above and below your eyes) really helps in making the illusion feel more “believable.”  Here’s a quick video of me watching some cubes bounce around on my kitchen table.

As with all AR projects, the illusion is dependent on the presence of a marker of some kind.  Wearing the goggles gives a user much more freedom of movement, which means that you tend to lose sight of the marker more often than you would while using your laptop’s fixed webcam.  My next project will be to try and overcome this problem by creating a set of redundant markers that can act in concert to help provide overlap coverage for when a single marker goes out of view.

Google Analytics is an amazing tool. For the price of adding a small bit of embed code to your blog you can glean an amazing amount of insight into the makeup and behavior of your visitors. For example, using data gathered over the last 30 days I was able to discover the following about people who end up at my blog:

1. The vast majority of hits to my site came from locations within the USA. However it also seems as though I have a small but loyal fan club from Barcelona, Spain who have visited the site 205 times during the last month.
2. The search term “craig kapp girlfriend” was used to find my site 6 times in the last week. The computer that originated the request lives somewhere in Brooklyn, NY, and this individual stayed on my site for an average of 4:22 minutes, much longer than the average visit over that time (2.18 minutes.) I have no idea who this person is!
3. A surprising amount of traffic directed to my site is focused on finding information about “kid friendly virtual worlds.”

In the interest of further mining this rich set of data I have constructed a flash-based inspector that parses individual search terms used in locating my blog. The program breaks down search terms into individual words and ranks them by frequency of use through a dynamically generated bar chart. Mousing over a bar will display all search variations that use that particular word (i.e. “craig” has been used in the search terms “craig kapp,” “craig kapp itp” and “sex chat rooms blog.craigkapp.com” – yikes! Let’s hope that last one was a spambot …

The tool is currently running off a snapshot of last month’s data, but I’d like to hook it up to a live feed for up to the minute analysis. Feel free to check it out.

This week I decided to try and hook up an augmented reality scene so that it can be driven by SMS messages. The app below lets you vote for your favorite flower using your mobile phone – just text the appropriate code (”craigk red” or “craigk blue”) to 41411 to cast your vote!

The system is built using the public version of the TextMarks SMS aggregation system, a pair of back-end Perl scripts and, of course, Flash + Papervision.

Vote for your favorite flower!

Check out a video of the project in action:

The Carbon Peep Show is a project that I co-developed with Adam Harvey over the summer for Artist as Citizen. Developed as an entry for AAC’s  Burning Embers competition, The Carbon Peep Show is designed to showcase potential environmental scenarios that may play out based on the level of global warming that the Earth experiences over the next hundred years or so.

Upon visiting the Carbon Peep Show, visitors are presented with a closed set of theater curtains.  Behind the curtains lie a series of climate change themed videos that depict ecological consequences that may happen to our global environment if the world warms by a certain number of degrees.  The only way to watch a video, however, is to “pay” for it using a dollar bill.  Simply holding up a dollar bill to the highlighted rectangle on your screen (assuming that you have an active webcam on your computer) will advance the peep show to the next video in the playlist.

Hold up a dollar to watch the next video

In addition, the Carbon Peep Show is designed to be part of a larger conversation in which responses from the community can be showcased and broadcasted to the world at large.  By clicking on the ‘Participate’ button, visitors can submit their own video responses to be included in the project.  Will we be the victims of environmental disaster?  Will our cities become submerged under rising ocean levels?  Or, perhaps, will our ecosystem find its way towards a new equilibrium?  Submit your response and let the world know what you think!

The Carbon Peep Show wants to hear from you!