The Pixel Farm

Last spring I had the opportunity to work with Nisma Zaman on a series of assistive technology projects for inpatient kids at NYU’s Rusk Institute of Rehabilitation Medicine.  The overarching goal for this project was to try and find ways to create entertaining technology driven-experiences for patients to play with in the center’s main recreation room.  The following video shows our first game, called “PairBurst Memory,” in action.

PairBurst Memory from Nisma Z on Vimeo.

“PairBurst Memory,” is an augmented reality matching game that asks kids to try and find pairs of 3D animals that are “hidden” behind pieces of a wooden puzzle.  The game setup consists of a ceiling mounted projector, a Mac Mini, a wooden puzzle board and a webcam that is mounted slightly above and behind the player.  The game software is designed to track a series of nine symbols that exist underneath each puzzle piece.  When visible, the PairBurst software will augment the uncovered symbols with one of four 3D animals.  Upon finding a match the animals will fly off of their spaces and land in the center of the board.  When all four matches are found the animals fly back and celebrate by having a 3D dance party.

For more information please visit the project site at http://www.pairburst.com.

Over the last few weeks a number of ZooBursters have gotten in touch to ask if there was a way in which they could interact with their 3D pop-up books without having to print out an augmented reality marker.  Many people don’t have easy access to a decent quality printer, and while you can use an external device in place of a marker (see Greg Borenstein’s exploration into using an iPad in place of an AR marker) the glare issues that arise from using a backlit display can be a bit difficult to overcome.  Plus it’s hard to enjoy your 3D creation while holding something up in front of your screen!

In response to these issues I’ve built an experimental features that lets you display your ZooBurst book on the screen without needing to have an AR marker handy.  Once activated, this feature will cause your book to fall out of the sky and bounce into place along the bottom of the video field (see image below).  In addition, a new control tab also appears at the top of the page – this tab “listens” for “swipes” in the video feed.  When detected, these swipes can be used to control the page turning action.

Controlling a ZooBurst book using physical gestures

Here’s a quick video of how the new feature works:

T-minus 9 days until the premier educational technology event of the year – the annual New Media Consortium Summer Conference! This year’s program is looking better than ever with sessions and workshops focusing on a wide range of topics, such as educational gaming, virtual worlds, mobile applications, geolocation technologies, cloud-based learning systems, digital storytelling techniques and yes, of course, Augmented Reality!

NMC 2010 Summer Conference

This year I will be presenting a half-day hands-on session in which we will explore emerging educational applications of augmented reality as well as introducing ways in which educators can build their own AR environments.  I’m busily preparing a number of fun tutorials for this event, including:

• Hands-on activities that will let participants experience augmented reality first-hand (printed markers included!)
• A series of “quick start” code libraries that are designed to help educators construct their own AR learning applications, and step-by-step instructions on how to get up and running fast.  All participants will leave the session with their own working AR project on their blog or website.
• A preview of the new version of the ZooBurst digital storytelling tool which includes a number of new features, including live audio recording capabilities, classroom management features and even an augmented reality presentation mode.

If you’ll be in Anaheim CA next week and would like to join us feel free to register!

ZooBurst has been getting tons of use over the last few weeks, and I’ve been spending most of my free time making updates to the system and thinking of ways to move forward with the environment. It’s really amazing how fast it’s taken off – as of today we have over 1,000 people actively using the system to create their own 3D pop-up books!

There are a ton of features on my “to-do” list for ZooBurst, but one of the most pressing is to find a way to make the augmented reality feature more usable.  Right now you can watch as your own books fly out of the printed symbol and into the world around you, but in order to interact with the book (i.e. turn the page, make a character speak) you still need to use the mouse.

ZooBurst augmented reality mode

I’ve toyed around with a number of potential interaction methods, including using the rotation of the marker to control various elements, but I’m not 100% happy with them just yet.  I want to try and preserve the feeling of a real book as much as possible, and creating additional mappings that don’t necessarily apply to a physical book is something that I’d like to try and avoid.

One experiment I’ve been playing around with is the idea of using physical gestures to actually turn the pages of the book.  I whipped up a quick 2D demo of this in action – basically I use motion detection in very small regions of the screen to create a matrix of “touchable” sensors.  From there I can create a simple algorithm to interpret gestures based on the order in which these regions are activated.  Here’s an example that allows you to simulate a coverflow-style interface with a series of images.  I’m hoping to adapt this functionality into the 3D world of ZooBurst by the next release of its AR viewer component.

2D coverflow using a webcam

ZooBurst is a web-based digital storytelling tool that is designed to let anyone easily create their own customized 3D pop-up books.   As of today I am pleased to report that version 1.0 of ZooBurst is up and running, and is accessible to the world at http://alpha.zooburst.com.

Here’s an example ZooBurst book (requires the Adobe Flash player plugin – sorry iPad/Pod/Phone users!)

… and here are a few screen shots for those of you who are viewing this post via an aggregator:

Zooburst 3D Pop-Up Book

Zooburst 3D Pop-Up Book - Augmented Reality

The current version of ZooBurst supports the following features:

• Web-based access to user created books
• Embeddable content that allows a ZooBurst book to exist on any website that accepts HTML embed code
• Augmented Reality mode that lets you experience a book in the palm of your hand (requires a webcam)
• Web-based creation of new books
• Integration with the Open Clip Art Library of public-domain vector images
• Local file upload feature to allow you to use your own artwork within a book

Over the next week I will be contacting individuals who have signed up for the public beta version of ZooBurst with an invitation code.  We are still accepting new users on a rolling basis – to sign up for an account please visit this form and tell us a little more about yourself and why you would like to give ZooBurst a try.