As part of the process for making Wired, I worked with some students over the summer to create a load of gameplay prototypes.
Nearly all of these games took one person 2 weeks to make (though a couple took longer). They all have something interesting about them and are also all flawed to some extent. However, each prototype showed enough to know whether the core idea would work and where the potential risks lay.
I worked on a number of camera-based games for Playstation during my ten years at Sony.
This video demonstrate the book tracker my team developed. The player holds a real cardboard book in their hands while the PlayStation camera looks at them. On the TV, the book is transformed into a magic spell book. There are a number of systems working together to pull off the illusion including: Book tracking, hand segmentation, PS Move controller tracking and motion detection. Note that the PSEye camera is a fairly standard web camera and doesn’t have a depth sensor.
In this link you can see an overview presentation of how the book tracker part worked. The photos in the presentation show Tom Lucas-Woodley, one of the senior engineers who poured his soul into this to get it working: book-tracking_reduced.pdf
You can see adverts for the actual products the technology went into here:
SLAM stands for Simultaneous Localisation and Mapping. This was a system my team at Sony developed for mobile gaming devices. You look around an environment with the camera and it builds a 3D map. You can then use this map to place virtual objects in the environment or indeed warp the environment itself.
The table warping demo used the front and back touchscreen of the PlayStation Vita to interact.
While the team I managed developed this technology, I didn’t do any of the technical work myself. This was primarily developed by Antonio Martini, Simon Hall and Andrew Stoddart.
I created some interesting new user interface widgets using a camera. Most of the camera-based games we created did not require the player to hold a controller to play them so it was important that we could navigate menus without a controller too.
The first part of the video shows a “scroll bar” which could be moved with your hand. This suffered from the problem that it could be accidentally moved by someone standing behind it. The second part of the video is an attempt to get around this problem with a “dial controller” which only responds to circular motion.