Web apps a la Minority report?

Yes, I know, I have been lazy again with blogging. It was been a turbulent two months and I didn’t have much time. Anyhow, let’s go on. My past blog posts were all about AR, current and future developments with some examples. I didn’t get to play with it yet but something else popped up on my to do list.

While I was looking around for AR examples done with Flash and working with Papervision3D I had this idea… what if I could detect the motion of my hands through the web cam to move objects around in my Papervision3D application?

At first I thought it might be impossible to achieve with Flash but I still googled: flash motion detection. Woop I came across numerous post and most of the pointing me to two crazy guys Ohtsuka Masakazu and Mario Klingemann. Now Ohtsuka was crazy enough to port parts of OpenCV into ActionScript3 (the project is code named Marilena) and Mario improved parts of that port for performance, which enables Flash developers to perform facial recognition on still images and web cam. This technology even found its way into Facebook via an application.

Coming across this blew my mind literally and I started investigating on OpenCV and hand motion recognition which in turn blew my mind (or what was left of it) again, I realized that what I have been thinking is actually possible but yet to be implemented in ActionScript3. The most amazing part is that unlike Minority Report, this approach does not need any special gloves (although it would surely simplify development).

So here are a few videos I have found during my research. I am positive that the user experience on the web will drastically change in the next few years. AR, motion and object dedection, there is a huge amount of possibilities. Imagine a website with computer vision that would respond to your facial expression; try to cheer you up when you’re sad, calm you down when you’re angry and joke with you when you’re happy.

This first video is pretty much the most impressive that I found so far. Big kudos for this kid.

Could body motion recognition the next version of the Wii?