Web apps a la Minority report?

Yes, I know, I have been lazy again with blogging. It was been a turbulent two months and I didn’t have much time. Anyhow, let’s go on. My past blog posts were all about AR, current and future developments with some examples. I didn’t get to play with it yet but something else popped up on my to do list.

While I was looking around for AR examples done with Flash and working with Papervision3D I had this idea… what if I could detect the motion of my hands through the web cam to move objects around in my Papervision3D application?

At first I thought it might be impossible to achieve with Flash but I still googled: flash motion detection. Woop I came across numerous post and most of the pointing me to two crazy guys Ohtsuka Masakazu and Mario Klingemann. Now Ohtsuka was crazy enough to port parts of OpenCV into ActionScript3 (the project is code named Marilena) and Mario improved parts of that port for performance, which enables Flash developers to perform facial recognition on still images and web cam. This technology even found its way into Facebook via an application.

Coming across this blew my mind literally and I started investigating on OpenCV and hand motion recognition which in turn blew my mind (or what was left of it) again, I realized that what I have been thinking is actually possible but yet to be implemented in ActionScript3. The most amazing part is that unlike Minority Report, this approach does not need any special gloves (although it would surely simplify development).

So here are a few videos I have found during my research. I am positive that the user experience on the web will drastically change in the next few years. AR, motion and object dedection, there is a huge amount of possibilities. Imagine a website with computer vision that would respond to your facial expression; try to cheer you up when you’re sad, calm you down when you’re angry and joke with you when you’re happy.

This first video is pretty much the most impressive that I found so far. Big kudos for this kid.

Could body motion recognition the next version of the Wii?


Augmented Reality and… magic?!

As usual I’ve been browsing the web for AR (Augmented Reality) applications, articles and videos when I stumbled upon a video by virtualmagician on YouTube. It is the coolest AR-like video I have seen so far. I am not sure if he is using true real time AR on the trick he performs or if the video was just post-edited. In either way it is damn impressive. Check it out:

AR with FLARToolKit

Augmented Reality in practiceYesterday I gave you an introduction to AR (Augmented Reality) and what is possible with this technology. Today I want to introduce you to FLARtoolKit. FLARToolKit is an ActionScript 3 port of the open source library ARToolKit with which you can build your own AR applications. With FLARToolKit you can build easily great AR applications using Flash/Flex in combination with you favorite 3D Flash Engine which means that thanks to Saqoosha, a Japanese coder, we can experience beautiful AR in our browsers. View Saqoosha’s new year’s greetings in the video or print out a marker and try it yourself.

In the next few weeks I will try myself on the FLARTToolKit and post some examples on how to build AR applications with Flex and Papervision3D.


On Augmented Reality

Augmented Reality - This might be the near futureRecently I have been researching on AR (Augmented Reality) which is in my opinion currently the most interesting trend in the industry. According to Wikipedia AR is

… a term for a live direct or indirect view of a physical real-world environment whose elements are merged with-, or augmented by virtual computer-generated imagery – creating a mixed reality.

So what does THAT now mean? It is not virtual reality as some might think. Virtual reality is composed purely of computer generated elements whereas AR uses pictures (video feeds) of the real world and superimposes computer generated elements on it.

Some of you might have actually already experienced AR in one way or the other. The first time I got in touch with it was a game on my Nokia phone a few years back (can’t remember which one it was) that used the phone’s camera to build the environment of the game. Upon the camera feed were little 3D viruses generated that I had to shoot and by moving the phone I would move within the game’s environment.

AR has many applications and it will change a gamers experience soon. The guys from NVIDIA already started (with others) to develop a unique gaming experience. Let’s take “ARhrrr!”  for example: This game is a project from Georgia Tech and SCAD-Atlanta. The game is developed for the new Tegra by NVIDIA and uses the Tegra’s camera to build a gaming environment on a printed city map. In this city you, the player, have to protect citizens from zombies. The device represents a helicopter in the gaming environment and you can even use real-life Skittles as ammunition. I think it is a bit hard to explain the game itself so here’s a demo video of it:

But the use of AR is not limited to games only. It can be used in many different ways. One company working a lot with AR is Total-Immersion and has lots and lots of examples of use including digital marketing, publishing and education.

One of my favorite marketing applications using AR is “Living Sasquatch“. Many of you may know the advertisements “Messin with Sasquatch” and with Living Sasquatch you can make your own little video with Sasquatch in your own room. All you need is a webcam a printer and you’re good to go. Try it out.

In the future I think that AR will be present everywhere. Scientists are currently researching on digital contact lenses that can show you your vital signs directly in your sight. Though it will take a few years until a first prototype is ready, this idea alone brought up many other applications such as integrations with your phone or pocket PC or the like. Furthermore we can expect at least glasses with AR capabilities much earlier than the contact lenses, as the application of AR on glasses is much easier than on lenses. I myself will probably come up soon with my own little test applications and I already have several ideas how to use this wonderful technology to build interactive websites.


Finially… After a long long period of silence I got myself back together and now I’ll start to write again.

A few months ago I started to develop with Adobe Flex. Quite interesting and sort of easy to use but I got really interested when I stumbled upon Away3D and Papervision3D. Those two are 3D engines for Flash… yes… 3D and not like the good old Shockwave Flash. No, fully fledged “high” performance 3D engines even suitable to program little games with them. I was amazed by the capabilities of those two engines and directly dove into developing small test applications and I am currently working on two websites using Papervision3D.

I will post more in the next few weeks about my progress in developing those two applications but in the meantime you can check out some really cool websites developed with PPV3D and Away3D:


Eminem’s new album has broken records, and this amazing interactive adventure website lets you discover and immerse yourself in Eminem’s sick musical world.

Some superb 3D User Interface ideas can be seen at AIRFORCE.com’s Supercar site.

Powerflasher’s “PDF Book 3D” tool uses Away3D to breathe fresh air into your run-of-the-mill 2D page flip book viewer


Nascar and the United States Air Force, the work was performed for GSD&M Idea City out of Austin, Texas.

Site for a band that already broke up in 1986