What it is

An interactive experience that makes the user cut their own commercial, using the webcam to detect whenever they blink.

Visit experience
The mission
Get everyone up to date with ACOs new brand and products.
Face recognition
Let the user use their eyes to interact with the experience.
Localization
Communicate the same experience for two brands and in four languages.

Background

ACO, a Nordic skincare brand, recently made a complete makeover of their brand to focus more on beauty products and skin care. DDB Stockholm contacted us with an idea that revolved around Moisture. As ACO’s products moisturize the skin, the blink moisturizes your eyes. That was the starting point for this project.

Three videos were created with similar camera movements but with different shots. One revolved around the feeling of moisture, one revolved around skin and one focused on the product.

The Moisture
The Skin
The Product

When the user blinks we make a cut to another video, and that way every visitor’s experience of the new ACO commercial will be a completely different one. The number of possibilites are astonishing, with 4,724*10882 possible variations of the video.

3 possible videos every frame (25 frames * 74 sec)
325*74 ≈ 4,724*10882 possibilites

How it works

  1. After entering the experience, the user is prompted to engage in a physical manner by allowing the browser to gain access to the camera of the computer.

  2. Once the camera has been engaged and the application has access to it the scan for the users eyes begin immediately.

  3. This is where the fun begins. Through careful coding and countless hours of tweaking the camera we present the user a custom commercial, cut in real time based on when the user blinks.

Try it out

Live camera not working? Browser not supported
This demo requires a webcam and a desktop browser with native webcam support.
Google Chrome logo

Web Workers

A quite old technology that we never really had the chance to play around with in depth are web workers. Web workers are basically threading in Javascript. In this case we used it to take the heavy CPU load off the main thread that renders the UI.

The site runs a main thread, and two web workers. One worker handles the face detection; finding a square on the webcam canvas that contains your face. The other worker searches within that square to find the area where your eyes currently are, and communicates that back to the UI thread, which detects the actual blinks.

The face and eye recognition are the most expensive algorithms, and the blink detection is very cheap in comparison. When we started the project we were thinking about putting the blink detection in it’s own thread, but it was deemed unnecessary.

Algorithm

The algorithm we used for the face and eye tracking is the Viola-Jones object detection framework, that’s also used in OpenCV which we used in early testing. We took a great deal of inspiration from tracking.js and their implementation, but wrote most of the code to make it work with web workers.

Youtube Gadget

ACO Moist was also the first project in which we developed inside a Youtube Gadget, a custom tab available for Youtube partners. The development and functionality available is pretty similar to those of Facebook apps.

Localization

The site is available in four languages; English, Swedish, Norwegian and Finnish - we’re bloody excellent at localization now.

Alternative versions

Since the getUserMedia API isn’t available in Safari and Internet Explorer we had to come up with another solution for those two browsers. The client was adamant that it work in all browsers, and our only option was Flash, something that we normally don’t do so we were in a bit of a conundrum. With just a few hours till the whole project was taken somewhere else we got a tip about a guy who joined up with our team. It was a great collaboration.

We also had to create some fallback solution for those who didn’t want to allow access to their webcam or didn’t have one. We used the letter ‘B’ for blink to trigger the swaps between films.

And the poor sods who just couldn’t handle three videos at once at least got a view of the final cut commercial.

Afterwards

The site has just been launched. We’ll update you later on the success.

During development a cat accidentally sat in front of the blink detection.
It worked =^.^=
Sound awesome?

Get in touch

E-mail
hello@agigen.se
Address
Kaptensgatan 6
114 57 Stockholm
Call
+46 (0)8 20 1337
Zippity-zip
114 57 Stockholm