http://www.flickr.com/apps/slideshow/show.swf?v=71649

Last night I presented “Using Interfaces Without Interfaces” at Show and Tell Pdx. I showed off some Flash motion detection, Flash color detection and even demo’d a little voice speech recognition in Flash. Some of you asked for more info on how last one was accomplished. I’ll be posting more on this later.

Thank you to Substance for hosting, and thanks for all that attended and enjoyed the show. I had a lot of you come up to me and say that my work has insipred you to experiment more. Just wanted to say, that kicks ass.

http://vimeo.com/moogaloop.swf?clip_id=2829875&server=vimeo.com&show_title=0&show_byline=0&show_portrait=0&color=00adef&fullscreen=1
Webcam Swarm – Motion Detection from chris teso on Vimeo.

Reactive installation concept idea for future interactive installation using motion detection.

Concept: Using Particles to conform to typography.

Picture this large. Now take that image and double it. That’s how I envision it. Giant.

Try it for yourself : Reactive Motion Detection and Typography [webcam es necessitous]

Permalink: http://www.christeso.com/index.php/lab/motion-detection-and-typography/

http://vimeo.com/moogaloop.swf?clip_id=2566287&server=vimeo.com&show_title=1&show_byline=0&show_portrait=1&color=ffffff&fullscreen=1
Snow – Interactive Installation from chris teso on Vimeo. Commercial produced North.

Project Client

City of Portland Downtown Marketing Initiative in conjunction with North.

Project Goal

Create an outdoor reactive environment in which passersby could interact, based on their location and movements, with artists interpretations of snowfall in Portland.

Project Strategy and Equipment

Display – Custom made acrylic panels lined with photosensitive film built and erected to exact dimensions of installation window
Projection – Rear projection with flipped signal using a 5200 lumen Sanyo PDG-DXT10L Projector
Video Capture – Logitech QuickCam® Vision Pro
CPU – Mac Pro Quad
Application – Flash AS3
Industrial Design – Two large tarps sewn together to create light blocking canopy. Lining to seal off windows and acrylic. A shitload of Velcro.

Project Synopsis

As part of a larger campaign to brand Downtown Portland North was charged with creating an outdoor reactive environment in which passersby could interact, based on their location and movements, with artists interpretations of snowfall in Portland. Three traditional artists were commissioned by The Portland Institute for Contemporary Art to create their interpretation of snowfall in downtown Portland. These pieces were then transformed for use with my motion detection and tracking application built entirely in actionscript. The application was built to display 3 different scenes with randomly generated snowfall. The three scenes were set to rotate on a time interval. The application used motion detection to make the falling snowfall react and animate based on the location of an individuals movement. The application also incorporated and automated snapshot function that took a photo every 10 minutes and posted it to a private flickr account. This function was implemented solely for remote monitoring, ensuring the application was up and running. After going to the installation location and taking careful measurement of everything from window frame to projector distance, the installation was built and staged at North. We built a ‘to scale’ model of the window in the back of the office. This was necessary for accurate calibration of projector, and optimization of motion detection and tracking code. The staging was an extremely helpful and necessary step in eliminating early stage bugs such as projector calibration and camera positioning. We quickly learned the maximum distance for a USB signal, the proper use of DVI vs. DHMI to projector and the challenges of doing motion detection in every possible light range from complete darkness to bright sunshine. After testing stage was complete we moved the entire rig over to the installation spot located at 6th and Alder in Downtown Portland. We went about setting up a canopy to block out all extraneous light leak. The interior of the installation space was two stories of extremely large windows. The canopy we chose was a homemade compilation of two tarps strung up to the surrounding walls with rope. Since there was very expensive equipment residing under the canopy, the fear lingered of canopy failure collapsing down, resulting in devastating loss. We next set out to install the panels into the window frame. Since they were pre-cut to exact dimensions these flowed right in smoothly with only small light leaks needed to be sealed. We then hooked up the projector, cpu, camera, monitor, and application and set to testing. Testing in a staging environment is one thing. However, most tests are rendered moot when on-site. I quickly learned that reactive environments are just that, entirely dependent on their environment and all the challenges that come along with it. The motion detection is to run 24/7 for six weeks during all weather, lighting situations, and process location accurately with varying levels of distractions. Since the installation is located on an extremely busy downtown street corner, distractions ranged from traffic movement, pedestrians, bikers, storefront lights and weather. For testing purposes I built many functions that allows the system to be calibrated without actually going into the code, but rather using an external UI to update blurring, light detection levels, contrast, and other variables. After several days of calibration, code tweaks and testing the application has been running uninterrupted and has gained quality exposure for Downtown Portland. Many fine citizens of Portland have had a unique interactive experience, and have consequently spread kind words about the project. Furthermore, the project is a successful case study on the implementation of new technology and reactive medium into a larger branding campaign.

Project Challenges

Development – Create a motion detection application that worked in every possible light and weather condition.
Design – Making traditional raster art work work in a complex location detection application and animation.
Industrial Design – Create an environment that optimized presentation and functionality in varying environmental variables.

More information on the design and development process

Permalink: http://www.christeso.com/index.php/portfolio/snow-a-reactive-environment-inst…

photo by .:9:.
With just a couple days until the launch of the interactive installation in Downtown Portland I’ve been going mad tweaking code and functionality to run motion detection in rain, darkness and direct sunlight, with multiple object motion distractions such as pedestrians, bikers, segways and automobile headlights stopped at the traffic light precariously located directly in front of the installation camera. Simultaneously we’ve been crazy busy setting up the environment. Designing and developing the application for motion detection has kept me busy enough and has been an awesome learning experience using Flash as an interactive installation platform.

What I’ve underestimated are the challenges and learning curve associated with the industrial design end of the project. The challenges the environment has placed on us have been many. I’ve mentioned the lighting situation and the application challenges of varying weather, lighting and motion objects. There have also been other challenges such as getting the lens and projector right to cover the entire store front window from an elevation, keystoning [ keystonery? keystoningness? ] focal and clarity perspective. Properly aligning the webcam for best motion detection and display results within the application. [ viewers will be video overlayed in the application ].

Sewing together multiple tarps and hanging them one story above the installation to block out extraneous light that would dull the projection. Major issues with using an AT&T 3G card for connectivity in the absence of any wifi. [ the application takes a snapshot and posts the image to a private flickr account every 10 minutes so I can ensure the app. is up and running ]. Dealing with not being able to use an internal monitor while running the app with the projector. Adhering the specialized light sensitive acrylic pieces properly into each of the six window frames. The list goes on and is still mounting.

All this said, testing is moving along nicely. As you can see from the images below there have been people using the app. in its beta stage already. The feedback has been amazing. Standing on the street corner and watching people interact with the snowflakes and move around the area waving their hands and legs has been great. It’s been especially cool pretending to be an innocent onlooker to overhear the conversations and guessing of how the entire thing may be working. I saw more than a few people physically touching the glass guessing that they could effect the display ala touch screen. Several people danced in front of it. A few people looked semi frightened by the whole scenario. The best quote of the night came from a man who was simply amazed by the display. After carrying on about how cool it was he wrapped up the rant by exclaiming “this is downright amazing… but pfft… they’ve probably had this in Tokyo for the last 10 years”.

More testing tomorrow… and posting to ensue.

Watch the news report on the installation
Steve’s coverage of the install

Permalink: http://www.christeso.com/index.php/lab/interactive-installation-testing-tribu…

http://vimeo.com/moogaloop.swf?clip_id=2300282&server=vimeo.com&show_title=1&show_byline=1&show_portrait=1&color=00ADEF&fullscreen=1
Interactive Motion Detection Installation from chris teso on Vimeo.

Interface and motion design idea for future interactive installation using motion detection.

Words are chosen at random from a random pool of Flickr tags. The words are then broken down into individual characters. These characters are then animated based on the location of user interaction.

Picture this large. Now take that image and double it’s size. Picture it large enough to where you’d have to jump to reach different ‘doors’. That’s how I envision it.

Try it for yourself : Interactive Motion Detection Installation [webcam es necessitous]

Permalink: http://www.christeso.com/index.php/lab/interactive-motion-detection-installat…

http://vimeo.com/moogaloop.swf?clip_id=2334635&server=vimeo.com&show_title=1&show_byline=1&show_portrait=1&color=00ADEF&fullscreen=1
Untitled from chris teso on Vimeo.

Equilibrium The Human Mashup: A Follow Up, originally uploaded by NymphoBrainiac.

Equilibrium: A Follow Up

Last week I had a showing at Equilibrium, the Human Mashup. The show’s goal was to ask “How can art and technology and the synergy of the two address humanity’s next evolutionary step?”. Being a staunch singularitarian this was right up, or is it down, my proverbial alley. I built an application that allows a user to take a self portrait photograph and then draw themselves on a wall simply by using and moving their hand. The premise was to use the human as the sole interface to draw themselves, exploring the convergence of man and technology. A one day only installation, I didn’t have to high of hopes for it’s exposure. On the flip side, this being my first showing ever I was like a greedy 8 year old on Christmas morning, and extremely anxious to see how people would interact and respond to the installation. The methodology of controlling an interface without the normal peripherals such as a mouse, keyboard, touch screen, kiosk, or otherwise was foreign to all that attended. What transpired blew me away. The attendance, response and exposure far exceeded my hopes. I barely had to explain at all to users how to run the application. People just stepped right up to the installation and began to interact with it. They were actually learning on their own while performing the drawing. People were actually converging with the technology. Furthermore, those that were done performing their drawing became evangelists, telling those who followed them their idea of how best to use the application. At that moment it became cemented in my mind that I had succeeded in my experiment. People had invested themselves in the piece to the point of bringing out their own inner artist. Several times I stepped in to ‘coach’ people on how to get the most out of the motion detection while drawing their portrait. The response I got was unexpected. People took personal offense that I was pushing my artistic ideas on their own drawing. Again, success. I was also sent to the floor several times by attendees who said they had come to the event specifically to see my work which they had been following online. There were even several people who mentioned that they had hooked my work up to projectors at parties they were holding. Wha?! Seriously, amazed.

I’d like to thank several people for the night. Steve Benoit for all his help with testing the application, videography and film editing of the event. The result is the shweet movie posted above. Rachel Kjack and Adrienne Fritze from SAO and Working Artists Online respectively for their organization and inviting me to submit my work. All that attended, I met some inspirational folk. And finally, my wife Lisa for her unending support and diggin’ me even when I have late night dreaded ‘code eyes’.

Looking forward to the next event.

Related:

the online version of the installation

the flickr gallery of finished drawings

previous blog post

working artists online

software association of oregon

Permalink: http://www.christeso.com/index.php/lab/equilibrium-the-human-mashup-a-follow-up/

http://vimeo.com/moogaloop.swf?clip_id=2337699&server=vimeo.com&show_title=1&show_byline=1&show_portrait=1&color=00ADEF&fullscreen=1
Tracking Multiple Objects Using a Webcam from chris teso on Vimeo.

In continuing exploration into motion tracking using flash and a webcam I’ve created an application that enables you to track multiple objects based on color. The holy grail of the application is of course to track multiple objects without specifying color. However, this is a good step forward and actually opens up many other avenues for use. Color, after all, is ubiquitous.

The truly useful part of the code is getColorBoundsRect. If you’re unfamiliar with this function, it determines a rectangular region that either fully encloses all the pixels of a specified color within a BitmapData object, or fully encloses all pixels that do not include the color. The function returns a rectangle around the color area. You can then reference this rectangles x and y respectively.

var areaColor:Rectangle = SomeBitmapData.getColorBoundsRect( 0xFFFFFFFF, _colorArray[i].color, true );

The function is rather finicky about the depth of where the object is. Since it’s only searching for a certain RGB value it tends to lose it if the lighting or the object gets too small in the camera’s depth of field. To overcome this you could search for similar colors. Soulwire has written a sweet Color utility class that can aid you in this endeavor. However, searching for a range of colors will also come with it’s challenges, as the more range of color you search on the tougher it will be to pinpoint the exact object you want to track.

Check out the demo here: Tracking Multiple Objects Using a Webcam [webcam obv required]

Music: My Morning Jacket – I’m Amazed

Permalink: http://www.christeso.com/index.php/lab/tracking-multiple-objects-using-a-webcam/

This Friday -[Oct. 17th]- I will be part of SAO and the Portland Working Artists Network in presenting at Equilibrium, the Human Mashup, using the synergy of art and technology to revolutionize society. I am currently designing and developing an interactive art installation. A brief description of the project is below. I’m super excited to see what other installations are on display. If you’re in the Pdx area and interested in what Equilibrium has to offer please stop by and paint yourself on the wall.

Brief Description
In the spirit of Multiplicity and Convergence an interactive installation will be developed that allows viewers to draw a self portrait on a wall using a motion sensing application, a camera and projector. The application will allow a user to take a self portrait photograph. They will then have the opportunity to draw themselves on a wall simply by using and moving their hand as a paintbrush. The idea is to use the human as an interface to draw themselves. The drawing will be performed using code that analyzes and compares the location of the viewers hand to the location of pixels in their photograph. As the viewer waves their hand a paintbrush will paint their snapshot onto the wall. The resulting image will then be automatically uploaded to a photo gallery on Flickr. A different version that uses music as the paintbrush and an online image as the self portrait can be viewed here.

Photo by .:9:.

http://vimeo.com/moogaloop.swf?clip_id=2339145&server=vimeo.com&show_title=1&show_byline=1&show_portrait=1&color=00ADEF&fullscreen=1
AS3 Webcam Drawing from chris teso on Vimeo.

You have to admire a job that allows you to make a bumble-bee/owl (a bumblowl?) fly around a mural merely by waving your finger… all while drinking beer.

This is an elaboration on my actionscript webcam motion detection experiments, built as an installation for the Inverge party hosted by North. The idea was to take motion detection and combine it with art in the form of digital drawing and 2d space in the form of a wall mural. The concept is to allow human interaction to affect physical space, freeing oneself from all traditional user interface. The application works by constantly monitoring a webcam, taking snapshots of each frame every millisecond. The pixels in the current frame are compared to pixels in the previous. The difference in brightness within each section determines whether or not motion had occurred in this area of the screen. If motion is detected the coordinates of a rectangular grid around said motion is recorded and the owl is ordered to go to that point. The line drawings begin and follow the owl recording a ‘flight path’. A mask is drawn to prevent the owl and its drawing trail from going in front of some of the trees giving the mural a sense of dimension.

Thanks to the great Eric Natzke for the drawing inspiration and open source, and NORTH for being forward thinking enough to encourage digital experimentation.

Permalink: http://www.christeso.com/index.php/lab/as3-motion-detection-drawing-installat…