Da Young Ju

Art and Technology

Project I – SeeSaw

leave a comment »


figure1. examples of image database 

Keywords: digital arts, interactivity, HCI, sensors, computer vision

1. Introduction

This project is a participative and interactive video installation artwork, whose conceptualization is achieved by the spectator’s own engagement, and is catalyzed by reciprocal visual and sound extensions to express how communication works or perhaps more importantly, how it doesn’t work and to investigate a noble computer interaction scheme.

2. A Computer with personality

Computer devices were propagated all over the world. This project puts the computers into perspective in the future. The SeeSaw is software as personality for a computer. In other words, computers can be sad. This artwork can easily work on a computer, so users can communicate with their computer by using this artwork.

3. Typical user interaction

When visitors confront the monitor at eye level, this is the starting point of communication that is even further condensed and intensified by sound effects like talking, laughing, and shouting. A computer monitor interacts, communicates and changes its images. While changing the images in the monitor, the computer speaks, laughs and shouts through the speaker. The images on the monitor and the sounds on the speaker represent the rhythm that accompanies all the movements and expressions of the monitor. The sounds are recorded in advance by the artist, but the sounds are then also regenerated simultaneously.


 figure2. installation and user interaction

4. Installation

See figure2 and 3 . There is an installation. The equipments are common computer and common accessories. So this project can be played even on laptop computer. However, for best interaction, I use PC, LCD monitor, usb digital microphone and camera.



figure3. Equipments

5. Computer Interaction Methods

5.1. Recognition scheme

The artwork “SeeSaw” uses a specially designed human detection system. Once a human spectator is detected through a camera and a microphone, the system analyses the captured image and voice and then sends an order to an estimation part. The analyzing part uses face and motion detection methods of computer vision technology. 2D face detection technique is very popular skill in computer vision field. Using technique in this project is a model which detecting triangle of eyes and mouse in real-time movie clip (or detecting full face). I used ‘OpenCV facedetect.c’ (OpenCV Yahoo! Groups) for test of face detection, and ‘C#’ for test of sound detection. I am not sure about voice recognition now. This project only needs to recognize an appearance of a spectator, and a distance between the spectator and computer.

5.2. Estimation scheme

Estimation system is a part for choosing appropriate responses. There is an enormous volume of estimated response for interaction (conversation) with spectators. Even in reality, communication (conversation or dialog) is comprised of appropriate responses, unexpected responses and misunderstandings. In this part, the system analyses the most suitable response. but sometimes the response can be unnatural. The responses are categorized by a standard of this work, and can be randomly chosen among the setting responses in the category. This part is building using C++ now. See figure4 for an example of flow.

5.3. Response scheme

The SeeSaw is being with the machine’s personality. This is because the responses of the artwork can be distinguishable. The response scheme was designed to generate according to the spectator’s play (their motion or voice). So the responses are not static. The SeeSaw answers by her/his own feelings and thoughts about the spectator’s play.             flow.jpg            figure4. Over flow

6. A definition of the personality The interaction of this project is a dialog (conversation) between human (spectator) and computer. However, an expectation of the dialog is definitely hard work. So I set a flexible personality of this software (SeeSaw). The computer is owner of lively, sensitive personality like teen-agers. Moreover he/she feels nostalgia for their past days and home country. I tried to set interesting dialog as many as possible.

7. Conclusion

The most salient point of this artwork is the fact that a personality has been given to a machine such as a computer. This project suggests a noble computer interaction scheme as “personality” and “emotion-based generative responses” which is different from previous interactive art work. As a future work, the artist is meditating a “gender recognition system” and a “language recognition system”. These technologies are for more specific, sensitive, and minute interaction.This project aims to use the latest technology, which communicates with humans. This suggests a direction of coexistence between computer technology and humans as well.


Written by dayoung Ju

March 16, 2008 at 2:14 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: