Emerging field shifts perceptions of human, machine limits

By Nicolas Mokhoff
EE Times
August 09, 2004 (9:00 AM EDT)

MANHASSET, N.Y. ? Timed to this week’s Siggraph in Los Angeles, a pioneering company in the percetion technology field rolled out a commercial development kit for its 3-D-sensor machine vision chip.

Electronic perception technology has already been used to create projection keyboards, where finger movements on a flat surface are interpreted as keystrokes by a PDA or similar device projecting a keyboard “in thin air,” and numerous other applications are being evaluated or developed.

http://www.eetimes.com/at/news/showArticle.jhtml?articleID=26806416

MANHASSET, N.Y. ? Timed to this week’s Siggraph in Los Angeles, a pioneering company in the percetion technology field rolled out a commercial development kit for its 3-D-sensor machine vision chip.

Electronic perception technology has already been used to create projection keyboards, where finger movements on a flat surface are interpreted as keystrokes by a PDA or similar device projecting a keyboard “in thin air,” and numerous other applications are being evaluated or developed.

Canesta’s platform includes a single-chip 3-D camera, USB interface and Windows-based software development environment. Developers will be able to prototype their application and code on a Windows PC and then embed the same application, along with an Equinox chip, in the end product.

“Up to now, computer vision has been [limited to] the labs,” said Canesta vice president of marketing Jim Spare. “We aim to commercialize it.”

The class of applications possible using Canesta’s electronic perception technology include size and depth detection, image segmentation, object classification, object tracking and location analysis, and human interaction. “A factory robot could accurately pick up or avoid a package, or a ‘smart truck’ could evaluate and monitor its cargo load,” said Spare.

Comparing Canesta’s technology with conventional camera perception, Spare said a conventional camera setup would have difficulty recognizing a human form in the passenger seat of a car, particularly if that person’s clothing and the car’s upholstery had no discernible texture. “But with our technology, the location, size and shape of the person would be instantly determined,” he said. “This would be invaluable for making an airbag that could choose to deploy or not, depending on whether the occupant was an adult or child.”

Additionally, the technology’s object classification feature could take the “wire frame” view of a scene and distinguish between, say, a child and a grocery bag. Moving objects could be tracked and their locations analyzed. “With the technology, human gestures or actions can control numerous types of systems from a distance,” said Spare.

Canesta’s breakthrough centers on its Equinox chip, built in low-cost, standard CMOS. “True 3-D image sensing is extremely useful for applications that need to look at a scene and decompose it into objects,” Spare said. “If you not only can resolve objects from the background but also can acquire information about their size, shape, and distance from the sensor, then you open up applications that are otherwise difficult or impossible to do.”

The Equinox chip resolves a scene into pixels, as does an ordinary camera chip; but instead of simply providing the brightness of each pixel, Equinox also provides the distance from each picture element to the sensor chip. In effect, this renders the scene into three-dimensional objects that are easily processed by computing chips in devices such as cell phones or PDAs.

“We call this primary 3-D data,” said Spare. “It eliminates the need for massive calculations of one or more 2-D images to accomplish the same tasks.”

The chip uses a time-of-flight principle to determine the distance from each pixel in the array to the corresponding feature in the scene. The time of flight is measured by detecting the phase difference between a modulated light source (an array of laser diodes or LEDs near the sensor) and the returning light at every pixel in every frame. The phase directly relates to the distance from the object to the camera.

While it is theoretically possible to determine time-of-flight measurement mathematically, through demodulation, by combining two reference signals with the returning light signal, the digital or analog circuitry to perform such a calculation would be too expensive and bulky to implement in a CMOS chip, said Spare. “Rather than compute this mathematically, Canesta invented a way to leverage the properties of the CMOS silicon semiconductor material to do the demodulation automatically at every pixel. The distance value is provided right out of the hardware and does not require expensive signal or image processing to generate the value.”

Canesta’s sensor comprises a 64 x 64 array of light-sensitive elements (pixels), each consisting of two gates. To determine the phase delay of the incoming light, each gate is alternately switched in synchronization with the modulation frequency of the light emitted from the source.

While one of the gates is switched in phase with the frequency of light from the photodiode array, the other is switched out of phase. In this way, a charge will build at each gate, proportional to the amount of light received during the period. Effectively, this CMOS silicon structure has performed the signal demodulation. Thus, the differential output voltage between the two gates provides an effective measure of the phase shift of the light and the distance of the object from the sensor. Each of the dual-gate pixels can thus provide a range value.

The chip was designed at Canesta and is built in a standard 0.25-micron CMOS process by Taiwan’s United Microelectronics Corp.

Canesta’s Electronic Perception Development Kit includes a light source and matching optics that may be ordered in three versions, with an 80°, 55° or 30° field of view. The included software development kit provides an application programming interface for writing and debugging C or C++ applications using Microsoft Visual Studio 6.0 on a Windows 2000 or Windows XP PC.

Through the API, developers can access raw brightness and depth image frames, as well as set such sensor parameters as the frame rate, shutter speed, operation mode, window size or region of interest.

The development platform costs $7,500. But a $2,500 discount is offered to accredited educational institutions that agree to publish on applications developed using the kit and to contribute their software and design concepts to the public domain under the open-source principle.