A Pipeline for the Creation, Compression, and Display of Streamable 3D Motion Capture Based Skeletal Animation Data

Since its invention, motion capture technology has been used in the production of numerous feature films and video games. Motion capture technologies have also become a highly active and varied area of research. Motion capture has not, however, been utilized for sharing works of performance art over the internet with the same level of success as enjoyed by video and audio.

This dissertation presents the components of a pipeline for the creation, compression, and display of streamable 3D motion capture based skeletal animation data. These components were built to allow individuals skilled in motion-based art forms, such as dance and martial arts, to create performance-driven 3D animation of original work and stream it over the internet to online viewers.

The construction of these components was not an easy feat; the aforementioned elements have roots in human figure animation, motion capture, virtual reality, data compression, and streaming network transmission.

Using the novel compression scheme presented in this dissertation, it is possible to create streamable animation data that targets input bandwidth requirements while minimizing reconstruction error and limiting local data storage on the viewer's machine.

Previous approaches to motion capture data compression with an emphasis on bandwidth have either ignored error, assigned static importance to each of the animated character's joints, or work in a manner that is complimentary to the approach developed here. With the approach to error minimization taken in this work, the saliency of an animated character's individual skeletal joints is assumed to dynamically vary over time. Although other compression methods that minimize error under similar assumptions exist, those methods are based on file size reduction and not network transmission requirements.

As a secondary contribution, Labanotation—a commonly used form of dance notation—is presented and studied here, for the first time, as an error metric. While Labanotation has been used in computer animation applications before, that use was limited to the areas of motion editing and automated summary creation. Using Labanotation, error is assigned based on differences in each motion's goal. This differs from existing error metrics, which assign error based on instantaneous differences in posture.