数据驱动的3D面部动画技术与应用

4星 · 超过85%的资源需积分: 13 20 浏览量更新于2023-03-03 1 收藏 8.25MB PDF 举报

身份认证购VIP最低享 7 折!

30元优惠券

"Data-Driven 3D Facial Animation" 是一本由 Zhigang Deng 和 Ulrich Neumann 编辑的专业书籍，聚焦于利用数据驱动的方法进行三维面部动画的创建、编辑与合成。这本书涵盖了一系列相关主题，如计算机面部动画、视觉语音生成、眼动模拟、多角色对话以及从语音中学习人类头部运动序列等。在这本书中，"Data-Driven" 概念是指通过收集真实的人脸表情、动作和语音数据，然后用这些数据来训练模型，以创建逼真的3D面部动画。这种方法与传统的手工制作动画相比，能更准确地捕捉和再现人类面部表情的细微变化，使得动画角色的表演更加生动和自然。 "3D Facial Animation" 是指在三维空间中对人脸进行动画化处理，包括嘴唇同步、眼球运动、眉毛和脸颊的动态等。书中的内容可能涉及到如何使用数据驱动的技术来实现这些效果，如通过纹理合成实现真实的眼球运动，以及分解和编辑口型和脸部运动，以匹配特定的语音片段。 "Expressive Visual Speech Generation" 关注的是生成与语言内容相符且具有表现力的视觉表现，例如通过分析语音信号来驱动3D面部模型，使角色在说话时不仅有正确的唇形，还能表现出情感和语调。 "Data-Driven Expressive Speech Animation Synthesis and Editing" 部分可能探讨了如何利用数据来合成和编辑具有表现力的语音动画，包括将语音信号分解成不同的运动组件，便于编辑和控制。 "Eye Movements, Saccades, and Multiparty Conversations" 这部分涉及如何模拟真实的眼球运动，包括快速的眼跳（saccades）以及在多角色对话中的复杂眼神交流，这对于创造可信的交互场景至关重要。 "Realistic Eye Motion Synthesis by Texture Synthesis" 介绍了一种通过纹理合成技术来生成逼真眼睛运动的方法，这有助于提升观众对3D角色的沉浸感。 "Learning Expressive Human-Like Head Motion Sequences from Speech" 可能讲述了如何训练算法从语音中学习并生成符合人类行为的头部运动序列，使得角色的头部运动与说话内容和情感相协调。 "A User Interface Technique for Controlling Blendshape Interference" 提到了一种用户界面技术，该技术允许制作者更好地控制面部混合形状的干扰，以创建更精确和细腻的表情动画。 "Sketching Articulation and Pose for Facial Animation" 可能是关于如何使用草图来预览和设计面部的关节运动和姿势，以便在动画制作过程中实现所见即所得的效果。 "Speech Motion Decomposition and Editing" 部分可能深入讨论了将语音运动分解为可编辑的部分，这样可以独立调整每个组件，从而实现更精细的动画控制。这本书是数据驱动3D面部动画领域的宝贵资源，涵盖了从基础理论到高级应用的各种技术，对于游戏开发者、影视特效艺术家、动画师以及人工智能和机器学习研究者来说，都是极具价值的参考资料。

资源详情

资源推荐

8 Z. Deng, J. Noh

1.6.2 Vector Muscle

A very successful muscle model was proposed by Waters [30]. A delineated defor-

mation ﬁeld models the action of muscles upon skin. A muscle deﬁnition includes

the vector ﬁeld direction, an origin, and an insertion point (the left panel of Fig. 1.3).

The ﬁeld extent is deﬁned by cosine functions and falls off factors that produce a

cone shape when visualized as a height ﬁeld. Waters also m odels the mouth sphinc-

ter muscles as a simpliﬁed parametric ellipsoid. The sphincter muscle contracts

around the center of the ellipsoid and is primarily responsible for the deformation

of the mouth region. Waters animates human emotions such as anger, fear, surprise,

disgust, joy, and happiness using vector-based linear and orbicularis oris muscles

utilizing the FACS. The right panel of Figure 1.3 shows the Water s’ muscles embed-

ded in a facial mesh.

The positioning of vector muscles into anatomically correct positions can be a

daunting task. The process involves manual trial and error with no guarantee of

efﬁcient or optimal placement. Incorrect placement results in unnatural o r undesir-

able animation of the mesh. Nevertheless, the vector muscle model is widely used

because of its compact representation and independence of the facial mesh struc-

ture. An example of vector muscles is seen in Billy, the baby in the movie Tin Toy,

who has 47 Waters’ m uscles on his face.

1.6.3 Layered Spring Mesh Muscles

Terzopoulos and Waters [39] proposed a facial model that models detailed anatomi-

cal structure and dynamics of the human face. Their three layers of deformable mesh

correspond to skin, fatty tissue, and muscle tied to bone. Elastic spring elements

Fig. 1.3 The left panel shows the zone of inﬂuence of Waters’ linear muscle model. The right

panel shows muscle placement in Waters’s work [30].

1 Computer Facial Animation Survey 9

connect each mesh node and each layer. Muscle forces propagate through the mesh

systems to create animation. This model is faithful to facial anatomy. Simulating

volumetric deformation s with three-dime nsional lattices, however, requ ires exten-

sive computation. A simpliﬁed mesh system reduces the computation time while

still maintaining similar quality [40].

Lee et al. [41,42] present face models composed of physics-based synthetic skin

and muscle layers b ased on earlier work [39]. The face model consists of three com-

ponents: a biological tissue layer with nonlinear deformation properties, a muscle

layer knit together under the skin, and an impenetrable skull structure beneath the

muscle layer. The synthetic tissue is modeled as triangular prism elements that are

divided into the epidermal surface, the fascia surface, and the skull surface. Spring

elements connecting the epidermal and fascia layers simulate skin elasticity. Spring

elements that affect muscle forces connect the fascia and skull layers. The model

achieves b etter ﬁdelity. Tremendous computation is required, however, and exten-

sive tuning is needed to model a speciﬁc face or characteristic.

1.7 3D Face Modeling

An important problem in facial animation is to model a speciﬁc person, i.e., mod-

eling the 3D geometry o f an individual face. A range scanner, digitizer probe, or

stereo disparity can measure three-dimensional coordinates. The models obtained

by those processes are often poorly suited for facial animation. Information about

the facial structures is missing; measurement noise produces distracting artifacts;

and model vertices are poorly distributed. Also, many measurement methods pro-

duce incomplete models, lacking hair, ears, eyes, etc. Therefore, post-processing on

the measured data is often necessary.

1.7.1 Person-Speciﬁc Model Creation

An approach to person-speciﬁc modeling is to painstakingly prepare a generic

animation mesh with all the necessary structure and animation information. This

generic model is ﬁtted or deformed to a measured geometric mesh of a speciﬁc

person to create a personalized animation model. The geometric ﬁt also facilitates

the transfer of texture if it is captured with the measured mesh. If the generic model

has fewer polygons than the measured mesh, deformation is implicit in the ﬁtting

process.

Person-speciﬁc modeling and ﬁtting processes use various approaches such as

scattered data interpolations [5, 43, 44] and projections onto the cylindrical coor-

dinates incorporated with a positive Laplacian ﬁeld function [42]. Some methods

attempt an automated ﬁtting process, but most require manual interventions.

Radial basis functions are capable of closely approximated or interpolated

smooth hypersurfaces [45] such as human facial shapes. Some approaches morph

10 Z. Deng, J. Noh

a generic mesh into speciﬁc shapes with scattered data interpolation techniques

based on radial basis functions. The advantages of this approach are as follows.

First, the morph does not require equal numbers of nodes in the involved meshes

since missing points are interpolated [43]. Second, mathematical support ensures

that a morphed mesh approaches the target mesh if appropriate correspondences are

selected [45, 46].

A typical process of 3D volume morphing is as follows. First, biologically

meaningful landmark points are manually selected around the eyes, nose, lips, and

perimeters of both face models. Second, the landmark points deﬁne the coefﬁcients

of the kernel of the radial basis function u sed to morph the volume. Finally, points

in the generic mesh are interpolated using the coefﬁcients computed from the land-

mark points. The success of the morphing depends strongly on the selection of the

landmark points [27,43].

Instead of morphing a face model, a morphable model exploits a pre-constructed

set of face databases to create a person-speciﬁc model [47]. First, a scanning pro-

cess collects a large number of faces to compile a database. This example 3D

face models spans the space of any possible human faces in terms of geometry

and texture. New faces and expressions can be represented as a linear combina-

tion of the examples. Typically, an image of a new person is provided to the sys-

tem; then the system outputs a 3D model of the person that closely matches the

image.

1.7.2 Anthropometry

The generation of individual models using anthropometry

attempts to produce

facial variations where absolute appearance is not important. Kuo et al. [48] pro-

poses a method to synthesize a lateral face from one 2D gray-level image of a frontal

face. A database is ﬁrst constructed, containing facial parameters measured accord-

ing to anthropomorphic deﬁnitions. This d atabase serves as priori knowledge. The

lateral facial parameters are estimated from frontal facial parameters by using min-

imum mean square error (MMSE) estimation rules applied to the database. Speciﬁ-

cally, the depth of one lateral facial parameter is d etermined by the linear combina-

tion of several frontal facial parameters. The 3D generic facial model is then adapted

according to both the frontal plane coordinates extracted from the image and their

estimated depths. Finally, the lateral face is synthesized from the feature data and

texture-mapped.

DeCarlo et al. [49] construct various facial models purely based on anthropom-

etry without assistance from images. This system constructs a new face model in

two steps. The ﬁrst step generates a random set of measurements that character-

ize the face. The form and values of these measurements are computed accord-

ing to face anthropometry (Fig. 1.4). The second step constructs the best surface

The science dedicated to the measurements of the human face.

12 Z. Deng, J. Noh

Fig. 1.5 Real-time tracking is performed without markups on the face using Eyematic Inc. ’s face

tracking system. Real-time animation of the synthesized avatar is achieved based on the 11 tracked

features. Here (a) shows the initial tracking of the face features, (b) shows features are tracked

in real time while the subject is moving, and (c) shows an avatar that mimics the behavior of the

subject.

1.8.1 Snakes and Markings

Snakes, or deformable minimum-energy curves, are used to track intentionally

marked facial features [52] . The recognition of facial featur es with snakes is pri-

marily based on color samples and edge detection. Many systems couple tracked

snakes to underlying muscles mechanisms to drive facial animation [39, 53–56].

Muscle contraction parameters are estimated from the tracked facial displacements

in video sequences.

Tracking errors accumulate over long image sequences. Consequently, a snake

may lose the contour it is attempting to track. In [57], tracking from frame to frame

is done for the features that are relatively easy to track. A relia bility test enables a

reinitialization of a snake when error accumulations occu r.

1.8.2 Optical Flow Tracking

Colored markers painted on the face or lips [9,58–64] are extensively used to aid in

tracking facial expressions or recognizing speech from video sequences. However,

markings on the face are intrusive. Also, reliance on markings restricts the scope of

acquired geom etric infor mation to the marked features. Optical ﬂow [65] and spatio-

temporal normalized correlation measurements [66] perform natural feature track-

ing and therefore obviate the n eed for intentional markings on the face [19,67]. Chai

et al. [68] propose a data-driven technique to translate noisy, low-quality 2D track-

ing signals from video to high-quality 3D facial animations based on a preprocessed

facial motion d atabase. One limitation of this approach is that a p reprocessed facial

motion database is required, and its performance may depend on the match between

prerecorded persons in the database and target face models. Zhang et al. [69] pro-

pose a space-time stereo tracking algorithm to build 3D face models from v ideo

剩余302页未读，继续阅读

qjyu

粉丝: 0
资源: 3

数据驱动的3D面部动画技术与应用

图书 Computer Facial Animation Second Edition 计算机动画第二版

Data-Driven Face Cartoon Stylization

Image-Based 3D FaceModeling System

Data-Driven Decision Making

使用Spring Cloud Alibaba实现分布式消息驱动与Event-Driven架构

Event-Driven Architecture in ARCO Design: An In-depth Overview

EJB Session Bean 和 Message-Driven Bean 的区别与应用场景

Data-Driven Crowd Design and Optimization for Product Innovation

data-driven approaches包含哪些方法

DATA-DRIVEN LEARNING OF NON-AUTONOMOUS SYSTEMS

Spike-driven Transformer

evy-driven随机过程

Knowledge-driven Egocentric Multimodal Activity Recognition

Boss-Driven Process有哪些特点，可能出现什么问题

mvc:annotation-driven 找不到 已经正确引入依赖包了

data-driven prediction of battery cycle life before capacity degradation

randomized subspace newton convex method applied to data-driven sensor selec

multi-driven net on pin

介绍Cuckoo-driven Way

最新资源

mvc:annotation-driven 找不到已经正确引入依赖包了