Early work on facial motion synthesis, facial motion recognition, and a psychological approach to facial coding:
Yuencheng Lee, Demetri Terzopoulos, and Keith Waters. (1995). Realistic Modeling for Facial Animation. Proc. ACM SIGGRAPH, 55-62.
Irfan A. Essa and Alex P. Pentland. (1997). Coding, Analysis, Interpretation, and Recognition of Facial Expressions. IEEE PAMI, 19, 7, 757-763.
2 comments:
Modeling for Animation:
Okay so this paper gets bonus points for super creepy renders.
My first question would be why they have to use the "generic face mesh" and conform it to the models. There are other geometric simplification methods out there so why did they choose this? I mention this because there are few examples so it is hard to tell if this generic face applies any limitations to variability of the generated models.
The end parts of the paper where eyes, teeth, etc. are discussed at first made me curious again about variability. Especially in the case of the teeth, it seems like everyone would get very similar results. However, once I thought about it, these are not super critical parts of what makes a face a face. After all, teeth and eyes look mostly the same except for color variations.
Facial Expressions:
I didn't find much I liked in this paper. They seemed more concerned with Kalman filtering and discussing FACS than their own stuff here.
One thing I wish they would have discussed more is their motion energy maps. At one point they say other methods do not capture the fact that facial expressions are "application, release, and relaxation." I like how well the energy map seems to capture this information but I never really got a good feel for how they were using it.
Realistic Modeling for Facial Animation:
I was impressed with the authors' attention to detail here...modeling the skin sliding over the bone surface is I think an important thing that is usually ignored. They even modeled the dilation of the pupils due to light levels - wow, didn't expect that.
Alex, conforming to a generic face mesh makes a lot of sense for several reasons.
1) By using the existing mesh points and just readjusting their position based on the spherical displacement texture, you can avoid most of the error/artifacts that were in the original scanning.
2) You know exactly how to place the bones and muscles within the face model, which would be difficult to figure out otherwise.
3) Most importantly, you get a mesh with good topology for facial animation...its hard enough to get a well triangulated mesh automatically, but for facial animation you not only need that, but you also need vertices to be well positioned so that they don't spike out in all the possible facial positions. This would be extremely difficult /nearly impossible to come up with from a single static view of a head.
4) You can take advantage of facial symmetry to reduce errors...ie, maybe the 3d scan had a weird bump or was missing a region on one side of the face..this could be averaged out and fixed by considering the other symmetrical side.
I thought their results looked really good, the expressions actually looked pretty real to me.
Coding, Analysis & Interpretation:
Using optical flow onto a mesh with Kalman filtering seems like the best intuitive way to interpret facial expression from images, it's what I would have used, so I was glad to see itwas the method they used too.
I found their equations and figures difficult to understand, so I'm not totally sure on all the details they were doing..unfortunately.
They definitely needed to show more results. They showed plenty of optical flow fields, but they didn't really show any examples of how this translated into moving their 3D mesh. My guess is that this is because it didn't work for some reason!
Also, I thought that the 3D mesh they used for the face was terrible. Not enough vertices, and poorly triangulated...did not look like it would be capable of being animated properly even manually.
Post a Comment