Three-dimensional imaging has become increasingly prevalent in the fields of plastic surgery and dermatology.1–7 The ability to capture images in three dimensions has opened up new avenues for analyzing patients before and after procedures, and offers the promise of quantitative analysis of postoperative or posttreatment results by providing soft-tissue volumetric data and surface topography with submillimeter precision.3 , 8
Several studies have attempted to quantify volumetric changes related to face lifting,9 fat grafting,10 and fillers.11 However, it is unclear whether these reports are valid because three-dimensional imaging currently lacks standardization. Intuitively, one could imagine how various facial expressions and head positions could mimic the outcome of some cosmetic procedures. For example, a slight smile can lift the cheek,12 and neck extension and mandible protrusion give the impression that patients have undergone facial rejuvenation and neck liposuction.13 Undoubtedly, subtle facial expressions have been inadvertently and/or purposely used to modify the appearance of surgical outcomes in photography.14
Standardization is essential to all forms of photography, regardless of the imaging modality used. In two-dimensional photography, variables such as lighting, background, camera distance, use of makeup, and subject positioning can alter the critical analyses of outcomes, especially for patients undergoing facial rejuvenation.15–18 For this reason, many efforts to establish two-dimensional photographic standards have been reported, concerning patient positioning, camera angles, light settings, and placements.16 , 17 , 19 Furthermore, with the increasing popularity of nonsurgical procedures, it is even more important to standardize three-dimensional imaging because results in these cases can be subtle and easily misrepresented.
To obtain accurate surface measurements and assess volumetric changes by means of three-dimensional imaging, we must, first, understand which facial expressions can cause volume changes and to what degree. Second, we must identify a method to standardize three-dimensional images, such as the use of soft-tissue landmarks to identify discrepancies in facial animation and head position that may affect facial volume. This study sought to evaluate these aims in the mid and lower face, as most facial rejuvenation procedures affect volumes in these areas.
PATIENTS AND METHODS
With institutional review board approval, 20 subjects without a history of facial surgery or facial abnormality were enrolled. Because facial asymmetry at rest and with animation is very common,4 each hemiface was considered independently.
Standardization of Image Capture
Each subject was photographed under standard conditions. Ambient light was provided by fluorescent lamps. Subjects were seated upright in an adjustable-height chair without head support, facing the camera positioned 1 m away from their cheek. Chair height and head position were adjusted to align each subject’s Frankfort horizontal plane with a horizontal line drawn behind them to maintain a consistent camera angle.
Three-dimensional facial images were captured using a VECTRA-3D dual-module camera system (Canfield Scientific, Inc., Fairfield, N.J.) and analyzed using Mirror Digital Photography System version 5.2 (Canfield Scientific). Images were reviewed immediately after capture to ensure absence of acquisition errors.
Volumetric Changes with Facial Expression
The first section of the study evaluated volumetric changes of the mid and lower face with facial expressions and head position changes. After subjects were positioned, they were asked to perform one neutral face and 22 animations:
- Neutral: concentric relation with lips closed, eyes open, and mimetic muscles relaxed.
- Oral movements: smile, frown, lip pursing, puffed cheeks, lip biting, and lower lip depression.
- Mandibular movements: protrusion/retrusion, concentric occlusion, and opening.
- Nasal movements: snarling and alar flaring.
- Periorbital movements: elevation, depression, and wink.
- Head movements: neck flexion/extension, protrusion/retrusion, rotation, and tilt.
- Neck movements: hyoid depression and platysmal strain.
Each subject was trained for 15 minutes and demonstrated full understanding of the instructions before image capture. Facial expressions were instructed to be subtle with no muscle strain, whereas head movements (e.g., flexion, extension, rotation, tilt) were 30 to 45 degrees off the central axis.
All images were processed using Mirror Digital Photography System software. For each subject, the 22 facial expression photographs were superimposed onto their neutral photograph.
Accurate alignment was achieved by surface referencing the three-dimensional structure of the forehead, brow, and nasal root, which remain relatively fixed with facial animations. When evaluating periocular and nasal movements, uninvolved portions of the face were used for registration.
After each facial expression image was appropriately referenced, a subtraction process was used to identify volume changes across the mid and lower face by creating three-dimensional color maps, which were then measured in milliliters. Changes occurring in the upper and lateral cheek were noted as the “malar region,” whereas those occurring along the perioral jawline were noted as the “jowl region.” The malar and jowl “areas of interest” were defined by a perimeter of “zero” change, per the setting available in the Mirror image-analysis software, such that all positive or negative volume changes around the malar or jowl area were included until a perimeter of no volume change from baseline was encountered. Areas of volume change in the cheek area were ascribed to the malar area, and volume changes inferiorly or along the angle of the mandible were attributed to the jowl area.
Volumetric Change with Variable Degrees of Animation
The second section of the study evaluated the relationship between volume change and the degree of expression. Using the same image-acquisition technique, subjects were asked to perform smiling and frowning gestures in varying degrees of animation. Specifically, subjects were instructed to perform the following:
- Neutral: mandibular concentric relation with lips closed, eyes open, and mimetic muscles relaxed.
- Minimal smile: soft smile without showing teeth.
- Moderate smile: hard smile without showing teeth.
- Maximum smile: hard smile showing minimal teeth but without depression of the lower lip.
- Minimal frown: soft frown with lips closed.
- Moderate frown: hard frown with lips sealed closed.
- Maximum frown: hard frown attempting to show lower teeth.
Additional digital landmarks were placed at the oral commissures for the purpose of measuring commissure excursion (Fig. 1, left). Animation images were superimposed onto the neutral image, and “malar region” and “jowl region” volumes were obtained as described above, as in Figure 1, second from left, second from right, and right, and Figure 2. Oral commissure excursion was measured as the distance between oral commissure markers on neutral and animated images. Malar and jowl volume changes were compared to oral commissure excursion.
To assess all patients as a group, volume and distance ratios were used. A subject’s volume change with any degree of smiling or frowning was expressed as a fraction of the volume obtained during maximal smiling or frowning; in a similar manner, excursion could as be expressed as a percentage of the maximum.
The third section of the study analyzed which soft-tissue surface landmarks can be used to identify movements caused by facial expressions. Fourteen standardized soft-tissue surface landmarks were analyzed: glabella, pronasale, subnasale, alar crease, labrale superioris, cheilion, labrale inferioris, pogonion, endocanthion, exocanthion, palpebrae superioris, palpabrae inferioris, laryngeal prominence, and sternal notch. Each of the subject’s facial expression photographs was superimposed onto the subject’s neutral image, and the distances between the listed landmarks was measured in neutral versus animation. If greater than 90 percent of hemifaces (36 of 40) demonstrated more than 1 mm of landmark translation, the landmark was considered to be an identifier of the respective facial animation.
Twenty subjects (10 female and 10 male), aged 20 to 66 years, participated in the study, for a total of 40 hemifaces.
Volumetric Changes with Facial Expression
Sixteen of the 22 animations and head positions tested had a significant effect on mid and/or lower facial volumes, as summarized in Table 1.
The effects of smiling and frowning had the greatest overall effects on facial volume. There were no head positions that affected malar volumes, but most had a significant effect on the jowl region. Alar flaring, brow elevation/depression, eye closure, and hyoid depression did not affect mid or lower facial volumes even though they had local effects around the nose, forehead, and periorbital and cervical region, respectively.
Volumetric Change with Variable Degrees of Expression
During maximal smile, mean excursion of oral commissures was 13.7 ± 3.0 mm. The malar region demonstrated a 17.2 ± 5.7 ml increase in volume, whereas the jowl region demonstrated a 1.7 ± 0.9 ml decrease in volume. During maximal frowning, mean excursion of oral commissures was 7.4 ± 2.3 mm. The malar region demonstrated a 3.7 ± 2.3 ml decrease in volume, whereas the jowl region demonstrated a 5.4 ± 0.9 ml increase in volume.
Volume ratios were calculated for minimal and moderate smiling and frowning, and graphed against the oral commissure excursion ratio. As shown in Figure 3, left, in the act of smiling, there was significant volume augmentation of the malar region with small degrees of oral commissure excursion, such that a quarter smile (excursion ratio, 0.25) could induce a volume change equivalent to 75 percent of a maximum smile.
The jowl also decreases in volume with smiling, but does so in a linear fashion such that a quarter smile yields a volume change equivalent to 25 percent of a maximum smile. Frowning exhibited the opposite, but similarly significant changes in mid and lower facial volume. A quarter frown led to jowl volume change equivalent to 75 percent of a maximum frown and a malar volume change equivalent to 50 percent of a maximum frown. To demonstrate the significant impact of subtle facial expression on absolute volume changes, excursion and volume ratios were multiplied by the average maximal volume changes and graphed in Figure 3, right. Here, we note that a 1-mm excursion during smiling can produce 5 to 10 ml of malar augmentation, whereas 1-mm excursion during frowning can produce 2 to 4 ml of jowl augmentation. Representative three-dimensional color-maps of volume changes associated with variable degrees of animation are shown in Figure 4.
All facial expressions except platysmal straining could be identified with the 14 landmarks studied. The cheilion or oral commissure was the most sensitive landmark, identifying 10 of the 22 facial expressions, followed by the labrale inferioris (nine of 22, lower lip anterior projection point), pogonion (seven of 22), and laryngeal prominence (seven of 22). Excluding platysmal strain and facial expressions that do not affect mid or lower facial volumes, the minimum number of landmarks to standardize both sides of the face was five—glabella, bilateral cheilion, pogonion, and laryngeal prominence—as shown at rest in Figure 5 and in animation in Figure 6.
The cheilion was able to identify all movements of the perioral region and mandible that affected facial volumes, with the exception of jaw opening. The pogonion could identify all movements of the mandible that affected facial volumes, including jaw opening. The glabella was able to identify all nasal movements that affected mid facial volumes and all brow movements, which did not have an effect on mid or lower facial volumes. The sternal notch was able to identify all positions of the head, whereas the laryngeal prominence could identify all positions of the head plus hyoid depression. However, no landmark, including laryngeal prominence, could identify platysmal strain.
There is a relative lack of ability to quantify postoperative changes and conduct rigorous comparison of different rejuvenation techniques. The use of three-dimensional photography represents a valuable development with its ability to quantify volumetric changes with great precision.3 , 7 , 8 However, without proper understanding of the effects of normal facial expression and head position on volumetric changes of the face and a method to standardize these three-dimensional images, the information gained from such technology can be misleading. To address this deficiency, this study quantified the volumetric changes of the mid and lower face associated with facial animations and head position. In addition, this study identified five standardized three-dimensional imaging soft-tissue surface landmarks that can identify discrepancies in facial animation that can impact facial volume.
With the ability of three-dimensional facial photography to objectively calculate volumes, it is even more imperative to standardize patient positioning and facial expressions. Here, we demonstrated that 16 of the 22 evaluated facial expressions and head positions had a significant impact on mid and lower facial volumes. In particular, smiling and frowning had the greatest effect. Smiling increased malar volume by 13.4 ± 3.1 ml and decreased jowl volume by 1.2 ± 0.5 ml, although frowning had the opposite effect, increasing jowl volume by 3.3 ± 1.1 ml and decreasing malar volume by 2.2 ± 0.9 ml. Malar volumes were also most augmented by snarling (5.1 ± 1.5 ml), whereas jowl volumes were most augmented by lower lip depression (3.2 ± 1.2 ml), mandible retrusion (2.8 ± 1.1 ml), neck flexion (1.9 ± 0.7 ml), and platysmal strain (2.6 ± 0.9 ml). The volume changes exhibited are equivalent to or a magnitude larger than one may expect from routine facial rejuvenation procedures, especially nonsurgical procedures.
We also observed that subtle animations with smiling and frowning can elicit volume changes similar to more pronounced expressions. In this study, subjects expressed three degrees of smiling and frowning. Because these levels are arbitrary, we used oral commissure excursion as a measure of the degree of animation. In addition, because patients exhibited different degrees of excursion and volume change, we used ratios as a method of internally controlling all measurements. We found that subtle smile and frowning led to substantial nonlinear increases in malar and jowl volume, respectively. Specifically, a subtle expression with a 0.25 excursion ratio led to volume changes equivalent to 75 percent of a maximum expression. We believe these results are related to isometric contraction of mimetic muscles and their influence on fat compartments. As mimetic muscles contract, they may first do so in a manner that does not lead to translation of the oral commissure. However, these muscles constrict the floor and sidewalls of fat compartments. As a result, the fat compartment bulges in a manner that increases facial volume without visible animation or minimal expression.
However, not all volumetric changes related to facial expression and head positions follow this nonlinear change. During smiling, the jowl volume decreases in a nearly linear fashion. Similarly, the malar region decreases in volume with frowning in a nearly linear fashion. These changes appear to be related to skin stretch and tension resulting from animation that can have a flattening effect over the underlying fat compartments. Although all volumetric changes are important to note, we believe that volume changes related to direct contraction of mimetic muscles and bulging of fat compartments are the most difficult to identify because significant volume changes can be present with minimal expression.
These results underscore the necessity for standardizing three-dimensional facial photography. To that end, we identified five landmarks that are useful in referencing three-dimensional photographs: glabella, pogonion, laryngeal prominence, and bilateral cheilion. The glabella, pogonion, and laryngeal prominence are located on the face, jaw, and neck and are mobile relative to one another. As such, these three reference points can correct for all changes in head and mandible position. The influence of facial animation is assessed by the addition of two reference points at the oral commissures (cheilion). Most facial expressions will lead to translation of the oral commissures (Table 3). Alignment of the oral commissure is essential, as even a 1-mm excursion can lead to 5 to 10 ml of malar augmentation during smiling or 2 to 4 ml of jowl augmentation during frowning.
First, isolated animation of the nares (nasal flaring) and periorbita (eyelid and brow) do not have an impact on mid and lower facial volumes but can have local effects that were not assessed here. Second, none of the landmarks evaluated could identify platysmal straining, which can influence jowl volume. Finally, simultaneous facial expressions may confound and invalidate results.
It is also important to note that some landmarks may be substituted. For instance, the glabella maybe substituted by the nasion. Although the nasion is easier to identify on three-dimensional contours, it does not allow for assessment of all forehead and eyebrow movements, as does the glabella. Similarly, the laryngeal prominence often cannot be identified on women. In this case, one may use the sternal notch, with the caveat being the inability to accurately assess the cervicomental angle because hyoid depression could not be excluded. In principle, all landmarks may be substituted, with the exception of the oral commissure.
Although we included male and female volunteers ranging in age from 20 to 66 years and of varying ethnic background, the large number of patients needed to control for these variables and others (e.g., body mass index) in a statistical analysis was outside the scope of the current study. More research will be needed to further investigate the relationship of these variables to facial volume changes with expression.
Standardization of two-dimensional photography in plastic surgery has been described extensively.14 , 17 , 20 The accuracy of soft-tissue landmark identification on three-dimensional images appears to be as good or better when compared to two-dimensional images,21 supporting the use of soft-tissue landmarks in standardization protocols. The most intuitive initial application for these methods is in assessment of facial rejuvenation using autologous fat grafting or injectable fillers, where three-dimensional imaging has already been used to measure facial volume changes and fat graft retention.22–24 However, none of the studies using three-dimensional imaging for facial change assessment have codified a formal procedure to control for facial expressions. Follow-up results can be easily confounded by different expressions in the before-and-after photographs, as the smile of a happy patient or the frown of a disappointed patient may produce enough volume change to completely overwhelm the magnitude of the injectable’s effect and give falsely positive data regarding its efficacy. Until now, there has been no rigorous way to control for this variable.
We recommend a standard protocol of capturing and comparing preprocedure and postprocedure three-dimensional facial photographs that we have adopted in our practice. In a fashion similar to two-dimensional photography, all three-dimensional images are acquired under standard conditions, with attention to room lighting, camera distance, angle, and alignment of the Frankfort horizontal plane. The patient is asked to maintain a neutral expression, with no head tilt. After images are obtained, preprocedure and postprocedure photographs are superimposed. Optimal alignment is obtained using the three-dimensional structure of the forehead, brow, and nasal root, which is relatively fixed between most facial expressions. Lastly, the five points of reference are used to confirm that the two images are comparable and unaffected by subtle changes in head position or facial expression. Once this is confirmed, appropriate measurements and volumetric data can be obtained.
As the use of three-dimensional photography becomes more prevalent, such a method should be adopted universally to ensure the reliable comparability of photographs. By incorporating it into their practice, surgeons can immediately improve the accuracy of their result evaluation and modify their current technique. Eliminating the variable of facial expression will also build a stronger foundation for future studies comparing different techniques.
Subtle facial expressions may cause significant volume changes in the mid face that can mimic the desired outcomes of facial rejuvenation procedures. Three-dimensional imaging is a highly promising technique in quantifying postoperative results, but for imaging to be meaningful, standardization would be highly beneficial. The five-point referencing system using the glabella, pogonion, laryngeal prominence, and bilateral cheilion allows one to identify changes in head position and facial expression and may aid in the standardization of three-dimensional images.
Patients provided written consent for the use of patients’ images.
1. Lin SJ, Patel N, O’Shaughnessy K, Fine NA. Three-dimensional imaging in measuring facial aesthetic outcomes. Laryngoscope 2008;118:1733–1738.
2. Al-Anezi T, Khambay B, Peng MJ, O’Leary E, Ju X, Ayoub A. A new method for automatic tracking of facial landmarks in 3D motion captured images (4D). Int J Oral Maxillofac Surg. 2013;42:9–18.
3. Ayoub AF, Siebert P, Moos KF, Wray D, Urquhart C, Niblett TB. A vision-based three-dimensional capture system for maxillofacial assessment and surgical planning. Br J Oral Maxillofac Surg. 1998;36:353–357.
4. Giovanoli P, Tzou CH, Ploner M, Frey M. Three-dimensional video-analysis of facial movements in healthy volunteers. Br J Plast Surg. 2003;56:644–652.
5. Honrado CP, Larrabee WF Jr. Update in three-dimensional imaging in facial plastic surgery. Curr Opin Otolaryngol Head Neck Surg. 2004;12:327–331.
6. Okada E. Three-dimensional facial simulations and measurements: Changes of facial contour and units associated with facial expression. J Craniofac Surg. 2001;12:167–174.
7. Weinberg SM, Scott NM, Neiswanger K, Brandon CA, Marazita ML. Digital three-dimensional photogrammetry: Evaluation of anthropometric precision and accuracy using a Genex 3D camera system. Cleft Palate Craniofac J. 2004;41:507–518.
8. Ayoub A, Garrahy A, Hood C, et al. Validation of a vision-based, three-dimensional facial imaging system. Cleft Palate Craniofac J. 2003;40:523–529.
9. Jacono AA, Malone MH, Talei B. Three-dimensional analysis of long-term midface volume change after vertical vector deep-plane rhytidectomy. Aesthet Surg J. 2015;35:491–503.
10. Gerth DJ, King B, Rabach L, Glasgold RA, Glasgold MJ. Long-term volumetric retention of autologous fat grafting processed with closed-membrane filtration. Aesthet Surg J. 2014;34:985–994.
11. Davila AA, Buck DW II, Chopp D, et al. A novel prospective three-dimensional analysis of nasolabial fold augmentation. Aesthet Surg J. 2012;32:488–494.
12. Riml S, Piontke A, Larcher L, Kompatscher P. Quantification of faults resulting from disregard of standardised facial photography. J Plast Reconstr Aesthet Surg. 2011;64:898–901.
13. Sommer DD, Mendelsohn M. Pitfalls of nonstandardized photography in facial plastic surgery patients. Plast Reconstr Surg. 2004;114:10–14.
14. Parker WL, Czerwinski M, Sinno H, Loizides P, Lee C. Objective interpretation of surgical outcomes: Is there a need for standardizing digital images in the plastic surgery literature? Plast Reconstr Surg. 2007;120:1419–1423.
15. Pham AM, Tollefson TT. Objective facial photograph analysis using imaging software. Facial Plast Surg Clin North Am. 2010;18:341–349.
16. DiBernardo BE, Adams RL, Krause J, Fiorillo MA, Gheradini G. Photographic standards in plastic surgery. Plast Reconstr Surg. 1998;102:559–568.
17. Persichetti P, Simone P, Langella M, Marangi GF, Carusi C. Digital photography in plastic surgery: How to achieve reasonable standardization outside a photographic studio. Aesthetic Plast Surg. 2007;31:194–200.
18. Scomação IR, Graf RM, Maluf I Jr, Forte AJ, da Silva Freitas R. Evaluation of photographic variables in patients with indication for facial rejuvenation. Aesthetic Plast Surg. 2013;37:1114–1119.
19. DiSaia JP, Ptak JJ, Achauer BM. Digital photography for the plastic surgeon. Plast Reconstr Surg. 1998;102:569–573.
20. Kinney BM. Neligan PC. Photography in plastic surgery. In: Plastic Surgery. 2013:New York: Elsevier Saunders; 104–123.
21. Zogheib T, Jabobs R, Bornstein MM, et al. Comparison of 3D scanning 2D photography for the identification of facial soft-tissue landmarks. Open Dent J. 2018;12:61–72.
22. Wu R, Yang X, Jin X, et al. Three-dimensional volumetric analysis of 3 fat-processing techniques for facial fat grafting: A randomized clinical trial. JAMA Facial Plast Surg. 2018;20:222–229.
23. Meier JD, Glasgold RA, Glasgold MJ. 3D photography in the objective analysis of volume augmentation including fat augmentation and dermal fillers. Facial Plast Surg Clin North Am. 2011;19:725–735, ix.
©2019American Society of Plastic Surgeons
24. Simunovic F, Schlager S, Montanari M, Iblher N. Prospective 3D analysis of facial soft tissue augmentation with calcium hydroxylapatite. J Cosmet Laser Ther. 2017;19:283–289.