Since 2001, the Centers for Disease Control and Prevention's Public Health Emergency Preparedness cooperative agreement has supported state, territorial, and local public health departments in preparing for and responding to public health emergencies. This conceptual article describes complexities identified and lessons learned in developing community preparedness performance measures for the Centers for Disease Control and Prevention's public health preparedness program. Challenges arose in (a) defining community; (b) measuring meaningful community engagement; and (c) determining a strategy for collecting, aggregating, and analyzing data from diverse state, territorial, and local health departments. This article contributes to prior work describing conceptual challenges in developing standardized measures of performance at the federal level and suggests ways to potentially mitigate general performance measurement challenges as well as measurement complexities specific to community preparedness. It may be informative for those state, territorial, and local health departments currently implementing (or contemplating implementing) community preparedness activities and for individuals more generally engaged in performance measurement.