Institutional members access full text with Ovid®

Share this article on:

Procedures for Determination of Detection Limits: Application to High-performance Liquid Chromatography Analysis of Fat-soluble Vitamins in Human Serum

Browne, Richard W.a; Whitcomb, Brian W.b,c

doi: 10.1097/EDE.0b013e3181ce9a61
Methodologic Issues in Environmental Exposures: Mixtures and Limits of Detection: Original Article

Background: Problems in the analysis of laboratory data commonly arise in epidemiologic studies in which biomarkers subject to lower detection thresholds are used. Various thresholds exist including limit of detection (LOD), limit of quantification (LOQ), and limit of blank (LOB). Choosing appropriate strategies for dealing with data affected by such limits relies on proper understanding of the nature of the detection limit and its determination. In this paper, we demonstrate experimental and statistical procedures generally used for estimating different detection limits according to standard procedures in the context of analysis of fat-soluble vitamins and micronutrients in human serum.

Methods: Fat-soluble vitamins and micronutrients were analyzed by high-performance liquid chromatography with diode array detection. A simulated serum matrix blank was repeatedly analyzed for determination of LOB parametrically by using the observed blank distribution as well as nonparametrically by using ranks. The LOD was determined by combining information regarding the LOB with data from repeated analysis of standard reference materials (SRMs), diluted to low levels; from LOB to 2–3 times LOB. The LOQ was determined experimentally by plotting the observed relative standard deviation (RSD) of SRM replicates compared with the concentration, where the LOQ is the concentration at an RSD of 20%.

Results: Experimental approaches and example statistical procedures are given for determination of LOB, LOD, and LOQ. These quantities are reported for each measured analyte.

Conclusion: For many analyses, there is considerable information available below the LOQ. Epidemiologic studies must understand the nature of these detection limits and how they have been estimated for appropriate treatment of affected data.


From the aDepartment of Biotechnical and Clinical Laboratory Sciences, University at Buffalo, State University of New York, Buffalo, NY; bEpidemiology Branch, Division of Epidemiology, Statistics & Prevention Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, MD; and cDivision of Biostatistics and Epidemiology, School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA.

Submitted 1 October 2008; accepted 31 July 2009; posted 11 March 2010.

Supported in part by the American Chemistry Council and by an intramural contract with the National Institute of Child Health and Human Development (Contract No. ADB-N01-HD-4–3394).

Supplemental digital content is available through direct URL citations in the HTML and PDF versions of this article (

Correspondence: Richard W. Browne, Department of Biotechnical and Clinical Laboratory Sciences, University at Buffalo, State University of New York, 26 Cary Hall, Buffalo, NY 14214. E-mail:

© 2010 Lippincott Williams & Wilkins, Inc.