Laboratory Administration & Management of Pathology Practices

Quality management

Validation of reference intervals and reportable range

Last author update: 30 August 2022
Last staff update: 17 January 2023

Copyright: 2022-2023,, Inc.

PubMed Search: Establishing reference intervals

Duy K. Doan, M.D.
Lewis A. Hassell, M.D.
Page views in 2022: 272
Page views in 2023 to date: 68
Cite this page: Doan DK, Hassell LA. Validation of reference intervals and reportable range. website. Accessed February 3rd, 2023.
Definition / general
Essential features
  • Nearly 80% of physicians' medical decisions are based on information provided by laboratory reports (Am J Clin Pathol 2010;133:180)
  • RIs given in laboratory reports have an important role in aiding the clinician in the interpretation of test results as compared to values for healthy populations (Biochem Med (Zagreb) 2016;26:5)
  • Reference ranges are population specific and are also influenced by instrumentation and testing methods
    • Hence reference ranges used within a given lab may vary from those of another lab using different testing systems and serving different patient populations
  • Reference population is a group of healthy individuals that is served by the laboratory
  • Outlier is an observation that lies an abnormal distance from other values, in a random sample from a population
  • Normal distribution, also called Gaussian distribution, is a probability distribution that is symmetric about the mean, showing that data near the mean are more frequent in occurrence than data far from the mean; the normal distribution data show a bell shaped curve
  • Parametric method is a statistical test that assumes that observed values follow the normal distribution
  • Nonparametric method is a statistical test that makes no specific assumption about the distribution of the data (Bishop: Clinical Chemistry - Principles, Techniques, and Correlations, 9th Edition, 2022)
  • Box-Cox transformation is a mathematical method used to convert nonnormal distribution data into normal distribution data
  • Systematic error is a consistent or proportional difference between the observed and true values
  • Random error is a chance difference between the observed and true values
  • Validation of reference interval is a study to establish that an assay works as intended
    • This applies to non-FDA cleared tests (e.g., laboratory developed methods) and modified FDA approved tests
    • Modifications refer to changes to the assay that are not specified as acceptable by the manufacturer
  • Verification of reference interval is a one time study meant to demonstrate that an assay performs in line with previously established performance characteristics; this applies to unmodified FDA approved or cleared tests
  • Transference refers to adoption of previously established RI by a laboratory
  • Reportable range refers to the span of results within which the method performs acceptably in terms of sensitivity, linearity and reproducibility
Diagrams / tables

Contributed by Duy K. Doan, M.D. and Lewis A. Hassell, M.D.
Normal distribution, serum sodium

Normal distribution, serum sodium

Reportable range

Reportable range

Validation of reportable ranges
Standard values Observed values Means
Replicate 1 Replicate 2 Replicate 3
0 2 0 4 3
200 220 210 200 210
400 400 400 400 400
600 620 630 610 620
800 760 760 790 770
1,000 920 900 880 900

  • In this example:
    • Expected reportable ranges: 0 - 1,000 mg/dL
    • Number of concentration levels: 5
    • Number of replicates: 3
Establishing reference intervals
  • Direct approach: subjects representing the reference population are selected and sampled and the specimens are analyzed to determine the reference intervals
  • Indirect approach: results from specimens collected for routine purposes, such as for screening, diagnostic or monitoring purposes, are used to determine the reference intervals
  • Reference: Clin Chem Lab Med 2018;57:20

Direct approach Indirect approach
  • New data are generated from the population
  • Higher cost to perform
  • Preanalytical and analytical conditions may not match the routine conditions
  • Hard to repeat in different locations
  • Ethical issues
  • Requires basic statistical knowledge and expertise
  • Use the data that are already generated
  • Lower cost to perform
  • Preanalytical and analytical conditions match the routine conditions
  • Easy to repeat in different locations
  • No ethical issues
  • Requires significant statistical knowledge and expertise

Direct approach
1. Selection of reference individuals
  • Determine healthy candidates by medical history, physical examination and laboratory investigations
  • Implement inclusion, exclusion and partitioning criteria
  • Obtain informed consent from participants
  • Take samples from reference individuals; 2 sampling approaches:
    • A priori approach: inclusion of reference individuals is determined before samples are collected
    • A posteriori approach: the definition of reference individuals includes information applied after the samples have been collected
  • Reference: Turk J Biochem 2020;45:1
2. Analysis of the samples
  • Preanalytical considerations:
    • Biological factors: circadian rhythms, fasting status, physical activities
    • Methodological factors: collection techniques, types of additives, specimen handling, transportation, storage
  • Analytical aspects: methods of measurement, equipment, reagents, calibration standards and calculation methods
3. Evaluation
  • Outlier removal: very important step to make RIs reliable
    • Dixton's Q test: simple and effective
      • Q = D/R (D is the absolute difference between the outlier in question and the closest number to it; R is the entire range of the observation)
      • If D/R > 1/3, the outlier should be discarded
      • Drawback of Dixton's Q test: if there is more than one outlier, the test become less sensitive
    • Tukey fence method: more sophisticated
      • Q1 = lower quartile, Q3 = higher quartile; IQR (interquartile range) = Q3 - Q1
      • If number < Q1 – 1.5 IQR or > Q3 + 1.5 IQR, it is an outlier and should be removed
    • RIs calculation: parametric and nonparametric methods
      • Parametric method: only apply to Gaussian distribution data
        • Reference values need to be a normal distribution or transformed to a normal distribution
        • Box-Cox power transformation is often used
        • RIs = mean ± 1.96 SD
      • Nonparametric method: applied to data irrespective of distribution characteristics
        • IFCC recommended method for estimating reference intervals is a nonparametric method that essentially involves simply excluding the lowest and highest 2.5% of reference values
    • Partitioning: stratification by age and gender is important for establishing more accurate RIs
    • Calculate the confidence intervals of the lower and upper RIs; at least 120 samples are needed for analysis
  • References: Biochem Med (Zagreb) 2016;26:5, Am J Clin Pathol 2010;133:180, CLSI: EP28 - Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory, 3rd Edition, 2010

Indirect approach
1. Select the population
  • Data from outpatient settings are preferred; pathological changes in admitted patient populations make the laboratory results less reliable
  • Number of subjects: more is better; 1,000 may be small number, 10,000 can be seen as large number
  • Stability: it is imperative to ensure that the analytical method and the population are stable during the time data are collected
  • Partitioning: all data should be evaluated for cofactors such as age and gender; if standard deviation ratio between subgroups exceeds 1.5, a separated reference interval is highly recommended (Clin Chem 1990;36:265)
  • Exclusions: eliminate results from subjects that are more likely to have diseases based on hospital settings, clinical information and other laboratory results
2. Statistical evaluation
  • Standard parametric and nonparametric statistics can be used in indirect RI studies; outlier removal is very important because standard statistic techniques are significantly affected by these extreme results
  • Hoffmann method is a simple graphical method for estimating reference intervals from routine laboratory data
    • Distribution of results could be represented by a mixture of 2 underlying Gaussian distributions, corresponding to the healthy and diseased subpopulations, with the healthy subpopulation dominating the sample
    • Drawback of the Hoffmann procedure is that it is influenced by the size of the diseased subpopulation (Am J Clin Pathol 2019;151:328)
  • Bhattacharya method is also a graphical method for estimating reference intervals
    • Bhattacharya method has been shown to be less influenced by data not included in the Gaussian distribution compared with the Hoffmann method (Am J Clin Pathol 2019;151:328)
  • Special programs: Arzideh developed a method that is more sophisticated than Hoffman's and Bhattacharya's
    • First, kernel density estimation is used to generate the distribution of the combined data of the nondiseased and diseased subjects
    • Central part of the distribution of all data should represent the nondiseased population
    • Then, Box-Cox transformation is used to produce a truncated Gaussian distribution of the nondiseased values
    • Finally, a goodness-of-fit statistic is applied to find the main part of the data
    • Reference limits are calculated as the 2.5 and 97.5 percentiles of the estimated power normal distribution for the nondiseased values (Clin Chem Lab Med 2018;57:20)
Transference of reference intervals
  • Laboratories are not required to establish their own RIs; most laboratories use previously established RIs
  • If the patient population and methodology are comparable, the RIs can be transferred
  • Like establishing RIs, parametric and nonparametric methods are also used for comparing data sets in transference
  • Nonparametric methods such as Mann-Whitney, Run and Monte Carlo tests are preferred because they can be applied to data irrespective of distribution characteristics (Laboratory Medicine 2006;37:306)
Validation of reference intervals
  • CLSI document C28-A recommends 3 methods for meeting the CLIA specified requirement:
    • Inspection method
      • This is a not a statistical approach
      • If there is no evidence to show that the patient population served by the laboratory differs from the reference population, the use of reference intervals is acceptable
      • Decision can be made by the laboratory medical director
      • This method can be used only if the local population data are unavailable
    • Limited validation
      • 20 samples are obtained from the healthy subjects served by the laboratory
      • If no more than 2 values (10%) fall outside the reference interval, the range is validated
      • If ≥ 3 reference specimens are outside of the reference range, 20 additional reference samples can be obtained; if ≥ 3 of the second reference sample are again out of the reference interval, the laboratory should consider establishing its own reference range (Laboratory Medicine 2006;37:306)
    • Extended validation
      • 60 samples are obtained from healthy subjects served by the laboratory
      • Reference interval of the local population is generated (parametric method, Gaussian distribution)
      • Another 60 samples are randomly collected from the same population; if the reference interval calculated in this study is similar to that of the above, the range is validated
      • This method is not recommended because this is nearly the same as the sample size required for an RI study (Clin Chem Lab Med 2018;57:30)
Validation of reportable range
  • Procedure to confirm the linear relationship between the analytical results of a method and the concentration of the analyte over the range of measurement
  • Materials: patient specimens, commercial kits, standard reference materials, calibrators
  • Number of concentration levels: CLSI recommends at least 4, preferably 5 different concentration levels; 1 minimum value, 1 midpoint value and 1 maximum value should be included
  • Analyze 3 samples at each value in replicate
  • Data analysis: plot observed values against standard values
    • Check visually for linearity
    • Calculate the slope and intercept
    • Coefficient of variance (CV) = SD/mean; CV < 10 is very good, 10 - 20 is good, 20 - 30 is acceptable and CV > 30 is not acceptable
  • Reference: Arch Pathol Lab Med 2014;138:1173

Reference intervals, the basics

Board review style question #1

Reference interval Sample size
Ceruloplasmin (mg/L) 159 - 414 603

A group of investigators from Saudi Arabia was studying the reference intervals of serum ceruloplasmin in adult healthy individuals. The results of their study are shown above. How many of the 603 subjects are expected to have a ceruloplasmin concentration of greater than 454 mg/L?

  1. 3
  2. 6
  3. 9
  4. 12
Board review style answer #1
A. 3. The data show a normal distribution; mean: 286.5. By definition, reference interval = mean ± 1.96 SD → 1 SD = 65. Ceruloplasmin levels of more than 454 will be approximately 2.58 SD from the mean / median (454 - 286.5 = 167.5 = 2.58 x 65 = 2.58 SD). 99% of values will be between (mean - 2.58 SD) and (mean + 2.58 SD), 0.5% values will be < (mean - 2.58 SD) and 0.5% values will be > (mean + 2.58 SD). Number of values > 454 will be 0.5% x 603 (n) = 3.

Comment Here

Reference: Validation of reference intervals and reportable range
Board review style question #2
Which of the following is a disadvantage of using the indirect approach to establish reference intervals?

  1. Calculation method is complicated
  2. More expensive to perform
  3. Need informed consent to use samples
  4. Repeating the evaluation is technically challenging
Board review style answer #2
A. Calculation method is complicated

Comment Here

Reference: Validation of reference intervals and reportable range
Back to top
Image 01 Image 02