Analytical validity, clinical validity and clinical utility

Analytical validity, clinical validity and clinical utility are separate bodies of evidence, and the classification regulation for a device type states which of them its special controls demand. In 21 CFR Part 862 the two that are named and required are analytical performance and clinical performance, listed as distinct items within design verification and validation, so a study of one does not discharge the requirement for the other. Evidence that the test improves patient management appears nowhere in these special controls as a required study. Several prognostic classifications instead require labeling that expressly disclaims the management uses a utility claim would support.

The three terms as Part 862 states them

Analytical performance is the ability of the device to measure what it measures. The regulation expresses it through the following characteristics.

• precision
• reproducibility
• accuracy
• limit of detection
• interference
• carryover
• reference intervals

Clinical performance is the link between the measured analyte and the condition. The phrase clinical validity is used directly in § 862.1488(b)(1)(i), which requires study results demonstrating the clinical validity of a lysosomal storage disorder newborn screening test, including information supporting the link between the analyte measured and the condition being screened.¹ The same evidence appears elsewhere as a clinical validation study in § 862.1506(b)(1) and as clinical performance in § 862.1223(b)(1)(i)(A) and § 862.1602(b)(1)(i).¹

Clinical utility, in the sense of evidence that use of the result changes what happens to the patient, has its closest counterpart in § 862.1164(b)(1)(i). That special control requires documentation of studies bridging the efficacy of setmelanotide in the clinical trial population identified by the clinical trial assay to its efficacy in the device intended use population, using the clinical trial samples.¹ The utility rests on the approved therapeutic product labeling, and the device evidence links the device to the trial assay.

Analytical performance required of test systems and instruments

Every classification set out here places analytical studies in its own subparagraphs. Section 862.1488(b)(1) requires the reference interval in the normal newborn population, study results on carryover or drift, and concentrations of the limit of blank, limit of detection and limit of quantitation.¹ Sample concentrations below the limit of quantitation must not be reported by the device.¹ Imprecision must be shown with sample panels from at least three reagent lots and at least three instruments over more than 20 testing days.¹

Section 862.3364(b)(1)(i) requires data on precision, reproducibility, accuracy, limits of detection and interferences for a pharmacogenetic assessment system, including the minimum amount of input DNA that consistently produces accurate results.¹ Interference data must include an assessment of potentially interfering genetic sequences such as pseudogenes and variants proximal to the variant of interest.¹ For a setmelanotide eligibility gene variant detection system, § 862.1164(b)(1)(ii) requires accuracy evaluated at the variant level and the sample level by comparison to validated bidirectional Sanger sequencing or another method FDA determines appropriate.¹ Precision under § 862.1164(b)(1)(iii) uses multiple reagent lots, operators and instruments over multiple days.¹

Instrument classifications carry analytical special controls alone. A high throughput genomic sequence analyzer under § 862.2265 is class II and exempt from premarket notification subject to the limitations in § 862.9. Its special controls address the following.¹

• specimen types validated
• nucleic acid types
• variant types
• read depth
• reproducibility across instruments and sites
• accuracy against well characterized samples from at least two sources

At least one of those reference sources must have sequence generated independently of the manufacturer with respect to technology and analysis.¹ Reagents for molecular diagnostic instrument test systems are class I and exempt under § 862.3800, again subject to § 862.9.¹

Where a clinical study is the named requirement

Screening and prognostic classifications require clinical data in the intended use population, and they specify the design. Section 862.1488(b)(1)(i) requires a clinical validation study using well characterized prospectively or retrospectively obtained specimens from the intended use population.¹ Testing must be performed by operators representative of the intended users, and the design must assess the effects of sample collection and processing.¹ Confirmed positive specimens must have a diagnosis based on confirmatory diagnostic methods or clinically meaningful information regarding the status of the subject.¹ Section 862.1506(b)(1)(i) requires results demonstrating that the analyte measured identifies a population of newborns who should be subject to follow up diagnostic testing for the condition being screened.¹ Predictive value must be demonstrated under § 862.1506(b)(1)(ii), and upper reference intervals must include sufficient samples to calculate 97.5th and 99.5th percentile information.¹

For an acute kidney injury test system, § 862.1220(b)(3) requires clinical data demonstrating positive predictive value, negative predictive value, sensitivity and specificity in the intended use population, submitted with the premarket notification.¹ Section 862.1223(b)(1)(i) requires a clinical study covering the different reported risk categories for chronic kidney disease progression, drawn from multiple intended sample collection sites.¹ The measured outcomes and the length of follow up must be clinically relevant to the progression of the disease.¹ Section 862.1622(b)(1) sets the parallel requirement for liver related disease progression, including information that the clinical criteria for determining whether the target disease is present are appropriate.¹ Section 862.1602(b)(1)(i) requires clinical performance evaluated across multiple intended use sites and broad demographics representative of intended use patients in the United States.¹

Pharmacogenetic classifications state the clinical requirement in terms of association. Section 862.3364(b)(1)(i)(E) requires clinical data generated in intended use patient populations demonstrating the pharmacogenetic association between the genetic variant tested and any clinical claims or therapy-related recommendations associated with that genotype.¹ Several of these provisions allow an alternative approach determined to be appropriate by FDA, which is stated in the regulation itself.¹

Limits the classifications place on management claims

The prognostic classifications permit risk assessment and require labeling that keeps the claim there. Section 862.1602(b)(2)(ii) requires limiting statements that the result is an aid in management and not a replacement for clinical judgement, that it is not to be used to aid in decisions of hospital discharge, and that it is not to be used to aid in decisions of pregnancy delivery.¹ A further limiting statement must say that the test is not intended to inform the healthcare provider about whether changes in immediate treatment are needed.¹ Section 862.1223(a) states that the device is not intended for diagnosis, for serial monitoring of kidney disease progression or for monitoring the effect of any therapeutic product, and § 862.1223(b)(2)(i) carries the same statements into the labeling.¹ Section 862.1622(b)(2)(i) requires a warning that results are not intended to diagnose disease or to monitor the effect of any therapeutic product.¹

Direct to consumer pharmacogenetic labeling is constrained in the same way. Section 862.3364(b)(2)(iii) requires a warning that the patient should not use the test results to stop or change any medication.¹ Section 862.3364(b)(2)(vi) requires a prominent limiting statement that the test provides a preliminary result needing confirmation by an independent pharmacogenetic test, unless design verification and validation demonstrate that the test can be used to make well-informed clinical decisions.¹ That alternative is an analytical showing, and it lifts a labeling statement rather than adding a utility claim.¹

Where one form of evidence will not stand for another

Two provisions decide this. Section 862.1(b) states that a manufacturer submitting a premarket notification cannot show merely that the device is accurately described by the section title and identification provisions of a classification regulation, and must state why the device is substantially equivalent to other devices as required in § 807.87.¹ Matching the generic type analytically is therefore the start of the argument.

Section 862.9 governs the second point. An exemption from premarket notification holds only to the extent that the device has existing or reasonably foreseeable characteristics of commercially distributed devices within the generic type, and for in vitro diagnostics only to the extent that misdiagnosis would not be associated with high morbidity or mortality.¹ A premarket notification is required where the device is intended for a use different from the intended use of a legally marketed device in that generic type.¹ It is also required for devices intended for screening or diagnosis of familial or acquired genetic disorders and for diagnosis, monitoring or screening of neoplastic diseases other than immunohistochemical devices.¹ The same applies to devices for measuring surrogate markers of life-threatening disease and for assessing cardiovascular risk.¹ Devices intended for diabetes management and for near patient testing also require a premarket notification.¹ An exempt analytical instrument that acquires a clinical claim moves into one of these categories.

Design controls and where the studies are documented

The special controls place the studies inside design verification and validation. That location is used in the following provisions.¹

• § 862.1164(b)(1)
• § 862.1223(b)(1)
• § 862.1488(b)(1)
• § 862.1506(b)(1)
• § 862.1602(b)(1)
• § 862.1622(b)(1)
• § 862.3364(b)(1)

Section 862.1220(b)(2) requires the end user training program to be documented as part of the risk management activities performed within the design and development activities under 21 CFR 820.10(c).¹ Section 862.1602(b)(1)(iii) sets a comparable requirement for a licensed practitioner training program on proper use of the device and proper interpretation of results.¹

Questions people also ask

Does analytical performance data satisfy a special control that calls for clinical validity?

The two are listed separately. Section 862.1488(b)(1)(i) requires study results demonstrating clinical validity, including support for the link between the analyte and the condition screened, while paragraphs (b)(1)(ii) through (v) of the same section set the reference interval, carryover, detection limit and imprecision requirements.¹ Each paragraph stands on its own.

Which Part 862 classifications call for predictive value?

Section 862.1220(b)(3) requires positive predictive value, negative predictive value, sensitivity and specificity in the intended use population for an acute kidney injury test system.¹ Section 862.1506(b)(1)(ii) requires predictive value for a muscular dystrophy newborn screening test, demonstrated with prospectively or retrospectively obtained clinical specimens from the intended use population.¹

Can a sequencing instrument exempt from premarket notification support a clinical claim?

The special controls for a high throughput genomic sequence analyzer under § 862.2265 address analytical matters and require the labeling to reference legally marketed reagents and analytical software for use with the system.¹ The exemption operates subject to § 862.9, under which a device intended for a use different from that of the generic type requires a premarket notification.¹

What must the labeling of a risk assessment test say about diagnosis?

Section 862.1223(b)(2)(i)(A) requires a limiting statement that the results are not intended to diagnose any disease or condition.¹ Section 862.1488(b)(2)(i) and § 862.1506(b)(2)(i) require warnings that the screening test is not intended to diagnose the disorders in question, and § 862.1602(b)(2)(ii)(B) requires a statement that the result is not to aid in diagnosis of preeclampsia.¹

Does any Part 862 classification require evidence tied to a drug’s efficacy?

Section 862.1164(b)(1)(i) requires documentation of studies bridging setmelanotide efficacy in the clinical trial population identified by the clinical trial assay to efficacy in the device intended use population.¹ The device is intended to identify patients with obesity who may benefit from treatment in accordance with the approved therapeutic product labeling.¹

Footnotes

1. 21 CFR Part 862, Clinical Chemistry and Clinical Toxicology Devices. https://www.ecfr.gov/current/title-21/chapter-I/subchapter-H/part-862 ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶ ↩¹⁷ ↩¹⁸ ↩¹⁹ ↩²⁰ ↩²¹ ↩²² ↩²³ ↩²⁴ ↩²⁵ ↩²⁶ ↩²⁷ ↩²⁸ ↩²⁹ ↩³⁰ ↩³¹ ↩³² ↩³³ ↩³⁴ ↩³⁵ ↩³⁶ ↩³⁷ ↩³⁸ ↩³⁹ ↩⁴⁰ ↩⁴¹ ↩⁴² ↩⁴³ ↩⁴⁴ ↩⁴⁵ ↩⁴⁶ ↩⁴⁷ ↩⁴⁸ ↩⁴⁹ ↩⁵⁰