Use case · builds on Clinical AI Evaluation
Clinical validation and reader studies for regulatory submissions.
DeepLabel designs and runs clinician reader studies and clinical validation that produce the evidence behind a healthcare AI submission: study design, sample size, a statistical analysis plan, a defined reference standard, blinded independent reads, adjudication and the analysis itself. We design and run the study and produce the clinical evidence. We are not a regulatory consultancy, and you, as the manufacturer, and your regulatory advisers are responsible for the regulatory strategy, the submission and any claims made to a regulator.
- Who it's for
- Healthcare AI and medical device teams preparing evidence for a regulatory submission or a clinical claim.
- The problem
- A credible clinical claim needs a study designed and read properly, not a quick internal benchmark.
- Clinical and data science expertise
- Clinicians matched to the specialty and reading task, with PhD data scientists who design the study and the analysis.
- How quality is assessed
- Blinded independent reads, a defined reference standard, adjudication and pre-specified endpoints and analysis.
NDA and data processing agreement on every project. How we handle your data.
What evidence can a reader study give you?
Standalone performance
How your model performs against a defined reference standard, on a representative or deliberately enriched set of cases.
Reader studies with and without AI
Multi-reader, multi-case (MRMC) designs comparing clinicians reading with and without your model, with a washout period between reads.
Reference standard build
A documented ground truth from expert reads, consensus, or a confirmed outcome where one exists.
Case set design
A representative or enriched case set, with inclusion criteria and case mix agreed and written down.
Agreement and reader variability
How much readers agree, and how much of the result is the model rather than the reader.
Readers matched to intended users
Readers chosen to reflect who will actually use the device and the market it is for. If the device is built for general radiologists, the study uses general radiologists, not only subspecialists.
Review of an existing study
A clinical read of a study you have already run, to show what it supports and where it is thin.
- Study protocol and reading plan
- Sample size and statistical analysis plan
- Defined reference standard
- Blinded independent reads
- Adjudication records
- Pre-specified statistical analysis
- Reader agreement and variability analysis
- A clear study report
Need the reference labels built first? See medical imaging annotation and radiology annotation.
We design and run the study and produce the clinical evidence for your submission. We are not a regulatory consultancy and do not provide regulatory approval, guarantee acceptance by any regulator, or certify clinical safety. You, as the manufacturer, and your regulatory advisers are responsible for the regulatory strategy, the submission and any claims made to a regulator.
How does a reader study run?
- 01
Define the question and reference standard
We agree the clinical question, the intended use and how ground truth is established, with your team and your regulatory advisers.
- 02
Design the study
PhD data scientists and clinicians set the case set, the reading design (including MRMC where it fits), blinding, the sample size, the statistical analysis plan and pre-specified endpoints.
- 03
Recruit and check readers
We match readers to the specialty and task, and to the device's intended users, then check their professional registration and experience before they start.
- 04
Read, adjudicate and analyse
Readers work independently and blinded, disagreements are adjudicated, and we run the pre-specified analysis and report it plainly.
What does a reader study look like?
Reader study · Illustrative example
Each case is read independently by matched readers who are blinded to the model's output and to each other. Where they disagree, a senior reader adjudicates, and the agreed read sets the reference standard.
The model is then measured against that reference standard, using the endpoints agreed before any reading began.
- Reader A · independent read
- Reader B · independent read
- Disagreement highlighted
- Senior adjudication
- Reference label locked
Clinical validation, answered.
Anything else, just ask on a scoping call.
Can you run a reader study for an FDA submission?
Yes. We design and run clinician reader studies end to end, including multi-reader, multi-case (MRMC) designs: the study design, the sample size, the statistical analysis plan and the analysis itself, alongside the reference standard and the blinded reads. We produce the clinical evidence; we do not submit to the FDA, give regulatory advice or guarantee acceptance. The regulatory strategy, the submission and any claims made to a regulator stay with you and your regulatory advisers.
Can you support evidence for FDA 510(k), De Novo, UKCA or CE marking under the MDR?
We support evidence for submissions in the UK, US, Europe and elsewhere. What a given route needs varies and changes over time, so we agree the study design with you and your regulatory advisers rather than assuming the requirements. Our job is to keep the clinical and statistical work rigorous, pre-specified and well documented.
What is the difference between validation and evaluation?
An evaluation tells your team how the model performs so you can improve it. A validation study is designed up front, with a defined reference standard, pre-specified endpoints and blinded reads, to produce evidence that can stand up to outside scrutiny. We do both, and we are clear which one you are buying.
What is an MRMC reader study?
Multi-reader, multi-case: several clinicians each read the same set of cases, often once with your model and once without, with a washout period in between. It helps separate how much of the performance comes from the model from how much comes from the reader.
Who are the readers and how are they checked?
Clinicians matched to the specialty and reading task, and to the device's intended users, from the BiteLabs network of 30,000+ UK and US clinicians. We check professional registration and experience before work begins. Reads are blinded where the design calls for it. DeepLabel is a sister company to BiteLabs.
Does a reader study guarantee regulatory approval?
No. No study can do that. A well-designed study gives you credible clinical evidence to support your own submission and claims. It does not confer approval, certify clinical safety or replace the formal requirements of any regulator.
Related services.
Let's solve your next clinical AI challenge.
Tell us what you're developing and we'll reply the same working day.