Patient-Focused Drug Development: Collecting Comprehensive and Representative Input

Administrative guidance, not enacted legislation. This captured publisher edition preserves the listed document and attachments; later developments are not independently consolidated.

Patient-Focused Drug
Development: Collecting
Comprehensive and
Representative Input
Guidance for Industry, Food and Drug
Administration Staff, and Other
Stakeholders
U.S. Department of Health and Human Services
Food and Drug Administration
Center for Drug Evaluation and Research (CDER)
Center for Biologics Evaluation and Research (CBER)
June 2020
Procedural

Patient-Focused Drug
Development: Collecting
Comprehensive and
Representative Input
Guidance for Industry, Food and Drug
Administration Staff, and Other Stakeholders
Additional copies are available from:
Office of Communications, Division of Drug Information
Center for Drug Evaluation and Research
Food and Drug Administration
10001 New Hampshire Ave., Hillandale Bldg., 4th Floor
Silver Spring, MD 20993-0002
Phone: 855-543-3784 or 301-796-3400; Fax: 301-431-6353
Email: druginfo@fda.hhs.gov
https://www.fda.gov/drugs/guidance-compliance-regulatory-information/guidances-drugs
and/or
Office of Communication, Outreach and Development
Center for Biologics Evaluation and Research
Food and Drug Administration
10903 New Hampshire Ave., Bldg. 71, Room 3128
Silver Spring, MD 20993-0002
Phone: 800-835-4709 or 240-402-8010
Email: ocod@fda.hhs.gov
https://www.fda.gov/vaccines-blood-biologics/guidance-compliance-regulatory-information-biologics/biologics-guidances
U.S. Department of Health and Human Services
Food and Drug Administration
Center for Drug Evaluation and Research (CDER)
Center for Biologics Evaluation and Research (CBER)
June 2020
Procedural

Contains Nonbinding Recommendations
i
TABLE OF CONTENTS
INTRODUCTION .......................................................................................................... 1
Overview of Guidance 1 ................................................................................................. 3
Patient Experience Data ................................................................................................ 4
GENERAL CONSIDERATIONS FOR COLLECTING PATIENT EXPERIENCE
DATA .............................................................................................................................. 7
Overview ......................................................................................................................... 7
Defining the Research Objectives and Questions ........................................................ 8
Who to Collect Information From ................................................................................ 9
Defining the Target Population ....................................................................................... 9
Determining Who Will Be Providing Patient Experience Data....................................... 9
Determining the Study Design and Research Setting ............................................... 10
Sampling Methods .......................................................................................................... 10
Representativeness ......................................................................................................... 15
Sample Size .................................................................................................................... 16
Constructing a Sampling Frame ................................................................................. 17
Additional Considerations ........................................................................................... 18
Missing Data/Non-Response.......................................................................................... 18
Leveraging Existing Data .............................................................................................. 19
OPERATIONALIZING AND STANDARDIZING DATA COLLECTION AND
DATA MANAGEMENT ............................................................................................. 19
Locating Patients/Sites ................................................................................................. 19
Human Subjects Protection ......................................................................................... 19
Sampling Strategy ........................................................................................................ 20
Collecting Data ............................................................................................................. 20
Interviews, Focus Groups, and Facilitated Discussions ............................................... 20
Observational Methods .................................................................................................. 21
Documents...................................................................................................................... 21
Survey Instruments ......................................................................................................... 21
Audiovisual Materials .................................................................................................... 21
Social Media and Verified Patient Communities ........................................................... 22
Recording Information ................................................................................................ 25
Resolving Site/Field Issues ........................................................................................... 25
Data Management ........................................................................................................ 26
Data Standards ............................................................................................................. 26
Monitoring and Quality Assurance ............................................................................ 27
Storing Data .................................................................................................................. 27

Contains Nonbinding Recommendations
ii
Confidentiality .............................................................................................................. 27
Data Analysis ................................................................................................................ 27
CONCLUSIONS........................................................................................................... 28
REFERENCES ............................................................................................................. 29
APPENDICES .............................................................................................................. 33
Appendix 1. Standards and Requirements Pertaining to Submission of Data .................. 33
Appendix 2. Glossary ............................................................................................................... 34

Contains Nonbinding Recommendations
iii
TABLE OF TABLES
Table 1. Methodological Approaches for Collecting Patient Experience Data .............................. 5
Table 2. Types of Sampling, Examples and Some Potential Limitations ..................................... 12
Table 3. Common Methods for Gathering Patient Input Using Online Methods (Including Social
Media) ........................................................................................................................................... 22
TABLE OF FIGURES
Figure 1. General Steps for Conducting Studies About Patient Experience................................... 8
Figure 2. Factors to Consider To Achieve Sufficient Representation .......................................... 16

Contains Nonbinding Recommendations
1
Patient-Focused Drug Development: Collecting Comprehensive and
Representative Input
Guidance for Industry, Food and Drug Administration Staff, and
Other Stakeholders1
This guidance represents the current thinking of the Food and Drug Administration (FDA or Agency) on
this topic. It does not establish any rights for any person and is not binding on FDA or the public. You
can use an alternative approach if it satisfies the requirements of the applicable statutes and regulations.
To discuss an alternative approach, contact the FDA staff responsible for this guidance as listed on the
title page.
INTRODUCTION
This guidance is intended to address both a statutory requirement under the 21st Century Cures
Act of 2016 (hereafter referred to as “Cures Act”) Section 3002 (c) and a commitment made
under the Prescription Drug User Fee Act (PDUFA) VI (authorized under the FDA
Reauthorization Act of 2017 (FDARA), Title I) to issue methodological guidance to support
patient-focused drug development. The guidance to be issued under Cures Act section 3002(c)(1)
shall address:
“Methodological approaches that a person seeking to collect patient experience data for
submission to, and proposed use by, the Secretary in regulatory decision making may use, that
are relevant and objective and ensure that such data are accurate and representative of the
intended population, including methods to collect meaningful patient input throughout the drug
development process and methodological considerations for data collection, reporting,
management, and analysis.”
In addition to this Cures Act provision, FDA committed to publish a series of guidances under
Section I.J of the PDUFA VI Reauthorization Performance Goals for “Enhancing the
Incorporation of the Patient’s Voice in Drug Development and Decision-Making,” and the first
of the series (referred to as “Guidance 1”) covers the same methodological topics as Cures Act
sections 3002(c)(1), with the additional commitment to include standardized nomenclature and
terminologies.
The Cures Act2 defines the term “patient experience data” to include data that:
1 This guidance has been prepared by the Office of New Drugs and the Office of Translational Sciences in the
Center for Drug Evaluation and Research (CDER), in cooperation with the Center for Biologics Evaluation and
Research (CBER), at the Food and Drug Administration.
2 Patient experience data is defined for purposes of this guidance in Title III, Section 3001 of the 21st Century Cures
Act, as amended by section 605 of the Food and Drug Administration Reauthorization Act (FDARA) of 2017.

Contains Nonbinding Recommendations
2
(1) are collected by any persons (including patients, family members and caregivers of
patients, patient advocacy organizations, disease research foundations, researchers and
drug manufacturers); and (2) are intended to provide information about patients’
experiences with a disease or condition, including (A) the ‘impact (including physical
and psychosocial impacts) of such disease or condition or a related therapy or clinical
investigation; and (B) patient preferences with respect to treatment of the disease or
condition.
This expansive definition of “patient experience data” includes a wide range of opportunities for
the collection of information that might be used to inform and provide a greater patient focus in
medical product development.
The range of patient experience data that would fit within the Cures Act statutory definition
includes: patient registry data, natural history study data, patient focus group or meeting reports,
patient survey data, clinical outcome assessment (COA) data collected during clinical trials, and
elicited patient preference data. For the purposes of this guidance, some methodological
considerations apply across the diversity of potential study objectives, but others will apply to
only a subset of these.
This guidance (Guidance 1) presents a general overview of methods and approaches for
collecting patient experience data rather than focusing on methods for a specific, single purpose,
e.g., to support collection of COA data or patient preference information. Specific issues related
to COAs and patient preference information are addressed in other published FDA guidances.3
FDA is publishing a series of guidances intended to facilitate the advancement and use of
systematic approaches to collect and use robust and meaningful patient and caregiver input that
can better inform medical product development and regulatory decision-making.
The topics and questions that each guidance document will address are described below.
Guidance 1: From whom do you get input, and why? How do you collect the information?
Guidance 1 discusses sampling methods that could be used when planning a study to collect
patient input. It also provides a general overview of the relationship between potential research
question(s) and method(s) when deciding from whom to get input (including defining the target
population and development of the sampling strategy).
Guidance 2: What do you ask, and why? How do you ask non-leading questions that are well-
understood by a wide range of patients and other stakeholders?
Guidance 2 will discuss methods for eliciting information from individuals identified in
Guidance 1, gathering information about what aspects of symptoms, impacts of their disease, and
3 Issues related to COAs and patient preference information are addressed in the following guidance for industry:
Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims and Patient
Preference Information—Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device
Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling.

Contains Nonbinding Recommendations
3
other issues are important to patients. It will discuss best practices in conducting qualitative
research and reference-related resources.
Guidance 3: How do you decide what to measure in a clinical trial and select or develop fit-for-
purpose COAs ?
Guidance 3 will address refining the list of concepts of interest important to patients for
measurement. Given that not everything identified as important by patients, caregivers, and
clinicians can be addressed by an investigational treatment or be measured, this guidance will
address issues related to selecting what to measure in a medical product development program
and identification or development of fit-for-purpose COAs to assess outcomes of importance to
patients.
Guidance 4: How do you incorporate a given COA tool or set of measures into a defined clinical
study endpoint? How would you define a meaningful change in that endpoint?
Guidance 4 will address topics related to incorporating COAs into endpoints for regulatory
decision-making including COA-related endpoint development, defining meaningful within-
patient score changes, and collection, analysis, interpretation, and submission of data.
In general, FDA’s guidance documents do not establish legally enforceable responsibilities.
Instead, guidances describe the Agency’s current thinking on a topic and should be viewed only
as recommendations, unless specific regulatory or statutory requirements are cited. The use of
the word should in Agency guidances means that something is suggested or recommended, but
not required.
Overview of Guidance 1
Section I.B discusses patient experience data, how they are collected, and the types of methods
that may apply.
Section II discusses general considerations for collecting patient experience data. It presents a
logical sequence of questions for stakeholders to address to define the research questions of
interest to them, the relevant study population, and considerations for the design of the study.
Both quantitative and qualitative methods are presented to cover the range and diversity of study
aims and approaches for collecting patient experience data. To satisfy the statutory requirement
to address representative data collection in the context of both quantitative and qualitative
methods, Guidance 1 also provides an overview of a range of potentially relevant sampling
approaches of varying complexity.
Section III discusses considerations related to collection and management of the data. Depending
on the type of patient experience data and how the data will be used in medical product
development, different content and formats may be appropriate for submission. When patient
experience data are submitted to the Agency, the submitter should usually include the intended
purpose of the patient experience data being submitted (i.e., how the data are intended for use in
supporting medical product development and regulatory decision-making), and a study report

Contains Nonbinding Recommendations
4
and protocol from the research study, as well as additional information including the mode of
primary data capture (see Appendix 1).
Finally, some of the topics covered in the main body of Guidance 1 are further discussed in a
series of appendices, including standards and requirements related to submission of data
(Appendix 1) and the glossary of terms (Appendix 2). Hypothetical case examples are also
included. To illustrate important concepts presented in this guidance, they are also posted on the
CDER PFDD webpage.4
Patient Experience Data
Patient experience data include the experiences, perspectives, needs, and priorities of patients
related to:
• The signs and symptoms patients experience and how these signs and symptoms affect
their day-to-day functioning and quality of life
• The course of their disease over time, including the effect the disease has on patients’
day-to-day function and quality of life over time, and the changes that patients experience
in their symptoms over time
• Patients’ experience with the treatments for their disease: the symptoms and burdens
related to treatment
• Patients’ views on potential disease or treatment outcomes and how they weigh the
importance of different possible outcomes
• How patients view the impact of the disease, treatment, and outcomes, and their view of
potential tradeoffs between disease outcomes and treatment benefits and risks
The patient experience in a medical product development context incorporates their journey
throughout the course of their disease or condition, including patient views, feelings, needs,
actions, preferences, and interactions (e.g., clinical trials, home life, social life) with respect to
their disease and its treatment (Wolf et al. 2014; McCarthy et al. 2016).
There are different parts of the patient experience to collect and/or measure. These may include:
• Impact of the disease and its treatment on the patient
- Signs/symptoms of disease or condition
- Chief complaints (most bothersome signs/symptoms)
- Burden of living with or managing a disease or condition (including effect of the
disease or condition on activities of daily living and functioning)
4 https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-
series-enhancing-incorporation-patients-voice-medical.

Contains Nonbinding Recommendations
5
- Burden of treatment (including the effect of treatment on activities of daily living
and functioning)
- Burden of participating in clinical studies
• Patients’ perspectives about potential and current treatments
- Expectations of benefits
- Tolerance for harms or risks
- Acceptable tradeoffs of benefits and risks (i.e., patient preference)
- Attitudes towards uncertainty
• Views on unmet medical needs and available treatment options
• Enhanced understanding of the natural history of the disease or condition, including
progression, severity, and chronicity
Patient experience data may be collected throughout medical product development, beginning at
the launch of a discovery program, or may be independent of any specific medical product
development program. Data can be collected in a variety of settings, including clinical trials,
observational studies, advisory boards, public meetings (such as Patient-Focused Drug
Development (PFDD) meetings), and other novel settings (e.g., social media, online patient
communities).
Depending on study or research goals and the research questions, qualitative, quantitative, or
mixed methods may be appropriate for collection of patient experience data. Table 1 presents
some basic distinguishing features of qualitative, quantitative, and mixed methods research
approaches.
As noted earlier, Section II provides an overview of these methodological approaches and
discusses factors to consider when selecting an appropriate methodological approach.
Table 1. Methodological Approaches for Collecting Patient Experience Data
Research Approaches
Study
Element Qualitative Research Quantitative Research Mixed Methods Research
Common
research
objectives
Description,
understanding and
exploration/confirm
ation
Numerical
description, causal
explanation and
prediction
• Multiple objectives;
provide complex and
fuller explanation and
understanding;
understand multiple
perspectives

Contains Nonbinding Recommendations
6
Research Approaches
Study
Element Qualitative Research Quantitative Research Mixed Methods Research
Common
study
characteristics
Attempt to
understand
participant views,
perspectives and
meanings of
concepts; study
groups and
individuals in
natural or
controlled settings
Study behavior
under controlled
conditions; isolate
the causal effect of
single variables
• Study multiple contexts,
perspectives or
conditions; study
multiple factors as they
operate together
Data
collection
Qualitative data
such as in-depth
interviews,
participant
observations, field
notes and open-
ended questions
Quantitative data
generated using
structured data
collection
instruments
• Both qualitative and
quantitative data
Data analysis Use descriptive
analysis to identify
patterns, themes
and holistic features
of qualitative data
Identify statistical
relationships
among variables
• Quantitative and
qualitative analysis,
used separately and in
combination
Examples A group of patients
are interviewed to
describe their
experience with the
disease or condition
A group of
patients are
surveyed about
their experience
with the disease or
condition with a
survey instrument
that uses closed-
ended questions
with distinct
response options
to quantify
information
• A group of patients are
given a survey
instrument with both
open-ended and closed-
ended questions to
describe their experience
with the disease or
condition
• A group of patients are
first surveyed about
their experience with the
disease or condition
with a survey instrument
and then are later
interviewed to obtain
additional information

Contains Nonbinding Recommendations
7
GENERAL CONSIDERATIONS FOR COLLECTING PATIENT EXPERIENCE
DATA
Overview
The selection of people from whom to collect input depends upon the specific questions and
issues to be addressed (Figure 1). Thus, the selection process starts by considering the research
question: what are the specific objectives to be addressed by collecting patient input? Are the
objectives focused on understanding the most burdensome symptoms, the impact of current
therapies, patients’ tolerance for risk, or the course of the disease over time? Each of these may
require different approaches to patient selection and input collection.
Some factors that are important to consider when selecting a research approach include:
• Research goals or questions to be addressed
• Target population and availability of people in that population
• Most valuable information that should be generated through the study to achieve your
research goals or answer your research questions
• Feasibility of leveraging existing literature and data (See Section II.F.2)
• Expected short-term and long-term impacts of the information you intend to gather
through the study
• Amount of time to conduct your studies
• Study budget (including staffing, travel time, facilities costs, remuneration, data storage,
management, and analysis)

Contains Nonbinding Recommendations
8
Figure 1. General Steps for Conducting Studies About Patient Experience
Defining the Research Objectives and Questions
When formulating your research objectives, you should be specific. A broader research goal can
be outlined by a set of specific research objectives, each reflecting a component of the broader
goal. For example, the broader goal may be to evaluate how a treatment affects a particular
symptom as well as the burden the administration of that treatment places on the patient. In this
example each of these (the impact on the symptom, the burden on the patient) may be individual
objectives of the research. Your research questions will further refine the research objectives into
answerable questions. Your research objectives and questions should inform which
methodological approaches you use in your research.
You should examine previously conducted studies and other relevant research literature and
consult subject matter experts (e.g., clinicians, social scientists, patients, advocates, caregivers)
to help determine the most appropriate questions and to decide:
• Which methods are better suited to meet your research goals and provide evidence to
support your research questions
• The design of study materials (e.g., study protocol, interview guides, coding dictionary;
refer to Guidance 2 for more details).
8. Report study results
7. Analyze and interpret the data
6. Collect the data and perform data management tasks
5. Construct the study sample
4. Determine which analyses are required to achieve the research objectives
3. Determine the study design and research setting, including instruments
2. Determine the target patient population from whom to collect information
1. Define the research objective(s) and questions

Contains Nonbinding Recommendations
9
Who to Collect Information From
Defining the Target Population
The group of patients whose experience you wish to learn about is the target population. It is
important that the study enrollment criteria (inclusion and exclusion criteria) are carefully
designed to properly select the target patient population. For example, if there is interest in
understanding the perspective of patients newly diagnosed with a particular disease, defining this
population carefully (e.g., what are the criteria for “newly diagnosed?”, what diagnostic criteria
are necessary to ensure patients have the target condition?) is important. If the target population
is intended to represent the broad range of patients with a particular disease, the enrollment
criteria and sampling plan should ensure that study enrollment is not limited to patients with
recent onset, or longer-duration of disease, but is a sample of the full spectrum of patients with
the disease.
Example: If you wish to understand the views and preferences of all individuals with
Parkinson’s disease (PD) in the world, the target population could be defined as the set of all
individuals who have been diagnosed with PD. If you are interested in a subset of PD patients,
such as patients diagnosed within the last 5 years, the target population could be restricted
accordingly. Note, however, that patient experience in a restricted target population might be
different from that in a broader target population. For example, PD patients diagnosed within the
last 5 years may have different views and preferences than the set of all PD patients, which
includes those living with PD for over 10 years.
Determining Who Will Be Providing Patient Experience Data
For the collection of patient experience data, FDA recommends direct reports from patients,
unless they are unable to reliably report on the concept of interest (e.g., young children,
individuals with cognitive problems). The ability to provide self-report also depends on the
methods used to elicit the input and the complexity of the concepts. Methods of data collection
can be tailored to specific situations and patient groups, for example by eliciting patient
experience through play and drawings from young children. When collection of direct patient
experience is limited, valuable, but distinct, information may still be obtained from caregivers,
patient advocates, clinicians, and others.
In situations where direct report from the patient is limited (e.g., patients with cognitive
limitations), alternatives to patient-reported outcome (PRO) measures may be needed. Such
measures can provide useful information on observable patient experience, such as signs of
disease or condition and functioning (FDA, 2015). However, research has repeatedly
demonstrated lack of agreement, often markedly so, on the severity and frequency of signs and
symptoms when patient-reported and non patient-reported measures are compared.
Factors to consider when deciding whether and how patient self-report may be used include:
• Level of cognitive development, function, or mental status

Contains Nonbinding Recommendations
10
• Language skills
• Numeracy skills (e.g., these are needed if a quantitative scale is to be assessed)
• Health literacy (including basic literacy)
• Health state
• Co-morbidities
Prior to study initiation, it is important to set the criteria for determining who the reporter is. For
example:
• To what extent can patients reliably and validly self-report?
• What are the scenarios under which multiple reporters may be required?
• How might this change over the duration of the study?
The reporter should be recorded for each individual in your study and, if necessary, at each time
point of the report.
When there is the potential for patient report to be limited or compromised by any of the factors
noted above, the researcher should consider engaging with subject matter experts (e.g.,
clinicians, social scientists, patients, advocates, caregivers) in the specific disease area of interest
to determine the appropriateness of self-report in the target population.
Determining the Study Design and Research Setting
The design of the patient experience study is largely determined by the study objectives (as well
as resources and setting) and includes determination of a number of key features including:
• Study type (e.g., a clinical trial/study, observational study, survey study)
• Methodological approach (e.g., qualitative, quantitative, or mixed research methods)
• Patient selection/sampling method
• Sample size
• Methods for diagnosis determination (e.g. self-report, clinician report, other source)
• Subgroups – whether special considerations are needed for subgroups of interest
Sampling Methods
An initial step is the approach to selecting the patient population that will participate in the study,
referred to as the sampling scheme. As discussed previously, the research objective defines the
appropriate population to be included (factors such as disease, stage of disease, treatment and
treatment complications, disease status, disease complications), and selection of the appropriate
population is essential to getting information relevant to addressing the research objectives.
There are many sampling approaches, and they vary in complexity. Which approach you use
may depend on your research objectives and resource constraints. You should engage with
subject matter experts (e.g., statisticians, psychometricians) when determining which sampling
methods to use.

Contains Nonbinding Recommendations
11
Table 2 provides a non-exhaustive listing of possible sampling approaches that can be used to
obtain patient experience data. They can be classified under two broad categories: probability
and non-probability methods. Studies using probability sampling use some version of random
sampling (see Table 2) to select from a larger population to create a sample, for example to select
who will participate in a survey. When probability sampling is used, the patients who participate
in the study and the results of the study are more likely to reflect those in the target population.
Non-probability sampling uses non-random processes to select the study sample, such that the
selected sample may not be representative of the target population. When choosing a method, it
is important to consider resource or other constraints and carefully consider how the sample
selected may impact the interpretation of the patient experience data.
Probability sampling includes the following important components:
• A well-defined target population
• Listing of individuals within the target population
• Use of a random number generator
The listing of individuals in the target population is referred to as the sampling frame. Ideally,
the sampling frame should enumerate all individuals in the target population. Where this is not
feasible, the sampling frame should be representative of the target population. A random number
generator can be used to randomly sample individuals from the sampling frame, which in
principle produces a sample of patients whose experiences can be interpreted as being
representative of the target population.
Example: Suppose the target population consists of 100,000 patients with Parkinson’s Disease
(PD) alive in the United States, and each individual is enumerated in a sampling frame with a
label of 1 to 100,000. Using a random number generator, a sample of 2,000 patients is randomly
selected from among the 100,000 patients, and their experiences are ascertained. Random
sampling provides a mechanism for extending statements made about patient experience based
on the individuals in the sample to the entire PD population. In practice, additional steps such as
stratification may be needed to induce a sample having the desired characteristics.
It is beyond the scope of this document to discuss sampling schemes in greater detail. However,
more in-depth discussions with respect to advantages and disadvantages can be found in the
literature (Rothenberg 1995; Heckathorn 1997; Korn and Graubard 1999; Fricker 2008; Groves
et al. 2009; Levy and Lemeshow 2013; Valliant et al. 2013; Johnson and Christensen 2014;
Johnson 2015).
As noted, the appropriate sampling scheme is that which enables you to address your research
objectives and can be implemented within the scope of your resource constraints.

Contains Nonbinding Recommendations
12
Table 2. Types of Sampling, Examples and Some Potential Limitations
Types of Sampling Selection Strategy Examples Potential
Limitations
Probability Sampling
Simple random
sampling
A sample drawn by a
procedure in which every
member of the
population has an equal
chance of being selected.
A simple random sample is
taken from a population of
patients admitted to a
hospital in the first six
months of 2015.
• Can be expensive
or infeasible to
conduct.
• Samples can fail
to reflect the
heterogeneity in the
target population.
Stratified random
sampling
A sample drawn by
dividing the population
into mutually exclusive
groups and then selecting
a random sample from
within each group.
Population of patients with
a disease are stratified by
disease subtype and sex,
and a random sample is
taken within each
combination (stratum).
• Requires the
stratification factors
to be known for
each individual in
the population.
Multiplicity
sampling
A sample drawn by first
taking a probability
sample from the target
population followed by
drawing a sample from
the set of individuals who
belong to the network of
those initially sampled.
Approach used to
oversample those with rare
diseases when conducting a
household survey by
including a question about
people with a rare disease
who previously lived in the
household.
• The initial
probability
sampling phase
may not be feasible.
• Relies on the
initial respondents
to identify members
in their network.
Cluster sampling A sample drawn by
which clusters (i.e., a
collective type of unit
that includes multiple
elements, such as clinical
sites in different
geographic areas) are
randomly selected and
either complete- or sub-
sampling of individuals
within the selected
clusters are taken.
A probability sample of
hospitals in a state is taken,
from which a probability
sample of patients from
each hospital is taken.
• Often requires
information about
cluster size as
selection
probabilities can
depend on such
information.
• Units within
cluster tend to be
homogeneous.

Contains Nonbinding Recommendations
13
Types of Sampling Selection Strategy Examples Potential
Limitations
Multistage
probability
sampling
Generalization of cluster
sampling to include
multiple levels/stages of
cluster sampling.
CDC Medical Monitoring
Project.
• Stage 1, a probability
sample of states.
• Stage 2, a probability
sample of facilities within
each sampled state.
• State 3, a probability
sample of HIV patients
from each sampled facility.
• Often requires
information about
cluster size as
selection
probabilities can
depend on such
information.
• Units within a
cluster tend to be
homogeneous.
Non-Probability Sampling
Clinical trials A sample that consists of
patients who volunteer to
participate in a clinical
trial.
Patients with iron
deficiency anemia are
recruited to participate in a
clinical study that
compares the efficacy of an
experimental therapy
against a standard of care.
• Trial results may
not be generalizable
to the population of
all iron deficiency
anemia patients for
whom the therapy
is indicated.
Convenience
sampling
A sample drawn by
including people who are
available, volunteer, or
can be easily recruited.
Patients who can travel to
attend PFDD meetings.
• Study results may
not be generalizable
to the target
population.
Purposive
sampling
A sample drawn by
which the researcher
specifies the
characteristics of the
target population and
locates individuals with
those characteristics.
Researcher is interested in
studying adult females with
acne and selects their
sample based on these
characteristics (i.e., adult
females with acne).
• Study results may
not be generalizable
to the target
population.

Contains Nonbinding Recommendations
14
Types of Sampling Selection Strategy Examples Potential
Limitations
Quota sampling A non-random sample
drawn by which the
researcher determines the
appropriate sample sizes
or quotas for the groups
identified as important.
Researcher chooses their
sample to consist of 45%
females and 55% males to
maintain the correct
proportions representative
of the target population.
• Study results may
not be generalizable
to the target
population.
Respondent-driven
sampling
Similar to snowball
sampling (see the next
type in this list). The
chain of referrals is often
longer than snowball
sampling and, under
certain conditions,
estimates can be
generalizable to target
population.
A convenience sample of
individuals with substance
use disorders is recruited.
Each individual in this
initial sample is provided a
fixed number of coupons
that he/she uses to recruit
others in his/her network.
The second set of
individuals recruited via
coupons by the first set of
individuals are also given a
fixed number of coupons
that they use to recruit
individuals in their
network. This is repeated
for a fixed number of
cycles after which
recruitment terminates. The
coupons serve as financial
incentives for the recruited
to recruit others in the
network.
• Requires long
recruitment chain
and socially-
networked
population.
• Study results may
not be generalizable
to the target
population unless
assumptions, which
are not verifiable,
are valid.
Snowball sampling A sample drawn by
which each research
participant is asked to
identify other potential
research participants. The
initial sample of
individuals is often
obtained via non-
probability sampling;
subsequent samples are
obtained by chained
referrals from the
previous sample.
Patients with sickle cell
disease participate in focus
groups to discuss
symptoms of the disease
and impacts of the
medications taken. Focus
group participants are
asked to identify other
people they know with
sickle cell disease who
may be potential research
participants so study staff
can invite them to join the
research study.
• Study results may
not be generalizable
to the target
population.

Contains Nonbinding Recommendations
15
Types of Sampling Selection Strategy Examples Potential
Limitations
Web-based
sampling
A sample drawn by using
the web as the contact
mode; can involve
multiple sampling
strategies (e.g.,
systematic sampling,
multiplicity sampling,
list-based, entertainment
polls, un-restricted self-
selected surveys,
volunteer (opt-in) panel).
Researcher selects patients
from a web-panelist (e.g.,
online polling panel) to
include in study.
• Limited by pre-
registered panelists.
• Study results may
not be generalizable
to the target
population.
• Potential response
bias* (e.g.,
measurement error,
misclassification).
* Although response bias such as measurement error or misclassification is listed as a limitation under web-based
sampling, it applies to all types of study.
Representativeness
When studying patient experience, it is important to obtain patient experience data that are not
only relevant, objective, and accurate, but also representative of the target population. Sufficient
representation may depend on the characteristics of the target population, the disease or
condition under study, and the intended use of study results. In this document, the term
representative or representativeness can be interpreted in the following ways.
(1) A sample is representative of the target population if statements made about patient
experience based on data from the sample of patients are generalizable to the target
population. In principle, probability sampling schemes enable you to obtain such
representative samples and often arise in the context of quantitative studies. However, if
there are subgroups of patients from the target population that are not adequately
represented in your study sample, your ability to generalize your research findings to the
target population may be limited, even if you use a probability sampling scheme.
(2) A sample is representative of the target population to the extent that patients in the study
sample consist of individuals with various characteristics that approximate the
heterogeneity of characteristics in the target population. If your sample does not reflect
the broad range of patient characteristics of the target population, patient experience
results may not be representative of the target population. Thus, statements made about
patient experience based on data from the study are not necessarily generalizable to the
target population. Whether this is acceptable depends on the research objectives. If your
research objective is to generate hypotheses or tools to collect patient experience data,
this extent of representativeness may be sufficient. It also may be possible to attain
generalizability through weighting to account for the over-sampling (or under-sampling)
of certain subpopulations if the sample is obtained with known sampling probabilities.
Regardless of how individuals are selected for the study, it is important to ensure that patients in
the study sample represent the target population, particularly with respect to the attributes that

Contains Nonbinding Recommendations
16
are associated with the endpoints of interest. Figure 2 provides some guidance regarding factors
to consider to achieve sufficient representation.
Figure 2. Factors to Consider to Achieve Sufficient Representation
Sample Size
Important to the design of any study is sample size determination. Sample size estimates are
driven by:
• Research objectives
• Type of outcomes/endpoints under consideration
• Study design, including whether the study is quantitative or qualitative in nature
• Planned methods of analysis
Insufficient sample size may produce unreliable or imprecise results. FDA recommends that if
the sample size is limited due to practical considerations (e.g., rare diseases), the research
objectives and/or methods should be adjusted accordingly, and any limitations should be noted in
the study report. Some practical considerations when determining sample size include:
• Drop-out rates or non-response. Sample size may need to be adjusted upward to
account for drop-outs or non-response.
• Subpopulations. In cases where there is also interest in one or more subpopulations
within the target population, sample size should be determined to ensure there is

Contains Nonbinding Recommendations
17
sufficient information to also learn about those subpopulations. Subgroup5 analyses of
interest should be pre-specified at the study design stage whenever possible. Care should
be taken with the number of subgroups being proposed for analysis and inference.
Subgroups of interest may be based on reporter type (e.g., patients versus primary
caregivers) and/or socioeconomic, demographic, cultural, linguistic, clinical or other
factors pertinent to the disease/condition of interest.
For qualitative studies, sample size determination is often based on concept saturation, which
means the point at which no new important concepts relevant to the research question are
emerging from iterative rounds of interviews and the group of patients thus far recruited appears
to be representative (Sandelowski 1995; Francis et al. 2010; Dworkin 2012).
For quantitative studies, sample size calculations for different sampling types, study types and
data types can be found in the literature (Thompson 1987; Chow et al. 2008; Levy and
Lemeshow 2013). For complex designs where sample size formulae are intractable to obtain,
simulation could be used.
Constructing a Sampling Frame
The existence of a sampling frame facilitates probability sampling. Without a sampling frame, it
is difficult to randomly sample from the target population. To the extent that disease registries
are inclusive and regularly updated, they may provide a natural sampling frame. The scope of
registries may vary, with some defined at the regional level, sub-regional (e.g., state, province)
level, or some level local to an organization such as a hospital or a chain of hospitals owned by a
particular organization or part of a network. For an example of patient registries and related
information, see https://ncats.nih.gov/radar. The following example discusses potential use of a
clinician registry as a sampling frame.
Example: In the United States, physician listings such as the AMA Masterfile or state
licensing board files have the potential to be used to create a sampling frame for the target
population in the sense that a sample of physicians from these sources may be used to elicit
members of the target population, for example, patients who have been diagnosed with
diabetes. Physicians who treat patients with diabetes may be sampled, and for each sampled
physician, his or her patients are sampled.
It is important to note that unless all relevant physicians are sampled, and all patients under the
care of each sampled physician are identified, the resulting sampling frame may exhibit
undercoverage in the sense that not every member of the target population is listed in the
frame.
5 A subgroup is a part of the sample (e.g., in a clinical trial you look at the results only in the women in the trial, that
is a subgroup analysis). A subpopulation is part of the whole population (e.g., if you only sample from women you
are sampling from a subpopulation). We draw an inference from a subgroup to make generalizations about the
subpopulation.

Contains Nonbinding Recommendations
18
Undercoverage, mentioned in the example above, occurs when a proportion of members of the
target population is not included in the sampling frame. Undercoverage may not be problematic
if members excluded from the frame could be reasonably viewed as not being substantially
different from those within the frame. Problems occur when we do not know if patients in the
frame are similar to those not in the frame, for example where patient experience data is
collected using only an online community. Attempts to augment the sample from other patient
populations found using some of the sampling methods mentioned in the table can help reduce
undercoverage.
Additional Considerations
Missing Data/Non-Response
Missing data are common in most types of studies. With respect to representativeness,
individuals who were selected for the study may decline to participate in the study (unit non-
response), stop participating during the study (dropout), or may decline to answer some
questions (item non-response) after consenting to participate in the study. Investigators should
anticipate the types of missing data that are likely to occur given the study design, logistics, and
the particular patient experience data that are being collected.
The March 2019 Clinical Trials Transformation Initiative workshop on “Enhancing the
Incorporation of Patient Perspectives in Clinical Trials”6 included discussion of how to
collaboratively establish plans (in the study protocols) delineating strategies to minimize missing
data, and where missingness cannot be avoided, to collect or determine the reasons for the
missingness. Brick (2013), Calinescu et al. (2012), Levy and Lemeshow (2013), and Schouten et
al. (2013) discuss design strategies for improving the response rate in the context of surveys.
O'Neill and Temple (2012) and The National Research Council (2010) discuss design strategies
for the prevention of missing data in clinical trials.
To help better understand the extent and impact of missing data FDA recommends the following:
• Provide a table summary of missing data; useful information includes frequencies,
percentages, stratification by important subgroups, and reasons for missingness.
• For longitudinal data, summarize missingness stratified by assessment visits or time
points.
In addition, methods for handling missing data in analysis should be addressed in the protocol.
See The National Research Council (2010) and International Council on Harmonisation (ICH)
E9(R1) for discussion of those methods.
6 https://www.ctti-clinicaltrials.org/news/recording-now-available-ctti-and-fda-hold-workshop-incorporating-
patient-perspectives-clinical

Contains Nonbinding Recommendations
19
Leveraging Existing Data
FDA encourages collaboration among multiple stakeholders and the use of methods to combine
and leverage existing data (e.g., national registry data, archival databases, published literature) to
fit the specific needs of the research questions and study goals. It is important to note that if you
decide to explore the use of existing data, you should demonstrate the representativeness,
methodological rigor of the data collection method and data integrity as outlined in Section III of
this guidance.
OPERATIONALIZING AND STANDARDIZING DATA COLLECTION AND
DATA MANAGEMENT
Research to collect patient experience data that can be used in regulatory decision-making not
only requires the study to be well-designed but also that the methods and standards for data
collection and analysis meet Agency expectations for quality. Section III of this guidance
provides an overview of standard approaches to consider regarding data collection and data
management.
You should standardize data collection activities and methods to support the best possible data
quality.
Locating Patients/Sites
A critical step in the process of data collection is to identify the appropriate sample and/or sites
(e.g., medical practice/center, academic institution, research consortia) to study. Including
patients from diverse sites helps to provide a representative patient sample. FDA acknowledges
that for some conditions (e.g., rare diseases) a limited number of sites may be available for
clinical trials and additional research.
Human Subjects Protection
Research involving access to patient information or directly engaging with patients requires
careful consideration of federal, state, and local laws, and institutional polices for the protection
of human subjects. Because this guidance focuses on sampling methods for collecting patient
experience data through a variety of research contexts (including, but not limited to, clinical
trials, observational studies, advisory boards, public meetings) a full discussion of the laws that
may apply to these collection methods is beyond its scope. Research subject to FDA regulations
must satisfy the requirements for informed consent at 21 CFR part 50 and the Institutional
Review Board (IRB) requirements at 21 CFR part 56.7,8 Research supported or conducted by the
7 Details on 21 CFR part 50 can be found at
https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=50
8 Details on 21 CFR part 56 can be found at
https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=56

Contains Nonbinding Recommendations
20
Department of Health and Human Services must satisfy the requirements at 45 CFR part 46.9
FDA recommends that researchers work with their IRBs and Health Insurance Portability and
Accountability Act Privacy Boards to determine which laws may apply.
FDA also recommends that research involving patient information be conducted in accordance
with the principles of good clinical practice, including the ICH Guidelines.
Sampling Strategy
Of importance to the data collection process is the determination of a strategy for the sampling of
patients or sites. Refer to Section II.D.1 on the different types of sampling.
Collecting Data
You should consider which data collection approach is most appropriate for their research
objective. Data collection methods can include:
• Interviews
• Focus groups
• Facilitated discussions at meetings
• Observational methods
• Documents (e.g., medical charts)
• Survey instruments
• Audiovisual materials
• Social media and verified patient communities
• Digital health technologies
Each of these data collection methods generates different types of data, each of which has its
own advantages and limitations. Additional details, including potential advantages and
limitations of each method, are discussed below.
Interviews, Focus Groups, and Facilitated Discussions
Different interview types are used to collect patient experience data, including one-on-one
interviews (semi-structured, structured, or unstructured) or group interviews (focus groups,
facilitated discussions at patient meetings). The method of interviewing (e.g., in person,
telephone, or video chat) may vary depending on the goals of the interview. For example, if
visual cues are important for the context of the research objective, an appropriate data collection
method may be face-to-face interviews either in-person or by video chat instead of telephone
interviews. Further details and considerations regarding the different interview methods will be
provided in future guidance in this series.
9 Details on 45 CFR part 46 can be found at https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-
46/index.html

Contains Nonbinding Recommendations
21
Observational Methods
Observation can be a tool to collect patient experience data, and can include the observation of
the interactions of a participant in particular setting, activity, or behavior (Creswell 2013).
Observations are helpful in situations for individuals who have barriers to communicating their
thoughts orally or in writing. Additionally, observations of individuals or groups often can be
made to supplement interviews (individual or group) by documenting cues from the environment
and behaviors such as facial expressions, gestures, tone of voice, and other non-verbal indicators.
Documents
Various types of documents can be used to collect patient and/or caregiver input on burden of
disease and treatment. These include:
• Scientific publications
• Public documents
• Medical records, chart audits
• Personal documents (e.g., patient journal)
• Other printed materials (e.g., literature)
• Graphics
• Photo elicitation (participants take photographs or videotapes)
• Archival records and physical artifacts.
Survey Instruments
Survey instruments generally consist of a standard set of questions or items administered in the
same order to each participant (Johnson and Christensen, 2017). They can be administered in
both observational studies and clinical trials, for example, to collect PRO measures that are
associated with study treatments. Data can be collected by survey instruments throughout the
study or at the end of the study (e.g., exit surveys). Refer to Guidance 2 for details regarding
survey instruments.
If data obtained via survey instruments are intended to be a study endpoint in a clinical trial,
FDA recommends that stakeholders adopt good measurement principles. Refer to subsequent
guidance documents in this series for discussion of factors to consider when administering
survey instruments in clinical trials. Refer to Appendix 1. Standards and Requirements
Pertaining to Submission of Data regarding standards and requirements pertaining data
submission to FDA.
Audiovisual Materials
Audiovisual materials can be used to collect data in characterizing the patient experience. These
may include recordings or photographs of individuals or groups, sounds (laughter or other
vocalized expressions), email or discussion board messages with audiovisual attachments, and
video chat/conferencing (e.g., Skype).

Contains Nonbinding Recommendations
22
Steps to consider when collecting audiovisual materials as data include:
• Standardizing data collection (e.g., information that can be derived from photographs can
vary based on lighting and camera settings)
• Obtaining the required permissions needed to use materials, including informed consent
• Obtaining permission to extract information from web content, if necessary (e.g., request
permission to join online forums and inquire whether there are restrictions on use of
information for research purposes)
Social Media and Verified Patient Communities
Social media tools (e.g., medical community blogs, crowdsourcing, social media pages) may
include information on patients’ perspectives regarding symptoms and impacts of a disease or
condition. Targeted social media searches may be useful during the preliminary stages of a study
to complement literature review findings, inform the development of research tools (e.g.,
qualitative study discussion guides), or as a supplement to traditional research approaches (e.g.,
literature, one-on-one interviews, focus groups, or expert opinion).
Common methods for generating patient input using online methods, including social media,
and their potential strengths and limitations are summarized in Table 3.
Table 3. Common Methods for Gathering Patient Input Using Online Methods (Including
Social Media)
Social Media Qualitative
Research Methodology Strengths Limitations
All social methodologies • May allow access to
hard-to-reach populations
• Cost and time saving for
researchers
• Relatively easy to
implement
• Accurate and automatic
capture of data
• Participant convenience
and comfort
• Might result in greater
self-disclosure
• Self-selection bias (e.g., social
media participants may include
a narrow band of patients with
regard to clinical or
demographic characteristics or
willingness to participate in
social media platforms)
• Regulations and laws
surrounding privacy and use of
public and private data

Contains Nonbinding Recommendations
23
Social Media Qualitative
Research Methodology Strengths Limitations
Asynchronous online
focus groups or interviews
(occur at different places,
different times)
• Can be conducted using
email, discussion forums,
and other forms of social
media
• Provide flexibility and
convenience of logging
in at time and place of
participant’s choosing
• Lack of time pressure
and greater reflection
• Lack of visual cues
• Underlying selection process
might be difficult, if not
impossible, to quantify
• Representativeness might be
questionable without strong
assumptions
Synchronous online focus
groups or interviews
(among younger
participants; occur at
different places, same time
[for focus groups])
• Data captured in real-
time (synchronous)
• Can be conducted using
the phone (SMS/text
messages), chat methods,
video messaging
• Interaction is often
dynamic, immediate,
conversational (similar to
every-day interactions)
• Assessment of visual
cues (through video or
emotions conveyed
through emoticon use)
• Less threatening
methodology for younger
participants
• Scheduling can be difficult;
must find a common meeting
time (for focus groups)
• Requires a fast internet
connection, webcam/
audio/video capabilities that
some participants may not have
readily available
• Technology-rich interface can
present more technical
difficulties
• Moderation can be difficult with
too many participants;
sometimes participants have
trouble taking turns (for focus
groups)
• Faster typing speed gives
participants an advantage, and
these participants can dominate
the conversation (for focus
groups)
• Increased likelihood of passive
participation (e.g., a participant
logging on and observing but
not participating)
• Groups with more than five
participants require two
moderators (for focus groups)
• Self-selection bias (social media
participants may include a
narrow band of patients with
regard to clinical or
demographic characteristics)

Contains Nonbinding Recommendations
24
Social Media Qualitative
Research Methodology Strengths Limitations
Verified patient
communities10 and social
media data collection
• Generated through
platforms such as online
support groups and
online educational groups
• Groups include
identifiable patients and
identifiable reporters
• Helpful for:
- gathering information
on health conditions
(Prieto et al., 2014)
- sharing treatments
and experiences of
care (McGregor et
al., 2014)
- recruiting research
participants
(O'Connor, Jackson,
Goldsmith, &
Skirton, 2014)
• Must have authorization to
obtain identifiable information
(e.g., Protected Health
Information (PHI))
Social media data
collection (unelicited data)
• Generated through easily
accessible platforms
• Low burden for people
providing data
• Helpful for gathering
information on health
conditions (Prieto et al.
2014)
• Participants are unknown;
Respondent identification not
verifiable
• PHI not verifiable
• Underlying selection process is
difficult if not impossible to
quantify
• Representativeness is highly
questionable without strong
assumptions
• Predominantly retrospective
data
Although social media tools can provide useful data, limitations related to sampling should be
considered. With most social media sources, there is no mechanism for verifying patient identity
or clinical and demographic characteristics; you are limited to relying on patient self-
identification and diagnosis, which can be inaccurate. Additionally, different demographic
groups tend to use different types of social media. Based on this variability, it may be important
10 Communities that provide personal information to allow verification of personal characteristics, such as identity,
diagnosis, or other patient characteristics.

25
Contains Nonbinding Recommendations
for you to use different social media tools to gather information from the demographic group(s)
you are targeting. Likewise, when submitting information for regulatory review, you should
demonstrate how the data collection methods used to generate data addresses these limitations
and to ensure rigor in methodology and data integrity.
Concerns around the lack of ability to confirm patient characteristics (e.g., diagnosis) can be
mitigated in various ways. For example, to have a verified patient, there should be enough
information to indicate the existence of a specific patient, including age (or age category, e.g.,
adolescent, adult, elderly), sex, initials, date of birth, name, or patient identification number. In
the most ideal case, clinical information would also be available, by permission, through a
central database (e.g., for patients who are members of patient advocacy group message boards,
social networking groups, or medical community blogs). An identifiable reporter can be a
family member, doctor, other health care practitioner, or other individual who has sufficient
information to indicate that they are an identifiable person who has knowledge about the
patient.
Refer to Guidance 2 for considerations for use of social media to collect patient experience data.
Recording Information
You should develop, as a part of study protocols, written forms to collect patient experience data
(e.g., a discussion guide or observational data collection form). A discussion guide or
observation data collection form is a pre-designed form used to record information collected
during an interview or observation (e.g., an interviewer may take notes on the discussion guide
or observational protocol). Patient experience data can also be recorded through various forms,
such as interview summaries and audio- and video-recordings.
Resolving Site/Field Issues
Standardized training should be provided to the members of the research team to improve
consistency of research. The roles and responsibilities of the team should be outlined in the
research protocol. This will help to prevent many site issues. FDA encourages stakeholders to
also have a troubleshooting guide. Researchers should anticipate and address site/field issues that
might arise during data collection. Some issues to consider are listed below (Creswell (2013):
Access to patients/sites
• Patients’ willingness to participate in research
• Patient responsiveness
• Appropriateness of a site
• Building of trust and credibility at the field site
• IRB unfamiliar with certain methodologies
Interviews, focus groups, facilitated discussions
• Mechanics of conducting interviews and discussions (unexpected participant
behaviors, sensitive issues, inexperienced researchers)

Contains Nonbinding Recommendations
26
Observational methods
• Consistency in the role of observer
• Mechanics of observing (remembering to take site notes)
• Recording accurate quotes/notes
• Managing information sufficiently at site
Documents and audiovisual materials
• Locating materials
• Obtaining permission from the participant to use materials (e.g., audio/video-recorder)
• Minimal noise disturbance
• Best location for video recorder/camera
Survey instruments
• Paper-based administration: Quality control at the visit (e.g., administering correct
version of the survey instrument, looking for non-response patterns such as not
completing a particular section)
• Electronic-based administration: Consistency in data monitoring procedures and
follow-up (e.g., monitoring for timely completion and attrition)
Ethical issues
• Informed consent, if required
• IRB oversight, if required
• Conflicts of interest
• Dishonest or hidden (secret) activities
• Confidentiality and privacy considerations
• Benefits and risks of research to participants
Data Management
Data management considerations should be addressed in the early stages of a research study.
Before initiating data collection, you should formulate a data management plan (DMP) — a
written document that describes the data you expect to acquire or generate during your research
study; how you intend to manage, describe, analyze, and store said data; and what mechanisms
you will use at the end of your study to preserve and share your data (Stanford University
Libraries n.d.). Creating a written DMP helps formalize the data management process, identify
potential weaknesses in the DMP, and provide a record of what you intend to do.
Data Standards
External stakeholders should use appropriate data standards when collecting, managing, and
reporting patient experience data. When planning a study (including the design of case report

Contains Nonbinding Recommendations
27
forms, data management systems, and data analysis plans), you should determine which FDA-
supported standards to use. See Appendix 1. Standards and Requirements Pertaining to
Submission of Data for some data standards resources.11
While compliance with these standards may not be required for studies other than those
conducted to support a regulatory medical product application (e.g., an Investigational New Drug
(IND), New Drug Application (NDA) or Biologics License Application (BLA)) or medical
product labeling language, we encourage researchers to, at a minimum, bear these standards in
mind, because patient experience data that are ultimately intended for use in clinical trials would
be subject to the applicable standards.
Monitoring and Quality Assurance
FDA expects that external stakeholders will be responsible for monitoring the study, ensuring
data integrity, and performing the data analysis. This includes assessing the quality of the design
and performance of the studies.
Storing Data
External stakeholders should plan how to store their data in advance of starting their study.
Researchers should decide how data will best be stored so that the data can be easily retrieved
and protected from any type of damage or loss. The approach to data storage should reflect the
type of data collected. Regarding duration of data retention, researchers should comply with their
IRB and applicable regulations.
Principles to consider regarding data storage and handling data include the following (Creswell
2013):
• Create back-up copies of computer files
• Use high-quality equipment for audio-recording information during interviews
• Protect the anonymity of participants by de-identification
• Create a data collection table or database to track and identify data
• Maintain a list of types of data collected
Confidentiality
All personal participant data collected and processed for research should be managed by the
research team with adequate precautions to ensure confidentiality of the data in accordance with
applicable national and/or local laws and regulations on personal data protection.
Data Analysis
The approach taken to data analysis depends on the type of data, study conducted, and research
questions and objectives. Sampling information should be taken in to consideration. Later
11 https://www.fda.gov/industry/fda-resources-data-standards/study-data-standards-resources

Contains Nonbinding Recommendations
28
guidance in this series will provide more discussion of potential data analysis methods used with
different types of patient experience data.
CONCLUSIONS
This document has provided an overview of methods to collect robust, meaningful, and
sufficiently representative patient input to inform medical product development and regulatory
decision-making. The proposed methods presented serve only as a basis for dialogue in the
evolving and growing discipline of the science of patient input. If you are considering collecting
patient experience data, FDA encourages you to have early interactions with FDA and obtain
feedback from the relevant FDA review division on appropriate research design and any
applicable regulatory requirements.

Contains Nonbinding Recommendations
29
REFERENCES
American National Standards Institute, 2016, ANSI/AAMI/ISO 14971:2007 (R2016): Medical
devices - Application of risk management to medical devices.
Brick, JM, 2013, Unit nonresponse and weighting adjustments: A critical review, Journal of
Official Statistics, 29(3):329-353.
Calinescu, M, B Schouten, and S Bhulai, 2012, Adaptive survey designs that minimize
nonresponse and measurement risk, Statistics Netherlands.
Chow, SC, H Wang, and J Shao, 2008, Sample Size Calculations in Clinical Research, Boca
Raton, FL: CRC Press.
Creswell, JW, 2013, Qualitative Inquiry and Research Design: Choosing Among Five
Approaches, Los Angeles, CA: SAGE Publications.
Dworkin, SL, 2012, Sample size policy for qualitative studies using in-depth interviews, Arch
Sex Behav, 41(6):1319-1320.
FDA, 2009, Patient-Reported Outcome Measures: Use in Medical Product Development to
Support Labeling Claims, accessed March 15, 2018,
https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Gui
dances/UCM193282.pdf.
FDA, 2013, Electronic Source Data in Clinical Investigations., accessed March 11, 2018,
https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UC
M328691.
FDA, 2014a, Providing Regulatory Submissions in Electronic Format -- Submissions Under
Section 745A(a) of the Federal Food, Drug, and Cosmetic Act, accessed March 11,
2018, https://www.fda.gov/downloads/drugs/guidances/ucm384686.pdf.
FDA, 2014b, Providing Regulatory Submissions In Electronic Format — Standardized Study
Data, accessed March 11, 2018,
https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UC
M292334.
FDA, 2015, Clinical Outcome Assessment (COA): Glossary of Terms., accessed March 11,
2018,
https://www.fda.gov/drugs/developmentapprovalprocess/drugdevelopmenttoolsqualifica
tionprogram/ucm370262.htm.
FDA, 2016, Patient Preference Information - Voluntary Submission, Review in Premarket
Approval Applications, Humanitarian Device Exemption Applications, and De Novo
Requests, and Inclusion in Decision Summaries and Device Labeling, accessed August
22, 2019, https://www.fda.gov/regulatory-information/search-fda-guidance-
documents/patient-preference-information-voluntary-submission-review-premarket-
approval-applications.

Contains Nonbinding Recommendations
30
FDA, 2017a, Expedited Programs for Serious Conditions: Drugs and Biologics, accessed,
https://www.fda.gov/regulatory-information/search-fda-guidance-documents.
FDA, 2017b, Providing Regulatory Submissions in Electronic Format — Certain Human
Pharmaceutical Product Applications and Related Submissions Using the eCTD
Specifications., accessed March 11, 2018,
https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Gui
dances/UCM333969.pdf.
FDA, 2017c, Use of Real-World Evidence to Support Regulatory Decision-Making for Medical
Devices accessed August 22, 2019, https://www.fda.gov/regulatory-information/search-
fda-guidance-documents/use-real-world-evidence-support-regulatory-decision-making-
medical-devices.
Francis, JJ, M Johnston, C Robertson, L Glidewell, V Entwistle, MP Eccles, and JM Grimshaw,
2010, What is an adequate sample size? Operationalising data saturation for theory-
based interview studies, Psychol Health, 25(10):1229-1245.
Fricker, R, 2008, Sampling Methods for Web and E-mail Surveys, The SAGE Handbook of
Online Research Methods, Fielding, N., R. M. Lee and G. Blank, Thousand Oaks, CA:
SAGE Publications, Inc., 195-216.
Grajales, FJ, S Sheps, K Ho, H Novak-Lauscher, and G Eysenbach, 2014, Social media: A
review and tutorial of applications in medicine and health care, J Med Internet Res,
16(2):e13.
Groves, RM, FJ Fowler Jr, MP Couper, JM Lepkowski, E Singer, and R Tourangeau, 2009,
Survey Methodology, Hoboken, NJ: John Wiley & Sons.
Heckathorn, DD, 1997, Respondent-driven sampling: A new approach to the study of hidden
populations, Social Problems, 44(2):174-199.
International Conference on Harmonisation, 2016, Revision OF M4E Guideline On Enhancing
the Format and Structure of Benefit-Risk Information in ICH: Efficacy - M4E(R2),
accessed August 21, 2019,
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/CTD/M4E_R2_Efficacy/
M4E_R2__Step_4.pdf.
Johnson, RB and L Christensen, 2014, Educational Research, Thousand Oaks, CA: SAGE
Publications, Inc.
Johnson, RB and L Christensen, 2017, Educational Research: Quantitative, Qualitative, and
Mixed Approaches, Thousand Oaks, CA: SAGE Publications, Inc.
Johnson, TP, 2015, Handbook of Health Survey Methods Hoboken, NJ: John Wiley & Sons.
Korn, EL and BI Graubard, 1999, Analysis of Health Surveys, Hoboken, NJ: John Wiley &
Sons.

Contains Nonbinding Recommendations
31
Levy, PS and S Lemeshow, 2013, Sampling of Populations: Methods and Applications,
Hoboken, NJ: John Wiley & Sons.
McCarthy, S, P O’Raghallaigh, S Woodworth, YL Lim, LC Kenny, and F Adam, 2016, An
integrated patient journey mapping tool for embedding quality in healthcare service
reform, Journal of Decision Systems, 25(Suppl 1):354-368.
Medical Device Innovation Consortium, 2015, Medical Device Innovation Consortium (MDIC)
Patient Centered Benefit‐Risk Project Report: A Framework for Incorporating
Information on Patient Preferences Regarding Benefit and Risk into Regulatory
Assessments of New Medical Technology.
National Research Council, 2010, The Prevention and Treatment of Missing Data in Clinical
Trials, Washington, DC: The National Academies Press. https://doi.org/10.17226/12955.
O'Neill, RT and R Temple, 2012, The prevention and treatment of missing data in clinical trials:
An FDA perspective on the importance of dealing with it, Clinical Pharmacology &
Therapeutics, 91:550-554.
Patrick, DL, 2013, What Does "Patient-Centered Outcomes" Mean?, International Society for
Pharmacoeconomics and Outcomes Research (ISPOR) 18th Annual Meeting, 18.
Prieto, VM, S Matos, M Alvarez, F Cacheda, and JL Oliveira, 2014, Twitter: A good place to
detect health conditions, PLoS One, 9(1):e86191.
Rothenberg, RT, 1995, Sampling in social networks, Connections, 18:104-110.
Sandelowski, M, 1995, Sample size in qualitative research, Res Nurs Health, 18(2):179-183.
Schouten, B, M Calinescu, and A Luiten, 2013, Optimizing quality of response through
adaptive survey designs, Survey Methodology, 39:29-58.
Stanford University Libraries, n.d., Data best practices, accessed March 14, 2018,
http://library.stanford.edu/research/data-management-services/data-best-practices.
Thompson, SK, 1987, Sample-Size for Estimating Multinomial Proportions, American
Statistician, 41:42-46.
University of California San Francisco, 2017, Clinical Trial Protocol Development, accessed
August 07, 2019, https://hub.ucsf.edu/protocol-development.
US Congress, 1991, Public Law 102-73: The National Literacy Act of 1991.
US Department of Health and Human Services, 2000, Healthy People 2010.
Valliant, R, JA Dever, and F Kreuter, 2013, Practical Tools for Designing and Weighting
Survey Samples, New York, NY: Springer.
Wilson, H, E Dashiell-Aje, M Anatchkova, K Coyne, A Hareendran, NK Leidy, CA McHorney,
and K Wyrwich, 2018, Beyond study participants: A framework for engaging patients in

Contains Nonbinding Recommendations
32
the selection or development of clinical outcome assessments for evaluating the benefits
of treatment in medical product development, Quality of Life Research, 27:5-16.
Wolf, JA, V Niederhauser, D Marshburn, and SL LaVela, 2014, Defining patient experience.
Patient Experience Journal, 1(1):7-19., Patient Experience Journal, 1:7-19.

Contains Nonbinding Recommendations
33
APPENDICES
Appendix 1. Standards and Requirements Pertaining to Submission of Data
Many existing FDA regulations, guidances, requirements, and other standards for data
submissions apply to patient experience data. These pertain to the capture/collection,
transmission, processing, storage, archiving, retention, and submission of data from clinical
studies conducted to support a regulatory medical product application (e.g., IND, NDA, BLA)
or medical product labeling language. The following is a partial list of such regulations,
guidance(s), standards, and requirements:
• FDA forms and submission requirements
https://www.fda.gov/drugs/developmentapprovalprocess/formssubmissionrequirements/
• The International Council on Harmonisation (ICH) Guidelines12, such as ICH
Harmonised Guideline for Good Clinical Practice: E6(R2) and the Electronic Common
Technical Document (eCTD)
• 21 eCFR, Volumes 1 – 813
• Guidance for Industry on Providing Regulatory Submissions In Electronic Format —
Standardized Study Data (FDA 2014b)
• Guidance for Industry on Providing Regulatory Submissions in Electronic Format —
Submissions Under Section 745A(a) of the Federal Food, Drug, and Cosmetic Act (FDA
2014a)
• Guidance for Industry on Providing Regulatory Submissions in Electronic Format —
Certain Human Pharmaceutical Product Applications and Related Submissions Using
the eCTD Specifications (FDA 2017b)
• Guidance for Industry on Electronic Source Data in Clinical Investigations (FDA 2013)
• The FDA Data Standards Catalog.
For current and more detailed information on study data standards resources, please see:
https://www.fda.gov/industry/fda-resources-data-standards/study-data-standards-resources.
12 https://www.fda.gov/RegulatoryInformation/Guidances/default.htm
13 https://www.ecfr.gov/cgi-bin/text-idx?tpl=/ecfrbrowse/Title21/21tab_02.tpl

Contains Nonbinding Recommendations
34
Appendix 2. Glossary
This glossary defines terms that will be used in the series of methodological Patient-Focused
Drug Development FDA guidance documents that are required by the Cures Act, and that are
part of the commitments made by FDA under PDUFA VI. The goal of this glossary is to
provide standardized nomenclature and terminologies related to patient-focused medical product
development. As appropriate, definitions from existing federal resources (e.g., “BEST
(Biomarkers, Endpoints, and Other Tools) Resource”)14 have been incorporated. External
resources were also used to define terms and have been cited.
Attribute: A feature or characteristic of a medical product — such as efficacy or effectiveness,
safety, means of administration, duration of effect, or duration of use — that may affect benefit-
risk considerations.
Benefit: The favorable effects of a medical product. Types of benefits include clinical benefit
(see clinical benefit). Benefits may also include important characteristics of the medical
product, such as convenience (e.g., a more convenient dosing regimen or route of
administration) that may lead to improved patient compliance, or benefits that affect those other
than the patient. (Sources: International Council on Harmonisation (ICH) Guideline, Revision
of M4E Guideline on Enhancing the Format and Structure of Benefit-Risk Information in ICH
(Efficacy – M4E(R2) (International Conference on Harmonisation 2016); ANSI/AAMI/ ISO
14971: 2007/(R)2016 Medical Devices—Application of Risk Management to Medical Devices
(American National Standards Institute 2016).)
Benefit-Risk Assessment: Evaluation of the benefits and risks of a medical product and
making a judgment as to whether the benefits outweigh the risks associated with specified
conditions of use.
Biomarker: A defined characteristic that is measured as an indicator of normal biological
processes, pathogenic processes, or responses to an exposure or intervention, including
therapeutic interventions. Molecular, histologic, radiographic, or physiologic characteristics are
types of biomarkers. A biomarker is not an assessment of how an individual feels, functions, or
survives. (Source: “BEST (Biomarkers, EndpointS, and other Tools) Resource”)
Caregiver: A person who helps a patient with daily activities, health care, or any other
activities that the patient is unable to perform because of illness or disability, and who
understands the patient’s health-related needs. This person may or may not have decision-
making authority for the patient and is not the patient’s healthcare provider.
Caregiver Preference: A statement of the relative desirability or acceptability to caregivers of
attributes by which alternative health interventions may differ.
Clinical Benefit: A positive, clinically meaningful effect of an intervention, e.g., a positive
effect on how an individual feels, functions, or survives. (Source: “BEST (Biomarkers,
EndpointS, and other Tools) Resource”)
14 “Available at https://www.ncbi.nlm.nih.gov/books/NBK338448/.

Contains Nonbinding Recommendations
35
Clinical Outcome: An outcome that describes or reflects how an individual feels, functions, or
survives. (Source: “BEST (Biomarkers, EndpointS, and other Tools) Resource”)
Clinical Outcome Assessment: Assessment of a clinical outcome can be made through report
by a clinician, a patient, a non-clinician observer, or through a performance-based assessment.
Types of COAs include patient-reported outcome, clinician-reported outcome measures,
observer-reported outcome, and performance outcome. (Source: “BEST (Biomarkers,
EndpointS, and other Tools) Resource”)
Clinical Relevance: The extent to which an endpoint can capture and measure an aspect of a
potential clinical benefit (improvement in how the patient feels, functions, or survives) that is
important from a clinical perspective and from the patient’s perspective.
Clinician-Reported Outcome (ClinRO): A measurement based on a report that comes from a
trained health-care professional after observation of a patient’s health condition. Most ClinRO
measures involve a clinical judgment or interpretation of the observable signs, behaviors, or
other manifestations related to a disease or condition. ClinRO measures cannot directly assess
symptoms that are known only to the patient (e.g., pain intensity). (Source: “BEST
(Biomarkers, EndpointS, and other Tools) Resource”)
Concept (also referred to as Concept of Interest): In a regulatory context, the concept is the
aspect of an individual’s clinical, biological, physical, or functional state, or experience that the
assessment is intended to capture (or reflect). (Source: “BEST (Biomarkers, EndpointS, and
other Tools) Resource”)
Data Analysis Plan: A roadmap for how the data will be organized and analyzed, and how
results will be presented. A data analysis plan should be established when planning a research
study (i.e., before data collection begins). Among other things, the data analysis plan should
describe: (a) the data to be collected; (b) the analyses to be conducted to address the research
objectives, including assumptions required by said analyses; (c) data cleaning and management
procedures; (d) data transformations, if applicable; and (e) how the study results will be
presented (e.g., graphs, tables).
Data Management Plan (DMP): A written document that describes the data you expect to
acquire or generate during the course of your research study; how you intend to manage,
describe, analyze, and store said data; and what mechanisms you will use at the end of your
study to preserve and share your data. (Source: Stanford University Libraries n.d.(b), “Data
Management Plans (DMPS)” available at https://library.stanford.edu/research/data-
management-services/data-management-plans)
Disease Burden: The impacts, direct and indirect, of the patient’s health condition that have a
negative effect on his or her health, functioning, and overall well-being. Disease burden
includes: the physical and physiologic impacts of the disease and its symptoms; co-morbidities;
emotional and psychological effects of the disease, its management, or its prognosis; social
impacts; effects on relationships; impacts on the patient’s ability to care for self and others; time

Contains Nonbinding Recommendations
36
and financial impacts of the disease and its management; and considerations on the impacts on
the patient’s family.
Endpoint: A precisely defined variable intended to reflect an outcome of interest that is
statistically analyzed to address a particular research question. A precise definition of an
endpoint typically specifies the type of assessments made, the timing of those assessments, the
assessment tools used, and possibly other details, as applicable, such as how multiple
assessments within an individual are to be combined. (Source: “BEST (Biomarkers, EndpointS,
and other Tools) Resource”)
Fit-for-Purpose: A conclusion that the level of validation associated with a medical product
development tool is sufficient to support its context of use. (Source: “BEST (Biomarkers,
EndpointS, and other Tools) Resource”)
Generalizability: The extent to which study findings can be reliably extended to the target
population of interest.
Health Literacy: The degree to which individuals have the capacity to obtain, process, and
understand basic health information and services needed to make appropriate health decisions.
(Source: US Department of Health and Human Services (2000)) Health literacy also includes
numeracy skills—such as calculating cholesterol and blood sugar levels, measuring medication
doses, and understanding nutrition labels — and knowledge of health topics.
Literacy: A person's ability to read, write, speak, and compute and solve problems at levels
necessary to: (a) function on the job and in society; (b) achieve one's goals; and (c) develop
one's knowledge and potential. (Source: Public Law 102-73. The National Literacy Act of 1991
(US Congress 1991))
Methodologically Sound: Assurance that the methods and processes used to obtain and analyze
patient experience data are rigorous, robust and adhere to scientifically established principles
and best practices for method development or implementation. Evidence generated by
methodologically sound methods and processes increases confidence that the results can be
trusted, interpreted and support the intended regulatory uses.
Mixed Methods Research: Research that uses both qualitative and quantitative research
methods. See definitions for qualitative and quantitative research methods.
Observer-Reported Outcome (ObsRO): A measurement based on a report of observable
signs, events, or behaviors related to a patient’s health condition by someone other than that
patient or a health professional. Generally, ObsROs are reported by a parent, caregiver, or
someone who observes the patient in daily life. ObsROs are particularly useful for patients who
cannot report for themselves (e.g., infants or individuals who are cognitively impaired). An
ObsRO measure does not include medical judgment or interpretation. (Source: “BEST
(Biomarkers, Endpoints and Other Tools) Resource”). Examples of ObsROs include a parent’s
report of a child’s vomiting episodes or a report of wincing thought to be the result of pain in
patients who are unable to report for themselves.

Contains Nonbinding Recommendations
37
Patient: Any individual with or at risk of a specific health condition, whether or not the
individual currently receives any therapy to prevent or treat that condition.
Patient Advocate: An individual or group of individuals, who may or may not be part of the
target population and who has a role in promoting an interest or cause to influence policy with
respect to patients’ health or healthcare.
Patient-Centered: See patient-focused
Patient-Centered Outcome: An outcome that is important to patients’ survival, functioning, or
feelings as identified or affirmed by patients themselves, or judged to be in patients’ best
interest by providers and/or caregivers when patients cannot report for themselves. (Source:
ISPOR Plenary, Patrick (2013))
Patient Engagement: Activities that involve patient stakeholders sharing their experiences,
perspectives, needs, and priorities that help inform FDA’s public health mission. Such activities
may include testimony at Advisory Committee meetings, submission to a regulations.gov public
docket, meetings attended by patients, FDA, and other stakeholders, other correspondence with
FDA, interactions through social media, and interactions with or information from patient
representatives or patient advocates.
Patient Experience Data: Defined in Title III, section 3001 of the Cures Act, as amended by
section 605 of the Food and Drug Administration Reauthorization Act of 2017,15 and includes
data that are collected by any persons and are intended to provide information about patients’
experiences with a disease or condition. Patient experience data can be interpreted as
information that captures patients’ experiences, perspectives, needs, and priorities related to: 1)
the symptoms of their condition and its natural history; 2) the impact of the conditions on their
functioning and quality of life; 3) their experience with treatments; 4) input on which outcomes
are important to them; 5) their preferences for outcomes and treatments; and 6) the relative
importance of any issue as defined by patients.
Patient-Focused (also referred to as patient-centered): Ensuring that patients’ experiences,
perspectives, needs, and priorities are meaningfully incorporated into decisions and activities
related to their health and well-being.
15 For purposes of this section, the term ”patient experience data” includes data that (1) are collected by any
persons (including patients, family members and caregivers of patients, patient advocacy organizations, disease
research foundations, researchers and drug manufacturers); and (2) are intended to provide information about
patients’ experiences with a disease or condition, including (A) the impact (including physical and psychosocial
impacts) of such disease or condition, or a related therapy, on patients’ lives; and (B) patient preferences with
respect to treatment of such disease or condition.” The definition is codified at section 569C(c)(4) of the Federal
Food, Drug, and Cosmetic Act, and applies to section 3002 of the Cures Act, which directed FDA to issue certain
guidance documents regarding the collection of patient experience data, see section 3002(b).

Contains Nonbinding Recommendations
38
Patient-Focused Drug Development (PFDD) (also referred to as patient-focused medical
product development): A systematic approach to help ensure that patients’ experiences,
perspectives, needs, and priorities are captured and meaningfully incorporated into the
development and evaluation of medical products throughout the medical product life cycle.
Patient Input: Information that captures patients’ experiences, perspectives, needs, and
priorities. See also patient experience data.
Patient Partner: An individual patient, caregiver, or patient advocacy group that engages other
stakeholders to ensure the patients’ wants, needs, and preferences are represented in activities
related to medical product development and evaluation. (Wilson et al. 2018)
Patient Perspective: A type of patient experience data that specifically relates to patients’
attitudes or points of view about their condition or management of their condition. Patient
perspectives may include perceptions, goals, priorities, concerns, opinions, and preferences.
Patient Preference: A statement of the relative desirability or acceptability to patients of
specified alternatives or choices among outcomes or other attributes that differ among
alternative health interventions. (Source: FDA Guidance for Industry: Patient Preference
Information –Voluntary Submission, Review in Premarket Approval Applications,
Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in
Decision Summaries and Device Labeling (FDA 2016))
Patient Preference Information (PPI): Assessments of the relative desirability or acceptability
to patients of specified alternatives or choices among outcomes or other attributes that differ
among alternative health interventions. The methods for generating PPI may be qualitative,
quantitative, or mixed methods. (Source: FDA Guidance for Industry: Patient Preference
Information –Voluntary Submission, Review in Premarket Approval Applications,
Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in
Decision Summaries and Device Labeling (FDA 2016))
Patient-Provided Input: Patient experience data or other information that comes directly from
patients.
Patient-Reported Outcome (PRO): A measurement based on a report that comes directly from
the patient (i.e., study subject) about the status of a patient's health condition without
interpretation of the patient's response by a clinician or anyone else. A PRO can be measured by
self-report or by interview, provided that the interviewer records only the patient's response.
Symptoms or other unobservable concepts known only to the patient (e.g., pain severity or
nausea) can only be measured by PRO measures. PROs can also assess the patient perspective
on functioning or activities that may also be observable by others. (Source: “BEST
(Biomarkers, EndpointS, and other Tools) Resource”)
Patient Representative: An individual, who may or may not be part of the target population,
who has direct experience with a disease or condition (e.g., a patient or caregiver) and can
provide information about a patient’s experience with the disease or condition.

Contains Nonbinding Recommendations
39
Performance Outcome (PerfO): A measurement based on a standardized task(s) performed by
a patient that is administered and evaluated by an appropriately trained individual or is
independently completed. PerfOs require patient cooperation and motivation. These include
measures of gait speed (e.g., timed 25-foot walk test), memory recall (e.g., word recall test) or
other cognitive testing (e.g., digit symbol substitution test). (Source: “BEST (Biomarkers,
EndpointS, and other Tools) Resource”)
Qualitative Research Methods: Methods associated with the gathering, analysis,
interpretation, and presentation of narrative information (e.g., spoken or written accounts of
experiences, observations, and events). Qualitative research methods may also include direct
observations (e.g., non-verbal communication and behaviors).
Quantitative Research Methods: Methods associated with the gathering, analysis,
interpretation, and presentation of numerical information.
Real-World Data: Data relating to patient health status and/or the delivery of health care
routinely collected from a variety of sources. (Source: FDA Guidance on Use of Real-World
Evidence to Support Regulatory Decision-Making for Medical Devices (FDA 2017c))
Real-World Evidence: The clinical evidence regarding the usage and potential benefits or risks
of a medical product derived from analysis of real-world data. (Source: FDA Guidance on Use
of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices (FDA
2017c))
Reporter: In research studies designed to collect patient experience data, the reporter is the
individual, group of individuals, or entity providing patient experience data. Reporters may be
patients, parents, sexual/romantic partners, caregivers, physicians, or other healthcare
professionals. Selection of an appropriate reporter in a given research study will depend on the
definition of the target population of interest. If a patient in the target population can be
reasonably expected to reliably self-report, one would expect the patient herself/himself to be
the reporter in that research study.
Representativeness: Confidence that a sample from which evidence is generated is sufficiently
similar to the intended population. In the context of patient experience data, representativeness
includes the extent to which the elicited experiences, perspectives, needs, and priorities of the
sample are sufficiently similar to those of the intended patient population.
Research Protocol: A document that describes the background, rationale, objectives, design,
methodology, statistical considerations, and organization of a clinical research project. (Source:
University of California San Francisco (2017)) A research protocol guides the study and
associated data collection and analysis in a productive and standardized manner.
Risk: Risks are adverse events and other unfavorable effects associated with a medical product.
Risks include drug interactions, risks identified in the non-clinical data, risks to those other than
the patient (e.g., fetus, those preparing and administering the medical product), and risks based

Contains Nonbinding Recommendations
40
on pharmacologic class or current knowledge of the product. Factors such as potential misuse,
abuse, or diversion of the product may also be considered. (Source: International Conference
on Harmonisation (ICH) Guidelines – Efficacy M4E(R2))
Risk Tolerance: The degree to which a patient would accept increased probability or severity
of a harm in exchange for a specific expected benefit. (Source: Medical Device Innovation
Consortium (2015))
Science of Patient Input: Methods and approaches of systematically obtaining, analyzing, and
using information that captures patients’ experiences, perspectives, needs, and priorities in
support of the development and evaluation of medical products.
Social Media: Web-based tools that are used for communication. Social media may include
blogs, microblogs, social networking sites, professional networking sites, thematic networking
sites, wikis, mashups, collaborative filtering sites, media sharing sites, and others. (Source:
Grajales et al. (2014))
Subgroup: A subset of the study population or study sample defined by specific baseline
characteristics. For example, demographic subgroups are commonly defined by subject sex,
race, and age.
Surrogate Endpoint: A type of endpoint used in clinical trials as a substitute for a direct
measure of how a patient feels, functions, or survives. A surrogate endpoint does not measure
the clinical benefit of primary interest in and of itself but rather is expected to predict that
clinical benefit or harm based on epidemiologic, therapeutic, pathophysiologic, or other
scientific evidence. From a U.S. regulatory standpoint, surrogate endpoints and potential
surrogate endpoints can be characterized by the level of clinical validation: (a) validated
surrogate endpoints; (b) reasonably likely surrogate endpoints; and (c) candidate surrogate
endpoints. (Source: “BEST (Biomarkers, EndpointS, and other Tools) Resource”)
Target Population (also referred to as the target patient population, the underlying population
or intended population): The group of individuals (patients) about whom one wishes to make
an inference.
Trade-off: The extent to which a change in the level of one or more attributes of a medical
product is offset by a change in one or more other attributes of that product. (Source: Medical
Device Innovation Consortium (2015))
Treatment Burden: The impacts of a specific treatment or treatment regimen that have a
negative effect on the patient’s health, functioning, or overall well-being. Treatment burden
includes: side effects, discomfort, uncertainty about treatment outcomes, dosing and route of
administration, requirements, and financial impacts.
Treatment Effect: The amount of change in a disease/condition, symptom, or function that
results from a medical intervention (as compared with not receiving the intervention or
receiving a different intervention).

Contains Nonbinding Recommendations
41
Treatment Outcome: The benefits or harms to a patient who receives an intervention; the
impact on a patient’s health, function, or well-being — or on a clinical indicator thereof — that
is assumed to result from an intervention.
Unmet Medical Need: A condition whose treatment or diagnosis is not addressed adequately
by available therapy. An unmet medical need includes an immediate need for a defined
population (e.g., to treat a serious condition with no or limited treatment) or a longer-term need
for society (e.g., to address the development of resistance to antibacterial drugs). (Source: FDA
Guidance for Industry. Expedited Programs for Serious Conditions––Drugs and Biologics (FDA
2017a)).