Research Tips by Dr Maxine Kuroda
Dr Maxine Kuroda
Clinical Research is essential for advancement of medicine. However, the benefit of any research activity is directly related to the quality of research, data obtained, and proper treatment of the data. Perhaps one of the most important steps in conducting clinical research is advanced planning with a statistician or epidemiologist. In this issue of the NYSORA newsletter, Dr Maxine Kuroda, PHD shares a few crucial steps in designing and conducting clinical research.
a. General concepts
i. What questions are driving the research?
ii. What are the clinical implications of the research?
iii. Will the research contribute new information? (Even if the intention is to expand on previous findings
- in what way is such 'expansion' likely to contribute to the field?)
1. To share a case report
2. To describe the experiences of a case series
3. To conduct a clinical trial
4. Note: There is nothing wrong with observational research. In fact, this is often the only type of
research that is feasible. Moreover, clinical trials are not generalizable beyond the
imposed parameters of the study and its research subjects.
ii. Specific aims
1. Typically, new investigators want to answer all the questions and solve all of the problems in their
2. However, most studies should be limited to 1 or 2 very specific aims.
3. Note: Once data collection has begun, these aims cannot be changed. This is because study
measures were pre-selected for their usefulness in addressing the specific aims. The same
measures may not be able to address specific aims that have been changed - or there may be
other measures that are better suited to answering specific aims that have been changed.
iii. Hypotheses to be tested
1. These are research ('alternative') hypotheses that are presented as simple declarative sentences.
Null hypotheses of no effect or no difference are not stated. This is not a silly point - for instance,
often times investigators want to look at whether a new procedure or drug has the same success
rate as an established procedure or drug. (Why would a new modality want to be statistically
similar to that of the established modality?)
2. To a statistician, this is one of the most important elements overall. It is extremely helpful when
investigators come prepared with written statements that can serve as a springboard for
discussion. Indeed, it should be a requirement that investigators come prepared with written
pecific aims and/or hypotheses.
3. Note: Once data collection has begun, hypotheses cannot be changed. This is because the Type I
error can be elevated when data have been previewed (even inadvertently).
c. Facility resources
i. Institutional support
iii. Personnel - Various levels and experience may be needed for the study. Clinical staff may need special
training. Research assistants will need to learn, understand, and practice the study protocol.
iv. IRB submissions - Since IRB requirements vary widely by institution, these submissions would need to
be the responsibility of the investigator and his/her research assistants. However, statistical input
is required as most IRBs require sample size estimates and a brief statistical analysis plan (SAP).
i. What data are needed to answer the research questions?
ii. Have instruments or questionnaires been developed that collect the information needed for the study?
For questionnaires - Have their reliability and validity been established? For instruments - How precise
are they and what are their standard errors of precision?
iii. What are the data metrics? This ensures that data are submitted to the appropriate statistical
approaches when testing hypotheses, and that findings are properly tabled and/or graphed.
iv. Are research subjects, volunteers, cadavers, or surveys to be used? What are the inclusion and
exclusion criteria? Rationale for same?
v. What are the advantages of the chosen study design, methods, and measures? What are its
limitations? It is useful to record why it was deemed possible to proceed with the study in view of
any limitations identified in the planning phase of the study.
vi. Sample size cannot be estimated without information obtained from this planning phase of the
research. Besides the usual parameters (study design, Type I error rate, power, size of effect),
estimated loss to follow-up and multiple comparisons are useful to consider. Sample size parameters
often come from the literature and/or pilot studies.
e. Additional notes
i. The planning phase is the most critical (make-or-break) phase of research. It is not too early to start
drafting the Introduction and Materials and Methods sections. It even makes sense to discuss journals
that might be interested in publishing the research.
ii. It is possible to prepare via emails among the investigators, however, much depends on how
like-minded and committed the investigators are. Besides potential for confusion, elements of research
will be weakened without consensus among investigators in the planning phase. For instance, will
methodology/equipment need to be modified? If so, are these acceptable to everyone?
a. Recruitment of subjects/volunteers should be tracked.
b. Returned surveys and follow-up phone calls should be tracked.
c. IRB approvals are typically renewed once a year and require a progress report.
d. Data quality
i. Holes in the data - If data are missing, why? Was there insufficient training of research assistants? Was
there a change of research assistants? Was there miscommunication between nursing staff and
research assistants? Were eligible subjects missed?
ii. If blinded, is the 'blind' in tact? Even 'close call' breaches should be identified in order to prevent future
iii. Outliers - Research assistants should investigate and correct (if typographical errors)
any values that seem to be aberrant (e.g., male on one form and female on another, age 45 on one form
and 54 on another, negative instrument/equipment values when only positive values are physiologically
possible). This will save time when resolving outlying values later identified by the statistical analyst.
e. Interim checks should be stated a priori. Unless already stated in the research and statistical plan, it is not
really kosher to conduct an interim analysis based on trends have been observed in the data. This is
because luck-of-the-draw could suggest a result that may differ from that which would actually obtain
if the entire sample was studied. Since the sample size was calculated to answer a specific question, why
would an investigator use just part of it? (For clinical trials of, say, a new drug, interim analyses would have
been incorporated in the research plan for patient safety.)
f. Protocol violations should be recorded. This will come in handy when writing the paper.
g. Ideas for the Discussion section (including advantages and limitations of the study) that occur to the
investigators and research associates should be recorded as the study progresses.
3. Analyzing results
a. Data analyses should follow the SAP and should be kept as simple and straightforward as possible. This is
because fancy statistics are usually not warranted, and even informed readers tend not to understand them.
The danger is that the readers will rely on the statistical reviewer's judgment and simply accept the
analyses at face value. Unfortunately, research that is not fully understood does not fully contribute to the
b. Tables are constructed.
c. Non-clinical figures are graphed. Note that these figures depend on the data metric, e.g., scatterplots are
used when both variables are continuous.
d. Results are written in a very straightforward manner that does not regurgitate the tables or figures.
e. Results should directly address the hypotheses that were stated at the outset.
a. A manuscript should flow so that the readers can follow without getting whiplash. But even with good flow,
paragraphs can get confusing if the writer adds information that is tangential and/or distracting.
b. Direct, active sentences are preferred.
c. Study findings should not be overstated or grandiose. For instance, 'superior anesthesia' should not be
claimed unless the study tested all of the factors that go into 'superiority.' A study may have tested time to
anesthesia onset, occurrence of adverse effects, and patient satisfaction, but may not have tested ease of
administration, need for expensive equipment or additional personnel, and cost.