Nothing in Research is Ever Definitive
When you are conducting research under the pressure to “publish or perish,” the likelihood that your findings will not be reproducible at the first attempt may be much higher than most research scientists would care to admit. If you receive notification that an unsuccessful attempt was made to reproduce your findings, the situation may not be as bleak as you would think.
You Are Not Alone
An immediate emotional reaction to the news is completely understandable, especially if you are still in the early days of your research career. That degree of emotion will also correlate to the extent to which your original findings had been publicized, and the corresponding rank of the academic journal in which they were published. The first thing to do is to take a deep breath and remind yourself of just how many variables are in play here. Let’s start with the researchers who attempted to reproduce your research study:
- Are they connected with the journal that originally published your research, or with a known institution? It’s not unheard of for unknown researchers to try and claim some media attention by allegedly disproving a notable study only to disappear back into the ether when pushed for specifics.
- Did they follow your methodology to the letter or did they have to make any accommodations for budget, time, population sample, etc.?
- If it was a quantitative study, have they triple-checked their results?
- If it was a qualitative study, how did they handle observer bias?
If the efforts of the researchers attempting to replicate your research study appear to be in earnest, then the next step of the investigation turns the spotlight on you and, if appropriate, your lab and your team:
- Were the researchers given access to all the raw data from your original study, or was some held back as a result of institutional policy?
- If you weren’t the Principal Investigator (PI) on the study, was your PI given adequate time to review the project before releasing the results for publication?
- Did the study stay on track and meet the planned deadline, or was there a last minute rush?
- Was there a substantive peer review or were the results published in an open access journal that didn’t quite take peer review seriously?
- Are the issues, as identified by the researchers, correctable?
An Imperfect World
Your last line of defense, though by no means the most ethical line to take, is to count on the unwillingness of journals to publish negative results. Fortunately for you, replication studies have a hard time finding a receptive journal to publish, because those journals are under as much pressure to publish positive results as you are under to publish on a regular basis.
The final decision to be made depends on the exact nature of the alleged error and the potential consequences for human well-being. If human test subjects were involved and follow-on protocols have been implemented based on your initial findings, the response, whatever the cost, has to prioritize the health and wellbeing of those test subjects and any other patients involved.
If, on the other hand, this was simply a foundational study, any and all personnel connected to any type of secondary study that incorporated that foundational data should be notified immediately so that the error isn’t allowed to compound.