{"id":23633,"date":"2019-08-01T15:18:39","date_gmt":"2019-08-01T09:18:39","guid":{"rendered":"https:\/\/www.enago.com\/academy\/?p=23633"},"modified":"2023-01-06T18:26:49","modified_gmt":"2023-01-06T12:26:49","slug":"preproducibility-preceeds-reproducibility-in-research-in-your-research","status":"publish","type":"post","link":"https:\/\/www.enago.com\/academy\/preproducibility-preceeds-reproducibility-in-research-in-your-research\/","title":{"rendered":"Preproducibility Preceeds Reproducibility in Research"},"content":{"rendered":"<p><a href=\"https:\/\/www.enago.com\/academy\/solving-the-reproducibility-crisis-in-scientific-research\/\" target=\"_blank\" rel=\"noopener\">Reproducibility<\/a> is vital to research practice. Fundamentally, it means that different researchers use the same method, under the same conditions, and they should get similar results. Reproducibility <strong>is the only thing that an investigator can guarantee about a study<\/strong>. In addition,\u00a0 it ensures transparency and helps in understanding the research methods used in the study.<\/p>\n<p>Research published in journals should be reproducible. However, many articles do not provide enough information for the work to be reproduced elsewhere. Leaving out important information also means that reviewers cannot assess whether or not the study is <a href=\"https:\/\/www.enago.com\/academy\/research-reproducibility-and-credibility-replication-studies\/\" target=\"_blank\" rel=\"noopener\">reproducible<\/a>. This dramatically reduces the scientific value of the work.<\/p>\n<h2>What is Preproducibility?<\/h2>\n<p>Until recently, there has been no word to describe research that is not reproducible due to a lack of information. In an article in <a href=\"https:\/\/www.nature.com\/articles\/d41586-018-05256-0\" target=\"_blank\" rel=\"nofollow noopener\"><em>Nature<\/em><\/a>, Philip Stark suggests a new term: <strong>preproducibility.<\/strong><\/p>\n<p>An experiment is preproducible if the author has given enough accurate information for others to repeat it. Stark compares a preproducible report with a recipe. He explains: To bake a loaf of bread, we need more than just a list of ingredients. We also need to know how those ingredients are combined and treated. Preproducibility can be described as a \u201cscientific recipe.\u201d If the recipe gives enough detail, anyone should be able to follow it and get a similar result.<\/p>\n<h2>What Happens When Research is Not Preproducible?<\/h2>\n<p>In some areas of science, as little as 25% of research papers are reproducible. This can have serious consequences. Failing to include enough detail in a report can waste researchers\u2019 money and effort. In medicine, irreproducible research can delay treatments, as they cannot be proven. This wastes the valuable time of both patients and doctors.<\/p>\n<p>Peer reviewers struggle to review papers that are not preproducible. How can reviewers assess an article when it is impossible to check whether the work can be reproduced? As Stark says, \u201cScience should be \u2018show me,\u2019 not \u2018trust me.\u2019\u201d At worst, research that is not preproducible could destroy confidence in the scientific process.<\/p>\n<h2>How to Report Research<\/h2>\n<p>Research must be reported in a systematic, organized way to ensure preproducibility. This helps reviewers to evaluate the work and allows other researchers to reproduce the study.<\/p>\n<p>Stark says that he will always publish his raw data (where possible) and the software he has used. This is an excellent start.<\/p>\n<p>Stark also <a href=\"https:\/\/www.stat.berkeley.edu\/~stark\/Seminars\/reproTAU15.htm#32\" target=\"_blank\" rel=\"noopener nofollow\">suggests<\/a> several points that researchers should include when writing a paper\/thesis. These should answer any questions a reader might have about how to repeat the research.<\/p>\n<p>Some of the points to consider:<\/p>\n<ul>\n<li>Have you included your raw data? How did you collect it?<\/li>\n<li>Are your materials and method detailed enough to allow repeat data collection?<\/li>\n<li>How did you process your raw data?<\/li>\n<li>Is your data analysis reproducible?<\/li>\n<li>What ad hoc choices did you make, and why?<\/li>\n<li>Why did you choose your method of analysis? Did you consider others?<\/li>\n<li>Have you included any code and data needed to reproduce tables and figures?<\/li>\n<\/ul>\n<p>These are just a few of the questions that researchers should consider when writing a preproducible report. Exactly what needs to be included will vary, depending on the research field and the type of study.<\/p>\n<h2>Challenges for Reproducible Research<\/h2>\n<p>Researchers often experience pressures that can hinder good scientific practice. Most individual scientists want to produce work with long-term value. However, they often face pressure to produce high-profile, \u201cflashy\u201d results instead. According to an article in <a href=\"https:\/\/www.nature.com\/news\/robust-research-institutions-must-do-their-part-for-reproducibility-1.18259\" target=\"_blank\" rel=\"noopener nofollow\"><em>Nature<\/em><\/a>, \u201cthe incentives to be first can be stronger than the incentives to be right.\u201d<\/p>\n<p>While higher salaries might tempt researchers in the industry, the pressures on academics can be varied. Publications, promotions, awards, and better funding prospects can all lead to \u201cbigger,\u201d rather than better, research. Most seriously, all these pressures can tempt scientists to ignore or re-work \u201cundesirable\u201d results.<\/p>\n<p>So, what is the solution? Some organizations are taking steps to tackle the problem. Journals, including <em>Nature<\/em> and <em>Science,<\/em> have updated their guidelines and introduced checklists. They inquire whether researchers have used good scientific practices such as randomizing, blinding, and calculating the right sample size. The US National Institutes of Health invites reviewers to decide whether experiment plans are soundly based on the details provided by the researcher<\/p>\n<p>Unfortunately, the research institutes have not yet come streamlined the research sharing process. To support strong science, universities could follow Good Institutional Practices (GIP).<\/p>\n<h2>Good Institutional Practices(GIP)<\/h2>\n<p>For some practices, such as animal testing and clinical work, researchers must be licensed and receive regular training. Institutions must have a member of staff to check that guidelines are met, who in turn is accountable to a regulator. This process ensures that the work meets very high standards.<\/p>\n<p>GIP could be the basis of a similar system for all research. This would help to support good scientific practices and avoid research that cannot be reproduced. If funding was linked to meeting GIP, institutions would be encouraged to comply.<\/p>\n<h2>How to Meet GIP<\/h2>\n<p>In summary, GIP should include these points:<\/p>\n<ul>\n<li><strong>Discussion of research methods;<\/strong>\u00a0Institutions should have formal processes in place to discuss methods and data collection. Researchers within the same institution should assess each other\u2019s work.<\/li>\n<li><strong>Reporting systems;<\/strong> should allow everyone, from students to staff members, to formally report any concerns with research practices.<\/li>\n<li><strong>Training and standards;<\/strong>\u00a0Often, researchers are assessed on the number and impact factor of their publications. Instead, they should be judged on their knowledge and application of sound scientific practices.<\/li>\n<li><strong>Records and quality management;<\/strong>\u00a0All records should be available for independent review.<\/li>\n<li><strong>Incentive and evaluation systems;<\/strong>\u00a0These should aim to encourage those who follow good scientific practices and excel as <a href=\"https:\/\/www.enago.com\/academy\/my-mentor-my-wizard-to-success\/\" target=\"_blank\" rel=\"noopener\">mentors<\/a>.<\/li>\n<li>Failures to meet GIP should be recorded and investigated.<\/li>\n<\/ul>\n<p>Researchers, funders, and journals can help to encourage GIP. However, achieving GIP must ultimately come from institutions and <a href=\"https:\/\/www.enago.com\/academy\/the-importance-of-scientific-transparency-and-reproducibility\/\" target=\"_blank\" rel=\"noopener\">reproducibility<\/a>, and preproducibility should be a common practice.<\/p>\n<p>What do you think of the new term \u201cpreproducibility\u201d? Is this something you already aim for in your work? Let us know your thoughts and experiences in the comments below.<\/p>\n<div style=\"display:flex; gap:10px;justify-content:\" class=\"wps-pgfw-pdf-generate-icon__wrapper-frontend\">\n\t\t<a  href=\"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/posts\/23633?action=genpdf&amp;id=23633\" class=\"pgfw-single-pdf-download-button\" ><img data-src=\"https:\/\/www.enago.com\/academy\/wp-content\/plugins\/pdf-generator-for-wp\/admin\/src\/images\/PDF_Tray.svg\" title=\"Generate PDF\" style=\"width:auto; height:45px;\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\"><\/a>\n\t\t<\/div>","protected":false},"excerpt":{"rendered":"<p>Reproducibility is vital to research practice. Fundamentally, it means that different researchers use the same&hellip;<\/p>\n","protected":false},"author":8169,"featured_media":23634,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"om_disable_all_campaigns":false,"footnotes":""},"categories":[747,2],"tags":[1618],"ppma_author":[1908],"class_list":["post-23633","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-content-structure","category-academic-writing","tag-tips-for-phd-students-and-postdocs"],"better_featured_image":{"id":23634,"alt_text":"Preproducibility","caption":"","description":"Providing exact information regarding the experiment is crucial for the research. Know more about preproducibility in the following article.","media_type":"image","media_details":{"width":750,"height":430,"file":"2019\/07\/agenda-analysis-business-plan-990818.jpg","sizes":{"thumbnail":{"file":"agenda-analysis-business-plan-990818-170x150.jpg","width":170,"height":150,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-170x150.jpg"},"medium":{"file":"agenda-analysis-business-plan-990818-401x230.jpg","width":401,"height":230,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-401x230.jpg"},"large":{"file":"agenda-analysis-business-plan-990818-750x430.jpg","width":750,"height":430,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-750x430.jpg"},"better-amp-small":{"file":"agenda-analysis-business-plan-990818-100x100.jpg","width":100,"height":100,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-100x100.jpg"},"better-amp-normal":{"file":"agenda-analysis-business-plan-990818-260x200.jpg","width":260,"height":200,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-260x200.jpg"},"better-amp-large":{"file":"agenda-analysis-business-plan-990818-450x300.jpg","width":450,"height":300,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-450x300.jpg"},"publisher-tb1":{"file":"agenda-analysis-business-plan-990818-86x64.jpg","width":86,"height":64,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-86x64.jpg"},"publisher-sm":{"file":"agenda-analysis-business-plan-990818-210x136.jpg","width":210,"height":136,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-210x136.jpg"},"publisher-mg2":{"file":"agenda-analysis-business-plan-990818-279x220.jpg","width":279,"height":220,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-279x220.jpg"},"publisher-md":{"file":"agenda-analysis-business-plan-990818-357x210.jpg","width":357,"height":210,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-357x210.jpg"},"publisher-tall-sm":{"file":"agenda-analysis-business-plan-990818-180x217.jpg","width":180,"height":217,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-180x217.jpg"},"publisher-tall-lg":{"file":"agenda-analysis-business-plan-990818-267x322.jpg","width":267,"height":322,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-267x322.jpg"},"publisher-tall-big":{"file":"agenda-analysis-business-plan-990818-368x430.jpg","width":368,"height":430,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-368x430.jpg"},"Book Review":{"file":"agenda-analysis-business-plan-990818-320x430.jpg","width":320,"height":430,"mime-type":"image\/jpeg","source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818-320x430.jpg"}},"image_meta":{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0","keywords":[]}},"post":23633,"source_url":"https:\/\/www.enago.com\/academy\/wp-content\/uploads\/2019\/07\/agenda-analysis-business-plan-990818.jpg"},"acf":{"faq_main_heading":"","faq_heading_one":"","faq_heading_two":"","faq_heading_three":"","faq_heading_four":"","faq_heading_five":"","faq_heading_six":"","faq_description_one":"","faq_description_two":"","faq_description_three":"","faq_description_four":"","faq_description_five":"","faq_description_six":""},"views":318,"single_webinar_page_date":null,"single_webinar_page_time":null,"session_agenda":null,"who_should_attend_this_session":null,"about_the_speaker_field":null,"co-webinar-sec":null,"co_webinar_sec_one":null,"speaker-name":null,"webinar-date":null,"webinar-time":null,"webinar-s-image":null,"custum_webinar_category":null,"authors":[{"term_id":1908,"user_id":8169,"is_guest":0,"slug":"eneditor","display_name":"Enago Academy","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/046a0ceeb5c38172654db93f9919593bc2e4e1391702eb8b7248865941ddbe18?s=96&d=identicon&r=g","author_category":"","user_url":"","last_name":"Academy","first_name":"Enago","job_title":"","description":"Enago Academy, the knowledge arm of Enago, offers comprehensive and up-to-date resources on academic research and scholarly publishing to all levels of scholarly professionals: students, researchers, editors, publishers, and academic societies. It is also a popular platform for networking, allowing researchers to learn, share, and discuss their experiences within their network and community. The team, which comprises subject matter experts, academicians, trainers, and technical project managers, are passionate about helping researchers at all levels establish a successful career, both within and outside academia."}],"_links":{"self":[{"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/posts\/23633","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/users\/8169"}],"replies":[{"embeddable":true,"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/comments?post=23633"}],"version-history":[{"count":0,"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/posts\/23633\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/media\/23634"}],"wp:attachment":[{"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/media?parent=23633"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/categories?post=23633"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/tags?post=23633"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.enago.com\/academy\/wp-json\/wp\/v2\/ppma_author?post=23633"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}