ABSTRACT

Graduate education in microbiology and other biological sciences has produced the scientific cadres that ushered in a revolution in medicine, agriculture, and industrial biotechnology, which has already brought great benefit to humanity. However, the graduate education enterprise is under pressure and criticism for overcapacity, for producing scientists who are too specialized, and for not being conducive to the training of versatile scientists capable of operating successfully in many parts of society, where scientific know how is desirable and needed. These challenges provide opportunities for reforming the graduate education enterprise to better align it with the needs of society and science in the 21st century.

Front Matter

Summary of Recommendations

• Broader knowledge
• Critical thinking
• Communication
• Interdisciplinary and collaborative work
• Competence in statistics and data analysis using today's new tools
• Good experimental design
• Effective people skills
• Risk assessment
• Project development from beginning to end
• Budget management and general business skills
• Effective mentoring and the research advisor
• Lifelong learners
• Ethics

How the field of graduate education is changing

The ability to train an individual to conduct deep research in a particular topic is one of humanity's great educational achievements. Today, practically every graduate program in the world can take an applicant with a background in science and, after completing a process of didactic education combined with mentored research, generate a candidate for a doctoral degree trained in scientific research. Such individuals can conduct deep research in their selected areas of expertise. The notion of investigating the natural world through experimentation is a relatively recent human advance. The existence of graduate programs on every continent creates a production pipeline to produce research scientists and experimentalists capable of working on the major challenges facing humanity.

Graduate programs are the only source of trained scientists in the biological sciences. These programs teach scientific knowledge through scholarly didactic methods and laboratory research through hands-on experience in research groups. Such programs produce individuals capable of addressing scientific problems through experimentation and data analysis. In recent decades, for example, microbiologists have studied microorganisms that can clean up contaminated soils and improve wastewater treatment, virologists have identified the most abundant biological entities in the ocean and the role these viruses play in enhancing ecosystem productivity, and the biomedical sciences have produced a dizzying series of advances that have greatly enhanced human life by reducing pain and suffering from disease. Notable advances in recent years include vaccines to prevent cancer, in vitro fertilization, antiretroviral therapy, organ transplantation, and immunotherapy for cancer. The continued success of the biomedical revolution is critically dependent on the excellence of the training pipeline for future scientists. Hence, if the 21st century is going to live up to the promise of being the biological century, then graduate training programs in microbiology need to produce superb future scientists capable of keeping the revolution going.

Figure

Graduate programs are the only source of trained scientists in the biological sciences

Despite the success of graduate programs in producing highly trained scientists capable of carrying out deep investigations, there are signs that graduate education needs reform. The focus on training individuals for research careers means that the training is highly specialized and not necessarily suitable for other employment opportunities in society. For example, there is a need for scientifically trained individuals in government, law, media, philanthropy, and nonprofits. However, the training available in graduate programs does not necessarily impart communication skills or the intellectual analytical tools necessary for versatility in approaching scientific problems outside of the laboratory. This has led to calls for modernizing the graduate curriculum while maintaining excellence in scholarship and research (1). Graduate programs have been criticized for being too focused on training scientists for academia at a time when the employment opportunities in that sphere are limited (2). The narrowness in the training contrasts with students' desire and enthusiasm for opportunities to learn about diverse jobs in science (3).

Graduate programs are notoriously poor at teaching communication skills (4). The difficulties involved in communicating science to the public and nonscientists pose a major problem for alumni of microbiology and other biology-related graduate programs, because the training limits employment opportunities outside of conventional scientific jobs. Poor communication skills of scientists can contribute to the widening chasm and disconnect that can occur between public and scientific opinion in certain issues relevant to policy, such as climate change, vaccination, and the need to fund fundamental research.

Figure

The difficulties involved in communicating science to the public and nonscientists pose a major problem for alumni of microbiology and other biology-related graduate programs

The teaching and learning experience in graduate biological science programs has also come under criticism (1, 5, 6). Didactic parts of the graduate program focus on specialized areas of biology that provide information-rich content without much attention to critical thinking. Many programs continue to produce graduates with inadequate training in statistics, computer programming, or quantitative biology despite the increased use of mathematics in all aspects of biology, as exemplified by the availability of “big data” in genetics and the rise of bioinformatics as a distinct field of study. The experimental part of the Ph.D. thesis usually takes place under the supervision of an advisor who provides direct input and guidance into the research project. This guild-like, apprentice model of training can be highly variable, depending on the student-advisor relationship, and there is no assurance that all combinations result in adequate mentoring in experimental techniques, critical thinking, and the process of research.

There is the concern that the production of individuals with Ph.D. degrees is much greater than the demand and that this is a worldwide problem (7). Furthermore, the length of training has increased in recent decades. A longer time to Ph.D., combined with lengthy postdoctoral training periods, means disproportionately longer times for scientists in training.

Another serious issue that emerged in recent years is the realization that there are significant problems with the quality of science produced in some biomedical research laboratories. The first hint came from two papers reporting poor reproducibility of published findings from academic laboratories by industry investigators (8, 9). This was followed by the reports that questionable research practices were common among investigators (10), that most retracted papers were the result of misconduct (11), and the finding that up to 4% to 6% of publications had an inappropriately reproduced image (12). Although graduate biomedical education is not directly linked to these problems, there is the concern that inadequate scientific training was producing inadequately prepared individuals, who in turn were producing scientific findings that were not reliable. This raises the possibility that reforms in graduate education that improve the training of scientists could also have a salutary effect on the quality of their scientific output.

The very nature of science, technology, engineering, and mathematics (STEM) research is also evolving as developments such as big data and artificial intelligence generate new demands for inter- and multidisciplinary research teams and for quantitative skills. This leads to new training pressures that challenge university infrastructures based on specific college disciplines. Given that 90% of data in the world today was created in the past 2 years (13) and with recent estimates placing growth of data at 40% per year (14), an emerging need exists for analytical training that enables future researchers to easily extract knowledge and insights from large datasets.

In addition to the changes described above, recent evidence suggests that graduate students and their expectations for education, as well as the way they are learning and communicating, are also changing. For example, an expanding body of literature provides evidence that today's students experience more stress in ways that differ from those of previous generations of students (15-18) and suggests that stress negatively impacts their ability to learn. Given that the population of students entering STEM graduate programs is now more diverse in many ways, including gender (19), race and ethnicity (20), disability, socioeconomic background, and country of origin and given that students with Ph.D.s now pursue many varied career paths (21), modernization of graduate programs would be expected to benefit graduate students in a significant number of ways.

Statement of Task

The need for changes in graduate education has been the subject of recent discussions, including a National Academies report titled “Graduate STEM Education for the 21st Century” (22). The American Academy of Microbiology (Academy) convened a colloquium to explore the role of professional societies in graduate student training. Colloquium participants focused on the role that partnerships can play in training scientists who are deep thinkers and able to apply their training to broad application, strengthening the “Ph” in the Ph.D.

Issues and Recommendations

Figure

To create new paths in our chosen field, we must collaborate with people in other disciplines

Unlike scientists who were exploring the world before the computer era, so much new information is presented to us every day that it is impossible for one person to incorporate or use it all. Furthermore, practically the totality of scientific information gathered by humanity is instantaneously accessible through smart mobile devices to anyone interested in science. This in turn has created a new challenge in using and integrating scientific information. To create new paths in our chosen field, we must collaborate with people in other disciplines, not only to ensure that we have all the facts in hand but also to ensure that we are in touch with all the new ways to work with those facts. Unique computational and analytical tools are being created all the time, and scientists in all areas of endeavor, from research to industry, must be able to use them to pursue their goals. Scientists need to be able to work closely, rather than competitively, with their colleagues and communicate their findings to others in their field, to policymakers in the government, and to the public. Additionally, the current training model selects for students from similar backgrounds and often discourages graduate studies by a broader spectrum of the population. Increasing student diversity brings varied approaches to scientific problems and is critical to increasing participation in science.

The colloquium participants focused on several key areas: how the field of microbiology is changing, the new skills graduates must learn in order to change with it, and how professional societies and graduate programs can help them gain these skills. Below, we have identified key components of a modern graduate education and identify roles that can be played by the graduate program and professional societies in training students in these skills.

The colloquium participants focused on several key areas:

1. how the field of microbiology is changing
2. the new skills graduates must learn in order to change with it
3. how professional societies and graduate programs can help them gain these skills

Some graduate programs are addressing the decline in academic research positions by reducing the number of students they admit. Instead of discussing that option, the participants in this colloquium explored ways to revamp master's and doctoral education systems so as to prepare students for the more diverse workforce needs outside of academia and research. The participants noted that, at a time of rapid scientific and technical progress, society needs scientifically trained individuals in many fields, including administration, law, government, etc. Revamped programs would include courses on critical thinking, innovative thinking, responsible conduct of research, error analysis, statistical methods, tackling logical fallacies, ethics, mentorship, teamwork, communication skills, collaboration across scientific disciplines, and recognizing the diversity of societal tasks that need science practitioners in our current society. Finding ways to stimulate curiosity, creativity, and the kind of self-direction common to Louis Pasteur, Esther Lederberg, Marie Curie, and Linus Pauling was also a major focus, as was an emphasis on the norms of good scientific practice: rigor, responsibility, and reproducibility, which are the keys to establishing the framework of evidence and moving easily between disciplines.

Interdisciplinary and collaborative work

Tackling a problem with scientists from other fields often means taking a systems-level approach, and that creates widely relevant solutions and innovative, transformative science. Interdisciplinary teams usually find more new ways to solve the problems that they address than do individuals working alone or only with others in their field, and such teams develop a broader field of vision and increase their productivity as well. Furthermore, diverse teams are more productive and more creative than those composed of individuals from the same field or with common backgrounds. Breadth does not mean sacrificing depth, because the more perspectives that are available, the deeper is the exploration.

People on interdisciplinary teams also tend to improve their communication skills because they must interact with others who do not use the same technical language. Students who have interdisciplinary experiences as graduate students are more likely to collaborate later in their career, and older scientists who work collaboratively on a regular basis have more job satisfaction and are less likely to burn out.

Collaborations, particularly interdisciplinary collaborations, often lead to more creative and profound solutions to problems. Students should have opportunities to explore collaborative research.

Potential problems associated with interdisciplinary work include increasing time to degree. However, this may not be a negative if it means less time as a postdoctoral fellow or if it eliminates the need for other training. Publications that result from interdisciplinary work usually have more authors than within-field studies. Also, it may be more difficult for people who make promotion and hiring decisions to determine each individual's contribution, with the exception of the first author. Creating authorship agreements before a project begins, like the ones common in industry, usually solves this problem. There may also be a perceived lack of depth in interdisciplinary work by reviewers who are not acquainted with all the contributing fields and who cannot adequately assess the enterprise. This is particularly true at the grant review stage or the publication stage of research.

Ethics

Students need a thorough grounding in ethics that includes proper ways to acquire and present information; reproducibility; responsible conduct of science, including working with living organisms; and evaluating the impact of the discoveries that are being made now and will be made in the future. Ethics programs should evolve as science evolves.

Roles of the Graduate Programs and Professional Societies

Recommendations for Change

The Role of The Graduate Program

Colloquium participants focused on the academic and professional development of graduate students. They recommended that institutions of higher learning rededicate themselves to more holistic education and training rather than emphasize getting students into laboratories to help with research. They also felt that everyone who works directly with students needs training in mentorship, supervision, and ways to help young scientists become innovative and independent. Good mentorship may mean guiding a student into a different field, and the needs of the student must always come before the needs of the PI/ advisor. Getting postdoctoral training in other areas may be the only way for newly minted doctoral students to achieve their intellectual and professional goals.

The participants agreed that revising graduate education could be accomplished with few administrative burdens and only modest increases in funding. Small changes in the graduate programs can have tremendous downstream effects and cost nothing.

Reforms must be pilotable, scalable, and measurable. Modules are a good idea, but some things — like project management — require hands-on experience and taking a project to completion.

Metrics of success should be thought out carefully. Graduate programs should start with clear learning objectives that are communicated at the beginning of the program. Milestones and competencies need to be assessed regularly during the program. Metrics of success should also be holistic. For example, one measure of success is whether everyone in a group can demonstrate knowledge of what everyone else in the group is working on. This would demonstrate a comprehensive understanding of how all components of the group's work fit together. A nonholistic metric, time to degree, was largely viewed negatively, and it was firmly stated that finishing the Ph.D. degree in four years was not an appropriate measure of success. However, many participants thought it important to keep track of time to degree data, because very long times to degree could signal a problem, with either the student or the supervisor, that merits attention. Other metrics of success discussed at the colloquium included diversity of graduate student output (such as peer-reviewed publications, patents, databases, precomputed data products, invited talks, etc.), permanent employment, retention of a science-based career, employment in the jobs for which the students trained, postgraduate school publications, and progression in later positions. It was agreed that students should be followed for at least 15 years with surveys that include salary questions.

The first step in changing graduate programs must be changing graduate admissions requirements

The first step in changing graduate programs must be changing graduate admissions requirements. The participants supported eliminating the GRE requirement and not using cutoff scores during the admission process. Admitting intellectually able older students and minority students whose life circumstances have kept them out of the top group on paper, then sending them to boot camps to bring them quickly up to speed or to prep programs so they will be ready for admission in a year, is proving to be a rewarding way to assemble a more diverse student body. In any case, all students' skills, competencies, and needs should be assessed as soon as they are admitted to a graduate program.

Student thinking should be challenged early. They should be asked to explain what they are doing and why they are doing it as often as possible. They should be able to show different ways to approach the problem at hand during these interviews and present the advantages and disadvantages of each plan of action. They should also learn to criticize and get critiques from their peers.

Programs should provide structured learning and feedback experiences from the beginning. Students should meet with their advisors/PIs regularly and develop charts that plot their projects from beginning to end. The skills they need to learn and the reasons they need to learn them should be made clear from the program's start, as well as how and when those skills will be assessed.

Figure

Students should meet with their advisors/PIs regularly and develop charts that plot their projects from beginning to end

Courses in communication skills, the ethical conduct of science, teaching, mentoring, and good research design should be introduced early as well. Students should learn the value of null results in research as soon as possible, even though the scientific community often fails to recognize null results as worthwhile reportable subject matter.

The faculty should not expect students' critical thinking skills to be fully developed when they enter the program. A metric should be set to evaluate student progress in this area, and critical thinking courses should continue to be required until students are able to meet specified critical thinking goals. Students should not be set adrift if they do not master these skills right away.

There should be opportunities for interdisciplinary projects, storytelling should be employed to practice dialog across disciplinary boundaries, and innovative journal clubs and field trips should enrich the graduate agenda. Seminars and meetings that promote interdisciplinary learning communities should be developed, and these communities should include regular brainstorming sessions.

The interlocking roles of chemistry, computer science, mathematics and applied statistics, and the biological sciences should be explained during the first year of graduate school. The institution's chemistry and computer science departments should be encouraged to develop courses tailored to the needs of life-science graduates.

Critical thinking, logical thinking, and lifelong learning should be modeled by the faculty. The faculty should encourage students with expertise in new technologies to run short courses to bring the faculty and their fellow students up to speed, and work groups could have journal clubs or go on weekend retreats to master a new subject or learn a new skill together.

Figure

Critical thinking, logical thinking, and lifelong learning should be modeled by the faculty

Some of the professional skill courses can be taught at a core facility on campus open to students from other graduate schools. Subject matter experts in each field should be included so that students can link the skills they are learning to their discipline.

Core facilities should be available to students at all times so that they can learn new technologies as soon as they need them. Statistics and quantitative biology should be required courses for everyone, as should courses on how to analyze datasets and basic computer programming.

Students should be acquainted with the resources elsewhere on their campus (such as business schools and writing centers) and in the community (professional societies, government, NIH resources). Institutions should enrich their stable of resources all the time.

PIs should look critically at their own skill sets so that good PI/student matches can be made. There is no ideal; both micromanagers and hands-off advisors have their place.

There was consensus on the fact that there is a significant power imbalance between students and their PIs and within work groups that must be addressed directly. Emotional support for students should be provided so that their voices can be heard, and the need for interpersonal boundaries must be emphasized. Crossing those boundaries results in significant risk for students and for the institutions where they are being trained. Providing the student with a mentor, in addition to the supervisor, from their department to increase the student's exposure to additional leadership styles and to allow the mentor to understand any issues that may arise is an approach that is gaining momentum in STEM fields (11). It would take only an hour of the mentor's time once a month and would not be expensive.

PIs should understand that students are part of the graduate program, not their personal employees. When the needs of the PI clash with the needs of the student, the student's needs must come first.

Advisors/PIs should not be present for the entirety of thesis committee meetings. Students need to be able to speak freely with the committee about their concerns and use the committee to help them solve conflicts with their advisor. They also need the experience of the meeting being “their show” with no crutches to fall back on.

Graduate students should be provided opportunities to meet people from a variety of careers, including industry and government, who can show them the range of career options outside of academic research as early as possible. Institutions should bring these people to campus and make it easy for students to access them. Students should also be able to intern in adjunctive areas if they choose, and administrative and financial barriers to switching fields as postdoctoral students should be removed.

Institutions and professional societies should make reasonable efforts to provide graduate students in the life sciences a chance to intersect with the legislators and program officers who regulate and fund their work. This helps students to understand real-world science and gives legislators an opportunity to meet the next group of upcoming scientists, some of whom may choose to join government agencies to interpret complex science for those who administer it and advise them on scientific concerns.

Graduate programs should make sure that all graduates have employable skills and receive the training appropriate to their career choice.

The Role of Professional Societies

The committee discussed the changing nature of professional societies and how some of them are responding to externalities such as open access to scientific information and decreased sales of hardcover books by developing new business models and operational strategies. Considering the widespread use of social media to communicate scientific findings and resources, such as Twitter, Research Gate, and LinkedIn, to generate networking opportunities, the committee discussed whether professional societies — traditional sources of scientific networks — are still worth joining and whether they might have a future role in graduate education. The answer, unequivocally, was yes.

A better-trained scientific workforce is of central interest to scientific societies, and societies can take a more active role in education

Professional societies can contribute to graduate education in a variety of ways. A better-trained scientific workforce is of central interest to scientific societies, and societies can take a more active role in education. In fact, at a time when the role of societies has been diminished by the emergence of non-society meetings and journals, taking a role in education could provide new missions and new vigor.

Figure

Society members can form networks

Societies can create guidelines on best practices in graduate programs. For example, the ASM could acknowledge Ph.D. programs that follow these practices.

Professional societies can create public policy and domain-intensive fellowship programs and short-term programs, including online courses that teach students specific skills.

Professional societies can create mentoring programs and create guidelines and training for mentors.

Society members can form networks that allow students to spend time in research labs outside their own institution during their graduate years.

Societies can create worldwide partnerships with other societies to provide enrichment experiences for graduate students at cross-disciplinary, jointly sponsored meetings, such as integrating microbiology and the environmental sciences.

A society's annual meeting can be made more useful to graduate students and early-career scientists. Some of the ways to do this might be:

• Creating more opportunities for younger scientists to present their work at meetings.
• Organizing tables of prominent scientists with whom students could visit and talk.
• Creating networking opportunities such as mixers for women, underrepresented minorities, and people with specific career interests.
• Creating mixers to bring alumni of a particular program together with current students in that program.
• Offering workshops or modules on different career paths.
• Offering workshops on the value of a Ph.D. and how it enables people to contribute to the social good.
• Letting graduate students organize sessions on their perceived needs or other topics that concern them, which will also allow them to develop organizational skills.
• Setting up job-finding, matchmaking activities at annual meetings and creating job fairs that include professional development workshops, science talks, and networking and job interviews paid for by employers who want to hire highly trained people.
• Holding workshops such as the Microbe Academy for Professional Development. This ASM workshop for undergraduates and graduate students is held the day before its annual meeting. It started as a way to prepare students to present at the meeting, but it now includes ethics, mentoring and being mentored, and a variety of other tracks that students can choose to suit their needs.

Interdisciplinary work should be incentivized with special awards by societies and special presentations at annual society meetings.

Societies can add sections to their websites especially for graduate students and graduate education that contain job listings and postdoc listings as well as links to materials warehouses and other resources. They could provide webinars that are free to graduate students and one of the benefits of membership, summaries of current technologies with links to in-depth information, and introductions to non-microbiology organizations that fund microbiological research. There should also be a section on the society's main website where new graduates can post summaries of their thesis papers.

Societies should be a bridge between training and the rest of a scientist's career and should provide services for every career stage: student, postdoctoral, early research, mid-level, senior level, and retirement.

Societies should create meetings on cutting-edge topics but maintain these meetings only as long as the topic remains cutting edge. When it becomes mainstream, the conference should be discontinued and another one should be set up on a current topic.

Regional meetings, such as those organized by the American Society for the Advancement of Science, could be created to foster geography-centered networking and provide regular training opportunities. ASM branch meetings provide a platform for regional interactions and career development for students.

Societies could create welcoming committees of their members in each community that help new graduates adjust to the area and find the resources it has to offer.

Examples of Model Graduate Programs

There are model graduate programs that have successfully experimented with new approaches to graduate education; a noncomprehensive set of examples is provided below.

There are model graduate programs that have successfully experimented with new approaches to graduate education

The Johns Hopkins Bloomberg School of Public Health has started a pilot graduate program that contains all the elements discussed in this paper. The course offerings for the first two years of this program are outlined in Tables 1 and 2 on the following pages.

The 17 institutions funded by the National Institutes of Health's Broadening Experiences in Scientific Training (BEST) program have successfully incorporated career opportunity-broadening experiences into their programs. Started in 2013, the BEST program was designed to develop innovative approaches to career exploration by Ph.D. students and postdocs that could be piloted, adapted, and adopted by the broader community of research institutions.

Predating the BEST program is the University of California at San Francisco's Graduate Student Internships for Career Exploration (GSICE). This program is philanthropically funded and supports mentored career exploration by providing career planning and full-time 3-month internship opportunities for advanced Ph.D. students (i.e., those who have passed all Ph.D. qualifying exams) to enable free and informed career decision-making before the students graduate.

Graduate Biomedical Science Education Needs a New Philosophy

Table 1.

R³ curricular plan for academic year 1: skill training in critically thinking and conducting science (5).

Table 2.

R³ curricular plan for academic year 2: continuous training in thesis-related background and professional practice skills (5).

References

1.

Hitchcock P, Mathur A, Bennett J, Cameron P, Chow C, Clifford P, Duvoisin R, Feig A, Finneran K, Klotz DM, McGee R, O'Riordan M, Pfund C, Pickett C, Schwartz N, Street NE, Watkins E, Wiest J, Engelke D. The future of graduate and postdoctoral training in the biosciences. eLife 2017; 6:e32715. doi: 10​.7554/eLife.32715 [PMC free article: PMC5648525] [PubMed: 29049023]

2.

Daniels RJ. A generation at risk: young investigators and the future of the biomedical workforce. Proc Natl Acad Sci U S A 2015; 112(2):313–318. [PMC free article: PMC4299207] [PubMed: 25561560]

3.

Mathur A, Chow CS, Feig AL, Kenaga H, Moldenhauer JA, Muthunayake NS, Ouellett ML, Pence LE, Straub V. Exposure to multiple career pathways by biomedical doctoral students at a public research university. PLoS ONE 2018; 13(6): e0199720. https://doi​.org/10.1371/journal​.pone.0199720 [PMC free article: PMC6014666] [PubMed: 29933412]

4.

Brownell SE, Price JV, Steinman L. Science communication to the general public: why we need to teach undergraduate and graduate students this skill as part of their formal scientific training. Journal of undergraduate neuroscience education: JUNE: a publication of FUN, Faculty for Undergraduate Neuroscience 2013; 12(1):E6–E10. [PMC free article: PMC3852879] [PubMed: 24319399]

5.

Bosch G, Casadevall A. Graduate biomedical science education needs a new philosophy. mBio 2017; 8:e01539–17. https://doi​.org/10.1128/mBio.01539-17. [PMC free article: PMC5736909] [PubMed: 29259084]

6.

Chabowski D, Kadlec A, Dellostritto D, Gutterman D. Adapt or perish: updating the predoctoral training model. Circulation Res. 2017; 120(7):1081–1083. [PMC free article: PMC5380225] [PubMed: 28360347]

7.

Cyranoski D, Gilbert N, Ledford H, Nayar A, Yahia M. Education: the PhD factory. Nature 2011; 472(7343):276–279. [PubMed: 21512548]

8.

Begley CG, Ellis LM. Drug development: raise standards for preclinical cancer research. Nature 2012; 483(7391):531–533. [PubMed: 22460880]

9.

Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov 2011; 10(9):712. [PubMed: 21892149]

10.

Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One 2009; 4(5):e5738. [PMC free article: PMC2685008] [PubMed: 19478950]

11.

Fang FC, Steen RG, Casadevall A. Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 2012; 109(42):17028–17033. [PMC free article: PMC3479492] [PubMed: 23027971]

12.

Bik EM, Casadevall A, Fang FC. The prevalence of inappropriate image duplication in biomedical research publications. mBio 2016; 7(3):e00809–16. doi:10​.1128/mBio.00809-16 [PMC free article: PMC4941872] [PubMed: 27273827]

13.

Hale T. How much data does the world generate every minute? IFLScience 26 July 2017. http://www​.iflscience​.com/technology/how-much-data-does-the-world-generate-every-minute/

14.

National Academies of Sciences, Engineering, and Medicine. Refining the concept of scientific inference when working with big data: proceedings of a workshop, 2017. The National Academies Press, Washington, DC, USA. doi: https://doi​.org/10.17226/24654 [PubMed: 28252890]

15.

Levecque K, Anseel F, De Beuckelaer A, Van der Heyden J, Gisle L. Work organization and mental health problems in PhD students. Research Policy 2017; 46(4):868–879. https://doi​.org/10.1016/j​.respol.2017.02.008

16.

Pain E. Trainees and mental health: let's talk! Science Careers Blog 11 April 2016. American Association for the Advancement of Science, Washington, DC, USA. doi:10​.1126/science.caredit.a1600058

17.

Pain E. Ph.D. students face significant mental health challenges. Science Careers Blog 4 April 2017. American Association for the Advancement of Science, Washington, DC, USA. doi:10​.1126/science.caredit.a1700028

18.

Tsai JW, Muindi F. Towards sustaining a culture of mental health and wellness for trainees in the biosciences. Nature Biotechnology 2016; 34(3):353–355. https://doi​.org/10.1038/nbt.3490 [PubMed: 26963557]

19.

National Science Board. Science and engineering indicators 2016. Appendix Table 2-24, S&E graduate enrollment, by sex and field: 2000-13. 2016. National Science Foundation, National Center for Engineering Statistics, Alexandria, VA, USA. https://www​.nsf.gov/statistics​/2016/nsb20161/#/

20.

National Science Board. 2016. Science and engineering indicators 2016. Appendix Table 2-26, S&E graduate enrollment by citizenship, field, race and ethnicity: 2000-13. 2016. National Science Foundation, National Center for Engineering Statistics, Alexandria, VA, USA. https://www​.nsf.gov/statistics​/2016/nsb20161/#/

21.

St. Clair R, Hutto T, MacBeth C, Newstetter W, McCarty NA, Melkers J. The “new normal”: adapting doctoral trainee career preparation for broad paths in science. PLoS One 2017; 12(5):e0177035. http://dx​.doi.org/10​.1371/journal.pone.0177035 [PMC free article: PMC5443479] [PubMed: 28542304]

22.

Leshner A, Scherer L (ed). Graduate STEM Education for the 21st Century. The National Academies of Sciences, Engineering, and Medicine. 2018. The National Academies Press, Washington, DC, USA. doi: https://doi​.org/10.17226/25038Downloaded by IP: 66.208.62.130 On: Tue, 24 Sep 2019 15:52:58