On October 3, 2007, the National Institute of General Medical Sciences convened a working group to assess the feasibility of computer modeling to guide policy makers in their efforts to increase the diversity of the scientific workforce. The group consisted of experts in workforce development; training, recruiting, and retaining minority students and faculty; sources of data; and computational modeling who made specific recommendations about next steps. These included:
Finally, the working group recommended that NIGMS hold a follow-up meeting of experts in computational modeling to determine if such a modeling effort is feasible and, if so, to make specific recommendations about how a modeling project could be developed. This follow-up meeting was held on October 17, 2008
Under the direction of Elias Zerhouni, NIH has begun an intentional effort to ground policy in data and quantitative analyses. Policies and programs that affect the dynamics of the scientific workforce are particularly important to NIH's and NIGMS's missions. Jeremy Berg described how modeling could improve decisions in two areas—enhancing minority participation in science and supporting new investigators. Even simple models that capture the relationship of when programs or decisions are enacted and when outcomes could be measured would be enormously helpful.
Evaluating the success of training programs is a huge challenge. Juliana Blome reported that most training evaluation projects have been driven by available data, not by the needs of policy makers. It is generally not possible to attribute specific outcomes to specific interventions. Clif Poodry pointed out that we need to know where to invest – in individuals, in institutions, or in curricula, for example. Further, we need to identify and challenge assumptions because debunking myths may be a significant outcome of a modeling effort. He urged the modelers to take account of such questions as "What makes a field desirable?," "Are there differential effects of programs and policies on minorities and women?," and "Are there ways to amplify the effects of interventions?"
Stephen Eubank summarized the views of the working group when he challenged the metaphor of a pipeline as a description of scientific workforce dynamics. The fallacy, he explained, is in assuming that people are aiming at a goal. In fact, we know little about what drives peoples' decisions at various stages of the development. He compared historical expectations to a Laffer curve, which purports to show the relationship of taxes and revenue. Although we may understand the relationship at the beginning and end, we may not know anything about what happens in the middle. The dynamics of the workforce, like those of economic systems, are complex and almost certainly depend on a variety of external forces and feedback. If the system is a network, the pipeline metaphor may lead us to focus attention on interventions that have little impact.
It is necessary, but insufficient, that models explain historical data and dynamics. Therefore, a milestone should be developing models that are consistent with existing data.
Flexible modeling strategies could allow for research and analysis from a variety of perspectives. Ross Hammond noted that agent based modeling allows for heterogeneities in the roles of actors (e.g., students, teachers, faculty), timing of interventions, economic landscape, institutional missions, and policy choices. The modeling process should start with breaking down the system into manageable pieces, possibly by identifying key transition points, and building up carefully and deliberately.
Eric Jakobsson pointed out that we know little about the structure of networks that produce and retain scientists. As an example, he explained that protein interaction networks are large and diffuse, but interactions of protein domains are much more compact and interconnected. Just as we can build protein interaction models from protein domain networks, it may be possible to study individuals as collections of social modules such as schools, neighborhoods, mentors, and so on. The challenge is finding the appropriate functional units.
The role of industry may be more difficult to study, but is critical to understanding workforce dynamics. Jack Muckstadt spoke for the entire working group in urging that industry be included in models. He further urged that we include cost/risk/benefit analyses to examine the real costs and real outcomes of programs and policies.
The modeling program envisioned by the working group is grounded in real data and focused on key policy and decision questions. Ultimately, models should provide policy and decision makers with additional information and analyses to contribute to informed decision making and to better, well-focused data collection and measurement of outcomes.
The goals of a modeling project should include the following:
Numerous reality checks are critical to the success of modeling projects and should be built into the infrastructure. For example, it is critical to involve advisors with a wide range of expertise and experience to guide the project. The working group pointed out that the MIDAS (Models of Infectious Disease Agent Study) organizational model has been successful in promoting constructive conversation about modeling questions and in developing a variety of modeling approaches suitable for different situations. They recommended that an NIGMS-supported program to model scientific workforce diversity should be organized, like MIDAS, as a collaborative network of individual groups.
The purposes to which models are put will define the scope of a modeling program. For example, if one wants to anticipate the demand for scientists, one would need a model with broad scope and long term focus. If one wants to study means for implementation, a narrower, targeted model is called for. Defining a clear set of questions in collaboration with key communities is fundamental.
What is the appropriate scope? Should a project include the medical and educational workforces? Technical staff? What areas of science should be included (or excluded)? How should industry be involved?
The 2008 working group agreed that available modeling approaches are suitable to development of a set of models of scientific workforce dynamics. Model results should contribute to a variety of questions related to training, diversity, incentives, life history, and impacts of policies and programs. The working group recommended that models include economic, social, and cultural factors as well as research on decision making strategies. Currently available data will be used to set parameters and to validate modeling approaches. One expectation is that models will inform future data collection efforts. The working group also agreed that this effort should include a diversity of modeling approaches and research groups who will work in close collaboration with NIGMS and its advisors.
This page last reviewed on
11/27/2019 11:25 AM
Connect With Us: