We want to share with you the results of an external evaluation of the Gordon and Betty Moore Foundation Marine Microbiology Initiative (MMI). The external evaluation was conducted by the American Association for the Advancement of Science (AAAS) and a summary synthesis of the detailed 165-page AAAS report can be found here. The summary was prepared by the Moore Foundation’s evaluation consultant. The major findings and recommendations of the evaluation from the perspective of MMI—the primary target audience for the evaluation results—are listed below, along with an update on recent actions we have taken that respond to the recommendations.
The Gordon and Betty Moore Foundation launched the Marine Microbiology Initiative in 2004 to significantly increase resources and effort to delve into the “who,” “how” and “where” of major marine microbial processes. Initially authorized for 10 years (2004–2013) and $145M, MMI defined as its ten-year objective:
“Marine microbiology is transformed into an integrated field of marine microbial ecology by applying novel molecular technologies and instruments with the goal of monitoring, modeling and generating new fundamental knowledge about representative microbial ecosystems in the ocean.”
To achieve this objective, during the period 2004–2010 MMI granted approximately 70 awards supporting individual researchers, development of methods and formation of community resources. An external evaluation of the first three years of the program was conducted in 2007.
Prior to approval of a second phase of MMI (2011–2019), the Moore Foundation commissioned AAAS to conduct an evaluation of 1) the strategic focus and design, 2) deployment, and 3) results of the initiative’s first phase. AAAS also was asked to weigh in on the forward direction of Phase II, both in light of learning and recommendations stemming from Phase I, and based upon broader knowledge of the field. Phase II objectives of MMI are to uncover the principles 1) that govern the interactions among microbes (who interacts with whom, how, when, where, and the consequences thereof) and 2) that influence the microbially mediated nutrient flow in the marine environment (who consumes and excretes what, where, how much, when, and the consequences thereof).
Four Major Findings
1) MMI-supported grants made between 2004–2010 transformed significant aspects of marine microbiology that may ultimately help to transform the field of marine microbial ecology more broadly.
- The most transformative aspect of the MMI funding was bringing “omics” techniques (genomics, metagenomics and metatranscriptomics) to the forefront of marine microbial ecology and to support major breakthroughs in microbial observation.
- Before the MMI was founded, many were skeptical that these approaches could successfully advance and refine knowledge of marine microbes. The pioneering work of the MMI grantees transformed methodologies and approaches deemed high-risk only eight years ago to routinely used tools and methods today, and are now part of the canon of marine microbiology.
2) A critical contributing factor to MMI’s success in transforming significant aspects of the field was the creation of an investigator program, which provided remarkably generous funding to a small and highly select group of established and emerging leaders in marine microbial ecology.
- Although there is not a way to determine whether the advances they achieved through MMI awards might have occurred without MMI funding, there is little doubt that the program significantly accelerated the rate of discovery on some of the major questions they addressed.
3) New tools developed by the MMI may well represent one of the most significant and enduring legacies of the program.
- MMI invested heavily in equipment infrastructure to ensure that scientists could work with cutting edge, high quality facilities. That strategy worked well, with genomics investments showing significant early returns and others showing great promise.
- The ability of MMI to respond quickly to evolving technologies and take higher risks than traditional funding agencies in providing new instrumentation significantly helped bring advanced, diverse technologies to the marine microbial world and to the scientific community.
4) The detailed assessment of CAMERA found that it has offered a range of unique services including exceptionally well curated datasets, capability for large BLAST searches (unavailable elsewhere due to excessive computational intensity), customizable workflows for making sophisticated data queries, and expert consulting services to help microbiology researchers.
- The vision for MMI-supported CAMERA was to create a central data repository together with bioinformatics tools to serve the needs of the marine microbiology community and other users.
- The assessment found that CAMERA faces several important challenges with regard to future scale-up and continued service to its targeted community.
Four Major Recommendations
1) The External Advisory Committee (EAC) suggests that MMI prioritizes the following opportunities as essential to attaining its Phase II objectives and define a strategy to address them: i) MMI consider expanding to address other environments (coastal, deeper waters, sediments, etc.); ii) extend microbial interactions with other organisms up the food chain; iii) support study of changing oceans and impact of human and natural perturbations
MMI Program Director Response:Precisely because MMI cannot encompass all aspects of marine ecology, we have developed our Phase II focus areas to emphasize new knowledge of microbial interactions and microbially mediated nutrient flow. The overarching goal of MMI during Phase II is to uncover generalizable ecological principles and they will be pursued in the environments most appropriate for the questions being addressed. Further, by supporting a deeper understanding of important ecological principles, the new knowledge gained will enhance the research community’s understanding of all marine environments, many ocean processes and many if not all types of environmental change.
2) The EAC recommends that workshops are more extensively used to bring together PIs, post docs, and students with the MMI investigators to promote cross-fertilization among and synthesis across projects and to add value to individual projects. MMI Program Director Response: We recognize the need for further integrating all our grantees to promote cross-fertilization and synthesis across projects, not just the investigators as was done in the first years of the Initiative. We also recognize the value of convening workshops. We have begun deliberate efforts to incentivize interactions and collaborations using multiple mechanisms. As one example we convened a modeling workshop last spring that included seventeen PIs from nine different projects and about fourteen additional scientists. We are also planning a symposium later this year that will bring together just postdocs and research associates from all the projects we fund.
3) The EAC does not recommend continuing CAMERA in its present form and suggests considering the possibility of evolving CAMERA into an entity that collaborates with other federal and foundation-funded projects for developing and managing databases. This strategy would deliver an extremely valuable contribution to the field as the size and types of data sets continue to increase.
MMI Program Director Response:CAMERA’s goal -- to create a cyberinfrastructure to enable researchers to query genomic and metagenomic data within an appropriate environmental context -- was visionary when it was established in 2004. Eight years later, there are few comparable entities that provide the exceptionally well-curated metagenomic datasets found within CAMERA and thus it continues to be a valuable resource to the community. Since CAMERA’s inception, sequencing and computing technologies have changed dramatically, requiring a greater diversity of bioinformatics tools and computing needs to address questions that were unimaginable only a few years ago. We agree with the EAC’s recommendation and will seek to encourage the development of a diverse and durable ecosystem of marine metagenomics cyberinfrastructures through partnerships with multiple grantees and funders.
4) The EAC recommends that the Moore Foundation find ways to bring the broader marine community into their advisory process. The broader marine microbial community needs to understand MMI’s decision-making process and how this tracks with the initiative’s long-term, targeted time-phased outcomes. In addition, the program staff ought to engage researchers from other areas such as biogeochemistry and nutrient dynamics.
MMI Program Director Response: We recognize the need for the community to better understand what MMI does and how we do it. Therefore, we are implementing ways for improved and increased communications with the community. As an example, we have started sponsoring social gatherings and town hall meetings at appropriate venues such as the Ocean Sciences meeting where we create opportunities for two-way conversations with the community about our activities. We engage with and use the multiple external advisors who assist us in grant reviews and provide us guidance in designing and managing our portfolio, although as with any grant review process, the identities of reviewers and contents of proposals are held strictly confidential. We see value in cultivating our grantees to be ambassadors of MMI to the science community and in maintaining close communications with other funders about our goals and processes.
A summary of the AAAS 2012 evaluation of the Gordon and Betty Moore Foundation’s Marine Microbiology Initiative can be found here. Please note that the 20-page linked document is a summary prepared by the Moore Foundation of the 41-page "MMI 2012 Evaluation by AAAS" submitted in 2012, which can be found here.