New Tool Primes Top Biomedical Research Resource for the Future

Data from over 300,000 clinical trials. Research on the human genome. Information on 55,000 drugs, 5,000 hazardous chemicals, and 80 genetic disorders. Studies on the efficacy of various treatment options for a wide range of diseases. Research grants for biomedical informatics and data science.

These are just some of the products and services the National Institutes of Health (NIH) provides through its National Library of Medicine (NLM)—the world's largest biomedical library and an engine for discovery for researchers across the globe. In fact, the NLM is so broad and information-rich that it became daunting to know exactly how many assets the Library had.

In 2018, NLM sought to gain deeper insights into their many offerings and asked for help from the Department of Health and Human Services' federally funded research and development center (the Health FFRDC), which MITRE operates and which was created by the Centers for Medicare & Medicaid Services (CMS). (FFRDCs serve as long-term strategic partners to the government, providing objective guidance in an environment free of conflicts of interest.)

That request led to the NLM Portfolio Analysis (NPA), a comprehensive effort to catalogue the products and services NLM offers and the resources needed to provide them.

"NLM wanted to obtain a clear view of their portfolio so that they could answer the questions that drive their decision-making processes," explains MITRE's Beth Melby, who led the NPA project.

For instance: What are the costs of providing each of NLM's products or services? What customer base does each of them serve? What IT infrastructures support each offering? And where do IT infrastructures or personnel overlap when it comes to providing these products or services?

With answers to those and other questions, NLM leaders could see what related offerings could be consolidated. They could see what services had outlived their usefulness and could be retired, freeing up resources for other, more relevant offerings.

Moreover, they'd have better data to support internal funding requests for new resources. And they would understand what NLM offerings would be affected by an IT failure, so they could build in resilience.

This task took on even greater importance when you consider the value of NLM. From PubMed to ClinicalTrials.gov and much more, the Library serves many populations—including researchers and clinicians, Congress and federal agencies, commercial entities, and the public.

Armed with new knowledge, the NLM staff knew they'd be better equipped to meet their constituents' needs into the future.

Developing a Framework for Standardized Data Collection

To deliver those insights, MITRE staff worked with leaders from across NLM to develop a framework for collecting standardized information about the Library's offerings, costs, resource usage, and interdependencies.

"We took our portfolio analysis and management expertise and applied it to resources that are fundamental to the biomedical research community," says Erin Williams, MITRE's Health Portfolio director.

The effort began with data collection about a variety of factors NLM leaders wanted to capture about their products and services.

"We came up with a framework for collecting the same information about each of the NLM offerings so that leaders could compare one to another—apples to apples," Melby explains. The framework also allows NLM leaders to see the intersections among the library's offerings, whether in terms of IT infrastructure, personnel, the databases used, or other factors.

"We used the framework to figure out what information we needed to collect for our initial baseline," Melby says. "And with that, we created a user-friendly portfolio dashboard using Tableau software."

A Picture Speaks Volumes

In February, NLM shared that early dashboard—using preliminary data—with its Board of Regents.

"The Tableau visualization of the baseline data stimulated our Board members to start asking questions and bringing up issues we hadn't considered before," says Michael Huerta, Director of NIH's Office of Strategic Initiatives and Associate Director of NLM.

"We realized then how powerful a tool the dashboard would be going forward."

But there was work still to be done.

"The initial dashboard showed NLM leaders what was possible," Melby says. "They then worked closely with us to refine the data they wanted to collect. We brought the know-how around portfolio management, but we relied heavily on our NLM partners to provide us with their specialized knowledge of NLM resources and their unique information needs from a management perspective."

A primary focus was cost data—by category. For example, how much is spent on personnel for a particular product or service? How much on IT?

Getting Ahead of the Curve on IT Cost Reporting

"One of the innovative things we did at that point in the process was to connect NLM leaders' goals for their portfolio dashboard with the Technology Business Management—or TBM—framework," Melby says.

The Office of Management and Budget is encouraging federal agencies to use the TBM framework to capture IT costs in several categories, including IT management, networking, and data.

"At some point, the government will require agencies to use this framework," Melby notes. "So, we helped NLM catalogue their costs using the TBM framework by incorporating it into the NLM portfolio framework we developed. That puts NLM well ahead of the curve on TBM adoption."

During this second phase of work on NPA, NLM leaders gained many insights.

"They were able to view their IT resources by type, by owning division, and by approximate total cost," Melby says. "They could see which IT services were being provided across the enterprise and which ones were being provided within only one division.

"And, for the first time, they could see where there were interdependencies among the offerings and the IT they used."

New collaborations between leaders emerged based on the insights they had gained about offering interdependencies.

Empowering NLM to Solve Strategic Problems

Today, an NLM portfolio team has taken the reins of the NPA framework and dashboard development process.

"We trained the NLM team so that they can change the framework depending on the information they want to collect, make the data calls, and update the dashboard," Melby says. "They're now doing everything on their own."

In December, NLM will honor the NPA team of NLM and MITRE staff with its Special Act or Service Award for their work.

"We see the framework and dashboard as an important tool to help us manage and enhance everything NLM does," Huerta says.

"NLM is very much at the center of the ecosystem for biomedical science and scholarship. We have a commitment to making our digital and physical research objects findable, accessible, interoperable, reusable, attributable, and sustainable—for all of our customers.

"This new portfolio analysis approach will help us continue to do that."

—by Marlis McCollum