Building a network of care

NCSA is so much more than just supercomputers – something that researchers and affiliates know well. This is why Crystal Reinhart, a senior research scientist at the University of Illinois Urbana-Champaign’s Center for Prevention Research and Development (CPRD), ended up working with NCSA on an important project her center is involved with – the Illinois Medicaid SUD Demonstration Waiver evaluation.

The Project

In 2018, an Illinois Medicaid Demonstration Waiver began – the five-year program temporarily expanded Medicaid services to include a wider variety of treatments for Substance Use Disorder (SUD) and provided better coverage and access to existing services. The idea was that if patients diagnosed with an SUD had access to community-based services, like case management and peer support programs, they would avoid using costlier crisis-based services, like emergency room (ER) trips. There was also a hope that this demonstration waiver would reduce the number of deaths due to overdose by removing some of the hurdles to treatment.

CPRD’s role was very specific. As Reinhart explains, “The Centers for Medicare and Medicaid (CMS) requires that an independent evaluator evaluate the effectiveness of the demonstration waiver. CPRD was hired to be the independent evaluator for Illinois. We have an evaluation plan that includes several metrics that we measure in our evaluation.”

Reinhart says these metrics include things like how many times a patient diagnosed with a SUD visited the ER, when their treatment was initiated, what treatments were used, etc. CMS and its partners provide the technical parameters for measuring these metrics using Medicaid claims data. CPRD had the know-how to perform the evaluation, but they needed a place to store the data for roughly 150,000 patients who have been diagnosed with an SD and a machine powerful enough to run statistical evaluations on this large dataset.

NCSA was able to help with that lack of HPC expertise that we couldn’t get with our other options. By going with NCSA, we would have adequate support to learn this system.

–Crystal Reinhart, senior research scientist, CPRD

“We knew that we wouldn’t be able to do this on any of the servers that we have,” said Reinhart. “Individual machines like laptops aren’t going to have enough computing power to run statistics on data for 150,000 people. Additionally, the data is complex. It’s not just one row of data for each person. Claims data includes every time a patient goes to the doctor, the pharmacy, convenient care or the ER. For anything related to a medical visit, there’s going to be a claim – sometimes multiple claims for one visit. So claims data is really big.”

Reinhart’s team spoke with several organizations before deciding who would provide their High-Performance Computing (HPC) needs. “We spoke with the University of Chicago, they’ve used Medicaid data for other projects in the past, so we got some advice from them on who we might reach out to for assistance. We also talked to a number of people on campus. In the end, we landed on two options. We could use the HIPAA-compliant server at NCSA, Nightingale, or we could use a commercial cloud service provider. Ultimately, we decided to go with NCSA.”

For Reinhart’s team, the big difference between NCSA and their other options was the support they would get by partnering with NCSA. “Obviously, we really liked that Nightingale is HIPAA (Health Insurance Portability and Accountability Act) compliant, that it’s a well-monitored system and very powerful. We knew that with NCSA, we had the opportunity to increase the size or computing capacity in the future if we needed to. However we were a little nervous because we only had one staff member that had ever used a Linux-based system before. NCSA was able to help with that lack of HPC expertise that we couldn’t get with our other options. By going with NCSA, we would have adequate support to learn this system.”

The Nightingale Experience

With their chosen system ready to accept Medicaid data, Reinhart’s team got to work. All along the way, NCSA was there to guide and mentor the team at CPRD as many of them tackled using their first supercomputer and Linux-based system. “They helped us with a number of aspects,” said Reinhart. “They helped build the SQL database. We also conduct a lot of different statistical analyses, so we needed several stats software programs installed, SPSS, Stata, R – those are the ones we most commonly use. NCSA got them installed for us to use on the system and taught us how to run them on a supercomputer like Nightingale because those programs don’t have the same interface as they do on a Windows system.”

NCSA’s support team consisted of experts familiar with HPC and the Nightingale system. Chris Pond, head of Research Data Engineering at NCSA, worked with Chris Stephens, NCSA lead research data engineer, to get the project rolling. “Chris Stephens and I set up databases for CPRD on the dedicated Nightingale database nodes, created custom database schemas based on the original Medicaid schemas provided by the Office of Medicaid Innovation (OMI), ingested data sets both from Crystal’s team and from OMI, and transformed and cleaned the data to make it suitable for CPRD’s use,” said Pond. “Early on, we also provided assistance writing the queries necessary to complete some of the early metrics before Crystal was able to bring additional database expertise directly onboard her team. We continued to sanitize and load additional datasets as they were provided, and new years of Medicaid data were available.”

This project is a good example of how the Nightingale Cluster enables new collaborations between University researchers and healthcare, government, or industry. Our experts help manage requirements around sensitive data so the users can focus on their research.

–Maria Jaromin, assistant director of healthcare research, NCSA

This level of support came with its own challenges. “One challenge early on was making sure that the OMI, CPRD, and NCSA teams were all on the same page,” Pond explained. “The Medicaid data schemas and queries are quite complex, and each team put forth a lot of effort to bridge the differences in each team’s domain knowledge, to come to a common understanding of what was needed to generate accurate metrics.”

While there was a lot to do, there were also some perks to working on a project like this. Many research projects that NCSA experts assist with are ongoing projects with long-term goals, each stage building upon the last. It can take a long time to see the results. The impact of the Medicaid waiver program was not only immediate but also directly assisted the Illinois community.

It’s quite rewarding to contribute to a project that has such a direct impact on those who will benefit from the continuation of the Medicaid waiver as a result of the CPRD findings.

–Chris Pond,  lead, Research Data Engineering, NCSA

Through it all, Reinhart had a contact at NCSA that guided her to the right person to ask her questions. “Maria Jaromin is my go-to contact at NCSA. I always reached out to her first. She’s going to know the answer, or she’s going to know who knows the answer. She’s been a great coordinator to work with.”

Maria Jaromin is the Assistant Director of Healthcare Research in the Health Innovation Program Office (HIPO). HIPO’s main role at NCSA is to coordinate efforts around healthcare and health sciences research. They connect experts to create powerful teams of researchers, often across disciplines. Their results speak for themselves. Since HIPO launched nearly five years ago, Jaromin has had a specialized role within the office. “Another part of my job right now is to coordinate with researchers on campus and inform them about the Nightingale Cluster.” When she heard about CPRD’s project, she knew Nightingale would be a great option for them.

“This project is a good example of how the Nightingale Cluster enables new collaborations between University researchers and healthcare, government, or industry,” said Jaromin. “Our experts help manage requirements around sensitive data so the users can focus on their research.”

Learning to Fly

While Reinhart’s team leaned heavily on NCSA’s expertise, she expressed hope that as the project evolves and CPRD takes on new responsibilities, they’ll be able to handle it on their own. “We’re much more comfortable with the idea of working with these types of systems now,” said Reinhart. The Medicaid Waiver Program was a five-year pilot program that ended in 2023, but Reinhart’s team is still involved with Medicaid evaluations. “We’ll be adding new types of data in the very near future to prepare for our final report and future projects. The SUD Demonstration Waiver was the first waiver that CPRD was hired to be the independent evaluator for, but we are already working on the evaluation plan for a second waiver regarding Continuity of Care. We’re using what we’ve learned from the first project to take on this additional project. It involves all Medicaid beneficiaries, not just those diagnosed with an SUD, so there will be even more data.”

CPRD will also collaborate with NCSA on that waiver. Reinhart and her team are excited to continue the working relationship and have nothing but positive things to say about the teamwork.

“Overall, our experience with NCSA has been really great,” said Reinhart. “There have been challenges on both sides because we have some big asks for NCSA. But they were always there, ready to answer our questions and lend a hand.”

ABOUT HIPO

Established in 2020, NCSA’s Healthcare Innovation Program Office (HIPO) enables healthcare partnerships that leverage NCSA’s dedicated technology and expertise in areas such as HPC, software development, visualization, data analytics, AI, and information security.  HIPO guides industry clients and academic researchers to solutions and expertise that meet their computing needs.  The people of HIPO are proud partners and stewards of NCSA’s resources, poised to assist users on their journey with NCSA.