Quick Buzz Feed
Implementations of a centralized platform at Inova Health and Mount Sinai Health System show how unifying AI management can support ongoing AI implementation and governance efforts.
As health systems deploy more AI tools, the processes for evaluating and monitoring them are becoming harder to manage at scale. At Inova Health, headquartered in Fairfax, Virginia, the challenge has become pressing as its AI portfolio has grown. The health system now has about 70 AI features in production, with its inventory continuing to expand. According to Jon McManus, Inova’s chief data and AI officer, that AI inventory is growing by 12 or more features per month. New York City-based Mount Sinai has a similarly large AI portfolio. Robbie Freeman, chief digital transformation officer, said the health system currently has more than 100 live AI use cases. About one-third were developed in-house, while others include AI features available through its Epic EHR, imaging tools and applications supporting revenue cycle and back-office operations. As the number of AI applications soars in health systems nationwide, managing each through separate evaluation and monitoring processes can become difficult to sustain. Consequently, health systems are looking for ways to bring those functions together across the enterprise. One platform designed to support that approach is Signal 1's AI Management System, which brings AI inventory, governance workflows and ongoing performance monitoring into a centralized platform. Inova and Mount Sinai are among the health systems using Signal 1 to manage AI governance. "We recognized the moment that the amount of AI we were dealing with was starting to outpace our ability to deal with it by hand," McManus said. Before implementing the Signal 1 platform, Inova employed various tools to manage its AI inventory. The health system used Word documents to collect information from vendors as well as ethics assessments completed by the Inova leader sponsoring the use case. Inova also tracked its AI inventory in spreadsheets and used qualitative tools to monitor safety and reliability. While performance reports from vendors were available, they were specific to individual AI features, making it difficult to compare performance across the organization. As Inova's AI portfolio grew, leaders recognized that these processes would become difficult to sustain. McManus highlighted that Inova's experience reflected a broader challenge for large health systems as their AI portfolios expand. "Volume [of AI tools is] going to require a platform approach for enterprise-size companies," he said. Mount Sinai also faced AI governance scaling challenges despite already having a centralized AI governance structure. As part of that structure, the health system has an independent AI assurance lab -- a dedicated team that evaluates higher-risk AI tools before deployment and monitors them once they are in use. The assurance lab tests tools for performance, bias and potential unintended consequences or harm before making implementation recommendations to the health system's AI governance team. However, applying that assurance process across Mount Sinai's AI portfolio could be labor-intensive. For scaling individual AI tools, staff may need to map workflows, design evaluations, configure and run tests and establish ongoing monitoring, according to Freeman. So, Mount Sinai is testing whether the Signal 1 platform can reduce the work required to conduct assurance across its AI portfolio. "We can't yet automate that full process, but we can shorten the time, energy, effort it takes to do assurance on each one of those AI tools that require it," Freeman said.
Inova's implementation of the Signal 1 system involved a great deal more than deploying an off-the-shelf platform. As a co-development partner, the health system had worked with Signal 1 for about 10 months at the time of the interview to develop governance workflows, automation and approaches for measuring AI safety and value. When standing up the platform's monitoring module, for instance, Inova began onboarding individual AI applications. The process initially required Inova teams to determine which data connections and performance measures were appropriate for each AI use case -- including tools for identifying sepsis, voice-based workflows, automated patient message responses and computer vision technologies. But, establishing those measures individually made it time-consuming to bring new AI applications into the monitoring process. "We were seeing a time to market that was lengthy and really reinventing the wheel each time," McManus said. In response, Inova and Signal 1 developed baseline sets of measures that McManus said cover about 95% of what is typically needed to monitor an AI application. Signal 1 said via email that the measures vary by type of AI, and cover performance, fairness and safety when relevant. For example, a model designed to classify something into one of two categories might be evaluated on how accurately it distinguishes between those categories, while a generative AI tool might be assessed on whether its output is complete and supported by the source information, Signal 1 said. Teams can then add the measures specific to an individual use case rather than building their monitoring framework from scratch. "We're not losing any integrity because people are still making the fine-tune adjustments, but they're doing it on top of 95% of the work done instead of working with a blank palette and trying to build it all out each time," McManus said. According to McManus, adding AI tools to the monitoring module took roughly six to eight weeks while the broader co-development work continued. Mount Sinai's implementation is at an earlier stage. At the time of the interview, the health system was about six weeks into the project and had brought its first few use cases live on the management platform. Signal 1 is being incorporated into the health system's existing assurance workflow, with the AI assurance lab team beginning to use the platform. The team will start with four substantially different technologies, including a generative AI feature to help with in-basket messages, an internally developed predictive model and a medical imaging application. The health system will evaluate whether Signal 1 can match or exceed its existing assurance processes while reducing the staff time and resources required before deciding how broadly to expand the platform. "You have to have a really robust evaluation kind of capability to just know how these tools are working with your patient population," Freeman said. Both health system leaders hope that a more centralized approach to AI management will help governance keep pace with their growing portfolios without sacrificing rigorous evaluation and monitoring. Elizabeth Stricker, BSN, RN, comes from a nursing and healthcare leadership background, and covers health technology and leadership trends for B2B audiences.