First-gen AI in healthcare will create more clutter, not clarity
Sanjay M. Udoshi MD
Every few years, healthcare experiences a technology hype cycle that promises to revolutionize patient care. Electronic health records were going to eliminate medical errors. Telemedicine was going to solve access disparities overnight. And now, artificial intelligence — particularly large language models and generative AI — is being heralded as the force that will finally transform healthcare from a reactive, inefficient system into a predictive, personalized, precision-medicine utopia.
It won't. At least, not yet. And the sooner we acknowledge the gap between the promise and the present reality, the better positioned we will be to build AI systems that actually help patients rather than simply generating more noise in an already overwhelmed clinical environment.
The fundamental challenge of healthcare AI is not algorithmic sophistication. Modern machine learning models are extraordinarily capable. The challenge is the environment into which these models are deployed. Clinical settings are characterized by fragmented data, inconsistent documentation practices, legacy IT infrastructure, competing workflow demands, and a workforce already suffering from burnout and information overload.
Consider the current state of clinical alerts. The average physician in a large health system receives hundreds of electronic alerts per day — medication interaction warnings, order entry reminders, quality measure prompts, and billing compliance notifications. Studies consistently show that clinicians override 90% or more of these alerts. The system has created so much noise that the signal is lost.
Now imagine layering first-generation AI tools on top of this already cluttered landscape. AI-generated clinical summaries that may or may not be accurate. Predictive risk scores derived from models trained on biased historical data. Ambient listening tools that generate draft notes requiring careful human review. Chatbots that provide patients with plausible-sounding but potentially misleading medical information. Each of these tools, deployed without rigorous integration into clinical workflows, adds another layer of information that clinicians must evaluate, verify, and manage.
The most capable AI model in the world is only as good as the data it consumes. And in healthcare, the data landscape remains deeply fragmented. Despite decades of investment in electronic health records, most health systems still struggle with basic data interoperability. Patient records are scattered across multiple systems, coded in incompatible terminologies, and stored in formats that resist standardization.
This is precisely why standards like the OMOP Common Data Model and initiatives like OHDSI matter so much. Before we can meaningfully apply AI to clinical data, we need that data to be standardized, validated, and governed. An AI model trained on inconsistent, poorly coded data will produce inconsistent, poorly grounded outputs — with the added danger that those outputs will carry the patina of algorithmic authority.
The uncomfortable truth is that most health systems have not yet done the foundational work of data standardization. They are being asked to adopt AI tools before they have built the infrastructure to support them. This is equivalent to installing a high-performance engine in a car with no steering wheel.
In clinical medicine, we do not approve drugs based on manufacturer claims. We require randomized controlled trials, peer-reviewed evidence, and post-market surveillance. Yet the current deployment of AI in healthcare often proceeds with far less rigor. Models are developed on retrospective datasets, validated (if at all) on convenience samples, and deployed into production with minimal ongoing monitoring.
The consequences of this approach are predictable. Algorithmic bias — models that perform well for majority populations but poorly for underrepresented groups — is well documented. Distribution drift — the degradation of model performance as the real-world data diverges from the training data — is inevitable. And the "automation bias" that leads clinicians to trust algorithmic recommendations over their own judgment introduces a new category of medical error that we are only beginning to understand.
Rigorous validation requires prospective evaluation in real clinical settings, transparent reporting of model performance across demographic subgroups, continuous monitoring for drift and degradation, and honest acknowledgment of the limitations of any given model. This is expensive, time-consuming, and unglamorous work. It does not generate headlines or investor enthusiasm. But it is absolutely essential.
Even a well-validated AI model will fail to improve outcomes if it is not integrated into clinical workflows in a way that is intuitive, timely, and actionable. The history of clinical decision support is littered with examples of technically sound tools that were ignored because they interrupted workflow, presented information at the wrong moment, or required additional clicks in an already cumbersome electronic record.
Effective AI integration requires deep understanding of how clinicians actually work — not how we imagine they work. It requires co-design with end users, iterative testing in real environments, and a willingness to adapt or retire tools that do not deliver measurable value. It requires, in short, the same attention to implementation science that we would bring to any other clinical intervention.
None of this is an argument against AI in healthcare. The potential is real. Machine learning can identify patterns in clinical data that no human could detect. Natural language processing can extract structured information from unstructured clinical notes. Predictive models can identify patients at risk of deterioration before clinical signs become apparent. These capabilities are genuinely valuable.
But realizing this value requires a different approach than the one currently dominating the market. It requires:
At Acumenus, our approach to AI in healthcare is deliberately pragmatic. We build platforms on standardized data foundations — OMOP CDM, FHIR R4 — because we know that the quality of any analytical output depends on the quality of its inputs. We focus on clinical decision support that integrates into existing workflows rather than disrupting them. We believe in open-source development because transparency and reproducibility are ethical imperatives in clinical technology.
The AI revolution in healthcare will come. But it will arrive through disciplined, evidence-based implementation, not through hype-driven adoption. It will be built on solid data infrastructure, rigorous validation, and deep respect for the complexities of clinical practice. And when it arrives, it will be far more powerful and far more trustworthy than anything the current hype cycle is producing.
Don't believe the hype. Believe the evidence. And if the evidence isn't there yet, build the infrastructure to generate it.
Dr. Udoshi is Medical Director of Informatics at Acumenus Data Sciences. His career spans leadership roles at Oracle Health Sciences, Geisinger Health System, and multiple health IT organizations.
More perspectives from the Acumenus team.