A preprint on the evaluation of a Generative AI (GenAI) tool at Radboudumc was recently published. This application automatically generates draft responses to patient questions via Epic’s In Basket messaging system. An example of the user interface is shown in the image. In January 2025, 237 healthcare providers from four departments (dermatology, ENT, medical oncology, and pulmonary medicine) were granted access to the tool. The deployment was evaluated using a hybrid Type 1 implementation study, which examined feasibility and effectiveness while identifying barriers and enablers for implementation. To measure this, questionnaires were distributed to end users to gather their expectations and experiences with the application.
A total of 8,410 draft responses were generated between January and June. The study showed that perceived clinical efficiency declined slightly over time, while perceived well-being remained stable. Adoption of the tool was high initially but declined thereafter. The same was true for the user experience and intention to continue using the tool. Healthcare providers cited positive aspects such as time savings, the clear structure of the draft responses, and patient-centered language. At the same time, challenges emerged, including limited added value and time savings in the daily workflow, content inaccuracies, and stylistic differences between AI-generated responses and personal style preferences.
These findings underscore the importance of a robust workflow to quickly identify user feedback and incorporate it into model development. Healthcare providers also request clear guidelines regarding clinical responsibilities when using GenAI, as well as transparent communication about the capabilities and limitations of GenAI applications.
In addition, the study highlights the importance of determining in advance which quality criteria and threshold values should be applied. These results identify key considerations that we can incorporate into future implementations of GenAI in healthcare practice.
Read full the article here: Kim J.M. Bladder, Arie C. Verburg, Milou Arts-Tenhagen, René Willemsen, Guido B. van den Broek, Chantal M.L. Driessen, Rieke J.B. Driessen, Bas Robberts, Arthur R.T. Scheffer, Arjen P. de Vries, Tim Frenzel & Julie E.M. Swillens. AI-Generated Responses to Patient’s Messages: Effectiveness, Feasibility and Implementation. medRxiv (2026 )