Scilight Press

Background: Dermatology chatbots are increasingly used for patient engagement and triage, generating large volumes of patient-reported data. Objective: To examine the nature of patient-reported adverse reactions and evaluate machine learning approaches for early detection of unresolved chatbot interactions. Methods: A dataset of 103 anonymized chatbot conversations was analyzed. Latent intent topics were identified using non-negative matrix factorization. Interactions were labeled as resolved or unresolved using rule-based heuristics. Logistic regression models with TF-IDF features and conversation-level metadata were evaluated using 5-fold cross-validation. Early-warning models were restricted to the first one or two user messages. Kaplan–Meier analysis was used to assess time-to-unresolved outcomes. Results: Reported side effects were predominantly dermatologic and generally moderate in severity but lacked specificity regarding treatment, dosage, and duration. Machine learning models demonstrated moderate predictive performance (ROC-AUC ≈ 0.72), improving to ≈0.78 using the first two user messages. Unresolved interactions were most likely to occur within the first 3–5 conversational turns. Conclusion: Dermatology chatbots capture valuable patient-reported signals but lack structured data for clinical interpretation. Early detection of unresolved interactions using machine learning is feasible and may support real-time intervention and improved patient safety.