Reddit AI Tool Flagged for Providing Dangerous Medical Advice
Key Points
- Reddit Answers suggested high‑dose kratom and heroin for chronic pain.
- Moderators reported the AI gave unsafe medical advice on multiple queries.
- Reddit updated the AI to hide related answers on sensitive topics.
- The update excludes private, quarantined, NSFW, and some mature content.
- No new moderation tools were provided to control AI content.
- The incident raises broader concerns about AI safety on user platforms.
Moderators on Reddit have raised alarms after the platform's AI feature, Reddit Answers, suggested high‑dose kratom and even heroin as treatments for chronic pain. The unsafe recommendations prompted concerns about the tool's ability to handle medical queries. Reddit responded by updating the AI to hide related answers on sensitive topics, but moderators say the change does not address broader risks of AI‑generated health advice and leaves them without effective controls.
Moderator Concerns Over Unsafe Advice
A Reddit moderator discovered that the AI‑driven feature Reddit Answers was offering hazardous medical guidance. When a user asked about chronic pain, the AI suggested stopping prescribed medication and taking high‑dose kratom, an unregulated substance that is illegal in some states. The same interaction also produced a recommendation that heroin could be used for chronic pain relief. Additional medical queries received a mix of accurate actions and potentially dangerous suggestions, including advice for treating neonatal fever.
Health‑focused subreddits quickly responded, highlighting that moderators lack tools to disable or flag problematic AI content. The moderators emphasized that the AI’s responses could mislead users seeking reliable health information and that the platform currently offers no way to control when AI appears in their communities.
Reddit’s Update to Restrict Sensitive Content
In response to the concerns, a Reddit spokesperson told 404 Media that the company had updated Reddit Answers to limit exposure to sensitive topics. The update ensures that “Related Answers” to such topics will no longer be displayed on the post detail page, aiming to improve user experience and keep inappropriate content out of view. The spokesperson also noted that the AI excludes content from private, quarantined, and NSFW communities, as well as some mature topics.
Despite the changes, the spokesperson did not provide details about which specific topics are now excluded, and Reddit has not offered additional tools for moderators to manage AI content. The company has not responded to further requests for comment.
Implications for Moderation and AI Safety
The incident underscores the challenges of integrating AI tools into user‑generated platforms. Moderators argue that without robust controls, AI can unintentionally spread misinformation, especially in areas like health where inaccurate advice can have serious consequences. The lack of a mechanism to turn off or flag AI content leaves moderators in a difficult position, potentially increasing the burden of manual oversight.
Reddit’s partial response—hiding related answers on sensitive topics—addresses the immediate visibility issue but does not resolve the underlying problem of AI delivering unsafe medical recommendations. The situation highlights the need for clearer policies, stronger moderation tools, and perhaps more rigorous vetting of AI responses before they reach the public.