Confession to an Algorithm: Why ChatGPT Is the Worst Guardian of Your Medical Secrets
Imagine walking into a doctor's office, but instead of a person in a white coat, there's a huge mirror cabinet that records every word you say, only to retell it to millions of other people in a slightly altered form. It sounds like the opening of a dystopia, but for 230 million ChatGPT users, this has become a weekly routine. OpenAI proudly reports that people are increasingly seeing their chatbot as an "ally" capable of navigating the complexities of insurance plans and deciphering murky lab results. But here's one small detail: your digital ally didn't take the Hippocratic Oath and, frankly, doesn't understand what human life is or why it needs to be cherished. The phenomenon of AI use in medicine didn't arise out of nowhere. Modern medicine has turned into a complex bureaucratic maze, where patients often feel like an unnecessary cog.
AI-processed from The Verge; edited by Hamidun News
More than 230 million people turn to ChatGPT every week for advice on health and well-being — that's the figure cited by OpenAI. The story is reported by The Verge. The company says that for many users the chatbot has become an "ally": it helps them navigate the maze of insurance plans, fill out paperwork, and advocate more confidently for their own interests within the healthcare system. At the same time, tech companies are not bound by the same obligations as medical institutions — and that gap is exactly where the main risk lies.
Why users trust a chatbot with their health
Modern medicine often feels like a complicated bureaucratic labyrinth in which the patient feels like just another cog. Against that backdrop, a polite, instantly responsive, and always-available ChatGPT looks like a lifeline: it doesn't rush you, doesn't react with irritation to repeated questions, and is willing to discuss symptoms at any hour. OpenAI describes this role as that of an "ally" that helps people navigate correspondence with insurance companies, handle paperwork, and, overall, advocate more confidently for their interests as patients. The problem is that this responsiveness is a high-quality statistical imitation, not conscious care: the chatbot never took the Hippocratic oath.
Legal vacuum: a chatbot is not a medical institution
Doctors and clinics are bound by strict confidentiality protocols — such as HIPAA in the United States or GDPR in Europe. OpenAI and other tech companies operate under terms-of-service agreements that most people skim through in seconds. When a user uploads an MRI scan or a list of medications to the chat, that data stops being simply a personal secret — it becomes part of the dataset used to train future versions of the model. Companies promise anonymization, but the industry's history of data leaks shows that "anonymity" online is a fragile and temporary concept. Tech companies are not bound by the same obligations as medical institutions.
The
Google Nightingale precedent and the probabilistic nature of AI
Similar scenarios have already played out before: in the case of Google's Project Nightingale, the data of millions of patients was transferred to a tech company without the people themselves knowing. The difference now is that users are handing over their medical information voluntarily, drawn in by the convenience of the chatbot interface. At the same time, a language model is not a knowledge base — it's a generator of the most statistically likely sequences of words. In tasks like writing code or an essay, a wrong but "plausible" answer from the model might look amusing; in a medical context, a hallucination about a drug dosage or the interpretation of a heart rhythm is a far more serious risk.
Anthropomorphism and the absence of accountability
A polite, well-structured answer is subconsciously perceived as a sign of competence — this is the anthropomorphism effect: people tend to trust the bot more than they should. But behind that facade there is none of the accountability that characterizes medicine: if a doctor makes a mistake, they risk their license and a lawsuit. If a chatbot gives bad advice, the company can point to a clause in the terms of service stating that the service is intended "for informational purposes only."
The bottom line: tech companies are not medical institutions, and the convenience of talking to a chatbot is not the same as the protection of personal medical data. Using AI to look up general information or organize notes is one thing; turning a chatbot into the keeper of your medical history and prescriptions is something else entirely.
How many people turn to
ChatGPT for health advice every week?
More than 230 million people a week — that's the figure cited by OpenAI.
How does
ChatGPT differ from a doctor in terms of obligations to the user?
Tech companies are not bound by the same obligations as medical institutions: they are not subject to confidentiality protocols like HIPAA and GDPR in the same way doctors and clinics are, and their terms of service state that the service is intended "for informational purposes only."
What happens to the medical data users upload to the chat?
It stops being a personal secret and becomes part of the dataset used to train future versions of the model; companies promise anonymization, but the industry's history of data leaks shows that this is a fragile and temporary concept.
Need AI working inside your business — not just in your newsfeed?
I build production AI for companies — custom CRM, internal tools, autonomous agents, workflow automation. Owned by you, shaped to your process, no per-seat tax. Built by Zhemal Khamidun, CPO of AlpinaGPT (AI platform, 6,000+ users).
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.