This is a most excellent place for technology news and articles.
Please just bring back traditional voice to text, with its dumb but decipherable errors instead of these plausible left field LLM "hallucinations".
I'd rather the computer write that I'm coming in for a "score throat and fever" than the sort of random ass shit LLMs can throw in there.
I'm looking at her and telling you she's done shrooms in her college days.
Wow. They are streamlining US "healthcare" by putting the AI claim-denier 5000 INSIDE the exam rooms.
TL;DR: AI transcript said I’d been a smoker since the age of 11, never smoked a day in my life.
My doctor asked me if she could use AI transcription software because she had hand surgery and couldn’t type. I said yes and reviewed the statements and while it was fairly accurate, it did have one egregious error on it.
When asked if I had ever smoked, in person, I replied with a simple “no,” and did not proceed to have any follow up conversation until her next question. The AI transcript for mysterious reasons decided my answer was “yes, since the age of 11.” 💀💀💀
Had to message them to fix it and while they did fix it asap, my first instinct wasn’t to check the transcript and I only did so out of curiosity. I can’t imagine what other people’s experiences may be.
Just say no.
We use a software to help us take notes is that okay? FUCK NO.
Article:
Error by AI scribe during medical appointment leaves patient devastated
By the Specialist Reporting Team's Paige Cockburn
Posted 13 Aug 2026 at 2:55pm13 August 2026 at 2:55pm, updated 13 Aug 2026 at 4:04pm13 August 2026 at 4:04pm
An error in Rebecca Green's medical records falsely claimed she had taken psychedelic mushrooms. (ABC News: Madeline Grace)
When Rebecca Green arrived for her first appointment with a urologist, she was asked whether she consented to having the appointment transcribed by artificial intelligence (AI).
She said yes without hesitation.
"I don't want to be in her bad books or be that annoying patient who takes up more of her time," she said.
But that decision would cause her a lot of stress, with the AI scribe making up the serious but false claim that Ms Green was taking illegal drugs.
"I was gobsmacked and in tears … it made up that I take [psychedelic] mushrooms. I have never done mushrooms in my life,"
she said.
More GPs and hospitals are using AI scribes. What are the risks?
AI tools are becoming more prevalent in healthcare, helping overworked doctors to carry the workload. But it has come with concerns.
New research by a privacy watchdog has raised serious concerns about AI being used in medical settings due to its propensity to "hallucinate" false symptoms or conditions, and warned patients were at risk.
Ms Green only picked up the mistake after her kidney stone surgery in March, when she read the post-operative letter her specialist sent to her GP.
It stated Ms Green micro-dosed mushrooms and that could be the reason for prior bleeding around the kidneys.
"For this letter to say something that wasn't based in any evidence but could potentially have huge consequences for me … it was quite scary," she said.
"It felt very impersonal to me. I wasn't getting the level of care and familiarity from the doctor that I would be hoping for in such a vulnerable situation."
Rebecca Green discovered the error on her medical record after reading a post-operative letter from her specialist. (ABC News: Madeline Grace)
Ms Green was particularly stressed as she was receiving workers' compensation and felt any mention of illegal drugs on her medical record could impact her case.
Australia's peak regulator of medical practitioners, AHPRA, said clinicians must always check all output from an AI scribe for accuracy to ensure they meet their professional obligations. Ms Green said it was clear her doctor did not do this.
"If you had checked that letter you would have been able to correct that before it was sent out … why aren't you in tune with each of your patients and what's going on?"
What to know before saying yes to your doctor using AI
Health practitioners are using generative AI to help them record and summarise patient information, but some patients are saying no.
After Ms Green made a complaint, the urologist sent a letter apologising.
"Please be assured that we take the accuracy of our documentation very seriously, and we are truly sorry that this occurred," the letter stated.
The doctor said she had not been able to determine how the claim about mushrooms came to be included, but it appeared to be from an error during the "dictation or transcription process".
The urologist corrected the correspondence and in her letter said she would review how AI was used in her practice.
The doctor did not provide any responses to the ABC's questions about what changes, if any, have been made since.
AI confuses which breast has cancer
Ms Green feels lucky she spotted the error, but is worried some people might not be aware of mistakes on their medical record, which could have repercussions for their health, their career or their insurance.
This week the ABC revealed instances where private health insurers were asking providers for their patients' full clinical history as part of "aggressive" auditing practices.
AI scribes have become prevalent within the healthcare sector. The Royal Australian College of GPs (RACGP) last year estimated that 40 per cent of GPs were regularly using them, but that number was considered conservative.
The peak body for GPs in Australia has estimated that 40 per cent are regularly using AI scribes at work. (ABC News: Tim Swanston)
Research shows having AI transcribe an appointment can reduce the administrative burden for GPs and allow them to focus more on their patient, rather than the computer.
But for every benefit, there are risks.
A recent report by advocacy group Digital Rights Watch (DRW) found Australians were being exposed to unregulated AI medical scribes, sometimes without their consent, and errors had the potential to change clinical decision-making.
Do you have a story to share about AI in health care?
Email Paige Cockburn at cockburn.paige@abc.net.au
Examples of mistakes found in the DRW report included a case where a scribe recorded the wrong breast in a diagnosis of breast cancer and said someone had epilepsy, when they did not.
Chief executive of the Consumers Health Forum of Australia Elizabeth Deveny said she often heard complaints of AI mixing up words and patients only finding out by chance.
"People report both frustration in finding the original source of the error and then getting the record changed," she said.
Elizabeth Deveny says it is frustrating for patients who have to get their records fixed when errors are made. (ABC News: Darryl Torpy)
Dr Deveny, who is currently involved in a major research project on AI medical scribes, said doctors may be too trusting of AI.
"There's evidence that for the first few weeks of using AI, [people] check things pretty carefully and then at a point they stop checking because they assume it will be right."
Perth GP Sean Stevens, who has been using a scribe for three years, said it was incumbent on the doctor to check every transcript, but believed the technology was becoming more accurate.
"Three years ago, I would have to correct multiple things on each transcription. These days it's like one correction every third or fourth transcription."
Sean Stevens says AI transcriptions are getting more accurate, but doctors still need to check them. (Supplied)
Dr Stevens is a big advocate of tech; he is the RACGP's Digital Health and Innovation chair as well as an advisor for one of the biggest AI scribes, but he admits AI has its weaknesses and can produce incorrect results that sound plausible.
"The classic is, it will get the side of the body incorrect and say right when you've said left … so you need to check things pretty closely, especially things like drug doses,"
he said.
He said using AI did not mean he could squeeze more patients in, but it allowed him to run on time more often and produce more thorough referrals.
Consenting to AI's involvement
The DRW found huge discrepancies in how consent is being obtained, with some doctors only putting a sign in the waiting room, which does not meet privacy law requirements.
Other times consent must be given when the appointment is booked online, and alarmingly some patients are even being refused appointments if they do not consent.
Dr Deveny said consent must be enthusiastic and fully informed, and patients should feel comfortable asking questions, even if it feels intimidating.
What the Consumer Health Forum recommends asking at your next medical appointment:
- Are you using AI today and if so, what are you using it for?
- Will the doctor review everything before it goes into my record?
- Will I be able to get a copy of the notes if I want them?
- What's the process if I think something's wrong?
The DRW report also identified that people might not be forthcoming about sensitive issues, like family violence, when they know AI is listening.
Dr Stevens said that was why he asked for consent in every appointment, even with regular patients.
"It's very important patients disclose any and all things that are relevant to their health … and that they know they can say no [to AI] … as a doctor you have to respect that," he said.
TGA flags breaches will be dealt with
There are around a dozen AI medical scribes available in Australia, but none are approved by the regulator — the Therapeutic Goods Administration (TGA) — because if the product claims to be solely for transcription and summarising, it is exempt from oversight.
The TGA only considers tools that do things like suggesting a diagnosis or treatment plan as a "medical device".
AI changing the way GPs tend to their patients
Some GPs are now relying on artificial intelligence to take notes during patient consultations. So, how safe is our sensitive information?
Currently, no AI scribes have gone through the regulation process, despite some claiming to offer clinical suggestions.
The TGA has acknowledged this means some scribes were potentially in breach of the law.
The TGA has spent the last year reviewing AI scribes and says its findings will be released in the coming weeks, and any breaches will be acted upon.
Chief executive of Patients Australia Lisa Robins believed all scribes should be regulated, as summarising a patient still involved decision-making by AI.
Dr Stevens had concerns that extending regulation would only stifle future innovation.
The DRW is calling on the newly established AI Safety Institute to test AI scribes and publish a list of approved tools.
This would be similar to New Zealand's approach, where the national AI advisory group reviewed tools and last year gave approval for clinicians to use two AI tools: Heidi Health and iMed.
Ms Deveny said ensuring doctors were using AI safely could also be enforced in other ways, such as compulsory checks and balances built into the software, or during practice accreditation, which happens every three years.
The error on her medical record has left Rebecca Green reconsidering if she would give consent to AI transcription again. (ABC News: Madeline Grace)
Ms Green said her trust in AI has been splintered by what happened and now would only give consent for AI if she had an established relationship with the doctor.
"I love AI … but the essential ingredient is the human. At the end of the day what's happened here is more about the lack of human involvement,"
she said.
As far as I'm concerned, using AI of any sort inside of a doctor's office is a grave violation of HIPAA
Let me tell you about how medical transcriptionists used to do it: Sister in law was one, she didn't seem to care about HIPAA too much, she and all her colleagues did lose their jobs about a 15-20 years ago when AI transcription software took over their jobs, but before that she was frequently given jobs because her predecessors made serious mistakes in their transcriptions.
If she didn't care about HIPAA, I'm glad she lost the job lol.
If only we could fire AI too.
She'd tell stories (without the names)- many of her colleagues would get deep into the personal details...
I don't think that's a hipaa violation, but I'm no expert.
EMTs, etc, do that too. You'd go crazy if you couldn't.
I work in the digital handling of PHI (HIPAA regulated) it's pretty insane what is considered HIPAA protected in that world.
In practice, yeah, real people can't just be all "oh, that's a national secret, I can't tell you anything about my work or the weird worms people had in weirder places...
Australia's peak regulator of medical practitioners, AHPRA, said clinicians must always check all output from an AI scribe for accuracy to ensure they meet their professional obligations.
Show me one single AI user that checks the output of their AI, and I will show you my unicorn
I just asked SlopGPT a question and checked the output. Can I see your unicorn?
(That was a lie. I just really want to see a unicorn.)
Not gunna get me with the ole "wanna see a unicorn" trick twice.
The patient was devastated? You should have seen how many times the AI apologized.
Luckily, I know something that would probably help her get over the trauma...
I read "medieval" appointment and was thoroughly confused, and intrigued.
leeches, blood letting, letting miasma out, holes in the skull, the usual.
Sounds like an AI hallucination
I've used AI for medical transcription before and it is absolutely incredible. It's not just a word for word text, but a detailed clinical breakdown of everything discussed.
All you have to do is simply read the damn thing thoroughly and correct whatever minor errors pop up.
Takes 95% less time.
Don't forget where you are. They kill people around here for that kind of talk.
All you have to do is fire the human who is trained and rarely makes a mistake and replace them with a paid machine that almost always makes a mistake and double check everything yourself (or hire another human to double check it) because someone’s life literally depends on the entirety of the text being absolutely correct.
AI does work at a level that makes unpaid interns look like career professionals. All for the low cost of completely fucking the environment.
Glad you like it.
I'm unsure if there's an ethical issue with putting patients medical records in the data machine.
There exists online and offline ML.
Transcription is not a task that needs a full blown LLM, summarization probably does, and if being done offline or on a server with trusted verifiable lack of logging should be fine. That does not mean send it to ChatGPT API.
I would hope they store the raw conversation file encrypted with patient records so they can retranscribe later as tools improve.
It needs to be a system and a contractual framework that complies with the medical privacy regulations of the place where it's being used (in the US, HIPAA). I have some doubts about whether specific companies or people will actually comply, or the enforcement of those laws by certain governmental authorities, but I don't think it's conceptually impossible to have this kind of tech be compliant with those types of regulations.
Exactly. It is a tool for professionals to use to speed up existing processes. It is not a replacement for people, regardless of what the marketers tell you so you buy their product. I don't understand how there aren't thousands of lawsuits against these AI companies for their blatantly false advertising.
AI keeps proving this day by day when people try to make it a replacement and have it blow up in their faces every day.
