Ask the principal of a small practice whether staff use AI and the answer is usually "not really". Ask the staff and the answer is usually "every day". That gap has a name: shadow AI, the use of AI tools the practice never approved, set up or checked.
It isn't a discipline problem. People reach for free chatbots because they genuinely help: a messy referral becomes a clean letter, a forty-page affidavit becomes a one-page summary, a blunt email becomes a polite one. The trouble is what goes in with the request.
Where the text actually goes
When someone pastes a document into a consumer chatbot, the text is sent to the provider's servers, which for the big names are mostly outside Australia. Depending on the product and the account settings, it may be:
- kept in the user's chat history, on a personal account the practice cannot see or delete;
- retained by the provider for abuse monitoring, sometimes for weeks;
- used to improve future models, unless someone found and switched off the right setting;
- reviewed by the provider's staff or contractors in some circumstances.
None of that is sinister. It is written into the terms. But the practice didn't agree to those terms; a staff member did, on their own phone, in a lunch break.
Three ordinary examples
These are illustrative, not real incidents, but anyone who has worked in a practice will recognise them.
- A receptionist tidies a referral letter by pasting in the patient's name, Medicare number and history. Health information has now been disclosed to an overseas company.
- A junior lawyer uploads a family law affidavit to get a quick chronology. Confidential, possibly privileged, material has left the firm.
- A bookkeeper asks a chatbot to draft a client email that includes income figures and a tax file number.
Why it matters
For practices covered by the Privacy Act 1988 (Cth), and every health service provider is covered regardless of turnover, a few of the Australian Privacy Principles come into play straight away:
- APP 6 limits using or disclosing personal information for purposes other than the one it was collected for.
- APP 8 deals with sending personal information overseas. In many cases the practice stays accountable for what the overseas recipient does with it.
- APP 11 requires reasonable steps to protect the information you hold.
The Office of the Australian Information Commissioner said as much in its 2024 guidance on commercially available AI products, recommending that organisations do not enter personal information, and particularly sensitive information, into publicly available AI tools. Professional bodies have said similar things: the Victorian Legal Services Board, for example, has warned lawyers against putting confidential client information into public AI chatbots.
For lawyers there is a second layer. Confidentiality is a professional duty, and putting privileged material somewhere outside the firm's control invites an argument about whether privilege survives.
Why a ban rarely works
The instinctive fix is a policy line: "Do not use AI." It tends to fail for the same reason shadow IT always has. The tools are useful, they are on everyone's phone, and nobody can see who is using them. A ban mostly moves the use somewhere less visible.
What works better is the boring combination:
- Find out what is actually happening. Ask staff, without blame, which tools they use and for what. The answers are usually tasks you'd be happy to support: letters, summaries, rewording, checklists.
- Give them a sanctioned tool that is just as easy. If the approved option is slower or worse, people will drift back.
- Write a short, specific policy. One page: which tool is approved, what must never go into anything else, and who to ask.
- Check in after a month. See whether the approved tool is being used and what people still go elsewhere for.
What "sanctioned" can look like
There are several reasonable options, and the right one depends on the practice. Enterprise versions of the major cloud tools offer contractual commitments about training and retention, and are a big step up from personal accounts. Some practices go further and want the assistant to run on hardware they control, so client and patient documents never leave the building or, at most, a dedicated server in Australia. That is the approach we take, but the first step, knowing what your staff already do, is worth taking whichever way you go.
This is general information, not legal advice. If you think personal information may already have been disclosed in a way that causes serious harm, the Notifiable Data Breaches scheme may apply, and it is worth getting advice promptly.
