When not to trust AI with your child's symptoms

When to skip the chatbot

AI can help you make sense of a calm situation: a question about teething, an explanation of what the nurse said, a list of questions for the next visit. It is not the thing to decide whether a sick child can wait.

The danger signs in the box above are the ones WHO teaches health workers through IMCI (Integrated Management of Childhood Illness). Any one of them means serious illness until proven otherwise. In those moments, every minute spent typing is a minute less of treatment. Type later, in the queue, if you want.

Also skip the AI when:

• your gut tells you something is very wrong, even if you can't explain it;
• your child is getting worse after being seen;
• a fever has lasted more than 2 days, or there's blood in the diarrhoea;
• your child has swallowed medicine, a cleaning product or paraffin;
• there's been a bad fall, a burn or an animal bite.

What AI can't do

It can't examine your child

A nurse or doctor counts breaths per minute, watches the chest, pinches the skin of the tummy to check for dehydration, looks at the palms and gums for paleness, feels the tummy, listens to the lungs. They can do a malaria rapid test. None of that comes through a text box. The AI only knows what you tell it — and parents, understandably, don't always know which details matter.

It can't see breathing or colour

Many of the most serious signs are visual: the chest sucking in under the ribs, nostrils flaring, blue lips, the pallor of severe anaemia. Even if you send a photo, a still image doesn't show breathing, and colour changes with the light and the screen.

It makes things up — confidently

Language models generate the text that looks most likely, not what is true. That's why they sometimes invent ("hallucinate") facts, studies or advice, and write the mistake in the same assured tone as the right answer. In its 2024 guidance on these models in health, WHO warns that they can produce false, inaccurate, biased or incomplete statements, and that people tend to over-trust the machine ("automation bias").

It doesn't know our context well

Much of the training data comes from wealthy countries, where a feverish child rarely has malaria and TB is uncommon. In South Africa, TB has to be considered for a cough lasting weeks, especially if someone at home has it — and malaria must be ruled out after any trip to a malaria area. The AfriMed-QA study, which tested 30 models on questions from medical schools in 16 African countries, found performance varied widely between specialties and regions, and lagged behind US medical exam results. If the AI says "just a virus" without asking where you've been, it's missing the basics.

What the research says

AI health results are often reported as impressive. Look closer and the picture is more modest:

Study	What it tested	What it found
Barile et al., JAMA Pediatrics, 2024	ChatGPT (version 3.5) on 100 children's case challenges	Correct diagnosis in 17. Wrong in 72, too vague in 11.
Bean et al., Nature Medicine, 2026 (Oxford)	1,298 adults, 10 scenarios, using GPT-4o, Llama 3 or Command R+	Alone, the models did well. With real people, fewer than 35% reached the right condition and fewer than 45% the right decision — no better than without AI.
AfriMed-QA, ACL 2025	30 models, 15,000 questions from 16 African countries	Big variation across specialties and regions; below US exam level.

Models are improving, and the children's study used an older version. But the Oxford study makes the key point: even a model that "knows" the answer falls down when the conversation is with a worried parent — answers mixed good and bad advice, and changed with small differences in wording. The authors conclude these systems need testing with real users before they reach the public.

Red flags in an AI answer

If an answer contains any of these, stop and speak to a health worker:

• Exact doses ("give 5 ml of...", "120 mg every 6 hours"). The dose depends on weight, the product's strength and other conditions. Only use the one the clinic prescribed or the one on the package insert.
• "No need to see a doctor" or "it's just a virus". The AI can't promise that without examining your child or doing a malaria test.
• Product or brand recommendations, supplements, "miracle" teas or online shops. Health advice isn't advertising.
• Harmful home remedies: alcohol or paraffin on the skin to bring down fever, honey before age 1, aspirin for children, enemas or purgatives for diarrhoea.
• One certain diagnosis, with no alternatives and no doubt.
• Studies or sources you can't find. If the AI cites a paper, look it up. If it doesn't exist, the whole answer is suspect.
• Advice that contradicts the Road to Health Book or what the clinic told you. The clinician saw your child; the AI didn't.
• It never asked age, weight or where you've been. It answered without the basics.

So is AI useless?

No — it's useful within limits. It can explain medical terms, help you organise the story of the illness, prepare questions for the visit and remind you what to watch at home once your child has been seen. See how to ask AI about your child's health and prepare for a clinic visit with AI.

The rule of thumb: AI can help you ask better questions; the answers about your child come from the health worker who examines them.

The short version

1. Danger sign, fever after a malaria area, or a sick baby under 2 months: clinic or casualty, not a chatbot.
2. AI can't examine, can't see breathing or colour, and can make things up confidently.
3. Research shows that in ordinary people's hands, chatbots don't lead to better decisions.
4. Be wary of doses, "no need for a doctor" and product recommendations.
5. When in doubt, go. Nobody at the clinic will think you overreacted.