Verfasst von Kevin Lancashire
When ChatGPT Interprets the Package Insert: What Recent Studies Reveal About AI in OTC Counseling

When ChatGPT Interprets the Package Insert: What Recent Studies Reveal About AI in OTC Counseling

Kevin Lancashire AI Answer Audits & Platform Strategy | Founder @ Lancashire Digital November 6, 2026

In the over-the-counter (OTC) pharmaceutical market, the preliminary decision for or against a product often happens at home—long before a consumer steps into a pharmacy or drugstore and receives professional advice. Alongside traditional search engines, consumers also turn to public AI assistants such as ChatGPT, Gemini, Claude, or Perplexity for symptom and medication questions.

For leaders in Medical Affairs, Regulatory Affairs, and Digital Strategy, this raises a critical question: How reliably do large language models convey medication guidance—even when given direct access to official product information?

1. What Empirical Research Shows About AI-Driven OTC Counseling

Peer-reviewed pharmaceutical studies provide a nuanced warning signal regarding autonomous AI counseling:

  • Severe Limitations in Unmodified Baseline Models: In an evaluation conducted at Keio University (Tokyo), researchers tested 264 typical consumer inquiries across 22 OTC drugs (12 queried by brand name, 10 by active ingredient name) and 12 clinical patient conditions. An unmodified baseline model (ChatGPT-3.5) satisfied all three evaluation criteria (coherence between question and answer, scientific accuracy, and appropriate instruction for action) simultaneously in only 20.8% of cases [1, 2]. Reproducibility was also limited: only 61.8% of correct responses were reproduced when the same question was re-entered on a different day. Instructed actions were weakest for questions about driving and concomitant medications [2].
  • Impact of Brand Names: The same research found that consumer prompts using commercial OTC brand names yielded poorer responses in baseline models than queries using generic active ingredient names [2]. In the customized setup with package insert texts, however, responses to brand-name questions improved significantly [1].
  • Combined Criteria Fulfillment (56.1%): In a 2025 follow-up study, researchers supplied a customized GPT-4 system (cGPT) directly with official package insert texts for all 22 OTC medications alongside five structured prompting instructions. While scientific accuracy reached 93.2% and logical relevance hit 100%, the third criterion (reliability with respect to package insert instructions) reached 60.2%. Across all three criteria combined, 56.1% of responses met every requirement [1].
  • Communication Strength vs. Structural Reliability: Newer models are undeniably articulate: in a 2026 bilingual study across 22 gastrointestinal OTC case vignettes, ChatGPT-4.5 achieved higher mean CLEAR counseling scores (4.8/5) than 58 community pharmacists (3.7–3.8/5) [3]. However, foundational LLM research highlights structural vulnerabilities—ranging from numerical errors [4] to sycophancy, where models change their answers when users push back. In one study, about 15% of all responses flipped from correct to incorrect after pushback (math and medical questions) [5].

2. Addressing the Skeptic's View: Do the Findings Transfer?

A discerning reader in Medical or Regulatory Affairs will rightly note: The Keio studies were conducted in Japan using earlier model generations—today's frontier models are more capable.

While model capabilities have advanced, two structural realities keep these findings relevant for OTC manufacturers today:

  • Context Alone Does Not Equal Compliance: The 2025 study shows that even when official package insert text is placed directly into the model's context, satisfying all evaluation criteria simultaneously still fails in a notable share of instances [1].
  • Real-World Prompts Lack Curated Context: In everyday use, consumers rarely supply verified regulatory texts. Instead, the AI relies on pre-trained weights or live web searches—without guaranteed localized safeguards across jurisdictions, languages, and brand extensions.

3. Practical Steps for OTC Manufacturers

Because pharmaceutical companies cannot control the model weights of OpenAI, Google, or Anthropic, practical risk mitigation relies on external data governance and systematic auditing:

  • Publish Clean, Machine-Readable Product Evidence: Official dosing tables, age limits, duration constraints, and contraindications should be published as clearly structured, easily parsable content on brand-owned websites, explicitly tagged by regulatory jurisdiction (e.g., CH / DE / AT). Retiring outdated documents from corporate domains helps search-augmented AI systems retrieve current primary disclosures.
  • Establish Systematic AI Answer Audits: Because brand-name queries were found to be more error-prone in baseline evaluations [2], OTC manufacturers should regularly test leading public AI platforms using realistic consumer and pharmacist queries across their brand portfolios. Auditing generated outputs against nationally approved product information (such as Swissmedic AIPS or national patient/healthcare professional leaflets) exposes omitted warnings, dosage errors, or product confusions early.
  • Align Cross-Functional Governance: Monitoring how public AI represents OTC products is not solely a digital marketing task. Medical Affairs (clinical risk assessment), Regulatory Affairs (alignment with approved labeling), Brand Management, and Digital Leadership should jointly review audit findings and translate them into structured content improvements on corporate digital channels.

Disclosure: Through Lancashire Digital, I develop and conduct AI Answer Audits for pharmaceutical and OTC companies.

Sources & References

Kevin Lancashire

Digital Communications and Innovation Manager.

https://www.a-jumpahead.com/blog
Weiter