What AI Therapy Can Collect
AI therapy systems may collect more than the words a user types into a chat window. Depending on the product and its settings, information can include session transcripts, voice recordings, emotional or health assessments, account identifiers, device information, IP addresses, referring websites, and timestamps that reveal when a person seeks support. Some systems also store progress notes, crisis flags, previous conversations, and information supplied during onboarding, such as medications, diagnoses, relationship problems, or suicidal thoughts. This makes mental health data unusually sensitive because it can reveal both a person's condition and the circumstances affecting them.
Also worth reading: How Do Private AI Health Tools Protect Your Medical Privacy in 2026? · Are AI Health Assistants Truly Private and HIPAA-Compliant in 2026? · How Can You Protect Your Health Data When Using an AI Benefits Assistant in 2026?
A free consumer chatbot, a clinician-operated note-taking tool, and an employer-sponsored wellness service can have very different privacy practices. A chatbot that generates replies in real time needs conversational context to work, while a clinician may use a separate system to summarize a live session. The key question is not simply whether the service uses AI, but whether each copy of the information is disclosed, necessary, encrypted, and governed by a policy the user can verify. “We do not sell your data” is not the same as “no third party ever processes your data.”
The legal status of a conversation also depends on where the product is offered and who operates it. Health information received by a covered entity, such as a licensed provider or participating health plan, may be protected by the U.S. Health Insurance Portability and Accountability Act, or HIPAA. Many direct-to-consumer apps are not covered by HIPAA, however, and may instead operate under general consumer, privacy, or state laws. International law adds another layer; the 2022 International Data Privacy Law article described differences among regulatory regimes, illustrating why identical products can face different obligations across jurisdictions.
How AI Therapy Systems Use Mental Health Information
AI is used in several ways, and the privacy risks differ by function. Some therapy apps use AI to create a supportive response or suggest exercises between appointments. Clinical systems may generate draft notes from a conversation that the clinician subsequently reviews and edits. Other products create summaries, match users to programs, identify changes in language, or analyze large quantities of records to improve an internal service. Not every system retrains a public model on private conversations, but users usually need confirmation rather than assumption.
The term “AI therapy data privacy” covers a chain of activities: collection, transmission, storage, analysis, model improvement, human review, retention, and deletion. A service may use reputable encryption in transit, yet still retain the original transcript indefinitely. It may promise not to sell personal information, yet use a cloud host, speech-recognition vendor, analytics provider, or moderation contractor. It may offer deletion from an app account, yet preserve records required for legal, safety, or clinical purposes. Privacy protections therefore have to be evaluated product by product and feature by feature.
Consumers should also distinguish between conversational memory and model training. Memory lets an AI remember prior preferences or events during a later conversation; training changes a system's behavior more broadly. The distinction is sometimes blurred because anonymized, de-identified, or aggregated data can still carry re-identification risks. A precise privacy policy should explain whether human staff can review chats, whether prompts are used to improve models, how long each category is retained, and whether a user can refuse training-related use. Broad statements such as “we value privacy” do not answer those operational questions.
What “HIPAA Compliant” Really Means
“HIPAA-compliant” can be a useful shorthand, but it needs context. It generally indicates that a participating organization has administrative, physical, and technical safeguards designed to meet HIPAA requirements and has signed a business associate agreement with vendors that handle protected health information. Compliance does not prove that a product is clinically safe, permanently private, or free from every possible misuse. It also does not mean that ordinary consumer chatbots automatically qualify merely because they mention health.
HIPAA applies principally to covered entities and their business associates, not to every online company that discusses health. A direct-to-consumer AI companion may fall outside HIPAA even if it asks about depression, trauma, or medication. In that situation, its handling of information may be governed by other federal rules, contracts, consumer-protection laws, and state privacy statutes, but those protections may be less uniform. Some states have introduced health-data privacy laws that apply more broadly than HIPAA, while their requirements and enforcement differ.
International privacy law also presents a second set of considerations. The European Union's GDPR generally restricts processing of personal data without a lawful basis, grants rights such as access and deletion, and imposes special protections for certain health information. Other countries use different national frameworks. A mental health consultant should therefore determine where a user lives, where the company is based, where servers are located, and where vendors process information before declaring that a product is private or compliant.
This terminology matters because users can make unsafe assumptions. Someone may avoid asking a HIPAA-covered portal detailed questions while typing anything imaginable into an unregulated consumer app. Another person may assume that deleting an account removes all legally retained records. The strongest answer is to ask which data elements are covered, under which law, by whom, for how long, and with what remedy when the service fails.
| Feature | Consumer AI Therapy Chatbot | Clinician-Connected AI Tool | In-Person Private Therapy |
|---|---|---|---|
| Who generally controls the platform | Private technology company | Licensed provider or health organization and contracted vendors | Treatment relationship usually limited to provider and authorized staff |
| HIPAA coverage | Often unclear or absent | May apply when connected to covered entities and proper contracts | Usually applies to covered providers |
| AI functions | Chat responses, memory, exercises, check-ins | Note drafting, summarization, scheduling, clinical support | Generally administrative; AI use is disclosed as applicable |
| Human review | Variable or none | Commonly clinician review, but automation levels vary | Professional treatment responsibility remains clearer |
| Main privacy question | “Who may see, retain, or train on my chats?” | “What enters the clinical record, and who can access it?” | “What is protected under my jurisdiction and provider policy?” |
| Approximate cost | Free to several hundred dollars per year, depending on the service | Often embedded in provider fees or covered insurance | Common, but regional prices and reimbursement vary greatly |
People routinely share extremely personal details with mental health services. A conversation may include childhood abuse, sexual experiences, substance use, domestic violence, grief, suicidal thinking, or details about family members who never consented. If that information is mishandled, consequences can include discrimination, embarrassment, financial harm, loss of employment, or danger to a person's physical safety. Confidentiality is therefore part of the therapeutic relationship rather than merely a product feature.
An AI system also faces risks that ordinary software can have at a larger scale. Unauthorized access, accidental configuration changes, hidden data exports, weak deletion controls, and excessive retention can expose many conversations at once. Model outputs can reproduce sensitive content or create an inaccurate summary that later shapes treatment. A privacy incident may also be difficult for a consumer to discover because the information can move among several vendors before it reaches the company's primary server.
Safety and privacy overlap in crisis situations. A well-designed service may review conversations for a credible threat of harm, but automatic escalation can involve a third party or emergency service. Users need to know whether crisis detection is automated, whether staff review reports, what location the system assumes, and what response it promises. A statement that a chatbot is “anonymous” may be inaccurate if the app collects device data or contacts a cloud vendor. These trade-offs should be explained before the user enters details, not discovered during an emergency.
Independent research and professional guidance have raised concerns about mental health chatbot safety, privacy, transparency, and regulation. The American Psychological Association's health advisory recommends careful evaluation of generative AI and wellness applications, including their limitations and risks. That guidance does not say AI tools have no value; it supports informed and cautious use. The safest approach is proportionate use: a service that helps someone journal between sessions is different from one that makes independent diagnostic or treatment decisions without professional involvement.
How to Evaluate an AI Therapy Service Before You Use It
Begin with the privacy policy, terms of service, clinical safety page, and vendor disclosures. Search for “model training,” “human review,” “retention,” “delete,” “third-party,” “HIPAA,” “business associate,” “server location,” and “minor data.” Look for a defined process that can be understood, not only vague language about security. A policy that names subprocessors and provides update notices is usually more informative than one claiming generic “best-in-class” protection.
Then test the service with non-sensitive information. Before sharing a real diagnosis or crisis history, ask the chatbot what it records, who can access prior conversations, whether chats may be used for model improvement, and how to delete them. Some answers may come from customer support rather than the policy itself, so important answers should be retained for later comparison. A user should not rely on a chatbot's statement that its own data are secure; chatbot-generated privacy claims can be inaccurate.
Next, review account controls. Strong options include a clear opt-out from training when one is available, short or selectable retention, two-factor authentication, deletion instructions, and warnings before permission changes. For a clinician-connected system, ask whether AI drafts require approval before entering the official record, whether clinicians are trained to check hallucinations, and whether raw audio or video is retained. Patients should receive this information without having to speculate about what software the provider bought.
For a real crisis, the evaluation should end before information is entered. If someone may harm themselves or another person or is in immediate danger, the appropriate action is to contact local emergency services, go to an emergency department, or use a U.S. crisis service such as 988 or the equivalent service in the user's country. A privacy review matters, but access to urgent help outranks a preference for an anonymous chatbot. AI should never be treated as the only option in a time-sensitive emergency.
Cost, Benefits, and Reasonable Alternatives
AI therapy services can be inexpensive or free, which explains their appeal. Many consumer products are offered at no charge, through a limited free tier, or at roughly $10 to $50 per month; enterprise clinical tools may be included in a provider's fees, insurance package, or institutional contract. These figures are not universal and may change. The “HIPAA-compliant” label may justify a higher price because compliance requires safeguards, contracting, monitoring, and oversight, but price alone does not demonstrate effectiveness or privacy.
The potential benefits are real but should be described accurately. AI tools can provide around-the-clock check-ins, reduce the friction of writing, help users practice journaling or cognitive behavioral exercises, and offer immediate language support when appointment slots are unavailable. They can also give clinicians a starting point for a summary. However, access, affordability, and personalization do not automatically equal clinical effectiveness. A chatbot cannot examine a person, verify a diagnosis, or reliably understand every cultural and family context, and a fluent response can conceal a serious error.
Alternatives range from private paper or encrypted journaling and structured workbooks to conventional therapy, telehealth, support groups, peer support, and clinician-led tools with limited AI functions. A person who wants privacy may prefer a clinician able to explain confidentiality rules, although they should still ask about technology use. A person who cannot afford therapy may use a vetted AI app as a limited support tool while seeking community, charity, training-clinic, or public-health services. The choice should reflect urgency, clinical need, budget, accessibility, and tolerance for technology.
If a service offers only a one-time low price, ask about automatic renewal and cancellation. The total price should cover subscriptions, add-ons, voice features, premium models, and cancellation terms. Users should avoid entering payment information or sensitive health details merely to obtain a vague “free” demonstration. Transparent pricing, clear control over data, and a realistic statement of capability are better indicators of quality than the word “free.”
Common Privacy Mistakes and When to Act
One common mistake is assuming a health label provides HIPAA protection. Another is typing details into an unidentified assistant and only later asking who owns the conversation. People also fail to distinguish account deletion from deletion from backups, vendor systems, fraud-prevention records, or legally required clinical records. Review dates are often missed because most accounts are accepted once and never examined again, despite product and vendor practices evolving.
A second mistake is providing unnecessary information. Sharing a name, address, employer, exact location, medication list, or daily routine may not be needed for a journaling prompt. Users can generalize a story or use a fictional alias, although the decision depends on the service's terms. They should avoid uploading identity documents, screenshots of medical charts, voice memos, or entire PDF records to a general-purpose chatbot. Uploading data also creates two concerns: what the tool knows now and whether the upload remains in its storage or conversation history.
A final mistake is assuming that a familiar brand can decide privacy settings for the user. Browser settings, third-party integrations, AI features added to messaging platforms, and employer accounts can alter data flows. Privacy may change after a launch, acquisition, redesign, or new business arrangement. A reasonable review interval is every six to twelve months and immediately before uploading new records.
Users should pause and act now if they uploaded highly sensitive records to a service they cannot identify, received an unexpected third-party login prompt, noticed an incorrect account or location, or received no clear deletion instructions. They can first open account settings and disable optional sharing, then contact the provider for confirmation of retention, training use, access logs, and deletion. For unauthorized disclosure, suspected fraud, or immediate safety threats, preserve screenshots and contact the relevant provider, privacy regulator, credit or identity service, healthcare organization, or law-enforcement agency as appropriate. Changing a password and enabling two-factor authentication can reduce account risk, but it does not erase previously exposed information.
For a therapist whose patient is considering an AI companion, the clinician can review the product together without demanding that every nonmedical function be rejected. The clinician should identify likely blind spots, emergency contacts, and questions for informed consent, while directing urgent or high-risk decisions to human care. As of October 1, 2026, no product should receive blanket approval simply because it uses encryption, claims AI development,, or carries a health-related description. Continuous reassessment remains the most defensible privacy practice.