Back to CaraLoom

AI safety, in writing.

Every AI surface on the CaraLoom platform is listed here — what it does, what data it sees, who reviews its output, and the active guardrails in place. We refresh each clinician-validated badge every 6 months. No marketing language, no hidden surfaces.

Total AI surfaces
8
Clinician-validated
0
Refresh due
0
Validation pending
8

Platform-wide guardrails

BAA-covered LLM chain

All PHI-touching AI calls route through Google Vertex AI under an active BAA. A runtime guard in _ai_send_phi blocks any OpenAI fallback. Whisper STT is wrapped in the same BAA chain for voice-to-SOAP.

PHI scrub before LLM boundary

Identifying fields (names, addresses, contact, IDs) are scrubbed at prompt-interpolation. Clinical values (vitals, diagnoses, scores) are preserved so the AI has the signal it needs without seeing identifiers.

Centralized PHI audit log

Every AI call against patient data emits an audit-log entry — caller, patient, surface, timestamp, request id — into the tamper-evident audit collection. Admins can produce a full chain-of-custody for any audit.

Clinician-in-the-loop

AI never autonomously charts, prescribes, submits a claim, or escalates care. A licensed clinician signs every final clinical record. AI outputs are drafts and advisory signals, not autonomous actions.

Every AI surface, in detail

8 surfaces total. Each runs under the platform-wide guardrails above plus the per-surface guardrails listed below.

Patient Intake Assessment
Onboarding triage scoring
POST /api/auth/register (patient flow)
high-risk Validation pending
Inputs
  • · Diagnosis
  • · Care level needed
  • · Functional limitations
  • · Mental health factors
  • · Location
  • · Budget
  • · Schedule
Outputs
  • · care_complexity_score (0–100)
  • · urgency_level (Routine|Moderate|High Risk)
  • · recommended_care_level (CNA|LPN|RN|NP|MD|Hospice MD)
  • · care_type_recommendation (telehealth|in_person|both)
  • · reasoning
Viewer
Patient + matched clinician
Per-surface guardrails
  • · PHI scrubbed at prompt interpolation
  • · Default fallback returned on AI failure
  • · No prescriptive medical advice in reasoning
Clinician Intake Fit Scoring
Clinician onboarding fit
POST /api/clinician/assessment
low-risk Validation pending
Inputs
  • · Years of experience
  • · Certifications
  • · Specialties
  • · Preferred care setting
  • · Weekly availability
  • · Comfort with diagnoses
Outputs
  • · competency_score (0–100)
  • · risk_tolerance_category (Low|Medium|High)
  • · care_intensity_capacity (0–100)
  • · strengths
  • · recommended_case_types
Viewer
Clinician (self-view) + admin
Per-surface guardrails
  • · Output never gates licensure or platform access
  • · Used as a soft signal for matching, not eligibility
SOAP Note · Text Appointment
Appointment-derived SOAP draft
POST /api/appointments/{id}/generate-soap
medium-risk Validation pending
Inputs
  • · appointment.notes (clinician dictation/typing)
Outputs
  • · subjective
  • · objective
  • · assessment
  • · plan
Viewer
Clinician (draft view; clinician signs the final note)
Per-surface guardrails
  • · Clinician-authored draft, AI restructures only
  • · AI never invents clinical facts (per system prompt)
  • · PHI scrubbed before LLM
  • · Clinician signature required before persistence
SOAP Note · Voice Transcription
Voice-to-SOAP via Whisper STT + Gemini structuring
POST /api/voice/transcribe-to-soap
medium-risk Validation pending
Inputs
  • · Base64 audio (clinician dictation)
  • · patient_id
Outputs
  • · subjective
  • · objective
  • · assessment
  • · plan
  • · suggested_title
  • · key_diagnoses
  • · vitals_detected
Viewer
Clinician (draft view; clinician signs the final note)
Per-surface guardrails
  • · BAA required at runtime (_require_phi_baa gate)
  • · Whisper STT covered by BAA chain
  • · Transcription scrubbed before LLM structuring step
  • · Vital signs flagged separately, never invented
Patient Clinical Risk Assessment
AI risk stratification from check-ins + profile
POST /api/risk/assess
high-risk Validation pending
Inputs
  • · Patient profile (complexity, conditions, care level, urgency)
  • · Last 7 daily check-ins (mood, pain, symptoms)
Outputs
  • · risk_level (low|moderate|high|critical)
  • · risk_score (0–100)
  • · risk_factors
  • · recommendations
  • · alert_clinician (boolean)
  • · suggested_interventions
Viewer
Clinician (caring for patient) + admin
Per-surface guardrails
  • · _check_phi_access enforces care-relationship before run
  • · PHI scrubbed at prompt interpolation
  • · alert_clinician flag drives in-platform notification, not automated escalation
  • · Audit log entry per assessment
Clinical Note Handoff Summary
AI-generated handoff / shift-change summary
POST /api/clinical-notes/ai-summary
medium-risk Validation pending
Inputs
  • · Last 7 days of clinical notes (or explicit note_ids list)
  • · summary_type (handoff|shift_change|care_plan|weekly)
Outputs
  • · Structured summary (Current Status / Key Changes / Active Issues / Recommended Actions per summary_type)
Viewer
Receiving clinician at handoff + admin
Per-surface guardrails
  • · _check_phi_access enforces care-relationship
  • · Patient name referenced as "the patient" in prompt; real name persisted only in stored doc
  • · Receiving clinician must acknowledge handoff in-platform
RCM Claim Scrub · Denial Prevention
Pre-submission AI claim audit
POST /api/rcm/claims/{claim_id}/scrub
low-risk Validation pending
Inputs
  • · Claim payload (CPT, ICD-10, modifiers, POS, units, provider NPI, etc.)
Outputs
  • · status (clean|hold|blocked)
  • · risk_score (0–100)
  • · risk_rationale
  • · issues[] (severity, location, rationale, suggested_fix)
  • · verified_clean[]
  • · recurrence_prevention
Viewer
Admin (RCM workflow); clinician sees status pill on claim row
Per-surface guardrails
  • · Output is advisory — submission gated by clinician, not AI
  • · Vertex AI timeout-bounded (≤55s) to stay under Cloudflare gateway
  • · Last result cached on claim for audit log
  • · AI-unavailable degrades to "pending" status, never auto-clean
Care Library Navigator
Free-text caregiver question → best 3-min clip
POST /api/care-library/navigate
low-risk Validation pending
Inputs
  • · Caregiver free-text problem statement (PHI-scrubbed)
Outputs
  • · best_item_id (catalog row id or null)
  • · reason (1–2 sentence warm intro)
  • · marketplace_thread (suggested clinician question)
Viewer
Caregiver / family-side users on /care-library
Per-surface guardrails
  • · Input is PHI-scrubbed before LLM (scrub_phi)
  • · System prompt forbids medical advice, prescriptions, or diagnostic claims
  • · Hard emergency-keyword shortcut routes to 911 (chest pain / stroke signs / SI)
  • · Audit log entry per call (phi_audit_log)
  • · AI-unavailable degrades to a warm fallback, never silently fails

Methodology: each clinician-validated badge is signed by a licensed clinician (MD, DNP, NP, or RN with relevant specialty) after a standardized review of golden-set test cases (typically 24–40 per surface). Pass-rate is the fraction of cases scored ≥3/5 on the per-surface rubric. Refreshes are scheduled every 6 months. See /app/memory/CLINICAL_VALIDATION_BRIEF.md for the rubric and golden cases.

Get started

Install CaraLoom

Add to your home screen for one-tap access.