Technical deep dive into Reynote: A collaborative swarm

Discover how a team of specialist AI agents allows Rey to focus on what matters

Technical Deep Dive: The Multi-Agent Architecture Behind Reynote

The Challenge of Integrative Therapy

When we first started building Reynote, we embraced the Therapeutic Palette framework as our guiding principle. This approach, developed by Dr. Peter Fraenkel, addresses the fundamental challenge that every relationship therapist confronts: how to seamlessly blend multiple therapeutic approaches along three key dimensions—Time Frame, Degree of Directiveness, and Change Entry Point—while maintaining genuine presence with clients. Effective relationship therapy isn’t about applying a single framework rigidly—it’s about knowing precisely when and how to shift between these dimensions based on what’s happening in the moment.

A skilled therapist might start with narrative techniques to understand a couple’s story, pivot to cognitive-behavioral approaches when unhelpful thought patterns emerge, then employ solution-focused methods to build on existing strengths. Throughout this dance between frameworks, they must simultaneously track time, maintain session structure, and note important themes for future exploration—all while staying fully present with their clients.

The art of integrative therapy lies in knowing exactly when to shift along the three key dimensions of the Therapeutic Palette:

• Time Frame Dimension:

  • Past-oriented work when understanding attachment patterns and relationship history
  • Present-focused techniques for addressing immediate interaction patterns
  • Future-oriented approaches when developing shared visions and goals

• Degree of Directiveness Dimension:

  • Low directiveness (facilitating) to mobilize existing strengths and resources
  • Medium directiveness (interpreting) to highlight patterns clients may not see
  • High directiveness (teaching) when couples need specific skills or education

• Change Entry Point Dimension:

  • Emotional entry points for addressing attachment and connection issues
  • Cognitive entry points for working with beliefs and relationship narratives
  • Behavioral entry points for creating immediate pattern changes
  • Physiological entry points (breathing, relaxation) when tension is high

This integrative approach is what makes human therapists so effective. They don’t just apply techniques mechanically; they weave them together organically, creating a therapeutic experience that’s greater than the sum of its parts.

The LLM Challenge: Divided Attention Means Diminished Performance

When we began experimenting with large language models like Claude and ChatGPT for relationship therapy, we quickly discovered their limitations when asked to juggle multiple complex tasks. Despite their impressive capabilities, these models struggle when their attention is divided across too many objectives simultaneously.

The constraints became clear: context windows limit how much information they can process at once, task-switching reduces effectiveness, and attention dilution occurs when tracking too many variables. When we asked a single LLM to maintain therapeutic presence while simultaneously tracking time, applying multiple frameworks, identifying future topics, and managing conversation flow, the quality of each function suffered noticeably.

It was like asking a single therapist to simultaneously be the primary clinician, take detailed notes, watch the clock, plan future sessions, and consult reference materials—all while maintaining eye contact and emotional connection with clients. No human therapist works this way, so why should we expect an AI to do so?

Our Solution: The Specialist Swarm Architecture

This realization led us to a breakthrough: instead of forcing a single AI to handle everything, what if we created a collaborative team of specialized AIs working together, just as human specialists might collaborate in a clinical setting?

The architecture we developed mirrors how human specialists collaborate. At the center is Rey, the primary therapist who directly interacts with users. Rey focuses on maintaining therapeutic presence and guiding the conversation flow—the most human-centered aspects of therapy.

Supporting Rey is a team of specialist observers, each dedicated to a specific therapeutic lens:

• Cognitive-Behavioral Observer: Identifies thought patterns and cognitive distortions
• Narrative Observer: Recognizes story elements and externalization opportunities
• Solution-Focused Observer: Spots strengths and exception patterns
• Emotional Process Observer: Tracks attachment dynamics and emotional undercurrents
• Communication Pattern Observer: Analyzes dialogue through communication theory frameworks

These specialists don’t interact directly with users. Instead, they observe the conversation and provide structured insights to Rey, who integrates them naturally into the therapeutic dialogue. This approach allows each component to focus deeply on its area of expertise without the cognitive load of managing the entire therapeutic process.

The Technical Implementation: Elixir’s Perfect Fit

Elixir’s actor model and concurrency primitives made it the perfect foundation for this architecture. The system we built leverages Elixir’s strengths in several key ways:

The primary therapist and each specialist observer run as separate GenServer processes, communicating through Phoenix PubSub. When a user sends a message, it’s received by Rey and broadcast to all specialist observers. After analyzing several messages, specialists formulate structured observations containing clinical insights and send them back to Rey.

This inter-agent communication happens in real-time, allowing Rey to incorporate insights from multiple therapeutic traditions seamlessly. The architecture naturally handles the asynchronous nature of these interactions—specialists can take their time formulating deep insights without blocking the main conversation flow.

The Collaborative Agent Team

Our multi-agent architecture consists of several specialized components working together:

• Primary Therapist (Rey): The central agent that directly interacts with users, maintains therapeutic presence, and integrates insights from other specialists.

• Specialist Observers:

  • Cognitive-Behavioral Observer: Identifies thought patterns, cognitive distortions, and unhelpful beliefs that may be affecting the relationship.
  • Narrative Observer: Recognizes story elements, externalization opportunities, and helps reframe relationship narratives.
  • Solution-Focused Observer: Spots strengths, exception patterns, and opportunities to build on existing resources.
  • Emotional Process Observer: Tracks attachment dynamics, emotional undercurrents, and deeper feeling patterns.
  • Communication Pattern Observer: Analyzes dialogue through communication theory, identifying destructive and constructive patterns.

• Session Management Components:

  • Time Keeper: Monitors session duration and provides regular updates to Rey.
  • Stage Progression System: Manages transitions between therapeutic phases.
  • Topic Tracker: Identifies and stores significant themes for future exploration.

• Partner Perspective Observer: When working with one partner individually, this component considers how the non-present partner might perceive the situation.

Each agent operates as a separate Elixir process, allowing them to work independently while communicating through a shared message bus. This architecture enables true parallel processing of therapeutic insights.

Session structure is maintained through a stage progression system that guides therapy through defined phases: Opening (5 min), Exploration (10 min), Understanding (10 min), Solutions (10 min), Integration (7 min), and Closing (3 min). The system tracks timing and provides Rey with regular updates, allowing for natural transitions between stages while maintaining therapeutic flow.

When significant themes emerge that can’t be fully addressed in the current session, Rey can note them for future exploration. Topics are categorized by priority level, with clinical notes capturing their therapeutic significance and suggestions for which therapeutic approaches might be most effective in addressing them.

Real-World Impact: More Than the Sum of Its Parts

The multi-agent architecture allows Reynote to provide therapy experiences that more closely mirror how human therapists work. Users experience more insightful observations that draw from multiple therapeutic traditions, better session pacing with natural transitions between stages, and continuity between sessions as important themes are tracked.

Most importantly, by freeing Rey from the cognitive load of juggling multiple frameworks simultaneously, we enable a more present therapeutic relationship. Rey can focus on what matters most—connecting with users and guiding the conversation—while the specialist team handles the analytical heavy lifting in the background.

This approach has proven particularly valuable for couples therapy, where the complexity of relationship dynamics often requires multiple therapeutic lenses to fully understand. A conversation that begins with communication patterns might reveal underlying attachment issues, which in turn connect to narrative themes about relationship expectations—all of which can be identified and addressed through our specialist swarm.

Future Directions: Evolving the Architecture

We continue to refine this architecture as we learn from real-world usage. We’re exploring adding specialized observers for additional therapeutic modalities, enhancing the communication protocols between agents, developing more sophisticated topic tracking and session planning, and improving how insights from multiple observers are integrated.

The beauty of our approach is its extensibility—adding new therapeutic perspectives doesn’t require rewriting the core system, just adding new specialist observers to the swarm. This allows us to continuously evolve Reynote’s capabilities while maintaining the focus and presence that effective therapy requires.

By embracing the specialist swarm approach, we’ve created a system that delivers relationship therapy combining the best of multiple therapeutic traditions—not by forcing a single AI to be a jack of all trades, but by orchestrating a team of specialists working in harmony toward a common therapeutic goal.