Layer 3 — Workflow Orchestration: Temporal.io

Durable, long-running, signal-driven workflows. One workflow execution per onboarding case.

---

Table of Contents

1. 6.1 Why Temporal for This Problem
2. 6.2 Workflow Hierarchy
3. 6.3 Signal-Based Human Task Pattern
4. 6.4 Error Recovery Pattern
5. 6.5 Temporal Worker Deployment on EKS

---

6.1 Why Temporal for This Problem

Client onboarding is, at its core, a durable long-running workflow with human task signals. Temporal maps cleanly onto every concept the domain requires:

Workflow Concept	Temporal Implementation
Process definition	Workflow definition (interface + implementation)
Automated system task	Activity (executed by a Worker)
Human approval step	Workflow waiting for a Signal
Process variable / state	Workflow state (persisted in Temporal’s PostgreSQL backend)
Nested subprocess	Child Workflow
Parallel fan-out	Promise.allOf() / async Activity fan-out
Integration error handling	Temporal’s built-in retry + ApplicationFailure + compensation
Conditional branching (segment gate)	Standard Java if/switch in workflow code
Process instance	Workflow Execution (uniquely identified by Workflow ID = case number)

---

6.2 Workflow Hierarchy

ClientServicingWorkflow          — main entry point, one per case
├── [optional] OrderApprovalGate  — ENTERPRISE only, waits for signal
├── ComplianceGateWorkflow         — child workflow, MEDIUM + ENTERPRISE
│   ├── DocumentReviewActivity    — creates human task, waits for signal
│   └── BSAAMLReviewActivity      — creates human task, waits for signal
├── CoreDataRetrievalActivity     — calls Core Banking System adapter
├── ProductProvisioningOrchestrator — fans out to N child workflows
│   ├── WireProvisioningWorkflow   ┐
│   ├── ACHProvisioningWorkflow    │ all run concurrently
│   ├── ARPProvisioningWorkflow    │ Promise.allOf() waits for all
│   ├── DigitalBankingWorkflow     │ segment + product-presence gate
│   ├── RTPWorkflow                │ determines which are started
│   └── ... (multiple products)   ┘
├── DigitalBankingSetupActivity   — Anchita Digital Banking Portal (user access provisioned; tier-based feature gating applied at runtime)
├── CredentialDeliveryActivity    — human task → Operations Team
└── CaseCompletionActivity        — Metrics write + notifications

EntitlementManagementWorkflow — day-2 operations
├── CurrentEntitlementLookupActivity
├── EntitlementChangeActivity
├── TreasuryPlatformUpdateActivity
├── [optional] DigitalBankingUpdateActivity
└── NotificationActivity

---

6.3 Signal-Based Human Task Pattern

Every operator action (approve, reject, attest, recover) is modelled as a Temporal Signal. The workflow reaches a human decision point, creates a task record in Aurora (which surfaces in the React workspace queue), and waits indefinitely — surviving restarts, deployments, and infrastructure failures:

@WorkflowInterface
public interface ClientServicingWorkflow {
    @WorkflowMethod
    CaseResult execute(OnboardingRequest request);

    // Human task signals
    @SignalMethod void submitOrderApproval(ApprovalDecision decision);
    @SignalMethod void submitDocumentReview(ReviewDecision decision);
    @SignalMethod void submitBSAAttestation(AttestationRecord record);
    @SignalMethod void submitErrorRecovery(RecoveryAction action);
    @SignalMethod void submitCredentialDeliveryConfirm();

    // Read workflow state without modifying (React UI polling)
    @QueryMethod WorkflowStageInfo getStageInfo();
}

Signal flow:

React UI → GraphQL Mutation submitApproval()
  → workflow-service.sendApprovalSignal()
  → Temporal client .signal("case-001", "submitDocumentReview", decision)
    → Workflow wakes, evaluates decision, advances to next phase
      → Temporal activity writes next task to Aurora
        → React workspace subscription fires → UI updates

---

6.4 Error Recovery Pattern

Integration failures are handled via Temporal’s compensation pattern. Failures create human-visible tasks in Aurora and wait for operator decisions:

// In an activity execution
try {
    integrationService.callCoreSystem(request);
} catch (IntegrationException e) {
    // Create error recovery task in Aurora (surfaces in React Error Recovery Panel)
    auditService.createErrorTask(caseId, e.getMessage());
    // Signal the workflow to wait for operator decision
    workflow.waitForErrorRecovery(); // blocks until submitErrorRecovery signal arrives
}

// Signal handler in workflow
@SignalMethod
public void submitErrorRecovery(RecoveryAction action) {
    switch (action) {
        case RETRY    -> retryFlag.set(true);
        case SKIP     -> skipFlag.set(true);
        case ESCALATE -> escalateFlag.set(true);
    }
}

Resilience chain:

1. Temporal built-in retry (up to 3 attempts, exponential backoff)
2. On retry exhaustion → ApplicationFailure → workflow receives error signal
3. Error recovery task created in Aurora → surfaces in React Error Recovery Panel
4. Operator submits RETRY, SKIP, or ESCALATE decision
5. Workflow resumes accordingly

This error recovery path is available for all three segments — Small, Medium, and Enterprise.

---

6.5 Temporal Worker Deployment on EKS

Temporal Workers are Spring Boot applications that poll the Temporal Task Queue and execute Activities. They are stateless and scale horizontally:

Temporal Server (EKS namespace: temporal)
├── Frontend service (gRPC endpoint)
├── History service (workflow state management)
├── Matching service (task queue routing)
└── Worker service (internal Temporal worker)

Application Workers (EKS namespace: workers)
├── servicing-worker          (ClientServicingWorkflow activities)
├── compliance-worker         (ComplianceGateWorkflow activities)
├── product-provisioning-worker (product sub-process activities)
└── integration-worker        (all external system calls)

KEDA ScaledObject: scale integration-worker based on Temporal task queue depth

Java 25 virtual threads: Each Temporal activity execution runs on a virtual thread (Project Loom). This allows thousands of concurrent in-flight activities — particularly activities waiting on external system responses — without OS thread exhaustion. The worker pod handles far higher concurrency per replica than a traditional thread-per-request model.

See Infrastructure for the full KEDA configuration and EKS cluster layout.