Request Flows

This document traces three core operations through the entire ThreatWeaver stack, from the frontend UI action to the database write and back.

---

Flow 1: Start a Scan

This is the most complex flow in ThreatWeaver. It spans the AssessmentWizard UI, the AppSec routes, the PentestCoordinator orchestrator, 56+ attack agents, AI validation, and SSE streaming.

Route Entry Point

From backend/src/routes/appsec.routes.ts, the scan start endpoint is:

POST /api/appsec/assessments/:id/start

This route requires authentication and the appsec module to be enabled (enforced by requireModule('appsec') middleware).

End-to-End Sequence

Pipeline Phases in Detail

The coordinator manages the assessment lifecycle through these states:

queued -> profiling -> planning -> attacking -> validating -> chaining -> completed

From pentestCoordinator.service.ts, each phase maps to specific services:

Phase	State	Service	Purpose
0	Bootstrap	ResourceBootstrapper	Budget allocation, auth, target validation
1	Profiling	targetProfiler.profile()	Endpoint discovery, crawling
2	Planning	pentestAiAdapter	AI attack plan generation
3	Attacking	56+ agent singletons	Parallel vulnerability testing
4	Validating	validationEngine.validate()	AI verification, dedup, enrichment
4.5		Sensitive data scan	AI scans HTTP responses for PII/secrets
4.6		Finding analysis	AI enriches critical/high findings
4.7		Remediation	AI generates framework-specific fixes
5	Chaining	chainBuilder.analyze()	Exploit chain discovery
5.5		Report narrative	AI executive summary
6	Completed	Finalization	Stats update, webhook notification

SSE Events

The coordinator extends EventEmitter and emits these event types (defined in AssessmentEvent):

type: 'phase_change' | 'progress' | 'agent_start' | 'agent_complete' |
      'agent_error' | 'finding_new' | 'chain_discovered' |
      'assessment_complete' | 'assessment_error' | 'warning' |
      'auth_test_user' | 'auth_test_users_complete' | 'resource_bootstrap'

The frontend subscribes to these via an SSE endpoint and renders live scan progress.

Agent Metrics

After each agent completes, the coordinator collects AgentMetrics:

export interface AgentMetrics {
    agentName: string
    requestsSent: number
    findingsCreated: number
    findingsValidated: number       // True positives
    findingsFalsePositive: number
    findingsDuplicate: number
    durationMs: number
    endpointsTested: number
}

---

Flow 2: View Dashboard

The dashboard flow is a read-heavy path that aggregates vulnerability data into KPIs, trends, and breakdowns.

Route Entry Point

From backend/src/routes/dashboard.routes.ts:

GET /api/dashboard/kpis
GET /api/dashboard/widget-data  (POST also supported)

Dashboard routes require authentication and the exposure_management module.

End-to-End Sequence

Widget Data Schema

Dashboard widgets use a validated request schema (from dashboard.routes.ts):

const widgetDataSchema = z.object({
    dataSource: z.enum([
        'vulnerabilities', 'assets', 'risk_scores', 'compliance',
        'remediation', 'sla', 'trends', 'appsec'
    ]),
    chartType: z.enum([
        'kpi', 'bar', 'line', 'doughnut', 'gauge', 'table',
        'matrix', 'stacked_bar', 'area', 'pie', 'number'
    ]).optional(),
    groupBy: z.string().max(50).optional(),
    filters: z.record(z.string(), z.unknown()).optional(),
    sort: z.object({
        field: z.string().max(50),
        order: z.enum(['ASC', 'DESC']),
    }).optional(),
    limit: z.number().int().min(1).max(1000).optional(),
})

Rate Limiting

Dashboard endpoints have their own per-user rate limiters, and the general /api/dashboard/* path is excluded from the global rate limiter to support high-traffic polling.

---

Flow 3: Tenable Sync

The Tenable sync flow imports vulnerability and asset data from Tenable.io using the Export API v2. This is the core data pipeline for the Vulnerability Management module.

Route Entry Point

From backend/src/routes/sync.routes.ts:

POST /api/sync/trigger    -- Start a new sync
GET  /api/sync/status     -- Get current sync status

Sync routes require authentication.

End-to-End Sequence

Sync Status Polling

The frontend polls GET /api/sync/status to track sync progress. The response includes ETA calculation (from sync.routes.ts):

// Only compute ETA after 15% progress -- earlier estimates are too volatile
if (progress > 15 && progress < 100) {
    const elapsedMs = Date.now() - startedAt.getTime()
    const fractionComplete = progress / 100
    const totalEstimatedMs = elapsedMs / fractionComplete
    const remainingMs = totalEstimatedMs - elapsedMs
    const etaSeconds = Math.ceil(remainingMs / 1000)
}

Data Flow Summary

Tenable.io Cloud
  |
  | Export API v2 (paginated chunks)
  v
syncService (normalize + transform)
  |
  | UPSERT (ON CONFLICT)
  v
PostgreSQL
  |
  | Aggregation queries
  v
aggregationService (KPIs, WeaverScore, trends)
  |
  | JSON responses
  v
Frontend Dashboard (React + Recharts)

Key Entities Involved

Entity	Table	Role
SyncJob	sync_jobs	Tracks sync status, progress, timestamps
Vulnerability	vulnerabilities	Normalized vulnerability records from Tenable
Asset	assets	Discovered assets (servers, workstations, etc.)
VulnerabilityStats	vulnerability_stats	Pre-computed aggregation statistics

---

Request Lifecycle Summary

Every API request in ThreatWeaver passes through this middleware chain (in order):

1. Helmet (security headers)
2. Permissions-Policy header
3. CORS
4. detectBaseUrl
5. Rate limiter (global)
6. Auth-specific rate limiters (login, MFA, etc.)
7. Body parser (JSON, 1MB limit)
8. Cookie parser
9. Body error handler (malformed JSON, too large)
10. JSON depth limiter (max 20 levels)
11. Decryption middleware (double encryption)
12. CSRF Origin validation (mutating requests)
13. Multi-tenant middleware chain:
    a. authenticate (JWT verification)
    b. resolveTenant (lookup schema from TLM)
    c. setTenantSchema (SET search_path on QueryRunner)
    d. enforceTenancy (license validation)
    e. tenantRateLimiter (per-tenant limits)
    f. usageMeter (request counting)
14. setupModeGuard (redirect if setup incomplete)
15. Route handler