Tenable Cloud Sync

The Tenable sync service connects ThreatWeaver to the Tenable.io cloud platform via the Export API v2. It pulls asset and vulnerability data in chunks, normalizes it into the ThreatWeaver data model, and stores it in PostgreSQL.

Sync Architecture

Sync Modes

Mode	Description
Full Sync	Exports all vulnerabilities and assets from Tenable.io. Used for initial setup or periodic full refresh.
Incremental Sync	Exports only records modified since the last sync timestamp. Used for regular scheduled updates.
Assets Only	Exports only the asset inventory, skipping vulnerabilities.
Vulnerabilities Only	Exports only vulnerability data, skipping assets.

Sync Configuration

Syncs can be triggered manually or run on a schedule:

• Manual sync -- Admin triggers via POST /api/sync/start with optional date range and scope
• Scheduled sync -- Configured frequency (in hours) runs automatically via the sync scheduler
• Resume -- A failed or interrupted sync can be resumed by passing the resumeJobId parameter

Chunk Processing

The Tenable Export API delivers data in chunks. The sync service processes each chunk sequentially:

1. Request chunk -- Fetch the next chunk from the Export API
2. Normalize -- Map Tenable field names to ThreatWeaver schema, classify OS categories (Windows Server, Workstation, Linux, Network), and standardize severity levels
3. Deduplication -- Prevent duplicate records via upsert logic keyed on Tenable plugin ID + asset ID
4. Persist -- Insert or update records in PostgreSQL using TypeORM
5. Progress update -- Emit SSE (Server-Sent Events) progress event with chunk count, percentage, and ETA

ETA Calculation

The sync status endpoint (GET /api/sync/status) computes a real-time ETA:

• Only calculated after 15% progress (earlier estimates are unreliable)
• Based on elapsed time divided by fraction complete
• Capped at 4 hours (14400 seconds) -- beyond this threshold, the estimate likely indicates a stalled sync
• Returned as etaSeconds in the status response

Data Normalization

During sync, raw Tenable data is transformed:

Raw Field	Normalized Field	Logic
asset.operating_system	os_category	Classified into Windows Server, Workstation, Linux, macOS, Network Device via regex rules
severity (1-4)	severity (info/low/medium/high/critical)	Mapped to standard severity labels
plugin_output	Vulnerability description	Sanitized and truncated for display
first_found / last_found	firstSeen / lastSeen	ISO date conversion

Sync History and Logging

Every sync operation is logged as a SyncJob entity with:

• Start and end timestamps
• Trigger type (manual / scheduled)
• Triggered-by user email
• Progress percentage
• Chunk counts (total, processed, failed)
• Final status (completed / failed / cancelled)
• Error messages if applicable

The sync history is viewable at /admin/sync in the UI.

API Endpoints

Method	Path	Description
GET	/api/sync/status	Current sync status with progress and ETA
POST	/api/sync/start	Start a new sync (admin only)
GET	/api/sync/history	List past sync jobs
POST	/api/sync/cancel	Cancel a running sync

Related Pages

• WeaverScore -- How synced data feeds into risk scoring
• Exposure Management Overview -- Module overview and architecture