insights

Purpose

insights is the main REST API server. It serves the endpoints the web dashboard uses to query captured traffic, hosts, certificates, compliance status, and SCA results, and it handles authentication (JWT sessions backed by an external identity provider, with optional two-factor authentication). It connects to ClickHouse for all data and to Redis for background task enqueueing (TLS/nmap scans triggered from the dashboard) and the WebSocket status feed.

Usage

insights -c <config.yaml> [-debug]

Flag	Default	Description
`-c`	`/etc/insights.yaml`	Path to the configuration file
`-debug`	`false`	Enable debug-level logging

The config file also carries a debug setting; either the flag or the config value turns on debug logging.

Configuration

The insights.yaml template is the starting point. Key sections:

name: insights
debug: false
listen: "localhost:3000"
uploadpath: /var/lib/insights/uploads
backup_path: /var/lib/insights/backups

db:                # ClickHouse (note: uses the generic `db` key, not `clickhouse`)
  host: localhost
  port: 19000
  database: insights
  username: <clickhouse-user>
  password: "<clickhouse-password>"

fusionauth:
  baseurl: http://localhost:9011
  apikey: <fusionauth-api-key>
  applicationid: <application-uuid>
  clientsecret: <client-secret>
  tenantid:

redis:
  host: localhost
  port: 6379
  password: "<redis-password>"   # Redis AUTH is required
  channel: insights

sca:
  webhook_base_url: ""
  oauth:
    encryption_key: "${SCA_ENCRYPTION_KEY}"   # 32-byte hex string
    github:
      client_id: "${GITHUB_CLIENT_ID}"
      client_secret: "${GITHUB_CLIENT_SECRET}"
      redirect_url: "https://your-domain/oauth/github/callback"
    gitlab:
      client_id: "${GITLAB_CLIENT_ID}"
      client_secret: "${GITLAB_CLIENT_SECRET}"
      redirect_url: "https://your-domain/oauth/gitlab/callback"

On startup, insights connects to ClickHouse, loads default settings, and then starts serving HTTP on listen.

API surface

All routes are mounted under /api and require a JWT (Authorization: Bearer <token>) except /auth/*, the OAuth callback endpoints, and the webhook endpoints. Route groups include:

/api/clients, /api/hosts, /api/handshakes, /api/certificates — query captured traffic and inventory
/api/hosts/{uuid}/tlsscan, /api/hosts/{uuid}/nmapscan — queue a scan for a specific host
/api/traffic/{hourly,sixhour,daily,weekly,monthly} — time-bucketed traffic volume for dashboard charts
/api/compliance/* — compliance stats, findings, framework/rule management (see Compliance)
/api/cbom/* — Cryptographic Bill of Materials generation/download
/api/sca/* — repository connections, OAuth, scan triggers and results
/api/chat/* — an AI assistant for ad-hoc questions about the data
/api/upload/pcap, /api/pcap/* — PCAP file upload and management (see pcapworker)
/api/status, /api/status/queues, /api/status/workers — service and queue health
/auth/login, /auth/refresh, /auth/two-factor — authentication
/oauth/github, /oauth/gitlab (+ /callback) — SCA repository OAuth
/webhooks/github, /webhooks/gitlab — repository push webhooks that trigger SCA scans

/api/* and /auth/* are rate-limited per client IP.

Example

insights -c /etc/insights.yaml

capture