hostsync

Purpose

hostsync is a daemon that periodically scans the tls_hellos table for destination IPs that aren’t yet in the hosts table (and source IPs not yet in clients), inserts them, and enqueues TLS and nmap scan tasks (via Redis/asynq) for each newly discovered host. It also refreshes last_seen timestamps for clients seen in the last hour. Without hostsync running, hosts observed by capture never get scanned or shown as “hosts” in the dashboard — they only exist as rows in tls_hellos.

Usage

hostsync -c <config.yaml> [-debug] [-once]

Flag	Default	Description
`-c`	—	Path to config file (required)
`-debug`	`false`	Enable debug logging
`-once`	`false`	Run a single sync pass and exit, instead of looping

Running with no flags at all prints usage and exits without doing anything.

Configuration

configs/hostsync.yaml:

name: hostsync
clickhouse:
  host: localhost
  port: 19000
  database: insights
  username: insights
  password: secret
redis:
  host: localhost
  port: 6379
debug: false
sync_interval: 5m   # e.g. "5m", "1h" — defaults to 5m if unset

On startup, hostsync validates that the hosts, clients, and tls_hellos tables exist in ClickHouse before starting the sync loop, and reports a Redis heartbeat (visible on the dashboard’s status page) if redis.host is set.

Examples

# Run as a daemon, syncing every sync_interval
hostsync -c configs/hostsync.yaml

# Run a single sync and exit (useful right after a fresh capture)
hostsync -c configs/hostsync.yaml -once

# Run with debug logging
hostsync -c configs/hostsync.yaml -debug

capture tlsscand