vulncheck/ARCHITECTURE.md

TrueVuln — Architecture Overview

Stand: Mai 2026 (dev-Branch, post-Migration 022). Diese Doku spiegelt den dev-Branch wider. Stable main ist eine Teilmenge — siehe README.md vs. README.DEV.md.

---

1. System Architecture

                       ┌──────────────────────────────────────┐
                       │            Browser (UI)              │
                       │   Next.js 14 App Router (SSR + CSR)  │
                       └──────────────────┬───────────────────┘
                                          │ HTTPS/JSON + JWT (Cookie+Bearer)
                                          ▼
┌──────────────────────────────────────────────────────────────────────────┐
│                       Backend — FastAPI (Python 3.12)                    │
│                                                                          │
│   ┌──────────────┐  ┌─────────────────┐  ┌─────────────────────────┐    │
│   │  Routers     │  │  Services       │  │  Integrations (clients) │    │
│   │  (HTTP API)  │──▶ (Business Logic)│──▶ Wazuh / Nessus / NVD /  │    │
│   └──────────────┘  └─────────────────┘  │ EPSS / KEV / EUVD /      │    │
│                                          │ Vulnrichment / cvelistV5  │    │
│                                          └─────────────────────────┘    │
│                                                                          │
│   ┌──────────────────────────────────────────────────────────────────┐   │
│   │  APScheduler — Background Jobs                                   │   │
│   │  - sync_schedules (every 60s, picks up DB changes)               │   │
│   │  - SLA breach checker (hourly)                                   │   │
│   │  - Daily threat-intel refresh (EPSS + KEV)                       │   │
│   │  - Nightly compliance SCA refresh (02:00)                        │   │
│   │  - Nightly Vulnrichment cascade (04:30)                          │   │
│   │  - Nightly URS recompute + snapshot prune (05:10)                │   │
│   │  - User-defined scan schedules (Wazuh or Nessus, cron/interval)  │   │
│   └──────────────────────────────────────────────────────────────────┘   │
└────────────────────┬──────────────────────────────┬──────────────────────┘
                     │                              │
              SQLAlchemy 2.x                  httpx clients
                     ▼                              ▼
        ┌────────────────────────┐       ┌────────────────────────┐
        │  PostgreSQL 16         │       │  External Sources      │
        │  - alembic-managed     │       │  - Wazuh Indexer/API   │
        │  - 22 migrations live  │       │  - Nessus REST API     │
        │  - JSON columns:       │       │  - NVD REST API        │
        │    sources, refs,      │       │  - FIRST.org EPSS      │
        │    enrichment_sources  │       │  - CISA KEV feed       │
        │  - Settings KV store   │       │  - ENISA EUVD          │
        └────────────────────────┘       │  - cisagov/vulnrichment│
                                         │  - cveproject/cvelistV5│
                                         │  - SMTP server (mails) │
                                         │  - LDAPS / OIDC / SAML │
                                         └────────────────────────┘

Stack

Layer	Tech
Frontend	Next.js 14 App Router, React 18, TypeScript, Tailwind CSS, Heroicons
Backend	FastAPI 0.115, Pydantic v2, Uvicorn
ORM	SQLAlchemy 2.x + Alembic
Database	PostgreSQL 16
Scheduler	APScheduler 3.x (interval + cron triggers)
Auth	Local (bcrypt + TOTP), LDAPS (ldap3), OIDC (authlib + JWKS), SAML 2.0 (python3-saml)
AI	Infomaniak AI proxy (configurable via env)
HTTP client	httpx (sync + async)
Container	Docker Compose (backend, frontend, postgres)

---

2. Security Architecture (OWASP Top 10 Mitigation)

OWASP risk	Defence in TrueVuln
A01 Broken Access Control	RBAC enforced via require_role() dependency on every router; admin / editor / viewer scopes. Asset-ownership filter on vuln listings.
A02 Cryptographic Failures	TOTP secrets encrypted at rest with AUTH_PROVIDER_CRYPTO_KEY (Fernet). LDAP bind password same. bcrypt cost-12 for local passwords. JWT secret rotated per deployment.
A03 Injection	SQLAlchemy parameterised queries everywhere. CVE-IDs and search inputs sanitised. Wazuh-Indexer queries built via OpenSearch DSL, not string interpolation.
A04 Insecure Design	Strategy-Pattern auth (LocalAuthStrategy, LdapStrategy, OidcStrategy, SamlStrategy) — single orchestrator, single JWT path. JIT provisioning + role mapping configurable, no hard-coded admins.
A05 Security Misconfiguration	verify_ssl=True default for upstream integrations. CORS restricted to configured frontend host. CSP header on Next.js.
A06 Vulnerable Components	pip-audit + npm audit in CI. Dependabot on dev.
A07 Auth Failures	Failed-login counter on users table, lockout after threshold. Constant-time bcrypt verify for unknown usernames (pre-computed dummy hash). MFA gate before JWT issue. SAML strict signature + XSW check.
A08 Data Integrity	created_at/updated_at mixin on every audit-relevant row. AuditLog row per security event. Alembic migrations idempotent.
A09 Logging & Monitoring	Structured logger across backend, NotificationLog table, AuditLog table, /api/v1/audit/logs admin view.
A10 SSRF	All outbound integrations whitelist-URL; no user-controllable URLs.

---

3. Component Details

3.1 Routers (app/routers/)

Each router groups a logical resource. All require JWT except /auth/* login endpoints.

Router	Resource	Key endpoints
auth.py	Local login + MFA	/auth/login, /auth/verify, /auth/mfa/{setup,activate,disable}, /auth/providers
auth_admin.py	Provider config + role mapping	/auth/admin/role-mappings, /auth/admin/{ldap,oidc,saml}/test
auth_oidc.py	OIDC SSO	/auth/oidc/login, /auth/oidc/callback
auth_saml.py	SAML 2.0 SSO	/auth/saml/login, /auth/saml/acs, /auth/saml/metadata
vulnerabilities.py	Vulns, enrichment, overrides, sync	/vulnerabilities, /vulnerabilities/sync/wazuh, /vulnerabilities/enrich/*, /vulnerabilities/override/*
nessus.py	Tenable Nessus	/vulnerabilities/nessus/{test,scans,sync,scan-host,cleanup-pseudo-cves}
intune.py	Microsoft Intune / Graph	/integrations/intune/{test,sync,sync/status}
vcenter.py	VMware vCenter / ESXi	/integrations/vcenter/{test,sync,sync/status,catalog/refresh}
igel.py	IGEL UMS (IMI API v3)	/integrations/igel/{test,sync,sync/status}
xenserver.py	XenServer pool (XAPI)	/integrations/xenserver/{test,sync,sync/status}
assets.py	Asset CRUD + assignment	/assets, /assets/{id}, /assets/bulk-update, /assets/{id}/assign
scans.py	Scan history + schedules	/scans, /scans/summary, /scans/clear, /scans/autoscan, /scans/schedules
compliance.py	SCA + URS	/compliance/summary, /compliance/assets, /compliance/{asset_id}, /compliance/refresh, /compliance/impacts/{import,stats}, /compliance/urs[/...]
policies.py	Security policies (SLA)	/policies, /policies/{id}, /policies/bulk-update
groups.py	User groups	/groups, /groups/{id}, /groups/default
settings.py	Key-value config	/settings, /settings/{key}
notifications.py	Mail log + test	/notifications/log, /notifications/test, /notifications/notify-critical
reports.py	CSV / PDF exports	/reports/csv, /reports/pdf
audit.py	Security audit trail	/audit/logs

3.2 Services (app/services/)

Pure business logic, no FastAPI imports. Reusable from scheduler + routers.

Service	Role
enrichment_service.py	EPSS / KEV / EUVD enrichment per CVE, with 24h cache in settings
kev_service.py	Merges the KEV sources (CISA KEV + ENISA EUVD exploited + VulnCheck KEV) into one catalog and annotates it with inventory impact (open vs remediated, active assets only)
kev_alert_service.py	Immediate mail when an actively-exploited CVE has open findings here; idempotent per CVE
vuln_override_service.py	3-stage CVSS / SSVC / fixed_version cascade (Vulnrichment → NVD → cvelistV5)
override_jobs.py	Async job tracker for the long-running "Correct CVSS" button
nessus_sync.py	Per-scan import, asset matching cascade, pseudo-CVE handling
compliance_service.py	Wazuh SCA pull + weighted-score computation per policy
compliance_impact_import.py	CIS-Benchmark CSV importer with loose-header matching
urs_service.py	AVS + ASS → URS formula, snapshot writes, daily prune
email_service.py	SMTP send, per-recipient templates, digest + single modes
vcenter_service.py	vCenter + ESXi inventory sync → assets, EOL and vSphere CVEs
igel_service.py	IGEL UMS server + endpoint-device inventory sync → assets and IGEL OS CVEs
xenserver_service.py	XenServer pool sync → host assets, CVEs from the security updates pending per host
vmware_release_service.py	vSphere release identity: build numbers, update lines, Broadcom KB 326316 catalog

3.3 Integrations (app/integrations/)

Thin HTTP clients, no DB writes.

Client	Purpose
wazuh_client.py	OpenSearch indexer DSL — paginated vuln + SCA fetch
nessus_client.py	REST API + plugin output parsing (fixed_version, VPR, host UUID)
ai_client.py	LLM dispatcher (currently routes to Infomaniak)
infomaniak_ai_client.py	Infomaniak chat-completion proxy
graph_client.py	Microsoft Graph (app-only) — Intune managed devices + detected apps
vcenter_client.py	vSphere SOAP (pyVmomi) — about + one PropertyCollector pass over every HostSystem
igel_client.py	IGEL IMI REST v3 — serverstatus, the firmware table, and every device in one ?facets=details call
xenserver_client.py	XAPI on the pool coordinator — XML-RPC pool/host records + GET /updates

3.4 Auth Stack — Strategy Pattern

   POST /auth/login (username + password)
              │
              ▼
   ┌────────────────────────────┐
   │ AuthOrchestrator           │  chain = AUTH_LOOKUP_ORDER
   │ (env: local,ldap)          │
   └──┬─────────────────────────┘
      │  on success
      ▼
   ┌────────────────────────────┐
   │ MFA gate (if totp_enabled) │
   └──┬─────────────────────────┘
      │  OK
      ▼
   JWT (httpOnly cookie + bearer in response body)

   GET  /auth/oidc/login    ── PKCE redirect ── IdP
   POST /auth/oidc/callback ── JWKS-validated id_token → orchestrator.complete_sso_login()

   GET  /auth/saml/login    ── python3-saml AuthnRequest
   POST /auth/saml/acs      ── strict signature + XSW check → orchestrator.complete_sso_login()

JIT provisioning (auth.jit_provisioner) auto-creates User rows on first SSO/LDAP login. RoleMapper re-evaluates role on every login from external_groups, so AD-side membership changes apply immediately.

3.5 Scheduler (app/scheduler.py)

Job ID	Cadence	What it does
scheduler_sync	every 60 s	Picks up DB changes to scan_schedules and re-registers jobs
sla_breach_check	every 1 h	SLA-overdue scan; honors sla_breach_enabled toggle + PolicyStatus.DISABLED skip; digest or single mode
threat_intel_refresh	every 24 h	Refreshes EPSS, KEV, EUVD across all open vulns; runs kev_alert_check straight after, since the catalogs just moved
kev_alert_check	hourly, :25	Mails actively-exploited CVEs (sources per kev_alert_sources, default CISA KEV + ENISA EUVD) that have OPEN findings on active assets; honors kev_alert_enabled, idempotent via kev_alert_state
advisory_feeds_refresh	every 6 h, :20	ZDI/CERT-EU/BSI/... RSS feeds into the advisories cache
https_certificate_check	03:45	Renews the auto-generated HTTPS certificate (< 30 days left, names changed); logs a warning for an uploaded one about to expire. Also runs in lifespan
new_vuln_digest_nightly	hourly, :05	Sends the new-CVE roundup at notification_nightly_hour (default 06) when notification_schedule is nightly
vuln_index_refresh_nightly	01:30	Rebuilds the cvelistV5 (+MFSA), GitHub-advisory, TeamViewer, IGEL-ISN and MSRC fixed-build indexes before any sync or scan reads them
compliance_sca_nightly	02:00	Wazuh SCA pull for every linked asset
intune_sync_nightly	02:10	Intune managed devices → assets + OS-EOL + per-device app-CVE scan, Defender TVM CVEs when enabled; holds an advisory lock, can run for hours
vcenter_sync_nightly	02:20	vCenter + ESXi hosts → assets, EOL, vSphere CVEs (refreshes the build catalog first)
igel_sync_nightly	02:30	IGEL UMS server + endpoint devices → assets, IGEL OS CVEs
exposure_nightly	02:30	Wazuh syscollector ports → network-exposure score
netdisco_sync_nightly	02:40	Netdisco switches/routers → assets, firmware CVEs
dockhand_sync_nightly	02:50	Dockhand container-image findings (rescan first)
xenserver_sync_nightly	02:55	XenServer pool hosts → assets, CVEs of the security updates pending per host
post_inventory_scans_nightly	03:00	Waits for the Intune sync, then in turn: EOL check → M365 Apps CVEs → app-CVE scan → MSRC fixed-build scan + remediation catch-up
vulnrichment_nightly	04:30	3-stage CVSS/SSVC/fixed_version cascade
msrc_refresh_weekly	Sun 04:40	MSRC CVRF remediations, 18 months back
exploit_intel_nightly	04:50	Exploit-DB / PoC-in-GitHub / Metasploit counts
urs_nightly	05:10	URS recompute + asset_risk_snapshots prune (>90d)
asset_lifecycle_nightly	05:25	Inactivates assets no source reported within the window, revives seen ones
audit_log_prune_nightly	05:40	Audit-log retention prune (audit_log_retention_days)
jev_triage_nightly	06:00	Jev config hint for new CVEs (off unless enabled)
user-defined	per schedule	Per-scan_schedules row, routes to run_wazuh_vulnerability_sync or run_nessus_sync based on scanner_type

---

4. Data Model

See DATABASE_SCHEMA.md for column-level detail. High-level entities and their main relationships:

   User ─┐                            Policy ── 1:N ──┐
         │ M:N (user_groups)                          │
   Group ┴── 1:N ──┐                                  │
                   │                                  │
                   ▼ M:N (asset_groups)               ▼
                Asset ──── 1:N ────► Vulnerability ──── 1:N ──► AIAnalysis
                  │                       │
                  │ 1:N                   │ 1:N
                  ▼                       ▼
                Scan                  NotificationLog
                  │
                  └── 1:N ──► ComplianceResult ── 1:N ──► ComplianceCheck

   ComplianceImpact  (cis_id, benchmark)   ◄── lookup for weighted_score
   AssetRiskSnapshot (asset_id, snapshot_date, avs, ass, urs, severity)

   AuditLog          (user_id, event_type, …)
   Setting           (key, value)         ◄── KV store: smtp, nessus_config,
                                              enrichment flags, sla toggle,
                                              cache for EPSS/KEV/EUVD JSON,
                                              role-mapping JSON, etc.

Vulnerabilities carry a sources JSON list (["wazuh","nessus"]) so one row holds the union per (cve_id, asset_id). Non-CVE Nessus plugins land under pseudo-IDs NESSUS-PLUGIN-{plugin_id}.

---

5. API Endpoints (summary)

Full reference: see Router section above + Swagger UI at /docs. JWT auth required on everything except /auth/login, /auth/oidc/*, /auth/saml/*, /auth/setup-admin.

Group	Examples
Auth	POST /auth/login, POST /auth/verify (MFA), POST /auth/mfa/setup, POST /auth/oidc/login
Vulnerabilities	`GET /vulnerabilities?source=wazuh
Assets	GET /assets, POST /assets/{id}/assign, POST /assets/{id}/rescan
Scans	GET /scans/summary, POST /scans/autoscan, POST /scans/schedules
Compliance / URS	POST /compliance/refresh, POST /compliance/impacts/import (multipart), GET /compliance/urs?avs_mode=hybrid
Nessus	POST /vulnerabilities/nessus/test, POST /vulnerabilities/nessus/sync
Settings	PUT /settings/{key} (admin)

Query parameters of note on GET /api/v1/vulnerabilities:

status, severity, source, cross_confirmed, kev_only, euvd_only, eu_critical, in_any_catalog, in_both_catalogs, epss_min, assigned_to_me, host, package, cve_id, sort_by (cve_id|priority_score|cvss_score|cpr_score|detected_at|published_date|updated_at), sort_order, limit, offset.

---

6. Deployment Architecture

   docker-compose.yml
   ├── postgres     (volume: pgdata)
   ├── backend      (FastAPI + APScheduler)  ── alembic upgrade head on start
   │                writes CA + server certificate to volume tls_data
   ├── frontend     (Next.js standalone build, served via node; 127.0.0.1 only)
   └── nginx        (profile "tls": TLS 1.2/1.3 on 443, 80 → 301; reads tls_data,
                     reloads itself when the certificate changes)

Environment configuration via .env (gitignored). Required keys:

DATABASE_URL=postgresql://vulnmanager:…@postgres/vulnmanager
JWT_SECRET=…                # rotate per deployment
AUTH_PROVIDER_CRYPTO_KEY=…  # Fernet, encrypts TOTP + LDAP bind pw

# Wazuh
WAZUH_API_URL=https://…:55000
WAZUH_INDEXER_URL=https://…:9200
WAZUH_API_USER=…
WAZUH_API_PASS=…

# Optional
DASHBOARD_URL=http://localhost:3000          # used in mail links
INFOMANIAK_API_KEY=…                          # AI analysis
LDAP_BIND_PASSWORD_BOOTSTRAP=…                # first-start LDAP setup
HTTPS_PROXY=…                                 # for outbound enrichment

Optional outbound HTTPS endpoints (firewall whitelist if locked-down):

• https://api.first.org (EPSS)
• https://www.cisa.gov (KEV)
• https://euvdservices.enisa.europa.eu (EUVD)
• https://raw.githubusercontent.com (Vulnrichment per-CVE + ZIPs)
• https://github.com (cvelistV5 ZIP)
• https://services.nvd.nist.gov (NVD REST)
• https://api.github.com (GitHub repo security advisories — Notepad++, Wazuh)
• https://www.teamviewer.com (TeamViewer security bulletins)
• https://nessus.your-internal/... (Nessus)
• https://graph.microsoft.com (Intune / Defender TVM)
• https://endoflife.date (EOL/EOS lifecycle)
• https://knowledge.broadcom.com (vCenter release → build catalog, KB 326316)
• https://vcenter.your-internal/sdk (vSphere SOAP, port 443)
• https://ums.your-internal:8443/umsapi/v3 (IGEL IMI REST)

Upgrade flow

git pull origin dev
docker compose build backend frontend
docker compose up -d backend frontend
docker compose exec backend alembic upgrade head
docker compose exec backend alembic current   # expect: 022 (head)

---

7. Performance & Scaling

• Wazuh-Indexer pagination beyond the OpenSearch 10 000-hit cap: search-after / scroll in WazuhClient.get_vulnerabilities.
• Vulnrichment cascade auto-routes to ZIP snapshot when > 25 CVEs requested — avoids 25 × N raw fetches.
• cvelistV5 ZIP (~557 MB) is on-disk-cached 12 h at /tmp/truevuln-cvelistv5-cache.zip.
• NVD stage caps at 100 CVEs per run (rate limit). >100 missing → falls through to cvelistV5 directly.
• sources / enrichment_sources JSON columns avoid table-join chatter for per-vuln source filtering.
• Indexed columns: cve_id, asset_id, cvss_score, severity, epss_score, kev_listed, euvd_listed, exploitation_status, ssvc_technical_impact, ssvc_automatable, nessus_vpr_score, nessus_plugin_id, package_name, detected_at.
• "Correct CVSS" runs as an async job (override_jobs.py) so the UI returns immediately with a job_id and a polling progress card — full-DB corrections take 5-20 min.

---

8. Monitoring & Observability

• Structured logger to stdout; docker compose logs backend is the primary view.
• AuditLog captures security-relevant events (login success / failure, role change, MFA reset, password change, deletes).
• NotificationLog records every mail attempt (status, error, recipient).
• /scans/summary surfaces sync history with per-run vuln deltas.
• /notifications/log lists outbound mail history with retry status.
• APScheduler logger prints next_run + last_run per job.

Recommended Prometheus exporters: postgres_exporter, node_exporter, and a thin /healthz endpoint on the FastAPI side (returns DB ping + scheduler status). Not yet shipped — placeholder for future ops work.

---

9. Development Roadmap

See README.DEV.md "Out of scope" sections for the live backlog. Big-ticket items currently parked:

• WebAuthn / FIDO2 second factor
• Per-framework URS breakdown (URS_CIS / URS_ISO27001 / URS_NIS2)
• SCIM v2 endpoint for IdP-driven provisioning
• endoflife.date integration for EOL/EOS package detection
• Real-time KEV push from Wazuh (currently poll-based)
• Two-way Nessus false-positive sync
• Concentration penalty on URS (≥ 3 high-impact failures → +10)
• Settings-UI toggle for sla_breach_enabled (currently DB-only)