MCM Platform — VM Sizing Calculation Guide

Confidential — For Internal Operations and RevDau Engineering Teams Only. This document contains the raw telemetry measurements, per-module analysis, formula derivations, and empirical sizing data used to produce the customer-facing VM Sizing Guide (published in mcm-website at /docs/v1.1/vm-sizing/vm-sizing-guide).

1. Telemetry Methodology

1.1 Test Environment

All measurements were collected from a live MCM Platform deployment on the following host:

1.2 Collection Method

Telemetry was collected using the mcm-telemetry.sh script (/opt/mcm/scripts/mcm-telemetry.sh) which queries:

• docker stats --no-stream for container CPU/memory snapshots.
• /proc/1/status inside each Java container for VmRSS (resident set size).
• mongosh aggregate queries using $bsonSize on $$ROOT to calculate exact document sizes.
• Elasticsearch and Wazuh Indexer REST APIs for cluster health and index sizes.
• du -sh on Steampipe cache directories inside the discovery container.
• docker system df -v and du -sh /var/lib/docker/volumes/* for volume measurements.

1.3 Test Runs

Two telemetry snapshots were taken:

2. Container Memory Analysis

2.1 Baseline RAM Consumption (All Containers)

The table below compares container memory usage between the baseline run (1 account idle) and the scaled run (3 accounts after discovery + module operations):

2.2 JVM Process Memory (VmRSS from /proc/1/status)

Key Finding — Discovery API Memory Scaling: The mcm-discovery-api container is the only service that exhibits significant memory growth under load. During active discovery of 3 cloud accounts (582 real resources), its VmRSS jumped from 830 MB to 2,254 MB (+1.4 GB). This is caused by Steampipe holding query result sets and resource metadata in JVM heap during bulk discovery sweeps.

Empirical RAM per Discovery Account: 1,424 MB growth / 2 new accounts = ~712 MB per actively polled account during discovery. Post-discovery the memory may settle after GC cycles.

2.3 Derived RAM Sizing Formula

Based on measurements:

Total RAM (GB) =
    Host OS & Docker Daemon ............... 2 GB
  + Core Platform APIs (mcm-api + keycloak
    + apisix + etcd + postgres) ........... 2.2 GB baseline
  + MongoDB (WiredTiger cache) ............ 2 GB minimum (scales to 50% of allocated RAM)
  + Elasticsearch ......................... 2 GB heap
  + Per Business Module ................... 0.7 GB baseline each
  + Discovery Module (active polling) ..... 0.8 GB baseline + 0.7 GB per concurrent account
  + Wazuh Stack (if SecOps enabled) ....... 3.5 GB (manager + indexer + dashboard)
  + Observability (if enabled) ............ 1.5 GB (fleet-server + kibana + filebeat)

Example — Standard (5 accounts, all business modules, no SecOps):

= 2 + 2.2 + 2 + 2 + (4 x 0.7) + (0.8 + 5 x 0.7)
= 2 + 2.2 + 2 + 2 + 2.8 + 4.3
= 15.3 GB → Recommend 16 GB minimum, 32 GB comfortable

3. MongoDB Storage Analysis — Per Module

3.1 Database-Level Summary

Key Finding — WiredTiger Compression: At higher data volumes, WiredTiger's block compression becomes dramatically more effective. The compression ratio improved from 1.08x (266 resources) to 4.87x (15,854 resources). This means per-resource on-disk cost drops significantly at scale.

3.2 Per-Collection Footprint (from Tenant Breakdown)

Discovery Module (cloud_resources_v2)

Breakdown by Resource Type (from Run 2):

Important Note — SYSTEM Catalog Resources: The discovery module stores global catalog data (e.g., all AWS instance types and AMIs per region) under a SYSTEM tenantId with accountId: GLOBAL. These are not customer resources but reference data. In this deployment, system catalog constitutes 15,272 out of 15,854 total resources (96.3%). Only 582 resources are actual customer-discovered infrastructure.

Per-Account Discovery Data (real resources only):

Derived: Average real resource BSON size = ~1.5 KB/resource

Governance Module

Derived: Governance baseline = ~265 KB per tenant (fixed seed) + ~0.72 KB per custom policy added

Orchestration Module

Derived: Orchestration baseline = ~48.5 KB per tenant + ~1.3 KB per additional deployed stack

Orchestration Docker Volume: The mcm_orchestration_data volume grew from 83.5 KB (Run 1) to 167 KB (Run 2) as new stack executions cached OpenTofu providers. At scale with many IaC deployments, expect this volume to grow to 50-200 MB.

FinOps Module

Note: FinOps stores billing data primarily in Elasticsearch indices (cloud-cost-index, subscription-cost-index), not in MongoDB. The MongoDB collections store only plan metadata and recommendations. Empirical FinOps sizing requires ingesting actual CUR/billing data, which was not performed in this test run.

Estimated FinOps Growth (from prior engineering analysis):

• Small account (< 100 resources, < $10K/month spend): ~50 MB/year in Elasticsearch indices.
• Large account (1,000+ resources, > $100K/month spend): ~500 MB to 2 GB/year.
• CUR Parsing Buffer: Temporary disk spike of 1-5 GB during large CSV file ingestion.

SecOps Module (Wazuh)

Key Finding — Wazuh Queue is the Largest Storage Consumer: Even with no external agents enrolled, the mcm_wazuh_queue volume consumed 11.78 GB within 7 hours of operation. This is the Wazuh Manager's internal event processing queue. With real agent telemetry, this will grow significantly faster.

Derived SecOps Disk Formula:

SecOps Disk (GB) = Wazuh Queue Baseline (12 GB) + (Enrolled Agents * 3 GB/month * Retention Months) + Wazuh Indexer Indices

Observability Module

Key Finding — Filebeat Index Growth: The filebeat-8.19.3 index grew from 30.2 MB (Run 1, 17,322 docs) to 33.5 MB (Run 2, 20,549 docs). Growth rate: ~3.3 MB for 3,227 new log entries = ~1 KB per log entry.

Core Platform (mcm-api)

MSP Tenant Footprint:

4. Steampipe Cache Analysis

Breakdown:

• Static Baseline: ~818 MB consumed immediately upon container initialization. This includes:
  • Steampipe internal PostgreSQL database and WAL files.
  • AWS plugin binary and schema definitions.
  • FDW (Foreign Data Wrapper) shared objects.
• Per-Account Incremental: ~3 MB per onboarded AWS account (connection config, cached credentials, query result caches).
• Note: The Ubuntu account does not use Steampipe (it is not a cloud provider), so its incremental footprint is negligible.

Derived: Steampipe Cache = 820 MB baseline + ~3-10 MB per cloud account

5. Elasticsearch & Wazuh Indexer Analysis

5.1 Elasticsearch (Observability + FinOps)

ES Cluster Status: yellow — 60 active shards, 17 unassigned replica shards (expected on single-node). No data loss risk.

ES JVM Heap:

• Heap Used: 211 MB (19% of 1 GB max) at Run 2.
• Non-Heap: 264 MB (Metaspace + code cache).

5.2 Wazuh Indexer (OpenSearch)

Cluster Status: green — 21 shards, all assigned, 100% healthy.

6. Docker Volume Sizing Analysis

Key Finding: The mcm_wazuh_queue volume alone accounts for ~95% of all Docker volume storage. Even without external agent enrollment, the Wazuh Manager generates a significant event queue from self-monitoring.

7. Docker Image Sizes

Total Docker image storage: 27.49 GB

Derived: Base Docker image storage = ~28 GB (required before any data volumes)

8. Host Disk Utilization Summary

9. Final Derived Formulas

9.1 Host Disk Formula (Refined)

Total Disk (GB) =
    OS + Docker Engine + Images ................ 45 GB (fixed baseline)
  + Steampipe Baseline Cache ................... 0.8 GB (fixed)
  + Steampipe per Account ...................... Accounts * 0.01 GB
  + MongoDB Data ............................... Resources * 0.005 MB (5 KB/resource compressed)
  + MongoDB System Catalog ..................... 0.04 GB (fixed one-time per AWS region)
  + Elasticsearch (Observability) .............. 0.05 GB baseline + 0.001 GB/day log growth
  + Orchestration State ........................ 0.2 GB (grows with deployed stacks)
  + FinOps Cost Indices ........................ Accounts * 0.5 GB/year (ES indices)
  + FinOps Parsing Buffer ...................... 5 GB (temporary during ingestion)
  + Wazuh Queue ................................ 12 GB baseline + Agents * 3 GB/month
  + Wazuh Indexer Indices ...................... Agents * 3 GB/month * Retention Months
  + Keycloak DB ................................ 0.07 GB (stable)
  + etcd ....................................... 0.15 GB (stable)
  + Buffer/Headroom ............................ 20% of total

9.2 Scaling Examples

Example A: Small Deployment (5 AWS Accounts, 2,500 Resources, No SecOps)

Disk = 45 + 0.8 + (5 * 0.01) + (2500 * 0.005 / 1024) + 0.04 + 0.05 + 0.2 + (5 * 0.5) + 5 + 0.07 + 0.15
     = 45 + 0.8 + 0.05 + 0.012 + 0.04 + 0.05 + 0.2 + 2.5 + 5 + 0.07 + 0.15
     = 53.9 GB * 1.2 (headroom)
     ≈ 65 GB → Recommend 100 GB

Example B: Medium Deployment (20 AWS Accounts, 10,000 Resources, SecOps with 5 Agents, 3 Months Retention)

Disk = 45 + 0.8 + (20 * 0.01) + (10000 * 0.005 / 1024) + 0.04 + 0.05 + 0.2 + (20 * 0.5) + 5 + 12 + (5 * 3 * 3) + 0.07 + 0.15
     = 45 + 0.8 + 0.2 + 0.05 + 0.04 + 0.05 + 0.2 + 10 + 5 + 12 + 45 + 0.07 + 0.15
     = 118.6 GB * 1.2
     ≈ 142 GB → Recommend 200 GB

Example C: Large Enterprise (50 Accounts, 50,000 Resources, SecOps with 20 Agents, 6 Months Retention)

Disk = 45 + 0.8 + (50 * 0.01) + (50000 * 0.005 / 1024) + 0.04 + 0.05 + 0.2 + (50 * 0.5) + 5 + 12 + (20 * 3 * 6) + 0.07 + 0.15
     = 45 + 0.8 + 0.5 + 0.24 + 0.04 + 0.05 + 0.2 + 25 + 5 + 12 + 360 + 0.07 + 0.15
     = 449 GB * 1.2
     ≈ 539 GB → Recommend 500 GB+ (SSD recommended for Wazuh I/O)

9.3 Host RAM Formula (Refined)

Total RAM (GB) =
    Host OS Overhead ........................... 2 GB
  + Core APIs (mcm-api + keycloak + apisix
    + etcd + postgres) ......................... 2.2 GB
  + MongoDB WiredTiger Cache ................... 2 GB minimum (auto-scales to 50% of available)
  + Elasticsearch JVM Heap ..................... 2 GB (1 GB configured + overhead)
  + Per Business Module (Gov, FinOps, Orch) .... 0.7 GB each
  + Discovery Module ........................... 0.8 GB idle + 0.7 GB per actively polled account
  + Wazuh Stack (if enabled) ................... 3.5 GB (manager 0.8 + indexer 1.5 + dashboard 1.0 + secops-api 0.8)
  + Observability (if enabled) ................. 1.7 GB (fleet 0.4 + kibana 0.7 + filebeat 0.1 + obs-api 0.6)

9.4 Host vCPU Formula (Refined)

Total vCPUs =
    Host OS & Docker ........................... 1 vCPU
  + Core APIs .................................. 1.5 vCPUs
  + Databases (MongoDB + ES) ................... 2 vCPUs
  + Per Business Module ........................ 0.25 vCPUs each
  + Discovery (active polling) ................. 0.5 + CEIL(concurrent_accounts / 2) vCPUs
  + Wazuh Stack (if enabled) ................... 2 + CEIL(EPS / 500) vCPUs
  + Observability (if enabled) ................. 1 vCPU

10. Tenant Lookup Issue — Engineering Note

Problem: "Unknown Tenant" in Telemetry Reports

The telemetry script's "Accounts Grouped by Tenant" and "In-Depth Disk Space Consumption per Tenant" sections show Unknown Tenant for all tenants. This occurs because:

1. The accounts collection stores tenantId as a plain string (e.g., "6a391a1f44b04432c4ad7810").
2. The enterprise collection uses enterpriseId as the lookup key, which may differ from the _id field.
3. The $lookup join between accounts.tenantId and enterprise.enterpriseId fails silently when the field values don't match.

Root Cause: The enterpriseId field in the enterprise collection stores a different identifier format than what the accounts.tenantId references. This needs further investigation against the Java entity model (@Field annotations in mcm-api).

Impact: Cosmetic only — the tenant data sizes are still calculated correctly; only the display name resolution fails.