KubeMQ
LearnEvents StoreHow-To Guides

Monitor Storage Utilization

Monitor disk usage and understand storage utilization thresholds that affect publishing.

KubeMQ includes a built-in storage utilization monitor that protects against disk exhaustion. When disk usage exceeds configurable thresholds, KubeMQ adjusts its behavior — from logging warnings to blocking all publish operations.

Utilization Thresholds

UtilizationLevelPolling IntervalLog LevelPublishing
0–80%Normal5 secondsAllowed
80–90%Warning3 secondsWARNAllowed
90–95%Critical2 secondsERRORAllowed
Above 95%Disabled1 secondERRORBlocked

The storage monitor transitions through escalating levels as disk fills; publishing is blocked only above 95% and resumes automatically once utilization recovers.

What Happens at Each Level

Normal (0–80%)

Everything operates normally. The monitor checks disk utilization every 5 seconds.

Warning (80–90%)

Publishing continues but KubeMQ logs warnings:

[WARN] storage utilization at 83.2% - consider increasing disk space or adjusting retention

The polling interval decreases to 3 seconds for faster detection.

Critical (90–95%)

Publishing still operates but KubeMQ logs errors:

[ERROR] storage utilization at 92.1% - approaching disabled threshold

The polling interval decreases to 2 seconds.

Disabled (Above 95%)

All Events Store and Queue publish operations are blocked. Clients receive an error:

storage has reached to 96.5% utilization and is not allowed

Publishing automatically resumes when utilization drops below 95%.

Recovery Steps

Identify the Issue

Check server logs for utilization warnings:

docker logs kubemq | grep "storage utilization"

Reduce Utilization

Use one or more of these approaches:

Option 1: Reduce retention — Lower the Store.MaxRetention value to purge older events faster.

docker run -d \  --name kubemq \  -p 50000:50000 \  -p 9090:9090 \  -p 8080:8080 \  -e KUBEMQ_TOKEN=YOUR_LICENSE_KEY \  -e STORE_MAX_RETENTION=60 \  europe-docker.pkg.dev/kubemq/images/kubemq:next

Option 2: Add disk space — Increase the volume size for the store path.

Option 3: Limit channel size — Set Store.MaxQueueSize to cap each channel's disk usage.

docker run -d \  --name kubemq \  -p 50000:50000 \  -p 9090:9090 \  -p 8080:8080 \  -e KUBEMQ_TOKEN=YOUR_LICENSE_KEY \  -e STORE_MAX_QUEUE_SIZE=1073741824 \  europe-docker.pkg.dev/kubemq/images/kubemq:next

Verify Recovery

Once utilization drops below 95%, publishing resumes automatically. Verify in the logs:

[INFO] storage utilization recovered to 89.3% - publishing re-enabled

Monitoring Best Practices

PracticeRecommendation
Set alertsMonitor for WARNING and CRITICAL log entries
Right-size retentionMatch retention to your replay window requirements
Use volume mountsAlways mount persistent volumes in production
Plan capacityCalculate expected event rate × retention window × average event size
Test thresholdsVerify your retention policies keep utilization well below 80% under peak load

The utilization monitor checks the filesystem where the store path is located. If you use a separate volume for the store, only that volume's usage counts toward the thresholds.

Capacity Planning Example

For an order processing system:

  • Event rate: 100 events/second
  • Average event size: 1 KB
  • Retention window: 7 days
Storage needed = 100 events/s × 1 KB × 86,400 s/day × 7 days
               = 60.48 GB

With 80% threshold target:
Disk size needed = 60.48 GB / 0.80 = 75.6 GB → provision 100 GB

Was this page helpful?

On this page