Monitor Storage Utilization
Monitor disk usage and understand storage utilization thresholds that affect publishing.
KubeMQ includes a built-in storage utilization monitor that protects against disk exhaustion. When disk usage exceeds configurable thresholds, KubeMQ adjusts its behavior — from logging warnings to blocking all publish operations.
Utilization Thresholds
| Utilization | Level | Polling Interval | Log Level | Publishing |
|---|---|---|---|---|
| 0–80% | Normal | 5 seconds | — | Allowed |
| 80–90% | Warning | 3 seconds | WARN | Allowed |
| 90–95% | Critical | 2 seconds | ERROR | Allowed |
| Above 95% | Disabled | 1 second | ERROR | Blocked |
The storage monitor transitions through escalating levels as disk fills; publishing is blocked only above 95% and resumes automatically once utilization recovers.
What Happens at Each Level
Normal (0–80%)
Everything operates normally. The monitor checks disk utilization every 5 seconds.
Warning (80–90%)
Publishing continues but KubeMQ logs warnings:
[WARN] storage utilization at 83.2% - consider increasing disk space or adjusting retentionThe polling interval decreases to 3 seconds for faster detection.
Critical (90–95%)
Publishing still operates but KubeMQ logs errors:
[ERROR] storage utilization at 92.1% - approaching disabled thresholdThe polling interval decreases to 2 seconds.
Disabled (Above 95%)
All Events Store and Queue publish operations are blocked. Clients receive an error:
storage has reached to 96.5% utilization and is not allowedPublishing automatically resumes when utilization drops below 95%.
Recovery Steps
Identify the Issue
Check server logs for utilization warnings:
docker logs kubemq | grep "storage utilization"Reduce Utilization
Use one or more of these approaches:
Option 1: Reduce retention — Lower the Store.MaxRetention value to purge older events faster.
docker run -d \ --name kubemq \ -p 50000:50000 \ -p 9090:9090 \ -p 8080:8080 \ -e KUBEMQ_TOKEN=YOUR_LICENSE_KEY \ -e STORE_MAX_RETENTION=60 \ europe-docker.pkg.dev/kubemq/images/kubemq:nextOption 2: Add disk space — Increase the volume size for the store path.
Option 3: Limit channel size — Set Store.MaxQueueSize to cap each channel's disk usage.
docker run -d \ --name kubemq \ -p 50000:50000 \ -p 9090:9090 \ -p 8080:8080 \ -e KUBEMQ_TOKEN=YOUR_LICENSE_KEY \ -e STORE_MAX_QUEUE_SIZE=1073741824 \ europe-docker.pkg.dev/kubemq/images/kubemq:nextVerify Recovery
Once utilization drops below 95%, publishing resumes automatically. Verify in the logs:
[INFO] storage utilization recovered to 89.3% - publishing re-enabledMonitoring Best Practices
| Practice | Recommendation |
|---|---|
| Set alerts | Monitor for WARNING and CRITICAL log entries |
| Right-size retention | Match retention to your replay window requirements |
| Use volume mounts | Always mount persistent volumes in production |
| Plan capacity | Calculate expected event rate × retention window × average event size |
| Test thresholds | Verify your retention policies keep utilization well below 80% under peak load |
The utilization monitor checks the filesystem where the store path is located. If you use a separate volume for the store, only that volume's usage counts toward the thresholds.
Capacity Planning Example
For an order processing system:
- Event rate: 100 events/second
- Average event size: 1 KB
- Retention window: 7 days
Storage needed = 100 events/s × 1 KB × 86,400 s/day × 7 days
= 60.48 GB
With 80% threshold target:
Disk size needed = 60.48 GB / 0.80 = 75.6 GB → provision 100 GBRelated
- Configure Retention for retention policy setup
- Events Store Reference for file store configuration
Was this page helpful?