# Monitor Storage Utilization (/learn/events-store/how-to/monitor-storage)



KubeMQ includes a built-in storage utilization monitor that protects against disk exhaustion. When disk usage exceeds configurable thresholds, KubeMQ adjusts its behavior — from logging warnings to blocking all publish operations.

## Utilization Thresholds [#utilization-thresholds]

| Utilization | Level    | Polling Interval | Log Level | Publishing  |
| ----------- | -------- | ---------------- | --------- | ----------- |
| 0–80%       | Normal   | 5 seconds        | —         | Allowed     |
| 80–90%      | Warning  | 3 seconds        | WARN      | Allowed     |
| 90–95%      | Critical | 2 seconds        | ERROR     | Allowed     |
| Above 95%   | Disabled | 1 second         | ERROR     | **Blocked** |

<Mermaid
  chart="stateDiagram-v2
    [*] --> Normal
    Normal --> Warning: utilization > 80%
    Warning --> Normal: utilization ≤ 80%
    Warning --> Critical: utilization > 90%
    Critical --> Warning: utilization ≤ 90%
    Critical --> Disabled: utilization > 95%
    Disabled --> Critical: utilization ≤ 95%

    Normal: 0-80% | Publish OK
    Warning: 80-90% | Publish OK + Warn
    Critical: 90-95% | Publish OK + Error
    Disabled: >95% | Publish BLOCKED"
/>

*The storage monitor transitions through escalating levels as disk fills; publishing is blocked only above 95% and resumes automatically once utilization recovers.*

## What Happens at Each Level [#what-happens-at-each-level]

### Normal (0–80%) [#normal-080]

Everything operates normally. The monitor checks disk utilization every 5 seconds.

### Warning (80–90%) [#warning-8090]

Publishing continues but KubeMQ logs warnings:

```text
[WARN] storage utilization at 83.2% - consider increasing disk space or adjusting retention
```

The polling interval decreases to 3 seconds for faster detection.

### Critical (90–95%) [#critical-9095]

Publishing still operates but KubeMQ logs errors:

```text
[ERROR] storage utilization at 92.1% - approaching disabled threshold
```

The polling interval decreases to 2 seconds.

### Disabled (Above 95%) [#disabled-above-95]

All Events Store and Queue publish operations are **blocked**. Clients receive an error:

```text
storage has reached to 96.5% utilization and is not allowed
```

Publishing automatically resumes when utilization drops below 95%.

## Recovery Steps [#recovery-steps]

<Steps>
  <Step>
    ### Identify the Issue [#identify-the-issue]

    Check server logs for utilization warnings:

    ```bash
    docker logs kubemq | grep "storage utilization"
    ```
  </Step>

  <Step>
    ### Reduce Utilization [#reduce-utilization]

    Use one or more of these approaches:

    **Option 1: Reduce retention** — Lower the `Store.MaxRetention` value to purge older events faster.

    <RunKubeMQ env="{ STORE_MAX_RETENTION: '60' }" />

    **Option 2: Add disk space** — Increase the volume size for the store path.

    **Option 3: Limit channel size** — Set `Store.MaxQueueSize` to cap each channel's disk usage.

    <RunKubeMQ env="{ STORE_MAX_QUEUE_SIZE: '1073741824' }" />
  </Step>

  <Step>
    ### Verify Recovery [#verify-recovery]

    Once utilization drops below 95%, publishing resumes automatically. Verify in the logs:

    ```text
    [INFO] storage utilization recovered to 89.3% - publishing re-enabled
    ```
  </Step>
</Steps>

## Monitoring Best Practices [#monitoring-best-practices]

| Practice             | Recommendation                                                                 |
| -------------------- | ------------------------------------------------------------------------------ |
| Set alerts           | Monitor for `WARNING` and `CRITICAL` log entries                               |
| Right-size retention | Match retention to your replay window requirements                             |
| Use volume mounts    | Always mount persistent volumes in production                                  |
| Plan capacity        | Calculate expected event rate × retention window × average event size          |
| Test thresholds      | Verify your retention policies keep utilization well below 80% under peak load |

<Callout type="info">
  The utilization monitor checks the filesystem where the store path is located. If you use a separate volume for the store, only that volume's usage counts toward the thresholds.
</Callout>

## Capacity Planning Example [#capacity-planning-example]

For an order processing system:

* **Event rate:** 100 events/second
* **Average event size:** 1 KB
* **Retention window:** 7 days

```text
Storage needed = 100 events/s × 1 KB × 86,400 s/day × 7 days
               = 60.48 GB

With 80% threshold target:
Disk size needed = 60.48 GB / 0.80 = 75.6 GB → provision 100 GB
```

## Related [#related]

* [Configure Retention](/learn/events-store/how-to/configure-retention) for retention policy setup
* [Events Store Reference](/learn/events-store/reference) for file store configuration