High Disk Usage in `/app-storage/global` — How to Troubleshoot and Resolve?

MulongDeng

Hi all,

I'm seeing high disk usage on `/app-storage/global` ( RVC4 standalone app ) and would appreciate some guidance on how to track down the root cause.

**My questions:**

1. **What are the common causes?**

2. **What are the recommended troubleshooting steps?**

Are there specific commands or tools the community uses to identify what's consuming space?

3. **What are typical fixes or preventive measures?**

Thanks in advance for any pointers!

---

OskarSonc

Hey MulongDeng - a useful first thing to note is that /app-storage/global is the app-visible mount for global persistent storage on RVC4, but it shares the same underlying device userdata partition with other oak-agent / overlay data. So if it shows 100% usage, it may be your app’s data, but it can also be stale app/container artifacts or downloads on the device.

Recommended troubleshooting:

On the device / inside the app shell:
df -h /app-storage/global /overlay /data              # Check usage of app and shared storage
du -xhd1 /app-storage/global 2>/dev/null | sort -h    # Find large directories in global app storage
du -xhd1 /overlay/oak-agent 2>/dev/null | sort -h     # Find large oak-agent directories

From the host with oakctl:
oakctl app list                                       # List installed apps
oakctl app prune-storage                              # Clean stale app files

Common causes are app data such as recordings, snapshots, logs, or caches, but also stale oak-agent downloads, packages, old container layers, or unused installed apps. Typical fixes are deleting unused apps with oakctl app delete <app-id>, running oakctl app prune-storage, adding retention/cleanup for app-created files, and avoiding global storage for large app-specific data unless it really needs to be shared. More on this here.

Thanks,
Oskar

MulongDeng

OskarSonc reten

My app's data doesn't seem too big; the log file retention was set.

MulongDeng

Deleting the active application did not reduce the disk utilization, which remained at 100%.

OskarSonc

This is helpful. It suggests /app-storage/global is not the main issue. In the container it shows only 62% usage, and du -xhd1 /app-storage/global is basically empty. So the disk usage is more likely coming from device-side overlay / oak-agent storage rather than your app’s own persistent data.

Could you check this on the device OS shell (not inside the app container) and share the output?

du -xhd1 /overlay/oak-agent 2>/dev/null | sort -h
du -xhd1 /overlay/oak-agent/downloads 2>/dev/null | sort -h
du -xhd1 /overlay/oak-agent/containers 2>/dev/null | sort -h

MulongDeng

Just to clarify—the 62% usage reading comes from a different device. Above is the device's outbut.

The two devices that were at 100% usage have already been reset by holding the factory reset button and are now back to normal. And they're now at only 4% usage.

OskarSonc

MulongDeng - I don't understand what you mean? So you already fixed the problem?

As it seems, the reset appears to have fixed the symptom, but we still do not know what originally filled the storage. If it happens again, the most useful thing would be to capture df -h, du -xhd1 /overlay/oak-agent, and oakctl app list before resetting.

MulongDeng

OskarSonc

Not fully resolved yet — currently just a temporary fix using the reset button. I'll observe for another day or two to see if it recurs.

The screenshot I just shared was from a device showing 62% usage, but the actual space consumed doesn't seem to add up to 62%. Below is the latest command output:

OskarSonc

MulongDeng Thanks, this helps.

The 62% usage is on the shared userdata partition (/dev/sda11), not mainly in oak-agent or app storage. In your output, /overlay/oak-agent is only about 3.8G, so it does not explain the full 46.2G used on the device.

So the remaining space is likely being used elsewhere on the same partition, such as /data, /home, /cache, or another overlay-backed path. Also, the repeated 62% lines in df -h are expected here, they are different mountpoints backed by the same underlying partition, not separate usage totals.

Which Luxonis OS version do you have?
To narrow down where the space is going, the next useful checks would be:

du -xhd1 /data 2>/dev/null | sort -h
du -xhd1 /home 2>/dev/null | sort -h
du -xhd1 /cache 2>/dev/null | sort -h
du -xhd1 /overlay 2>/dev/null | sort -h

MulongDeng

OskarSonc

From the output, it appears that /overlay/persistent-logs is the primary cause of the high disk usage.

Below is the latest command output:

MulongDeng

DepthAI: v3.5.0
Luxonis OS: 1.30.1
oakctl: 0.18.6

And is there a way to prevent this file from growing indefinitely?

MulongDeng

I tried deleting /overlay/persistent-logs/logcat.log and rebooting. The storage usage appeared to decrease afterward. Upon rechecking /overlay/persistent-logs/logcat.log, I noticed it was continuously flooding with the following error:

04-29 01:22:40.329 11153 11153 I depthai-device-rvc4: /local/mnt/workspace/lnxbuild/project/trees_in_use/free_tree_dir/checkout/build-qti-distro-rb-debug/tmp-glibc/work/kalama-oe-linux/adsprpc/13.0-r1/adsprpc/src/fastrpc_config.c:335: fastrpc_config_init: Couldn't find file depthai-device-rvc4.debugconfig, errno (No such file or directory) at /opt/qcom/aistack/qairt/2.41.0.251128/lib/hexagon-v73/unsigned/, /usr/lib, /dsp/cdsp, /dsp, /usr/lib/rfsa/adsp, /dsp/cdsp, /dsp,

OskarSonc

MulongDeng - thanks for digging into this. We checked internally and found this is Luxonis OS bug affecting log rotation on long-running systems, not expected app behavior.

It is planned to be fixed in Luxonis OS v1.31.0, which should be out in a few days.

In the meantime, you can use this workaround to clear the accumulated logs:

rm -f /overlay/persistent-logs/*
rm -f /var/volatile/log/*

Thanks for reporting, and sorry for any inconvenience it caused.
Oskar