Today we saw a strange problem on our Solaris hosts that NFS mount VxFS filestore from our Veritas cluster. The users were seeing “Disc quota exceeded” messages, whilst the quota command wasn’t showing they’d hit their limit. After some digging on the cluster node we found the following error message:
Sep 12 11:04:33 bes vxfs: [ID 702911 kern.warning] WARNING: msgcnt 10 mesg 089: V-2-89: quotas on /cluster/ResFS invalid; disk usage for group id 2805 exceeds 2147483646 sectors
Ah-ha! Group quota! We hadn’t even set group quotas, but it appears the system tracks the usage anyway when you mount with -o quota. Some googling turned up the following document:
http://seer.entsupport.symantec.com/docs/277535.htm
So it turns out there’s a 1TB maximum limit when using quotas. Since we weren’t using group quotas the simple option was to disable them:
vxquotaoff -gv /cluster/ResFS
Then I edited the Mount resource and changed the quota mount option to usrquota.
This only alleviates the problem for a while. Eventually someone will need to use 1TB of storage for themselves, but hopefully that’s a little way off yet. Maybe we’ll be using ZFS by then anyway 🙂
