GNU bug report logs - #18178
help

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18178 in the body.
You can then email your comments to 18178 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jermaine Golding <Jermaine.Golding <at> caribbean.fujitsu.com>
To: "bug-coreutils <at> gnu.org" <bug-coreutils <at> gnu.org>
Subject: help 
Date: Sat, 2 Aug 2014 20:31:42 -0700

[Message part 1 (text/plain, inline)]

Good Day;

I just deleted over 100gigs of logs and my var partition is still at 100%.

Please advise I am using checkpoint gaia.

[Expert <at> Fwall:0]# df --h
df: option `--h' is ambiguous
Try `df --help' for more information.
[Expert <at> Fwall:0]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_splat-lv_current
                       40G  8.5G   29G  23% /
/dev/sda1             145M   31M  107M  23% /boot
/dev/mapper/vg_splat-lv_log
                      768G  743G     0 100% /var/log
[Expert <at> Fwall:0]#

Regards

________________________________
Jermaine Golding
Systems Engineer
Core Services

Fujitsu Caribbean
18 Belmont Road
Kingston, Jamaica

Phone:    (876) 926-7567 extn 6293
Direct:     (876) 935-6293
Mobile:    (876) 486-2974
E-mail:    jermaine.golding <at> caribbean.fujitsu.com<mailto:jermaine.golding <at> caribbean.fujitsu.com>
Website: http://www.fujitsu.com/caribbean

[Message part 2 (text/html, inline)]

Message #8 received at 18178 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jermaine Golding <Jermaine.Golding <at> caribbean.fujitsu.com>
Cc: 18178 <at> debbugs.gnu.org
Subject: Re: bug#18178: help
Date: Sun, 03 Aug 2014 05:35:20 +0100

tag 18178 notabug
close 18178
stop

On 08/03/2014 04:31 AM, Jermaine Golding wrote:
> Good Day;
> 
> I just deleted over 100gigs of logs and my var partition is still at 100%.
> 
> Please advise I am using checkpoint gaia.
> 
> [Expert <at> Fwall:0]# df --h
> df: option `--h' is ambiguous
> Try `df --help' for more information.
> [Expert <at> Fwall:0]# df -h
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/mapper/vg_splat-lv_current
>                        40G  8.5G   29G  23% /
> /dev/sda1             145M   31M  107M  23% /boot
> /dev/mapper/vg_splat-lv_log
>                       768G  743G     0 100% /var/log
> [Expert <at> Fwall:0]#

If you delete a file which a process still has open
the blocks are not freed until the process closes the file.
This is what allows one to update a file which is already open.
I'll leave it to you to identify which services to restart etc.

Pádraig.

Message #15 received at 18178 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>, 
 Jermaine Golding <Jermaine.Golding <at> caribbean.fujitsu.com>
Cc: 18178 <at> debbugs.gnu.org
Subject: Re: bug#18178: help
Date: Sun, 03 Aug 2014 08:14:36 +0200

On 08/03/2014 06:35 AM, Pádraig Brady wrote:
> On 08/03/2014 04:31 AM, Jermaine Golding wrote:
>> Good Day;
>>
>> I just deleted over 100gigs of logs and my var partition is still at 100%.
>>
>> Please advise I am using checkpoint gaia.
>>
>> [Expert <at> Fwall:0]# df --h
>> df: option `--h' is ambiguous
>> Try `df --help' for more information.
>> [Expert <at> Fwall:0]# df -h
>> Filesystem            Size  Used Avail Use% Mounted on
>> /dev/mapper/vg_splat-lv_current
>>                        40G  8.5G   29G  23% /
>> /dev/sda1             145M   31M  107M  23% /boot
>> /dev/mapper/vg_splat-lv_log
>>                       768G  743G     0 100% /var/log
>> [Expert <at> Fwall:0]#
> 
> If you delete a file which a process still has open
> the blocks are not freed until the process closes the file.
> This is what allows one to update a file which is already open.
> I'll leave it to you to identify which services to restart etc.

Additionally to what Padraig said, if the files had 2 or more
hardlinks, then it's possible that you only deleted one hardlink
reference to the file.

Have a nice day,
Berny

Message #18 received at 18178 <at> debbugs.gnu.org (full text, mbox):

From: Jermaine Golding <Jermaine.Golding <at> caribbean.fujitsu.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Pádraig Brady <P <at> draigBrady.com>
Cc: "18178 <at> debbugs.gnu.org" <18178 <at> debbugs.gnu.org>
Subject: RE: bug#18178: help
Date: Sat, 2 Aug 2014 23:49:16 -0700

[Message part 1 (text/plain, inline)]

I also rebooted a few times

Sent from my Windows Phone
________________________________
From: Bernhard Voelker<mailto:mail <at> bernhard-voelker.de>
Sent: ‎8/‎3/‎2014 1:14 AM
To: Pádraig Brady<mailto:P <at> draigBrady.com>; Jermaine Golding<mailto:Jermaine.Golding <at> caribbean.fujitsu.com>
Cc: 18178 <at> debbugs.gnu.org<mailto:18178 <at> debbugs.gnu.org>
Subject: Re: bug#18178: help

On 08/03/2014 06:35 AM, Pádraig Brady wrote:
> On 08/03/2014 04:31 AM, Jermaine Golding wrote:
>> Good Day;
>>
>> I just deleted over 100gigs of logs and my var partition is still at 100%.
>>
>> Please advise I am using checkpoint gaia.
>>
>> [Expert <at> Fwall:0]# df --h
>> df: option `--h' is ambiguous
>> Try `df --help' for more information.
>> [Expert <at> Fwall:0]# df -h
>> Filesystem            Size  Used Avail Use% Mounted on
>> /dev/mapper/vg_splat-lv_current
>>                        40G  8.5G   29G  23% /
>> /dev/sda1             145M   31M  107M  23% /boot
>> /dev/mapper/vg_splat-lv_log
>>                       768G  743G     0 100% /var/log
>> [Expert <at> Fwall:0]#
>
> If you delete a file which a process still has open
> the blocks are not freed until the process closes the file.
> This is what allows one to update a file which is already open.
> I'll leave it to you to identify which services to restart etc.

Additionally to what Padraig said, if the files had 2 or more
hardlinks, then it's possible that you only deleted one hardlink
reference to the file.

Have a nice day,
Berny

[Message part 2 (text/html, inline)]

Message #21 received at 18178 <at> debbugs.gnu.org (full text, mbox):

From: Jermaine Golding <Jermaine.Golding <at> caribbean.fujitsu.com>
To: Jermaine Golding <Jermaine.Golding <at> caribbean.fujitsu.com>, "Bernhard
 Voelker" <mail <at> bernhard-voelker.de>, Pádraig Brady
 <P <at> draigBrady.com>
Cc: "18178 <at> debbugs.gnu.org" <18178 <at> debbugs.gnu.org>
Subject: RE: bug#18178: help
Date: Sat, 2 Aug 2014 23:52:53 -0700

[Message part 1 (text/plain, inline)]

That still did not resolve the issue

Sent from my Windows Phone
________________________________
From: Jermaine Golding<mailto:Jermaine.Golding <at> caribbean.fujitsu.com>
Sent: ‎8/‎3/‎2014 1:49 AM
To: Bernhard Voelker<mailto:mail <at> bernhard-voelker.de>; Pádraig Brady<mailto:P <at> draigBrady.com>
Cc: 18178 <at> debbugs.gnu.org<mailto:18178 <at> debbugs.gnu.org>
Subject: RE: bug#18178: help

I also rebooted a few times

Sent from my Windows Phone
________________________________
From: Bernhard Voelker<mailto:mail <at> bernhard-voelker.de>
Sent: ‎8/‎3/‎2014 1:14 AM
To: Pádraig Brady<mailto:P <at> draigBrady.com>; Jermaine Golding<mailto:Jermaine.Golding <at> caribbean.fujitsu.com>
Cc: 18178 <at> debbugs.gnu.org<mailto:18178 <at> debbugs.gnu.org>
Subject: Re: bug#18178: help

On 08/03/2014 06:35 AM, Pádraig Brady wrote:
> On 08/03/2014 04:31 AM, Jermaine Golding wrote:
>> Good Day;
>>
>> I just deleted over 100gigs of logs and my var partition is still at 100%.
>>
>> Please advise I am using checkpoint gaia.
>>
>> [Expert <at> Fwall:0]# df --h
>> df: option `--h' is ambiguous
>> Try `df --help' for more information.
>> [Expert <at> Fwall:0]# df -h
>> Filesystem            Size  Used Avail Use% Mounted on
>> /dev/mapper/vg_splat-lv_current
>>                        40G  8.5G   29G  23% /
>> /dev/sda1             145M   31M  107M  23% /boot
>> /dev/mapper/vg_splat-lv_log
>>                       768G  743G     0 100% /var/log
>> [Expert <at> Fwall:0]#
>
> If you delete a file which a process still has open
> the blocks are not freed until the process closes the file.
> This is what allows one to update a file which is already open.
> I'll leave it to you to identify which services to restart etc.

Additionally to what Padraig said, if the files had 2 or more
hardlinks, then it's possible that you only deleted one hardlink
reference to the file.

Have a nice day,
Berny

[Message part 2 (text/html, inline)]

Message #24 received at 18178 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Jermaine Golding <Jermaine.Golding <at> caribbean.fujitsu.com>
Cc: 18178 <at> debbugs.gnu.org
Subject: Re: bug#18178: help
Date: Sun, 3 Aug 2014 15:31:50 -0600

Jermaine Golding wrote:
> >> I just deleted over 100gigs of logs and my var partition is still at 100%.
> >> /dev/mapper/vg_splat-lv_log 768G  743G     0 100% /var/log
>
> I also rebooted a few times
>
> That still did not resolve the issue

If the file system is full then look for the files on disk.  I like
using the old 'xdu' command to visualize this.

  du -lmx /var | xdu

I like to do this in two steps.

  du -lmx /var >/tmp/du-lmx.var.out
  xdu /tmp/du-lmx.var.out

However I would also just start looking manually.

  du -lmsx /var/* | sort -n
  du -lmsx /var/cache/* | sort -n
  du -lmsx /var/local/* | sort -n

Somewhere you will find where that disk space is consuming your disk.
It is there.  Just keep looking.

Note that as others mentioned the file system is reference counted.
If you remove a file that only removes a link to it.  The space is not
freed until the last link is removed.  To force the file system to
free the space immediately it is typical to truncate the file
instead.  That is what I do before removing the file.

  $ ls -ldog somelargefile
  $ > somelargefile

The shell's ">" redirection to that file will truncate the file to
zero size.

Also if you have any mount points on top of /var then those mount
points would shadow and hide anything that was mounted below it.  Do
you have any mount points in /var?  If so then unmount those and look
underneath the mount point for hidden disk space.

  mount | grep /var

The bug tracker is not a good support forum.  I suggest using the
help-gnu-utils <at> gnu.org mailing list for such discussion in the
future.  Or depending upon your software distribution the user support
mailing list associated with it might be a good source of information.

Bob

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.