What are these large files and can I safely delete them?

Message boards : Questions and problems : What are these large files and can I safely delete them?
Message board moderation

To post messages, you must log in.

AuthorMessage
whore

Send message
Joined: 6 Mar 15
Posts: 18
United States
Message 60972 - Posted: 15 Mar 2015, 13:50:39 UTC

Since I started using BOINC, I've noticed some unusual hdd usage.

My hdd has a 500 gb capacity. It will soon fillup if this continues.

Here are my stats. This is a BOINC-only computer. It does nothing else so the /root should be a much smaller size:

- 64 bit Kubuntu 14.04
- A6-7400k
- 4 gb ddr3 system ram

I think I narrowed the large files to the following:


#1 - /var/log/syslog.1 (this is about 36 GiB)
#2 - /var/log/kern.log (this is about 126 GiB)
#3 - /proc/kcore (128 TiB)

Questions:

A) What are those 3 files?
B) May I safely delete them?
C) How can one file be 128 TiB if my entire hdd is only 500 GB?

If it helps, then here are the BOINC projects I work:

- RNA World
- The Lattice Project
- Citizen Science Grid
- Rosetta
- Mind Modeling
- Poem
- World Community Grid
- Einstien (uses GPU)

Thank you in advance for your time.
ID: 60972 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15483
Netherlands
Message 60973 - Posted: 15 Mar 2015, 14:10:41 UTC - in response to Message 60972.  
Last modified: 15 Mar 2015, 14:11:22 UTC

Since BOINC isn't an operating system, chances are high these are logs made by your OS.
I put it in a search engine and came up with http://ubuntuforums.org/showthread.php?t=1394909. Make sure you read all of it.
ID: 60973 · Report as offensive
whore

Send message
Joined: 6 Mar 15
Posts: 18
United States
Message 60983 - Posted: 15 Mar 2015, 22:00:16 UTC - in response to Message 60973.  

Good find. Thanks!
ID: 60983 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 801
Finland
Message 60984 - Posted: 15 Mar 2015, 22:09:47 UTC - in response to Message 60972.  

#1 - /var/log/syslog.1 (this is about 36 GiB)
#2 - /var/log/kern.log (this is about 126 GiB)


As the names say, these are log files. They may contain some useful information if you ever need to fix something in your system. The files you have are exceptionally large. Typical sizes are hundreds of kilobytes.

You can delete them but before you do that you may want to take a look at them and try to find out what program is spamming in them. Then make a bug report to Ubuntu.

#3 - /proc/kcore (128 TiB)


/proc is a virtual filesystem. None of the files there actually exist on a disk. Kernel creates the files and their contents on the fly. It looks like what exactly /proc/kcore is has changed over time. I think it is currently kernel's address space. Linux kernel (on x86-64) currently limits process' address space to 128 TB.


(Ignoring your other question since you already answered it yourself.)
ID: 60984 · Report as offensive
whore

Send message
Joined: 6 Mar 15
Posts: 18
United States
Message 60986 - Posted: 16 Mar 2015, 1:05:28 UTC - in response to Message 60984.  

I couldn't open the "log" files. It just froze and crashed. Of course, the smallest one was nearly 20 gb so I can understand why it froze and crashed.

Thus, I couldn't examine them. I did delete them though. So far, there is no ill effect.
ID: 60986 · Report as offensive
whore

Send message
Joined: 6 Mar 15
Posts: 18
United States
Message 60992 - Posted: 16 Mar 2015, 18:17:19 UTC

Update: I deleted the log files but now they are back. They are not yet as large as before, but they are rapidly growing.

How can I prevent the logs from growing so rapidly?
ID: 60992 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15483
Netherlands
Message 60993 - Posted: 16 Mar 2015, 18:34:17 UTC - in response to Message 60992.  
Last modified: 16 Mar 2015, 18:34:58 UTC

Which is why I said to read the whole thread I pointed to.
From that thread:
You can simply delete those log files. You can also stop the syslog daemon, so they won't come back. Logging everything is usually a good idea, but on a development system in a R&D environment, it has little to no value.

And also from that thread:
Seeing a "syslog.1" means your logs probably *did* get rotated, however log file sizes could be due to excessive errors or way verbose logging or unusual log rotation settings. For checking your log rotation see /etc/logrotate.conf and /etc/logrotate.d and /etc/cron.daily/sysklogd.

ID: 60993 · Report as offensive
whore

Send message
Joined: 6 Mar 15
Posts: 18
United States
Message 60994 - Posted: 16 Mar 2015, 20:11:34 UTC

My bad - I should have read that post more clearly. Having said that, I have no idea how to stop something like the syslog daemon. Is there a tutorial somewhere?

In the meantime, I will see what a google search reveals.

Thank you for the help.
ID: 60994 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 801
Finland
Message 60998 - Posted: 16 Mar 2015, 20:33:32 UTC - in response to Message 60986.  
Last modified: 16 Mar 2015, 20:34:06 UTC

I couldn't open the "log" files. It just froze and crashed. Of course, the smallest one was nearly 20 gb so I can understand why it froze and crashed.

You need to use some tool that doesn't try to read the file entirely into memory. Something like 'less' (command line program) should work.

You could stop the excessive logging by stopping syslog but that won't stop whichever misbehaving program from trying to spam the logs. I would recommend that you try to find out which program it is and get it fixed. (It takes CPU cycles to spam logs and I imagine you would like to give those cycles to BOINC.)

And of course, update the software in that box and reboot if you haven't done so already.
ID: 60998 · Report as offensive
whore

Send message
Joined: 6 Mar 15
Posts: 18
United States
Message 61005 - Posted: 17 Mar 2015, 2:37:21 UTC - in response to Message 60998.  

Forgive my newbie and noob ignorance: How could I determine which program is spamming it?
ID: 61005 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 801
Finland
Message 61038 - Posted: 17 Mar 2015, 19:44:33 UTC - in response to Message 61005.  

You take a look at the log file, identify repeating messages or blocks of messages (part of the message may vary), put the repeating part in Google and see what comes up.

Quite often the program name is included in the message and if you are lucky the name matches something you see in GUI.
ID: 61038 · Report as offensive

Message boards : Questions and problems : What are these large files and can I safely delete them?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.