Saturday, June 21, 2014

How to Enable/Generate/Debug Core Dump In Linux?

Today while browsing i came across to something interesting so thought to share with you all.

Thanks to Gordon.

What is Core Dump?

If the running programs were terminated abnormal or crashed, the memory status of the program will be recorded by OS and saved in a file, this file is called “Core Dump” . We also can think of core dump as “memory snapshots”, but in fact, in addition to the memory core dump information, some crucial state of program will also be dump down, such as register information (including the program pointer, the stack pointer etc.), memory management information, other processor state. Core dump is very helpful for programmers to diagnose and debug the program, since some bugs are difficult to reproduce, such as pointer exception, and we can analyze the core dump file to reproduce that program bugs.

How to generate Core Dump?

Under what circumstances the program terminates abnormally or crashes would happen, if we use the command “kill -9” to kill a process will trigger a core dump? the experiments show that this is impossible.

Linux signal is a kind of asynchronous event handling mechanism, each signal has its default operation, you can view the signal and the default processing way in here. The default action include Ignore (ignore signal ), Stop(suspend process), Terminate (terminate process), and Core (termination and core dump) etc.. If we are using the default action, then, the following signals will produce core dump:

 Signal  Action comment
 SIGQUIT   Core   Quit from keyboard
 SIGILL    Core   Illegal Instruction
 SIGABRT  Core   Abort signal from abort
 SIGSEGV  Core   Invalid memory reference
 SIGTRAP  Core   Trace/breakpoint trap

Of course, not limited to the above signals. This is why we use ”Ctrl+z” to suspend a process or “Ctrl+C” to terminate a process will not generate a core dump, because the former will send SIGTSTP signal to the process, its default action is to suspend process (Stop Process); the latter will send a SIGINT signal to the process, its default action is to terminate the process (Terminate Process). As mentioned above the “kill -9″ command will issue the SIGKILL signal, which defaults to terminate the process. But if we use the “Ctrl+\” to terminate a process, the process will be issued SIGQUIT signal , it will produce a core dump. There are other situations will produce core dump, such as: program calls abort () function, memory access error, illegal instruction and so on.

Two Examples as below:

1. compare both “Ctrl+C” and “Ctrl +\” in the terminal interface

[root@devops ~]# ulimit -c
[root@devops ~]# ulimit -c unlimited
[root@devops ~]# sleep 10
[root@devops ~]# ls
anaconda-ks.cfg install.log install.log.syslog
[root@devops ~]# sleep 10
^\Quit (core dumped)

[root@devops ~]# ls
anaconda-ks.cfg core.2329 install.log install.log.syslog

2. small program generate core dump

#include <stdio.h>

int main()
int *null_p = NULL;
*null_p = 10; //write to null pointer memory area,Segmentation fault will happen
return 0;
[root@devops ~]# gcc seg.c -o seg
[root@devops ~]# ./seg
Segmentation fault (core dumped)
[root@devops ~]# ls
anaconda-ks.cfg core.2410 install.log install.log.syslog seg seg.c

How to enable Core Dump function in linux?

1. enable core dump

If you enter “ulimit -c” command and get the result value as 0, it indicate that core dump is disabled by default, it would not generate core dump file.

We can use the command “ulimit -c unlimited” to enable the core dump function, and does not limit the core dump file size; if you need to restrict the size of the file, change the “unlimited” as you want to generate the core file maximum size, pay attention to the unit is blocks (KB).

Using the above command will only effective for terminal current environment, if you want to be permanent, you can modify the file /etc/security/limits.conf  file.  add the below one line:

# /etc/security/limits.conf
#Each line describes a limit for a user in the form:
#&lt;domain&gt; &lt;type&gt; &lt;item&gt; &lt;value&gt;
<span style="color: #008000;">   *       soft  core    unlimited</span>

2. Change The Saving Location Of Core Dump File

The default generated core file is saved in the executable file’s directory, file name is core.
By modifying the /proc/sys/kernel/core_uses_pid file to generate core file is automatically add pid number.For example: “echo 1 > /proc/sys/kernel/core_uses_pid“, the generated core file will become, wherein PID represents the process of PID.

How to use gdb to debug Core dump file?
When core dump, use the command “GDB programname corename” to view the core file, view sourceprint?

[root@devops ~]# gdb seg core.2410
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
Reading symbols from /root/seg...(no debugging symbols found)...done.
[New Thread 2410]
Missing separate debuginfo for
Try: yum --disablerepo='*' --enablerepo='*-debug*' install /usr/lib/debug/.build-id/80/1b9608daa2cd5f7035ad415e9c7dd06ebdb0a2
Reading symbols from /lib64/ debugging symbols found)...done.
Loaded symbols for /lib64/
Reading symbols from /lib64/ debugging symbols found)...done.
Loaded symbols for /lib64/
Core was generated by `./seg'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000400484 in main ()
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6.x86_64
(gdb) where
#0 0x0000000000400484 in main ()
(gdb) info frame
Stack level 0, frame at 0x7fff24e1a970:
rip = 0x400484 in main; saved rip 0x7f46a9a81d1d

Friday, January 18, 2013

How to run RAW I/O's on RHEL / CentOS (Testing Purpose)

For testing storage products or other software's we need to generate some I/O's to test it's functionalities

To run a raw I/O we generally use "dd" command in Linux or Unix system. 

# dd if=/dev/zero of=/dev/sdb bs=2048 count=5000

If you want it to keep running in loop;

# while true;do dd if=/dev/zero of=/dev/sdb bs=2048 count=5000;sync;done;

To come out of this loop use "Ctrl + C".