Monday, February 13, 2012

Module Management in RHEL / CentOS 7


The Linux kernel allows drivers and features to be compiled as modules rather than as part of the kernel itself. This means that users can often change features in the kernel or add drivers without recompiling, and that the Linux kernel doesn't have to carry a lot of unnecessary baggage. Want to learn how to manage your modules? It's easy to do, just keep reading.

In this tutorial, we'll walk through the steps of seeing what's already loaded in the running kernel, and adding and removing modules from the kernel.


What's Loaded

1. To see Loaded Kernel driver or module information for PCI components:

[root@localhost ~]# lspci -k
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
        Subsystem: VMware Virtual Machine Chipset
        Kernel driver in use: agpgart-intel
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
        Subsystem: VMware Virtual Machine Chipset
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
        Subsystem: VMware Virtual Machine Chipset
        Kernel driver in use: ata_piix
        Kernel modules: ata_piix, pata_acpi, ata_generic
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
        Subsystem: VMware Virtual Machine Chipset
        Kernel modules: i2c_piix4
02:01.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
        Subsystem: VMware PRO/1000 MT Single Port Adapter
        Kernel driver in use: e1000
        Kernel modules: e1000
.
.
.


2. List currently loaded modules:

[root@localhost ~]# lsmod
Module                  Size  Used by
xt_CHECKSUM            12549  1
ipt_MASQUERADE         12678  3
nf_nat_masquerade_ipv4    13412  1 ipt_MASQUERADE
tun                    31740  1
devlink                48345  0
ip6t_rpfilter          12595  1
ipt_REJECT             12541  4
nf_reject_ipv4         13373  1 ipt_REJECT
ip6t_REJECT            12625  2
nf_reject_ipv6         13717  1 ip6t_REJECT
xt_conntrack           12760  12
.
.
.


You can also find loaded modules list in the below file:

[root@localhost ~]# cat /proc/modules
xt_CHECKSUM 12549 1 - Live 0xffffffffc0819000
ipt_MASQUERADE 12678 3 - Live 0xffffffffc0814000
nf_nat_masquerade_ipv4 13412 1 ipt_MASQUERADE, Live 0xffffffffc0805000
tun 31740 1 - Live 0xffffffffc080b000
devlink 48345 0 - Live 0xffffffffc07f8000
ip6t_rpfilter 12595 1 - Live 0xffffffffc07f3000
ipt_REJECT 12541 4 - Live 0xffffffffc07ee000
nf_reject_ipv4 13373 1 ipt_REJECT, Live 0xffffffffc07e9000
ip6t_REJECT 12625 2 - Live 0xffffffffc07e4000
nf_reject_ipv6 13717 1 ip6t_REJECT, Live 0xffffffffc07df000
xt_conntrack 12760 12 - Live 0xffffffffc07cd000
ip_set 45644 0 - Live 0xffffffffc07d2000
nfnetlink 14490 1 ip_set, Live 0xffffffffc07c1000
ebtable_nat 12807 1 - Live 0xffffffffc07bc000
.
.
.



Installing Modules and get information about them

What if you have a module you want to load into the kernel? You can do that with insmod or modprobe.

The preferred method is modprobe, because it will also load any modules that the requested module depends on.

Load a module "vfat":

[root@localhost ~]# lsmod | grep -i vfat

[root@localhost ~]# modprobe vfat

[root@localhost ~]# lsmod | grep -i vfat
vfat                   17461  0
fat                    65950  1 vfat

Modules dependencies are recorded to "modules.dep" file using "depmod" command. Whenever we run "modprobe" command to load a particular module it gets the dependency list from "modprob.dep" file to know which are all other modules needs to be loaded before the one you asked for.


Get information about "vfat" module:

[root@localhost ~]# modinfo vfat
filename:       /lib/modules/3.10.0-957.10.1.el7.x86_64/kernel/fs/fat/vfat.ko.xz
author:         Gordon Chaffee
description:    VFAT filesystem support
license:        GPL
alias:          fs-vfat
retpoline:      Y
rhelversion:    7.6
srcversion:     A3254796A3CD9815ABDDC94
depends:        fat
intree:         Y
vermagic:       3.10.0-957.10.1.el7.x86_64 SMP mod_unload modversions
signer:         CentOS Linux kernel signing key
sig_key:        17:EA:5F:B9:16:4B:C2:26:55:5C:00:43:FA:D4:E5:86:CC:E8:A2:05
sig_hashalgo:   sha256


Removing Modules temporarily

To unload "vfat" module:

[root@localhost ~]# modprobe -r vfat

[root@localhost ~]# lsmod | grep -i vfat

When we run "modprobe -r <module_name>" command, it unload the module as well as it dependent modules as well. Again it refers "modules.dep" file to do so.


Uses of modprobe.d directory and dep files:


Now a days old "modprobe.conf" file is replaced with module configuration directories:

/usr/lib/modprobe.d/ - Default modules needs to be loaded at the time of host boot-up
/etc/modprobe.d/ - Custom modules needs to be loaded at the time of host boot-up


Where do we find "modules.dep" file:

/usr/lib/modules/3.10.0-957.el7.x86_64/modules.dep - List of modules and their dependencies, generated by depmod. Contains Human readable format.
/usr/lib/modules/3.10.0-957.el7.x86_64/modules.dep.bin - Binary file generated by depmod listing the dependencies for every module. It is used by kmod tools like modprobe.

In case if this "modules.dep" file is removed or corrupted, you can face issues while loading new module. So to re-create the "modules.dep" file you can run:

[root@localhost ~]# depmod -a

Adding multiple IP addresses on one Ethernet

If we have one network interface, eth0. And when you add IP addresses, you actually create virtual network interfaces named eth0:1, eth0:2, ...... eth0:n.


How to set it on temporary basis on all OS's:

#ifconfig eth0:1 up [IP address] netmask [chosen netmask, if unsure use 255.255.255.0].

You can add commands like this to your startup scripts to have them come up on reboot, but there is cleaner way which is maintioned below;


How to set it Permanently basis on RHEL / CentOS :


All interface configuration files are located in "/etc/sysconfig/network-scripts/". Each interface is represented by a file corresponding to ifcfg-eth<x> where <x> represents the unique interface number for that card (e.g. the first interface card is represented by ifcfg-eth0). To create an alias for that interface, you need to create a file in the format of ifcfg-eth0:<y> where <y> represents the alias number (e.g. the 2nd ip for the first card would be ifcfg-eth0:1). To set up two ip addresses on a ethernet edit/create the following files as noted (substituting your ip addresses where denoted by x's):



/etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
IPADDR=XXX.XXX.XXX.XXX
NETMASK=255.255.255.0
BROADCAST=XXX.XXX.XXX.255
GATEWAY=XXX.XXX.XXX.1



/etc/sysconfig/network-scripts/ifcfg-eth0:0

DEVICE=eth0:0
BOOTPROTO=none
ONPARENT=yes
IPADDR=XXX.XXX.XXX.XXX
NETMASK=255.255.255.0
BROADCAST=XXX.XXX.XXX.255

This assumes all ip addresses are in the same range. See the RedHat Enterprise Linux Reference Guide for details on the configurable parameters. Finally, running "service network restart" will restart the network interfaces (probably best to do this from lish). Hope it helps.

NIC Bonding : Bind multiple Network Interfaces (NIC) Into a Single Interface


The first step is to rename the server-change /etc/sysconfig/network to match your new server name.That’s easy :)

Now to the bonding driver. RHEL6 and OL 6 have deprecated /etc/modprobe.conf in favour of /etc/modprobe.d and its configuration files. It’s still necessary to tell the kernel that it should use the bonding driver for my new device, bond0 so I created a new file /etc/modprobe.d/bonding.conf with just one line in it:

alias bond0 bonding

That’s it, don’t put any further information about module parameters in the file, this is deprecated. The documentation clearly states “Important: put all bonding module parameters in ifcfg-bondN files”.

Now I had to create the configuration files for eth0, eth1 and bond0. They are created as follows:

File: ifcfg-eth0

DEVICE=eth0
 BOOTPROTO=none
 ONBOOT=yes
 MASTER=bond0
 SLAVE=yes
 USERCTL=no/
File: ifcfg-eth1

DEVICE=eth1
 BOOTPROTO=none
 ONBOOT=yes
 MASTER=bond0
 SLAVE=yes
 USERCTL=no
File: ifcfg-bond0

DEVICE=bond0
 IPADDR=192.168.0.126
 NETMASK=255.255.255.0
 ONBOOT=yes
 BOOTPROTO=none
 USERCTL=no
 BONDING_OPTS="<bonding parameters separated by spaces>"


Now for the bonding paramters-there are a few of interest. First, I wanted to set the mode to active-passive, which is Oracle recommended (with the rationale: it is simple). Additionally, you have to set either the arp_interval/arp_target parameters or a value to miimon to allow for speedy link failure detection. My BONDING_OPTS for bond0 is therefore as follows:

BONDING_OPTS=”miimon=1000 mode=active-backup”

Have a look at the documentation for more detail about the options.


The test is going to be simple: first I’ll bring up the interface bond0 by issuing a “system network restart” command on the xen console, followed by a “xm network-detach” command.The output of the network restart command is here:

[root@rhel6ref network-scripts]# service network restart
Shutting down loopback interface:  [  OK  ]
Bringing up loopback interface:  [  OK  ]
Bringing up interface bond0:  [  OK  ]


[root@rhel6ref network-scripts]# ifconfig
bond0     Link encap:Ethernet  HWaddr 00:16:1E:1B:1D:1F
          inet addr:192.168.99.126  Bcast:192.168.99.255  Mask:255.255.255.0
          inet6 addr: fe80::216:1eff:fe1b:1d1f/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:297 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:9002 (8.7 KiB)  TX bytes:1824 (1.7 KiB)

eth0      Link encap:Ethernet  HWaddr 00:16:1E:1B:1D:1F
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:214 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:6335 (6.1 KiB)  TX bytes:1272 (1.2 KiB)
          Interrupt:18

eth1      Link encap:Ethernet  HWaddr 00:16:1E:1B:1D:1F
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:83 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2667 (2.6 KiB)  TX bytes:552 (552.0 b)
          Interrupt:17

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)



The kernel traces these operations in /var/log/messages:

 May  1 07:55:49 rhel6ref kernel: bonding: bond0: Setting MII monitoring interval to 1000.
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: setting mode to active-backup (1).
 May  1 07:55:49 rhel6ref kernel: ADDRCONF(NETDEV_UP): bond0: link is not ready
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: Adding slave eth0.
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: Warning: failed to get speed and duplex from eth0, assumed to be 100Mb/sec and Full.
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: making interface eth0 the new active one.
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: first active interface up!
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: enslaving eth0 as an active interface with an up link.
 May  1 07:55:49 rhel6ref kernel: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: Adding slave eth1.
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: Warning: failed to get speed and duplex from eth1, assumed to be 100Mb/sec and Full.
 May  1 07:55:49 rhel6ref kernel: bonding: bond0: enslaving eth1 as a backup interface with an up link.


This shows an active device of eth0, with eth1 as the passive device. Note that the MAC addresses of all devices are identical (which is expected behaviour). Now let’s see what happens to the channel failover when I take a NIC offline. First of all I have to check xenstore which NICs are present:

Official Document on NIC Bonding

Enabling & Disabling Promiscuous mode on Ethernet


                     You can try below things to get it done;

In RHEL / CentOS :

#/sbin/ifconfig ethX promisc   (Temporary solution, You will loose setting if you reboot VM )

To set it permanently Edit file –

#cat /etc/sysconfig/network-scripts/ifcfg-eth1

BOOTPROTO=static
DEVICE=eth1
TYPE=Ethernet
PROMISC=yes
USERCTL=no
PEERDNS=no
HWADDR=xx:xx:xx:C8:67:0E


In SLES :

Temporary solution is same as above.

But for permanent change edit file “ /etc/sysconfig/network/ifcfg-rth-id-xx:xx:xx:xx:xx:xx”

Here you can add line “ifconfig eth0 promisc” OR if this doesn’t work add “PROMISC=yes


And to disable this mode either simply remove added line from the respective files or use command ;

#/sbin/ifconfig ethX -promisc   (Temporary solution)