The kernel acts as a mediator between your software programs and your systems hardware. This allows processes to be written with no clue as to what kind of hardware is installed on the computer. This is how linux is able to run on so many different hardware platforms. The kernel is basically composed of five subsystems: the process scheduler, the memory manager, the virtual file system, the network interface and the interprocess communication interface.
The process scheduler controls fair access to the CPU for all processes, it also ensures that hardware actions are performed on time.
The memory manager allows many processes to securely share the machines main memory.
The virtual file system takes all the details of the different hardware devices and creates a common file interface for all of them. The virtual file system also supports file system formats from other operating systems.
The network interface gives access to many networking standards and hardware.
The inter-process communication subsystem is used
for process to process communication on a single linux machine.
New kernels as apposed to old kernels have more device drivers, they try to be more stable, they can have better process management and they have previously reported bugs fixed. It is a good idea to upgrade your kernel when a new stable kernel is released. If you do not keep up to date you may find that when you buy that new fancy sound card your kernel will not support it.
1. Login as "root". Choose the kernel you want to install from the ftp site shown below. For this demonstration I have downloaded kernel version 2.2.17. When choosing your kernel be aware that an odd versioned kernel eg. 2.3.0 is an experimental kernel. The experimental series are fast moving versions which are used to test new features, algorithms, device drivers, etc. An odd versioned kernel may behave in strange unpredictable ways. You could expierence loss of data and or lock ups with experimental kernels. Even versioned kernels are known as the production or stable kernels. Production kernels are released less often then experimental kernels and are considered to be stable with few bugs. An even numbered kernel with the extension "test" for example "2.4.0-test11" is a beta version kernel. Beta kernels are considered unstable until they are released by Linus as stable kernels. If you want a bleeding edge kernel with the latest and the greatest; the developers encourage risk takers to try these beta kernels as well as give feedback on there performance.
ftp://ftp.kernel.org/pub/linux/kernel/
2. Download the the .tar file to a new directory. To avoid overwriting any existing kernel source trees you should place the tar file of your new kernel in it's own directory. I have chosen/usr/src/linux-2.2.17/where linux-2.2.17/ represents the version of the kernel I placed in the directory. If you were to untar your new kernel in /usr/src/ you could possibly overwrite an existing kernel source tree.
3. Untar the kernel file in its directory.
cd
/usr/src/linux-2.2.17/
tar
-xvzf linux-2.2.17.tar.gz
4. Change to the root directory of the kernel source tree. This is the directory you will make your kernel in.
cd /usr/src/linux-2.2.17/linux
5. Run one of the following make configure options to choose what options you want in your kernel:
a) make config, make menuconfig, make xconfig or make oldconfig.
1. make
config is rather old and cryptic. It's a one shot deal,
it gives you your
configure
options sequentially with no reverse. I don't recommend that you
use it.
make config
2. make
menuconfig is a text based menu for selecting your configuration.
make
menuconfig allows you to go back
to change any configuration if you feel you made
a bad choice.
Help is available for every option in make menuconfig. I
like
menuconfig
because it runs very nicely over a 56k modem for those times when
you are configuring
a kernel remotely. make menuconfig does require ncurses
to be available
on your machine.
make menuconfig
3. make
xconfig is for all of you who love GUI interfaces. It does everything
make menuconfig
does, but it requires "X" to be running.
make xconfig
4. When you
run one of the make configure options a file named .config is
created
if it doesn't
already exist. This file is created in the directory
/usr/src/linux-2.2.17/linux.
Sometimes you want to keep your current
configuration
while upgrading your kernel. This can be done by copying your .config
file from
a previous kernels source tree to the new source tree you are about to
compile.
When the old .config file is in place in the new source tree you
can run
make oldconfig.
cp
/usr/src/linux-2.2.14/linux/.config /usr/src/linux-2.2.17/linux/.config
make
oldconfig
6. Make dependencies.
make dep
7.Make the image. Be aware that compiling a kernel is a good stress test for your computer. If you have anything flaky in your setup this could be the time you find out about it. Overheating is common, especially if your processor is overclocked. This problem is so common that "Guru's " on the net when consulted with kernel problems will not talk to you if you are overclocked.
make bzImage
8. Make the modules you picked for your kernel.
make modules
9. Install modules into/lib/modules/2.2.17/(the directory will be created). Note: If you are creating an alternitively configured kernel with the same version number of an existing kernel; this step will overwrite the modules of the existing kernel. You may not want to do this if the existing kernel is your only working kernel. See the section below for adding a value for EXTRAVERSION in the Makefile to avoid wiping out your working modules.
make modules_install
10. Copy the kernel to the boot directory.
cp usr/src/linux-2.2.17/linux/arch/i386/boot/bzImage /boot/vmlinuz-2.2.17
11. Copy System.map to a location that LILO can find during startup. I prefer to keep the System.map with the rest of the compiled kernel. The System.map is not needed for your kernel to run, but it is a good debugging tool if you have problems involving a kernel fault. The kernel fault address can be looked up in the System.map file by klogd and then translated into something more meaningful.
cp /usr/src/linux-2.2.17/linux/System.map /lib/modules/2.2.17/System.map
Some people like to have it in the /boot/ directory alongside its matching vmlinuz file. It's your choice, but besure to name your system.map and your vmlinuz with the extensions shown below. Without these extensions if you have two different kernel images to boot from in the /boot/ directory it is possible that the wrong System.map will be used with the wrong kernel image.
cp /usr/src/linux-2.2.17/linux/System.map /boot/System.map-2.2.17
12. Reconfigure lilo to know about the new kernel.
vi /etc/lilo.conf
Append the following lines to lilo.conf to access your new kernel at boot time. The default image choice is already in the file. I suggest duplicating the default choice code and then changing the image variable in the duplicate code to the location of the new vmlinuz file. Change the label variable to something you feel represents the new kernel. The label value is what you will type at the LILO: prompt to load the new kernel. You can make the new kernel the default kernel by placing the lines of code above the existing default code, I do not suggest doing this until your sure that the new kernel works.
image=/boot/vmlinuz-2.2.17
label=2217
root=/dev/hdb7
append=""
read-only
13. Run lilo to update lilo with the latest configuration. If you do not do this you will not be able to select your new kernel when you reboot.
lilo
14. Reboot and enter the label of the kernel you want to start.
a) When you see "LILO:" press the <shift>
key then press the <tab> key. This should
display the lilo boot choices.
Enter your choice at the "LILO:" prompt.
LILO:2217 <enter>
15. If the kernel doesn't work properly you can shutdown and reboot recalling your original kernel at the LILO prompt.
LILO:Linux <enter>
You may want to have several kernels for different applications ( Although I would recommend buying another machine) and you may want them all to have the same "latest and greatest" kernel version. But... you do not want alternatively configured kernels of the same version to overwrite the modules chosen for each previously configured kernel of that same version. You can add an extra version number to the Makefile of your kernel to avoid this problem.
cd /usr/src/linux-2.2.17/linux
vi Makefile
At the top of this file you will see:
VERSION = 2
PATCHLEVEL = 2
SUBLEVEL = 17
EXTRAVERSION = NEW
Edit the line "EXTRAVERSION = " by adding what
ever you want your extraversion to be after the "=" sign. Now when
you make your bzImage and install your modules they will carry the new
version number you gave them in the Makefile. /usr/lib/2.2.17NEW/
will
be the directory created to place the new modules for Image vmlinuz-2.2.17NEW.
Kernel PrePatches are patches for a particular kernel version that are prior to upgrading to the next version of a kernel. For example, currently 2.2.17 is the latest production kernel released (2.2.18 is non existant at this time). You can get a number of patches that will make your 2.2.17 kernel closer to the 2.2.18 kernel they are building. These patches are named strangely. The latest kernel PrePatch is 2.2.18pre22, this means it is the 22nd patch to kernel version 2.2.17 prior to the release of kernel 2.2.18. If you truly want the latest and greatest you want to install this. Keep in mind if you are constantly upgrading to the latest and greatest PrePatch you will spend more time compiling your OS then using it.
1. Choose the kernel patch you want to install from the kernel ftp site.
ftp://ftp.kernel.org/pub/linux/kernel/people/alan/
You may be wondering why I chose alan/ instead of the other people in the directory people/. Alan Cox is the man in charge of stable kernels, when a production kernel is release by Linus it falls into the hands of Alan. Alan is in charge of maintaining the stable production kernels. The latest and greatest considered stable patches to these kernels are in his directory.
2. Download the file to the top directory of the kernel source tree you want to patch.
/usr/src/linux-2.2.17/pre-patch-2.2.18-19.bz2
3. Unzip the patch.
bunzip2 pre-patch-2.2.18-19.bz2
4. Alan changes his directory /usr/src/linux/ to /usr/src/linux.vanilla. You must also do this. The kernel version in the /usr/src/linux.vanilla directory must be 2.2.17 in this case. You can not apply a prepatch to a directory that already has had a prepatch applied to it. I also suggest that you do a make clean to remove all previously built *.o files from the source tree..
mv /usr/src/linux /usr/src/linux.vanilla
cd /usr/src/linux.vanilla
make clean
5. Apply the patch.
patch -p0 < pre-patch-2.2.18-19
6. Check if the patch was applied( you will see the version numbers matching the patch you just did).
head Makefile
7. Make the old .config file with the new updated kernel source code.
make oldconfig
8. Run make dep to find all the files that are dependant on the newly patched files so that they can be recompiled.
make dep
9. Make the new kernel image.
make bzImage
10. Make the modules.
make modules
11. Install modules into /lib/modules/2.2.18pre19/ (the directory will be created).
make modules_install
12.Copy System.map to a location that LILO can find during startup.
cp/usr/src/linux-2.2.17/linux.vanilla/System.map /lib/modules/2.2.18pre19/System.map
"or"
cp /usr/src/linux-2.2.17/linux.vanilla/System.map /boot/System.map-2.2.18pre19
13. Copy the kernel image to the boot directory.
cp usr/src/linux-2.2.17/linux.vanilla/arch/i386/boot/bzImage /boot/vmlinuz-2.2.18pre19
14. Reconfigure lilo to know about the new kernel.
vi /etc/lilo.conf
Append the following lines to "lilo.conf" .
image=/boot/vmlinuz-2.2.18pre19
label=2218p19
root=/dev/hdb7
append=""
read-only
When you boot you can choose the kernel image to load by entering the value of $label at the "LILO:" prompt. Run lilo to update lilo with the latest configuration.
lilo
15. Reboot and enter label of the kernel you want to start.
a) When you see "LILO:" press the <shift>
key then press the <tab> key. This should
display the lilo boot choices.
Enter your choice at the "LILO:" prompt.
LILO:2218p19 <enter>
16. If the kernel doesn't work properly you can shutdown and reboot recalling your original kernel at the LILO prompt.
LILO:2217 <enter>
The machine I rebuilt a kernel on had a very basic 13 disk installation of Debian. I do not believe any development tools were installed.
1. The first problem I encountered was while trying to run make menuconfig. ncurses was not installed on the machine. ncurses is required for the formatting of the text menu in make menuconfig.
Fix: Downloaded ncurses from Debian site. Ran dpkg on the file
http://packages.debian.org/stable/base/ncurses-base.html
dpkg ncurses-base_5.0-6.deb
2. The second problem I had was during my make bzImage, the compiler compiled right up until almost the end then it complained that a very basic assembler as86 was not available as well as a linker named ld86.
Fix: Downloaded bin86
from the Debian site. This package contains the assembler
and linker mentioned above. Run
dpkg on the file.
http://packages.debian.org/stable/devel/bin86.html
dpkg bin86_0.14.9-3.deb
3. My make bzImage was complaining that bzImage was to big when I used the .config file supplied with the initial install. One megabyte is the largest your kernel Image is allowed to be with Lilo. When your machine boots it start up in 8086 realmode. Realmode goes back approximately 10 - 15 years, this is the reason for the 1 meg size limit on your Image file. There are currently some changes going on with regards to this problem. I believe the beta kernel 2.4 deals with it. I have not looked into it very deeply.
Fix: The only
way around this was to do a make menuconfig or a make
xconfig and change as many
things to modules as was possible and turn
off things
I did not need. One area where I cut down on alot of size was
under network
cards. The .config file that came with the original
install
had all the
network card drivers built directly into the kernel instead of using
modules.
4. The fourth hangup was during my installation of a prepatch. I executed the line patch -p0 < pre-patch-2.2.18-19 and the machine complained about not knowing what patch was.
Fix: I actually
copied the binary named patch from my Mandrake machine
at home to
my Debian machine and to my amazement it worked! I would
recommend
executing an apt-get patch to aquire patch if you do
not have it.
If you have any serious problems you may want to check out the linux kernel mailing lists. These lists recieve 200 + emails a day regarding the linux kernel and problems encountered with it.
http://marc.theaimsgroup.com/?l=linux-kernel&r=1&w=2
http://boudicca.tux.org/hypermail/linux-kernel/this-week/
Debian help is available at:
http://www.debian.org/doc/FAQ/ch-kernel.html
The very top of the kernel source tree is usually found at /usr/src/linux. Here I have briefly described what each major directory in the source tree represents. I have also given a brief description of some of the important subdirectories/files you will find in each of these major directories.
arch/
Contains directories for each supported architecture, below is the i386/
architecture
directory and some of it's subdirectories.
arch/i386/mm/
Contains architecture specific memory
management code.
arch/i386/kernel/
Architecture specific kernel code.
arch/i386/kernel/irq.c Interrupt handling for
a specific architecture
arch/i386/lib/
Architecture specific library code.
include/ The include files that are needed to build the kernel.
include/linux/interrupt.h Kernel bottom half handling code.
include/asm-i386/irq.h
Interrupt handling code definitions.
include/linux/ext2fs.h
EXT2 data structure definitions.
include/linux/fs.h
Virtual File System data structures definition
init/ The initialization code for the kernel.
mm/ All of the memory management code.
mm/memory.c
Pagefault handling
mm/filemap.c
Memory mapping and cache code
mm/buffer.c
Buffer cache
mm/swap_state.c
Swap cache
drivers/ System device drivers
are in here, they are split into groups such as cdrom, pci,
drivers/ System device drivers
are in here, they are split into groups such as cdrom, pci,
drivers/block
This directory contains source code for
ipc/
Contains the kernels inter process communications code.
ipc/msg.c
System V messages implementation
modules/ The kernel module code is partially in
the kernel and partially in the modules package.
fs/
File system code. One directory is present for each supported file
system.
fs/ext2/
Here you'll find the source for the EXT2 file
kernel/sched.c
Scheduler
net/
Networking code for the kernel.
net/socket.c
BSD socket code.
lib/
Library code for the kernel.
scripts/ Contains scripts that are
used when the kernel is configured.
sound, etc... Each group has its own directory within /usr/src/linux/drivers/.
sound, etc... Each group has its own directory within /usr/src/linux/drivers/.
Drivers occupy the biggest percentage of lines in the overall kernel source
code.
block device drivers such as IDE and
SCSI based devices.
drivers/char
Character based devices such as ttys, serial
ports and mice are found in here.
drivers/cdrom
All of the CDROM code for linux.
drivers/pci
Source for the PCI pseudo-driver.
drivers/scsi
All the SCSI code and all the drivers for
SCSI devices.
drivers/net
This is the spot to find network device drivers
code.
drivers/sound
Sound card drivers source code lives here.
ipc/shm.c
Shared memory
ipc/sem.c
Semaphores
ipc/pipe.c
Pipes
The kernel code is all in kernel/modules.c with the data
structures in directory
include/linux/module.h. When you build modules they will be placed
in the
/usr/src/linux/modules/ directory.
system, the data structure definitions are in the
include/linux/directory.
kernel/ The kernel code.
kernel/fork.c
Fork code
kernel/modules.c
Kernel modules code
net/ipv4/af_inet.c
Contains the IP version 4 INET socket code.
net/core
Generic protocol support code.
net/ipv4
TCP/IP networking code
The linux kernel is monolithic which means it is one large program that lets all functional components have access to its internal data structures and routines. Linux allows components to be loaded as they are needed. Modules are pieces of code that can be linked and unlinked from the kernel dynamically with the help of the kernel daemon kerneld or manually linked and unlinked with modutils. Modules can also be usefull for trying out new code without having to re-compile the kernel. When you use modules there is a performance and memory penalty but it is very small and hardly noticed. Kernel modules are as much part of the kernel as the kernel and in fact share all the same rights and privileges as the kernel. A bad module can crash the kernel.
Loading a Module
A module can be manually loaded into the kernel with the insmod command.
A module can be demand loaded into the kernel which means it is only loaded when it is needed. When the kernel recognizes the need for a module it requests that the kernel daemon kerneld should load the module. kerneld does not perform these tasks itself. kerneld runs programs like insmod and rmmod to load/unload the modules. At boot time kerneld opens up an Inter-Process communication channel to the kernel; through this channel kerneld acts like an agent of the kernel and schedules work as if it was the kernel. When insmod is requested by kerneld to load a module insmod must first find the module to load. Demand loaded modules are normally kept in /lib/modules/kernel-version.
Unloading a Module
A module can be manually unloaded with the rmmod command.
Demand loaded modules are automatically unloaded as soon as the module has not been used for a set amount of time. This is done by kerneld making a system call requesting that all unused modules are removed from the system every time its idle timer times out. For example, If you mount a VFAT filesystem the VFAT module is loaded dynamically and used until the VFAT file system is unmounted. Shortly after the VFAT filesystem is unmounted the VFAT module will be removed from the kernel by daemon kerneld. If you have the VFAT file system mounted the FAT module (which is depended upon by VFAT module) will not be unloaded even if the FAT filesystem is not mounted. In short, a module depended upon by another module cannot be unloaded. The command lsmod will show you the currently loaded modules.
At this time kernel 2.4.0 has not been released as stable, I expect that it will be released very shortly. The number of significant changes per test version has been decreasing which makes me think they are getting close. Of course Linus will not let any one know when he thinks he will release the kernel because companies like Debian, Red Hat, Mandrake, etc will hold him to it and give him a hard time if he's late. Linux distribution companies want the new kernels released as quickly as possible so they can release hot new distributions with the latest and greatest kernels. I expect that shortly after this document is posted the 2.4.0 kernel will be released as stable and you are going to want to compile it. You will want to know what changes have been made and what the minimum requirements are for a successful installation. Here I will let you know what I have discovered about the up and coming latest and greatest kernel.
1. There is a good file that you should know about in the kernel source tree. The file /usr/src/2.4.0-test11/linux/Documentation/Changes tells you what has been changed since version 2.2.*. I suggest you look at this file, although some of what is in that file will be explained here. If you have a kernel version older then 2.2.* you will have to read the /usr/src/2.2.0/linux/Documentation/Changes file and make the appropriate changes before you can continue with the 2.4.0 changes file.
2. Kernel 2.4.0 requires that you have at least these versions of the following software tools. If you do not use some of the tools you may not need them (PCMCIA for example). The third column below is the command to execute to check what version you have.
o Gnu C
2.91.66
# gcc --version
o Gnu make
3.77
# make --version
o binutils
2.9.1.0.25
# ld -v
o util-linux
2.10o
# kbdrate -v
o modutils
2.3.18
# insmod -V
o e2fsprogs
1.19
# tune2fs --version
o pcmcia-cs
3.1.21
# cardmgr -V
o PPP
2.4.0
# pppd --version
o isdn4k-utils
3.1beta7
# isdnctrl 2>&1|grep version
Each of these software utilities is described in detail in the "Changes" file, it is good reading and I do suggest you read it. From that file I have robbed the links to the source code for the above software tools. Here they are:
Getting updated software
========================
Compilers
*********
egcs 1.1.2 (gcc 2.91.66)
---------
o <ftp://ftp.valinux.com/pub/support/hjl/gcc/egcs-1.1.2/egcs-1.1.2-glibc.x86.tar.bz2>
o <ftp://ftp.valinux.com/pub/support/hjl/gcc/egcs-1.1.2/egcs-1.1.2-libc5.x86.tar.bz2>
o <ftp://ftp.valinux.com/pub/support/hjl/gcc/egcs-1.1.2/egcs-1.1.2-alpha.tar.bz2>
Binutils
********
2.9.1 series
------------
o <ftp://ftp.valinux.com/pub/support/hjl/binutils/2.9.1/binutils-2.9.1.0.25.tar.gz>
2.10 series
------------
o <ftp://ftp.valinux.com/pub/support/hjl/binutils/binutils-2.10.0.24.tar.bz2>
System utilities
****************
Util-linux
----------
o <ftp://ftp.win.tue.nl/pub/linux-local/utils/util-linux/util-linux-2.10o.tar.gz>
Ksymoops
--------
o <ftp://ftp.kernel.org/pub/linux/utils/kernel/ksymoops/v2.3>
Modutils
--------
o <ftp://ftp.kernel.org/pub/linux/utils/kernel/modutils/v2.3/modutils-2.3.18.tar.bz2>
Mkinitrd
--------
o <ftp://rawhide.redhat.com/pub/rawhide/SRPMS/SRPMS/mkinitrd-2.5-1.src.rpm>
E2fsprogs
---------
o <ftp://download.sourceforge.net/pub/sourceforge/e2fsprogs/e2fsprogs-1.19.tar.gz>
LVM toolset
-----------
o <http://linux.msede.com/lvm/>
Pcmcia-cs
---------
o <ftp://pcmcia-cs.sourceforge.net/pub/pcmcia-cs/pcmcia-cs-3.1.21.tar.gz>
Jade
----
o <ftp://ftp.jclark.com/pub/jade/jade-1.2.1.tar.gz>
DocBook Stylesheets
-------------------
o <http://nwalsh.com/docbook/dsssl/>
Intel P6 microcode
------------------
o <http://www.urbanmyth.org/microcode/>
Network
*******
PPP
---
o <ftp://linuxcare.com.au/pub/ppp/ppp-2.4.0.tar.gz>
Isdn4k-utils
------------
o <ftp://ftp.isdn4linux.de/pub/isdn4linux/utils/testing/isdn4k-utils.v3.1beta7.tar.gz>
Netfilter
---------
o <http://netfilter.filewatcher.org/iptables-1.1.1.tar.bz2>
o <http://www.samba.org/netfilter/iptables-1.1.1.tar.bz2>
o <http://netfilter.kernelnotes.org/iptables-1.1.1.tar.bz2>
Ip-route2
---------
o <ftp://ftp.inr.ac.ru/ip-routing/iproute2-2.2.4-now-ss991023.tar.gz>
Here are some links that may help you with problems encountered that I have not addressed. Also feel free to contact me pashaross@home.com for assistance.
Mailing Lists:
http://marc.theaimsgroup.com/?l=linux-kernel&r=1&w=2
http://boudicca.tux.org/hypermail/linux-kernel/this-week/
LINKS:
My personal favorites:
http://members.aa.net/~swear/pedia/kernel.html
http://jungla.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html