If your going to put CentOS 5 on a machine with more than 4 gigs of ram you may need to use the PAE kernel. This is what I learned when moving from CentOS 4 to 5. I had a machine with dual Opteron 290's and 8 gig of ram (4 gig per processor). It was using CentOS 4 with the smp kernel from the CentOS plus repository. After upgrading it to CentOS 5 it put in a similar default kernel and the machine only saw 4 gig of ram.
I was trying to figured out why this was happening when I came across a note in the systems dmesg. Towards the top of the dmesg it said that it saw the 8 gig but was disabling the addressing of memory over 4 gig and to enable PAE. I have not heard of this PAE. So over to wikipedia I went.
I found this in the wikipedia entry:
Physical Address Extension (PAE) refers to a feature of x86 processors that allows for up to 64 Gigabytes (GB) of physical memory to be used in 32-bit systems, given appropriate operating system support. PAE is provided by Intel Pentium Pro and above CPUs (including all later Pentium-series processors except the 400 MHz bus versions on the Pentium M), as well as by some compatible processors such as those from AMD. The CPUID flag PAE is assigned for the purpose of identifying CPUs with this capability.
The processor hardware is augmented with additional address lines used to select the additional memory, and 36-bit page tables, but regular application software continues to use instructions with 32-bit addresses and a flat memory model limited to 4 gigabytes (GB). The operating system uses PAE to map this 32-bit address space onto the 64 gigabytes (GB) of total memory, and the map can be and usually is different for each process. In this way the extra memory is useful even though regular applications cannot access it all simultaneously.
For application software which needs access to more than 4 gigabytes (GB) of memory, some special mechanism may be provided by the operating system in addition to the regular PAE support. On Microsoft Windows this mechanism is called Address Windowing Extensions (AWE), while on Unix-like systems a variety of tricks are used, such as using mmap() to map regions of a file into and out of the address space as needed, none having been blessed as a standard.
So it looked like I needed PAE as my processors supported it (cat /proc/cpuinfo | grep pae) and I was using the 32bit version of CentOS. I remembered seeing a kernel with the PAE in it's name in the CentOS 5 plus repository. Sure enough the latest one was the kernel-PAE-2.6.18-8.1.15.el5.centos.plus.i686.rpm. After installing this kernel and rebooting the machine finally saw all 8 gig of memory.
