Dumping SMRAM

While investingating EIST on my GA-P35-DS3R motherboard, I found P-state changes use special I/O ports which trigger system management mode (SMM). To investigate further, it is necessary to look at SMM code that runs when accesses to ports 0x880 and 0x882 are intercepted.

The first step is to obtain the code. There are two possibilities: extracting it from the BIOS and dumping it from SMRAM. I chose to dump SMRAM. That will show what is undoubtedly actually being run, and it will also dump values of variables.

Since this deals with access to memory, the P35 northbridge chip needs to be examined. Intel provides the 3 Series Express Chipset Family datasheet (PDF). The DRAM controller registers described in section 5.1 can be dumped using "sudo lspci -xxx -s 00:00.0 > P35_DRAM_registers.txt". They are listed in table 5-1.

The value of the SMRAMC register at 0x9D is 0x0A. As expected, G_SMRAME is set to enable SMRAM, but surprisingly, the D_LCK bit is not set. This means that software is free to set the D_OPEN bit and access SMRAM. The compatible SMM space bits are hard-wired, causing memory between 0xA0000 and 0xBFFFF to be used as SMRAM. This overlaps with VGA graphics memory, but that is not a problem because the northbridge can direct accesses appropriately.

The next thing to check is the ESMRAMC register at 0x9E. It's value is 0x39. The H_SMRAME bit is not set, so the low SMRAM at 0xA00000 is not remapped. However, T_EN is set, so according to TSEG_SZ, the top 1 MB of memory is also reserved for use as SMRAM. This TSEG is set as uncachable via an MTRR, so that's the part that's probably being used as SMRAM.

$ cat /proc/mtrr
reg00: base=0x000000000 (    0MB), size= 2048MB, count=1: write-back
reg01: base=0x07ff00000 ( 2047MB), size=    1MB, count=1: uncachable

One could abuse caching to mess with SMM (PDF). However, the D_LCK bit is clear, so there is no need to resort to such trickery. It should be possible to simply set the D_OPEN bit and access that memory.

The first obstacle is the CONFIG_STRICT_DEVMEM. Ubuntu and most other Linux distributions enable it, because unrestricted root access to /dev/mem is considered a security hole. With it enabled, memory at 0xA0000 can be dumped because it is within the first megabyte, but the TSEG at the end of RAM cannot be dumped.

The protection is enabled when the kernel is compiled, and it can only be disabled by recompiling or otherwise modifying the kernel. However, it's possible to load a module which provides another similar device without the protection. It would be natural to base that driver on drivers/char/mem.c, and someone has already done this, creating fmem. After compiling the module and loading it via run.sh, memory can be freely dumped via /dev/fmem.

The only remaining obstacle is the inability to access SMRAM from outside SMM. Fortunately the D_LCK bit is not set, so one can simply set D_OPEN. Here are commands displaying the SMRAMC register, setting D_OPEN, and confirming that the register changed:

$ sudo setpci -s 00:00.0 0x9D.b
0a
$ sudo setpci -s 00:00.0 0x9D.b=0x4A
$ sudo setpci -s 00:00.0 0x9D.b
4a

Now, it is possible to dump SMRAM via dd:

$ sudo dd if=/dev/mem bs=64k skip=10 count=1 of=smram.a0000.bin
1+0 records in
1+0 records out
65536 bytes (66 kB) copied, 0.00113763 s, 57.6 MB/s
$ sudo dd if=/dev/fmem bs=1M count=1 skip=2047 of=smram.tseg.bin
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154848 s, 67.7 MB/s

It's probably a good idea to afterwards clear D_OPEN in the SMRAMC register, so that SMRAM has some protection again. This can be done via "sudo setpci -s 00:00.0 0x9D.b=0x0A". Feel free to try to dump the same memory while D_OPEN is cleared. Without D_OPEN, the resulting files have all bits set, showing how SMRAM cannot normally be accessed.

The dumps I got definitely contain the SMRAM. The first 64k of the TSEG dump contains structures documented in chapter 34 of Volume 3 of the Intel® 64 and IA-32 Architectures Software Developer Manual. The SMI entry point is at 0x8000. SMM starts out in a mode similar to real mode, and the code there sets up for an entry into protected mode. Saved state can be seen toward the end of the 64k segment. For example the SMBASE register is at offset 0xFEF8, containing 0x7FF00000 as expected.

That's all for now. I may post more if I find something interesting in the SMRAM.