Linux Magazine | July 1999 | GEARHEADS ONLY | How To Make Sure Your Driver Will Work On The Power Macintosh

There is an increasing number of people running Linux on their Power Macintoshes. One of the advantages of running Linux (as opposed to other variants of UNIX) on a Power Mac is that the Linux source includes drivers for a large number of PCI cards. All of the current Power Macs use the PCI bus as their internal I/O bus and many have PCI expansion card slots, allowing users to install any of the wide range of PCI expansion cards which are available today. The drivers in the Linux source will usually work on a Power Mac, but often only after some tweaking. The problem is usually that the driver author has made some assumptions that are true on an Intel PC but not on other machines. In this column, I will discusses some of those assumptions and how driver authors can make sure that their drivers will work not only on an Intel PC, but also on the Power Mac and other machines.

Figure 1 shows the block diagram of a typical Power Macintosh. There are some similarities with Intel PCs (such as the fact that they both have a PCI bus) and some differences, the most obvious of which is that they use different CPUs -- Power Macs use a PowerPC processor while Intel PCs use x86-based processors. Another difference is that the Power Mac does not have an ISA bus.

What difference does it make to the driver author that the CPU is a PowerPC? First, the PowerPC runs natively big-endian rather than little-endian. (The PowerPC can run in little-endian mode, but with possibly reduced functionality or performance. The Linux kernel on the PowerPC runs in big-endian mode.)

Secondly, all I/O on the Power Mac is memory-mapped. That is, I/O devices are accessed using normal load and store instructions rather than special I/O instructions. The PowerPC architecture, like all other modern architectures, allows memory accesses to be reordered for increased performance. For example, if a program stores one value to memory and then loads another, a PowerPC processor is allowed to perform the load before the store, because getting the value from the load is more critical for making forward progress with the execution of the program. (Of course, if the store and load were to the same memory location, the load is not permitted to return the old value in the memory location. It must appear to the program as if the loads and stores in the program are done in the order they appear.)

With normal memory, it doesn't matter which order loads and stores are performed in, but it certainly does matter for most I/O devices. For this reason, the PowerPC architecture defines the "EIEIO" instruction -- Enforce In-order Execution of I/O (with apologies to Old MacDonald -- ed.) The EIEIO instruction effectively inserts a barrier into the stream of memory accesses and prevents memory accesses from being reordered across that barrier.

Usually a device driver will use an I/O macro such as inb,outb, readl, or writel to access a device's registers. On the PowerPC, these macros are defined to do the access with a load or store, followed by the EIEIO instruction. Therefore, the programmer normally doesn't need to consider the possibility that the CPU may reorder memory accesses. If your device has memory or registers which are accessed directly (e.g. by dereferencing a pointer), you will need to be aware of this possibility. You may need to insert memory barriers at the appropriate points using the mb() or wmb() macro.

Along with not having an ISA bus, the Power Mac architecture doesn't include the Intel-style DMA controller that Intel PCs have. The I/O ASIC in Figure 1 includes a DBDMA (Descriptor-Based DMA) engine for the devices in the ASIC. PCI devices which are capable of bus mastering will have their own DMA engine which is programmed using the device's registers.

The PCI standard defines three address spaces for the PCI bus: memory, I/O, and configuration. PCI memory space is accessed using the read/write* family of macros. To get the address to pass to read/write*, you will need to look at the appropriate base address register in your device's configuration space to get the physical address of your device and then pass that to ioremap() to convert it to a virtual address that you can use with read/write*, as shown in the following code example:

struct pci_dev *pdev;
unsigned long base_phys;
char *base;
int reg_value;

pdev = pci_find_class(MY_DEVICE_CLASS, NULL);
base_phys = pdev->base_address[0];
base = (char *) ioremap(base_phys, 
       MY_DEVICE_SIZE);
reg_value = readl(base + MY_REGISTER_OFFSET);

The in/out* macros take a port number which is an address in PCI I/O space. Most systems only have one PCI host bridge and one PCI bus (or set of buses interconnected with PCI-PCI bridges). On these systems there is no ambiguity. However, a few systems have two or more PCI host bridges, and thus two or more completely separate PCI buses, each with their own distinct I/O space. On these systems, there is, unfortunately, no easy way to access the I/O space of the second or subsequent buses. The in/out*macros will access the I/O space of the primary PCI bus only.

PCI configuration space is accessed using the pci_ read/write_config_* (or pcibios_read/write_ config_*) functions, just the same as on other platforms. (There is no problem with multiple PCI host bridges with these functions.)

An important point to note is that it is an error to access a location in I/O or memory space where there is no device to respond. If you do, it causes a 'machine check' exception. The machine check exception is an imprecise nonrecoverable exception, which means that the processor doesn't guarantee to give you a coherent set of register values after the exception. (Most PowerPC exceptions are precise, recoverable exceptions, which means that the register state is consistent with all the instructions before a certain point having been completed, and none of the instructions after that point having been started. Being able to do this when you discover an exception condition in the middle of executing several instructions atonce, possiblyout of order, is actually a rather clever trick. Because a memory access can take an arbitrary length of time before it returns the error ack signal which causes the machine check, requiring the machine check to be a precise recoverable exception would reduce performance.)

Therefore, on the Power Mac, it is necessary to disable any code which probes for the presence of a device by accessing random I/O ports, where there may or may not be a device present. Usually it is only necessary to probe like this for ISA devices, since PCI devices have the configuration space registers which tell you where the device's registers are. So, it should not be necessary to probe for devices on a Power Mac. Simply enclosing the probing code in an #ifndef__powerpc__ ... #endif block would be a reasonable way to disable the probing code on the Power Mac, for now at least. I would like to see a CONFIG_ISA symbol defined on those systems which have an ISA bus, which can be used to enable probing code.

Similarly, it shouldn't be necessary to probe for IRQs on the Power Mac. The best thing is to use the irq field of the pci_dev struct for the device, rather than reading the PCI_INTERRUPT_LINE register in configuration space, as the latter is sometimes wrong. Another point is that your driver shouldn't assume the system only has 16 IRQs --most Power Macs have 32 or 64, and some have more. (Note that the 16 IRQ line assumption is wrong on modern Intel PCs also.)