NDB realtime options, choosing CPU and balancing interrupts

With the NDB realtime options, you can choose on which CPU the execution thread and the maintenance will be running. The point is, which CPUs to use.

The output of “cat /proc/interrupts” will help you determine which CPU to use. Here is an example of a dual quad-cores box:

$ cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7
  0: 2496072738          0          0          0          0          0          0          0    IO-APIC-edge  timer
  1:          3          0          0          0          0          0          0          0    IO-APIC-edge  i8042
  8:          1          0          0          0          0          0          0          0    IO-APIC-edge  rtc
  9:          0          0          0          0          0          0          0          0   IO-APIC-level  acpi
 12:          4          0          0          0          0          0          0          0    IO-APIC-edge  i8042
 14:        164    2889101   19195871     427134          0          0          0          0    IO-APIC-edge  ide0
 82:          0          0          0          0          0          0          0          0   IO-APIC-level  uhci_hcd:usb5
 90:        111    2419338   13748190    1148654          0          0          0          0   IO-APIC-level  uhci_hcd:usb6, hpqci
114:      13696   14929866   68053573        488          0       5138          0          0       PCI-MSI-X  cciss0
130:        132         21         23          0          0          0          0          0         PCI-MSI  qla2xxx
138:         96          0         58         21          0          0          0          0         PCI-MSI  qla2xxx
146: 3696757526          0          0          0          0          0          0          0         PCI-MSI  eth0
154:        113     150490     925101       2591      15200        105     152486          0         PCI-MSI  eth2
162:       1955    6540344   15094832    1555921          0          0          0          0   IO-APIC-level  ipmi_si
169:          0          0          0          0          0          0          0          0   IO-APIC-level  ehci_hcd:usb1, uhci_hcd:usb2
170:         36     134796     919147         15          0      18345       5252        120         PCI-MSI  eth3
177:          0          0          0          0          0          0          0          0   IO-APIC-level  uhci_hcd:usb3
178:    6507410          0          0          0          0          0          0          0         PCI-MSI  eth1
185:          0          0          0          0          0          0          0          0   IO-APIC-level  uhci_hcd:usb4
NMI:     242968      27253      29441      27173      75014      41090      41124      51655
LOC: 2496025119 2496047156 2496046223 2496046555 2496024866 2496046857 2496045965 2496046293
ERR:          0
MIS:          0

We can observe that CPU 6 and 7 are less busy servicing interrupts then the other but are still used for eth2 and eth3. Let’s say we want the NDB threads to run on these 2 CPUs and get rid of the interrupts. The Linux kernel allows to mask CPUs to interrupts. Considering the output above, eth2 is on IRQ 154 and eth3 is on IRQ 170. A bitmask of 6 “1”s equals 1 + 2 + 4 +8 + 16 + 32 = 63 so

echo 63 > /proc/irq/154/smp_affinity
echo 63 > /proc/irq/170/smp_affinity

should do it. Of course some scritpting will be needed to set these values at startup but the big picture is there. By the way, don’t assume the interrupts will always be the same.

About Yves Trudeau

I work as a senior consultant in the MySQL professional services team at Sun. My main areas of expertise are DRBD/Heartbeat and NDB Cluster. I am also involved in the WaffleGrid project.
This entry was posted in HA, mysql, NDB Cluster, yves. Bookmark the permalink.

Comments are closed.