DOIRQ, GENIRQ, and CHECKIRQ.
Dale Roberts



This file should help you get the DOIRQ and GENIRQ driver and console 
application running, and guide the curious through the source code.


(1) Support (NOT!)
------------------

This is *UNSUPPORTED* software.  Although I enjoy getting feedback about 
bugs, problems, and, yes, even happy experiences, and I am very happy to 
answer questions or help clarify issues that I may not have made clear 
in the article, I unfortunately do not have spare time to develop custom 
versions of the software.  If you have this file, then you should also 
have the source and executables, since they are together in the same zip 
archive.  This means that you have as much as I do.  Since this is a 
very specific diagnostic tool that highlights a specific bug on a 
specific version of NT using a specific HAL on a specific type of 
motherboard, I have no plans at all to expand or update the software.  
If any major bugs pop up, I'll make every effort to correct them and put 
a fixed version on the DDJ web site, but I do not have the resources to 
provide support directly.


(2) Running it
--------------

You will need a Pentium system with an Intel PCI chipset motherboard. 
The software *may* run on other motherboards as well, but this is all I 
have tried it on.  If you have a different type of computer and the 
program will not run on it, then this would be a great time to read 
through the source code and modify it to work on your system.

To see if your motherboard supports the edge/level registers, first run 
the CHECKIRQ program.  It twiddles the edge/level register bits and 
monitors the activity in the IRR (Interrupt Request Register) to see if 
any interrupts are generated.  If you see IRR bits set in the "after E/L 
SET" column, then you can run the program without modification.  If you 
see IRR bits set only in the "after E/L CLEAR" column, then you probably 
have a 486 system.  See the 486 section below for some tips on modifying 
the programs to work with your motherboard.  Unfortunately, I do not 
have a 486 version of the program myself that I can send to you, nor do 
I ever intend to try to get it working on a 486.

CHECKIRQ also shows you which interrupts you will be able to use for the 
interrupt overlap testing.  Note that CHECKIRQ may report more IRQs than 
DOIRQ will actually claim, since some may be in use by other drivers 
(CHECKISR is not driver-aware).

To run the interrupt timer delay portion of the program, you should make 
sure that GENIRQ is using a serial port that exists on your system.  It 
is set up to use COM2 by default.  If you want to use another COM port 
you must configure GENIRQ for the proper serial port and interrupt and 
recompile it.  There are two #define statements near the top of GENIRQ 
that specify the I/O port and interrupt numbers for the COM port.  You 
can get this information from the Windows NT Diagnostics application in 
the Administrative Tools folder.  Run it and click on the Resources tab.  
At the bottom of the screen, click on the Devices button.  Then double-
click on Serial (if you have done a "net stop serial," the serial driver 
will not appear).  You will see a list of the resources claimed by the 
serial driver.  Take any pair of port addresses and IRQs.  Typical 
addresses/IRQs are COM1: 0x3f8, IRQ4, COM2: 0x2f8, IRQ3.  It doesn't 
matter if the device is a modem or a regular serial port.  Either will 
work fine.  If a device is attatched to the serial port, the only thing 
that will happen is that it will receive a continuous stream of zero-
value bytes.

You need to get the GIVEIO.SYS driver from the May 1996 Dr. Dobb's 
Journal.  The source code and executables can be obtained from the DDJ 
web or FTP site (www.ddj.com).  They *are* there in the 1996/May 
directory.

Use INSTDRV to install the GIVEIO.SYS and DOIRQ.SYS drivers.  See the 
README with the GIVEIO file for how to use INSTDRV (which comes with 
full source code in the DDK), or you can use LOADDRV from the Paula 
Tomlinson article in the May 95 Windows Developer's Journal 
(www.wdj.com).

When using INSTDRV, you must give THE ENTIRE PATH AND SYS FILE NAME on 
the command line.  If you don't do this, you'll get error messages.  It 
should look something like this:

	instdrv doirq c:\foo\bar\irqfun\i386\free\doirq.sys

Of course, the path here is just an example.  Type in the correct one 
for your system.  This will declare the driver in the registry and start 
it up.

To free up some interrupts on your computer, you might want to try doing 
a "net stop serial" on a console text window ("MSDOS prompt") command 
line.  This will stop the serial driver and typically free up two or 
three interrupts.  If you have a serial mouse, it will keep working.  If 
you have a modem, it will stop working.  But you can always do a "net 
start serial" to get your modem back.  If you do "net stop serial" then 
"net start doirq", DOIRQ will take the serial interrupts.  So if you 
then try to do a "net start serial" before you do a "net stop doirq", 
the serial driver will not load because it will not be able to claim the 
serial IRQs because DOIRQ will have them!  If you want start up your 
serial ports, then, you should stop DOIRQ first.

You have to work with whatever interrupts you have available.  Your 
machine might have a different set of interrupts available than mine 
did.  To mimic the results shown in the article, try to keep the 
relative priorities the same, and the 1st/2nd 8259 choice the same.  For 
example, genirq 11 10 7 3 is equivalent to genirq 10 9 5 3 (since 10 and 
9 are both on the 2nd 8259, and are the same relative priorities as 11 
and 10, and 5 and 3 are on the 1st 8259 and are the same relative 
priorities as 7 and 3) or 11 9 7 5 (same reasons).  You get the idea.


(3) Compiling it
----------------

The executables are included in the ZIP archive.  You only need to 
recompile if you want to make changes.

Make sure the source code archive is unzipped with paths!  The 
INTHACK.ASM file must go in the .\i386 subdirectory in order for BUILD 
to find it.  If you have never compiled a device driver before, good 
luck!  You need to have the DDK installed, and use the BUILD utility to 
compile the driver.  Try to get a simple example to compile first.  Once 
you can compile a driver, go into the directory where the DOIRQ source 
is and type BUILD.  It should start building (if you have made any 
changes to the source).

The GENIRQ.C and CHECKIRQ.C programs can be recompiled without the DDK.  
You just need the 32-bit Microsoft C compiler.  I usually use the 
command line compiler to do these simple jobs.  Just type "cl GENIRQ.C" 
or "cl CHECKIRQ.C" to compile them.


(4) The Source Code
------------------

This is a very rough run through the source code.  It is fairly well 
commented, so you can get details by looking in the source code.  I'll 
try to give the "big picture" here, as well as fill in some details not 
mentioned in the comments.

  CHECKIRQ

CHECKIRQ is a good place to start if you want to understand how the 
edge/level registers work to generate interrupts.  This is not a 
documented feature of the Intel PCI chipset.  It is really more of a 
quirk of behavior than an undocumented feature, though.  It should not 
generally be relied on, and, in fact, the behavior is different in the 
486 PCI chipset.  CHECKIRQ requires the GIVEIO driver (from 5/96 DDJ) to 
be installed and running.  The first thing it does is call the GIVEIO 
driver to get access to I/O ports.  It then saves the 8259 IMR and sets 
the first 8259 IMR to 0xff.  This masks off all interrupts to the first 
8259, which is essentially the same as doing a CLI (Clear Interrupt 
Flag) assembler instruction.  Since all of the second 8259 interrupts 
are routed through IRQ2 on the first 8259, there is no need to mask IRQs 
on the second 8259.  We can't do a CLI from an application, which is why 
we use this trick.  Interrupts need to be masked off because we want to 
see the effect of the edge/level bit manipulations on the 8259 IRR 
(Interrupt Request Register), and we don't want NT to jump in and mess 
with the 8259s while we are trying to look at them.

Once interrupts are masked, CHECKIRQ sets up the 8259 so that we can 
read the IRR.  The first port on each 8259 has multiple purposes.  You 
must write a command to it to let it know which register you want to 
read.  Writing 0x0A to the port tells the 8259 that we want to read the 
IRR.  We could also write an 0x0B to read the ISR, but that is not 
necessary in CHECKIRQ (it is used in DOIRQ, however, to trace the state 
of the 8259s).  CHECKIRQ reads the initial state of the IRR, in case any 
interrupts are active.  Any initially active interrupts will be ignored 
in the code that follows.  It is possible for a bit in the IRR to remain 
active yet not cause an interrupt if NT has masked off the interrupt.  
After the initial IRR is read, CHECKIRQ sets the edge/level bits all 
high, then saves the IRR contents again.  It then clears the edge/level 
bits to all zeros and reads IRR yet again.  In each case, CHECKIRQ 
operates on both 8259s, not just one.  It then restores the original 
contents to the edge/level registers and enables interrupts.

We should not do any printing while the interrupts are masked off, so 
the above code just saves the IRR in some local variables.  Once the 
interrupts are enabled, it is safe to call the printf() function.  
CHECKIRQ shows the interrupt mask register, so you can see which 
interrupts are masked off.  It uses the XOR "^" operator to find which 
bits of the IRR have changed from the initial values.  It displays the 
IRR values from after the setting of the edge/level bits, and from after 
the clearing of the edge/level bits, so you can see what interrupts, if 
any, were activated by the manipulation of the edge/level bits.

  DOIRQ/GENIRQ

The DOIRQ kernel-mode driver and the GENIRQ console application are 
designed to work tightly together.  They violate most of the typical NT 
driver/application interface rules, and should not be used as examples 
of how to write good NT programs.  They are intended to be used together 
as a diagnostic tool to probe the operation of the NT kernel and HAL and 
their interactions with the motherboard interrupt hardware, and thus 
must, unfortunately, assume some knowledge of the system on which they 
are run.

The files that make up the driver are DOIRQ.C (the main driver file), 
INTINFO.H (information common to GENIRQ and DOIRQ), and INTHACK.ASM.  
SOURCES and MAKEFILE are used by the BUILD facility to compile the 
driver.  The files that make up the application are GENIRQ.C and 
INTINFO.H.

INTINFO.H contains the data structure that is shared between the 
application and the driver, as well as some other shared information.  
The structure contains the array of interrupts that are to be generated, 
the trace table that is created during the DOIRQ interrupt service 
routine, and the serial port statistics and port/IRQ information.

DOIRQ is started up independently of the GENIRQ application.  When DOIRQ 
starts it manipulates the edge/level registers to see which interrupts 
are active, then it registers the interrupts for its use.  It only uses 
interrupts that no other drivers have claimed.  The SetupAvailableIRQs() 
function is called from the DriverEntry() function, as DOIRQ is loading 
and initializing.  It twiddles the edge/level bits and registers the 
interrupts.  It uses essentially the same technique as CHECKIRQ.  But 
instead of displaying the information, it uses it to figure out which 
interrupts to claim.  It puts the interrupt numbers into a resource 
descriptor structure to submit to NT.  It calls the NT 
IoAssignResources() function to claim the interrupts, then checks to see 
which interrupts were successfully claimed.  Only the successfully 
claimed interrupts are used.  For each interrupt, SetupIRQDefault() is 
called to connect to the interrupt at its "default" priority.  
SetupIRQDefault() calls SetupIRQ() which calls NT's IoConnectInterrupt() 
function to set up the interrupt for operation.  SetupIRQ() passes the 
IRQ number as the "extension" to IoConnectInterrupt().  Normally, the 
extension is used to point to a driver-defined data structure that can 
hold any information that the driver might need about its device.  But 
we only need to remember a single integer, the IRQ number.  So we pass 
this integer directly as the extension, rather than passing a pointer to 
a data structure.  SetupIRQ() then calls SetDirectInt(), which calls 
PokeIDT() to poke the IDT (Interrupt Descriptor Table) to redirect the 
interrupt to initially call a small assembler function that shuts off 
the edge/level bit.  If this is not done, the first interrupt that comes 
along will lock up the system because it will stay continuously active.  
After this initial setup, DOIRQ remains idle until the interrupts start 
coming in.

It is unfortunate that the assembler file was needed.  But, as long as 
the driver is doing all of the other nasty stuff that it does, it might 
as well have an assembler file as well.  The assembler routine is called 
immediately after one of the claimed interrupts is acknowledged.  It is 
the first code that gets called as a result of the interrupt, before NT 
gets control.  The .ASM file contains 16 small mini-handlers, one for 
each IRQ level.  Although we never use all 16 mini-handlers, they must 
be there because we use their order to determine which interrupt was 
invoked.  When the mini-handler calls the main handler below, the main 
handler does some arithmetic to determine which mini-handler called it, 
and, thus, which IRQ caused it to be called.  With this information, it 
can figure out which bit to shut off in the edge/level registers.  All 
of this nonsense is necessary because when an edge/level bit is set high 
to generate an interrupt, the bit will STAY HIGH FOREVER, unless it is 
explicitly cleared.  This would cause the same interrupt to be generated 
continuously, locking up the computer and causing a crash when the stack 
space gets used up.  We can't clear the edge/level bit in the device 
driver interrupt service routine because NT gets control first, clears 
out the In Service Register with an EOI command and re-enables 
interrupts, all *before* it calls the driver's service routine.  So the 
"interrupt loop of death" would occur before DOIRQ's interrupt handler 
is ever called.  Normally, an edge type interrupt is cleared out 
automatically in the 8259 hardware when the processor acknowledges it.  
Since we have a "fake" edge interrupt, we must explicitly remove the 
source of the interrupt before NT gets control.

When GENIRQ starts up and connects to the DOIRQ driver, it first makes a 
DeviceIoControl() call to map the int_info data structure into its 
memory space, and also to possibly boost the priority of one of the 
IRQs.  The DeviceControl() routine checks the parameter passed by 
GENIRQ.  If it is non-zero, it takes the number as the IRQ that should 
receive a priority boost.  It calls SetupIRQBoost() to raise the 
priority of the interrupt to HIGH_LEVEL (the highest available IRQL on 
the system).  SetupIRQBoost() calls SetupIRQ(), which first disconnects 
the interrupt, then reconnects it.  It sets the "sirql" (synchronized 
IRQL) parameter of the IoConnectInterrupt() call to HIGH_LEVEL.  This 
boosts the priority of the interrupt.  It notes which IRQ was boosted in 
the BoostIRQ global variable so that it can be un-boosted when GENIRQ 
closes the DOIRQ driver.  After the BoostIRQ processing, DeviceControl() 
calls MapBuffer() to map the kernel buffer into the GENIRQ application's 
memory space.  Later, just before GENIRQ terminates, it closes the DOIRQ 
driver which unmaps the buffer, and, if necessary, un-boosts an IRQ.  
This is not the generally accepted way to map buffers, so don't take 
this as a good way to do it in a real driver.  Usually, the buffer is 
mapped as the result of a Read() or Write() call, and is only mapped for 
as long as the read or write transaction is in progress.  And usually 
the application buffer is mapped into kernel mode, not vice-versa.

When an interrupt occurs, control is passed to DOIRQ's IRQAllHandler() 
routine.  The same routine is called for all of the interrupts.  When 
the interrupt is called, NT passes two arguments to it: a pointer to the 
interrupt object created by IoConnectInterrupt(), and a pointer to the 
Extension.  You'll remember that we just pass the IRQ number as the 
extension, so that the interrupt routine can tell which IRQ caused it to 
be invoked.  We also could have just checked the IRQL (NT's term for 
Interrupt Request Level; there is a one-to-one correspondence between 
IRQ numbers and a subset of the IRQL designations).  Whenever an 
interrupt service routine is called, NT sets the IRQL to the level that 
matches the particular interrupt line.

In IRQAllHandler(), a trace message is generated indicating that the 
interrupt service routine has been entered.  It then checks to see if 
there are any more interrupts in the queue to be generated.  IntGenIdx 
in the int_info structure is used to point to the next interrupt to be 
generated.  A -1 indicates the end of the list of interrupts.  Since the 
index is global data, and IRQAllHandler() is reentrant, the index must 
be handled carefully.  Specifically, it must be incremented BEFORE any 
interrupt is allowed to be generated, otherwise the interrupt handler 
that is invoked as a result of the interrupt being generated will re-
invoke the same interrupt over again, which would be a bug.  While 
interrupts are disabled, the interrupt is generated by calling GenIRQ() 
which sets the appropriate edge/level bit high.  Since interrupts are 
disabled, the processor is not interrupted yet.  This allows us to enter 
a Trace message showing the effect of the interrupt on the IRR, before 
NT gets control.  After the trace message is entered, interrupts are 
enabled and an interrupt is possibly generated (if it is not explicitly 
masked by the IMR or implicitly masked by the priority levels imposed by 
the ISR - In Service Register).  If an interrupt is actually generated, 
then control passes through NT back to IRQAllHandler(), and the next 
message that the user sees will be Entering IRQ for the generated IRQ 
number.  If no interrupt is generated, then the user will see the 
Leaving IRQ message, or another After GenIRQ message (if there are more 
interrupts to generate).

The Trace8259() function, which is called only from IRQAllHandler(), 
reads the status of both 8259 interrupt controller chips and stores the 
information in the trace entry.  It reads the IRR, ISR, and IMR.  The 
IRR and ISR are accessed through the first 8259 port.  You must first 
tell the 8259 which register you want to read.  To do this, you send 
either an 0x0A (to read the IRR) or an 0x0B (to read the ISR) to the 
first 8259 port.  Then when you read the first port, you will get the 
correct register.  The IMR is always available for reading and writing 
at the second 8259 port.  The processor's interrupt flag (which tells us 
if processor interrupts are enabled) is determined by reading the flags 
register from the stack.  Trace8259() calls AddTrace() which actually 
puts the information into the trace structure and increments the trace 
index.  The trace structure is stored in the shared int_info structure 
so that GENIRQ can dump it for the user.

Once all of the interrupts in the queue have been generated and 
serviced, control passes back to the GENIRQ application which shows the 
trace table to the user.  Then GENIRQ starts up the serial interrupts so 
that we can look at interrupt delays.  In this mode, IRQAllHandler() 
just calls the HandleSerialIRQ(), and the rest of the function is 
unused.  HandleSerialIRQ() puts another byte in the serial output buffer 
to keep the interrupts going.  It then gets the current time stamp using 
the RDTSC() function.  It does a subtraction to calculate the elapsed 
time since the last time it was called.  If this time is greater than 
the current "max" time, the max time is updated.  If the time is greater 
than 400uSec or greater than 1000uSec, the appropriate counter is 
updated.  All of this information is stored in the int_info structure, 
which GENIRQ can access directly and display to the user.


(5) 486 Systems
---------------

The program should work on 486 systems with some modifications.  I began 
development of the program on a 486, but finished up on a Pentium.  I 
chose not to go through the trouble of providing 486 support in the 
program because it would have been a bit of a pain and would have made 
the program more complicated than necessary.  Also, the interrupt delay 
timing is not easily possible because the RDTSC assembler instruction is 
not present on the 486.

If you would like my help in getting the program working on your 486, 
you might want to carefully re-read section (1) again and consider it 
deeply before sending me email on this topic.

First, get rid of the RDTSC function.  It only works on Pentiums.  You 
can make a dummy RDTSC that just returns 0, or you can think up some 
clever way to generate timestamps of your own (maybe an outboard timer 
card, or reading the CMOS clock).

Note that the program WILL compile and run.  It won't crash as long as 
you remove the RDTSC instruction.  But it won't be very useful.  The 
interrupt tracing section of the program will not work, since the method 
of generating an interrupt on the 486 chipset is slightly different.  
The interrupt delay timer function should still cause serial interrupts 
to occur, but it won't be much use since the RDTSC() function can't be 
used, so you won't actually see any times.

The behavior of the edge/level registers on the 486 PCI chipset is 
*different* from the Pentium PCI chipset.  On the Pentium chipset, when 
you set an edge/level bit high (level mode), the interrupt is signaled 
(you can see it in the IRR) and remains signaled until you clear the 
bit.  On the 486 chipset (at least on the one I had), you must set the 
bit high (this does *not* signal an interrupt yet), then back low.  The 
interrupt is signaled in the IRR when the bit goes back low (edge mode).  
The interrupt remains signaled in the IRR until the processor 
acknowledges the interrupt, then the 8259 automatically clears it out 
(as it does with any edge sensitive interrupt).

You can observe this behavior using the CHECKIRQ program.  It shows you 
the state of the IRR after the edge/level bits are set high, then again 
after they are set low.  If your system makes the IRR bits go high after 
the edge/level registers are SET, then you can use the program as-is.  
If the IRR bits go high only after the edge/level registers are cleared 
back to zero, then you will have to modify the program.  If the IRR bits 
are never set at all, then you'll have to figure out another way to 
generate interrupts on your motherboard.

To make the program work on a 486 system you can start by pitching the 
assembler code and all of the functions in the driver that handle the 
direct accessing of the IDT.  Since the edge/level registers in the 486 
PCI chipset allow us to cause an *edge* interrupt, we don't need to 
clear out the bit during the servicing of the interrupt (which was the 
only reason for poking the IDT and pointing it to the assembler code).  
The IRR bit will be cleared automatically when the interrupt is 
serviced.  You also need to change the functions in GENIRQ and DOIRQ 
which manipulate the edge/level registers to generate interrupts, and 
the function in DOIRQ that detects which interrupts are active.  The 
change is very simple.  Just set the bit high, *then low* (with the 
interrupts disabled all the while) and the interrupt will be generated.  
That's it!  You can see how the CHECKIRQ program does it and copy its 
example.  Just do another set of _oupt()'s to the edge/level registers 
with an "& ~()" rather than an "|".

Presumably, Intel changed the behavior between the 486 and Pentium 
chipsets to make things easier when manipulating the registers.  In the 
486, you couldn't help but set off an interrupt whenever you switched 
modes from level to edge, and it could only be cleared by servicing the 
interrupt.  Although this is *easier* for the purpose of the diagnostic 
tool, it is a pain for an OS programmer.

