[Compuware Corporation] [Compuware NuMega home page]                [NuMega Lab]
[teal]

 [DriverStudio]    [Image][Image]
   Home                       [Image]      Canceling IRPs Waiting for Controller
 [Driver Products]                          Objects
   DriverStudio
   DriverBundle          This technical tip started out a lot more attractive, but
   Previews              as with most things involving cancel routines in Windows
   Compatibility         NT kernel mode drivers, it got a little ugly. The
 [Downloads]             approach presented here is viable, but its complexity
                          would most likely make it appropriate only in
 Wizards                  circumstances that demanded it. In laying out its
   Utilities             solution, however, this tip does discuss some interesting
   NT source             aspects of controller objects and cancel routines.
 examples
   VxD source            Controller objects can be powerful tools when designing
 examples                 Windows NT kernel mode drivers. They offer the ability to
   WDM source            synchronize driver resources between different device
 examples                 objects within the driver. To this end, each device
 [Resources]             object can queue a request on the controller object using
 Technical papers         the function IoAllocateController. When the controller
   Useful links          becomes available to service the needs of that device, a
   Technical tips        callback routine will be invoked. At this point, the
 [Support]               device that receives control owns the resource and can
                          perform its work. If this can be done within the context
 Support                  of the callback routine, the device can return
   Knowledge base        DeallocateObject, allowing the controller to go on to
   Problem               service other requests, or it can return KeepObject, and
 submission               at some later time call the function IoFreeController.
   Product
 registration             Typically, there may be several devices in a driver that
   Release notes         each contend for the resources managed by the controller
 [Shop NuMega]           object. For example, a piece of hardware may contain two
 Buy it!                  separate subsystems, but have only one interface that
   Price list            must be shared between them for control. Architecturally,
   How to buy            it may make sense to model each subsystem as its own
   Sales offices         device. The shared interface could be managed with a
                          controller object, allowing the two devices to exist
                          independently of one another.
 [Y2K Compliance]
                          It is standard practice for any IRP that is placed in a
                          pending state on a device, to provide a cancel routine
 [More information]       and make the IRP cancelable prior to entering the pending
                          state. The most common example of this occurs when an IRP
                          must be serialized, and is placed on a device queue to
                          await its turn to be processed on the device. By placing
                          the IRP into a cancelable state, the driver's
                          responsiveness and flexibility are enhanced. If the
                          issuer of the IRP determines that the IRP is no longer
                          necessary, for instance the process that issued the IRP
                          has been terminated, the IRP can be quickly disposed of
                          on the device. In the case of a device queue, the IRP can
                          be removed from the queue if it hasn't started on the
                          device yet, and completed in a canceled state. In the
                          above example of a terminating process, once all of the
                          outstanding IRPs have been completed or canceled, the
                          file handle to the device can be closed, and the process
                          termination can continue to completion.

                          Waiting on a controller object to process an IRP is in a
                          sense a pending state, for which one may consider trying
                          to employ a cancel routine and making the IRP cancelable
                          while waiting for the controller access to be granted. Of
                          course this is not required, and the driver could take
                          the approach that once a request has been queued to the
                          controller object, it has been started on the device and
                          cannot be canceled. This could pose a problem, however,
                          if some device holds onto the controller object for a
                          relatively long time, and outstanding IRPs from other
                          devices that have queued requests to the controller
                          cannot be canceled.

                          When attempting to make an IRP that is waiting on a
                          controller object cancelable, there are several potential
                          pitfalls that must be avoided. To understand what they
                          are, one must first understand what happens when a
                          request is queued on a controller object. Performing this
                          action is done using the function IoAllocateController,
                          which has the prototype:

                          VOID
                          IoAllocateController(
                          IN PCONTROLLER_OBJECT ControllerObject,
                          IN PDEVICE_OBJECT DeviceObject,
                          IN PDRIVER_CONTROL ExecutionRoutine,
                          IN PVOID Context
                          );

                          Notice that the function takes a pointer to the device
                          object, which is queued on the controller object. What
                          this actually means is important to understand. Within
                          the call to IoAllocateController, the system takes the
                          KDEVICE_QUEUE_ENTRY WaitQueueEntry member, located in the
                          WAIT_CONTEXT_BLOCK structure, which is part of the
                          DEVICE_OBJECT structure (see ntddk.h), and queues it to
                          the KDEVICE_QUEUE DeviceWaitQueue member located in the
                          controller's CONTROLLER_OBJECT structure.

                          An important point to note is that the DEVICE_OBJECT only
                          contains one such entry, meaning that it can only be
                          placed on the list once, otherwise the list will become
                          corrupted. Furthermore, a device object can only be
                          queued on a single list at any given time. Also note that
                          this same technique is employed in the operation of
                          DMA_ADAPTER objects, which make use of the same list
                          entry in the device object.

                          Because of the singular list entry, device objects must
                          serialize their attempts to queue themselves on
                          controller (or DMA adapter) objects. Thus, it is
                          important in any attempt to cancel an IRP that is pending
                          on an outstanding queued request, to try and remove the
                          request from the controller's queue. This must be done
                          atomically with respect to the system, which will also
                          try to remove the request from the queue for processing
                          on the device. This can be done using the function
                          KeRemoveEntryDeviceQueue, which has the prototype:

                          BOOLEAN
                          KeRemoveEntryDeviceQueue(
                          IN PKDEVICE_QUEUE DeviceQueue,
                          IN PKDEVICE_QUEUE_ENTRY DeviceQueueEntry
                          );

                          For the parameter DeviceQueue, one would use

                                  &(pController->DeviceWaitQueue)


                          and for DeviceQueueEntry, one would use

                                  &(pDeviceObject->Queue.Wcb.WaitQueueEntry)


                          If the device object cannot be successfully removed from
                          the controller's device queue, the IRP cannot be safely
                          canceled in the cancel routine, since starting the next
                          IRP on the device may try to queue the device again on
                          the controller object, thus corrupting the list.
                          Actually, most sane design methodologies would indicate
                          that an outstanding request on a device must be fully
                          resolved, and the device state known, before the IRP can
                          be completed or canceled. This ensures that there will be
                          no problems in either starting the next IRP, or with
                          extraneous processing occurring after the cancelled IRP
                          has been completed, which can be undesirable.

                          The following is pseudo-code for the steps that must be
                          taken in the various routines to make an IRP that is
                          pending on a controller object cancelable. The main
                          details are presented, but there are several "routine"
                          details that are omitted for clarity sake. The
                          DriverWorks controller example CONTROLR has been updated
                          using the same logic presented here. These fragments
                          assume that the IRP has simply been serialized prior to
                          attempting to gain control of the resources managed by
                          the controller, and that the device state has not been
                          altered by the IRP in any other way that needs to be
                          corrected in the cancel routine.

                          In StartIo routine:

                               Remove the IRP from the cancelable state it was
                               in while queued, testing to see if the IRP was
                               canceled, and dealing with it appropriately if
                               it was.

                               Enter QueueSync spinlock, a private spinlock
                               created for the sole purpose of allowing the
                               cancel routine to be set and the request queued
                               to the controller atomically, with no chance
                               that the cancel routine will proceed before the
                               request is queued.

                               Place the IRP into a cancelable state, using
                               the CancelInProgress routine as its cancel
                               routine, while testing to see if the IRP was
                               canceled, and dealing with it appropriately if
                               it was.

                               Queue our request on the controller object.

                               Leave the QueueSync spinlock.

                          In CancelInProgress routine:

                               Release global cancel spinlock.

                               Enter QueueSync spinlock.

                               Leave QueueSync spinlock. Now it is assured
                               that the request has been queued.

                               Try to remove the device's KDEVICE_QUEUE_ENTRY
                               from the controller's KDEVICE_QUEUE.

                               If successful, complete the IRP in the canceled
                               state, and start the next IRP on the device.

                               If not successful, signal an event,
                               CancelComplete. This will indicate to the
                               callback routine that the cancel routine has
                               finished executing. Note in this case, the IRP
                               is not completed, nor the next IRP started from
                               the cancel routine.

                          In ControllerCallback routine:

                               Remove IRP from the cancelable state it was in
                               while queued on the controller object, testing
                               to see if the IRP has been canceled.

                               If the IRP has been canceled, queue a work item
                               WaitForCancel and deallocate the controller
                               object.

                               If the IRP has not been canceled, proceed with
                               the work that the device must do to satisfy the
                               IRP, keeping the controller object until that
                               work is done and then releasing it.

                          In WaitForCancel routine:

                               Wait on the CancelComplete event.

                               Complete the IRP in the canceled state, and
                               start the next IRP on the device.

                          There are several interesting points to discuss
                          concerning the above logic. Most of these points involve
                          tricky timing issues that come to light when analyzing
                          the task of synchronizing two asynchronous threads of
                          execution, namely the cancel thread and the controller
                          callback thread. This endeavor leads to most of the
                          complexity shown in the pseudocode.

                          First, consider the spinlock QueueSync, which is used in
                          the StartIo and CancelInProgress routines. This spinlock
                          is necessary, since without it the cancel routine could
                          try to dequeue the device object from the controller's
                          queue before it has been queued, in which case it will
                          not find the request. This would result in queuing a
                          canceled request to the controller. Ultimately, this
                          situation would be resolved when the request is dequeued
                          by the system, but it will not happen in a timely manner,
                          which is the intent in the first place. One very
                          important point to note, with regards to this spinlock,
                          is that the ControllerCallback routine may be called with
                          the spinlock held if the system finds the controller is
                          not busy and calls the routine inline to the queuing
                          request on the controller. This is not a problem so long
                          as the callback routine does not try to reacquire the
                          spinlock, which would result in a deadlock.

                          Next, consider an IRP that is canceled and the possible
                          ordering of events. The IRP may be canceled while the
                          request is still queued on the controller object. In this
                          case the cancel routine will be able to successfully
                          remove it from the queue, complete it as canceled, and
                          start the next IRP on the device.

                          Another possibility is that the system dequeues the
                          request and calls the ControllerCallback routine before
                          the IRP is canceled. If the IRP is canceled after the
                          callback routine succeeds in making the IRP not
                          cancelable, the system will not call the cancel routine,
                          and the IRP will complete normally when processing is
                          completed on the device. Note that it is not compulsory
                          to complete an IRP that has been canceled with a canceled
                          status. Instead, that IRP should simply be completed
                          soon.

                          A final possibility is that the system dequeues the
                          request just as the cancel routine is being run. In this
                          case, both the cancel routine and the callback routine
                          will execute. When the cancel routine runs, it will not
                          find the request queued on the controller object. Instead
                          of completing the IRP and starting the next IRP, the
                          cancel routine will signal the CancelComplete event
                          indicating it has run. When the callback routine runs, it
                          will detect that the IRP has been canceled, and it will
                          queue a work item to wait for the CancelComplete event.
                          The reason a work item must be used is that the callback
                          routine may be running at DEVICE IRQL, precluding the
                          ability to wait. The reason to wait on the event in the
                          WaitForCancel routine is to ensure that both the cancel
                          and callback processing have been completed, before
                          completing this IRP or starting the next IRP on the
                          device. This ensures that our device state is fully
                          resolved before the next IRP is started.

                          The ability to cancel pending requests on controller
                          objects may not be required for most driver designs.
                          Certainly, any design in which requests will not remain
                          pending for very long before being serviced will not
                          benefit greatly from the added complexity. In a small
                          subset of designs, however, this ability might be
                          critical. It is somewhat unfortunate, therefore, that the
                          underlying mechanics are somewhat hidden in the call to
                          the IoAllocateController routine, since it forces our
                          solution to rely on that mechanism remaining constant.
                          While it is unlikely that this mechanism will change, the
                          fact remains that its design makes use of the driver's
                          device object in a semi-opaque manner, instead of opening
                          up the interface by exposing the underlying device queue
                          mechanism. Indeed, the design could have been made much
                          more generalized by the inclusion of a parameter allowing
                          the KDEVICE_QUEUE_ENTRY to be specified in the call to
                          IoAllocateController, eliminating the dependence on the
                          singular instance enmeshed in the driver's device object.

  DriverCentral  DriverStudio  Free downloads  Resources  Support and
                          Services  Shop NuMega
     Compuware NuMega  Tel: +1 603 578-8400  Updated: 9 August 1999 
                      Problems? Contact our webmaster.
