[NTExport logo] Copyright © 1997 Mark Russinovich and Bryce Cogswell last updated April 13, 1997 Access to any function or variable in NTOSKRNL Introduction Device driver writers have limited visibility into the functions and variables inside of NTOSKRNL, the core of the NT operating system. For instance, only a fraction of the NtXxxx and ZwXxxx calls are exported for use by a driver. There seems to be little rhyme or reason behind the selections Microsoft made. There are even a significant number of functions that are available from a user-mode program, but not a driver (see NtSetTimerResolution). Virtually none of the global variables in NTOSRKNL can be referenced from outside the kernel. Why is this important? Dozens of globals affect the tuning of a variety of critical NT subsystems including the Memory Manager, Cache Manager and scheduler. Many times these variables are hard-coded values based on the system memory size or whether the platform is a workstation or server. What is unclear is how these hard-coded values were obtained and in what environments they yield good performance. If NT provided some degree of control over these values, then falling back on reasonable defaults would be acceptable. However, in most cases NT provides no control over the internal tuning parameters, and in cases where it does, the control is at a very gross level. For instance, the only influence an administrator has over the system cache is to indicate that it should either be Large or Small (see NT Tips and Trivia for information on this). NTExport is a tool that generates a library that makes access to any variable or function in NTOSKRNL possible. A driver can then manipulate desired parameters or call functions that it requires. To use it, you must first determine which functions or variables you want access to. Then you must create intermediate files for each version of NT you want your driver to support, and then have NTExport generate an export library that you link with your driver. The rest of this documentation shows how to use NTExport in detail. Cacheman is a utility that we've written to accompany NTExport which allows you to tweak many of the Cache Manager's tuning parameters, and is used as an example. Note: This release of NTExport only supports NT 4.0 x86 versions of NT, and does not support checked builds. You should only use unexported functions and variables when absolutely required by your application. See the warning at the end of this page for more information. DBG files NTExport obtains information about function and variable addresses by combining information in a particular NTOSKRNL executable image and its corresponding Debug symbol file (.DBG file). Therefore, you must have access to both of these for each version (Service Pack) of NT you want your driver to support. Before using NTExport you should locate the .EXE's and .DBG files for all Service Packs that have been released. This includes NTOSKRNL.EXE, NTOSKRNL.DBG, NTKRNLMP.EXE and NTKRNLMP.DBG files. The .DBG files can be found under \support\debug\i386\symbols\exe (on the distribution CD). Specifying The first step in using NTExport is to determine which functions Things to or variables you want the generated library to export. The -l Export (list) option is used to direct NTExport to dump the names and sections of all unexported symbols in a particular Service Pack. To direct NTExport to the location of a .DBG file (each SP must have its .EXE and .DBG files in a separate directory to avoid name conflicts) use the -d switch. The -e switch points it at the executable. Below is an excerpt of a dump of unexported symbols in Service Pack 2 of NT 4.0. C:\>ntexport -l -d \nt40sp2 -e \nt40sp2 NTExport V1.0 Copyright (C) 1997 Mark Russinovich and Bryce Cogswell http://www.ntinternals.com Unexported symbols: .text: main .text: CcCopyReadExceptionFilter: 2 parameters .text: CcPostDeferredWrites: 0 parameters .text: CcPinFileData: 9 parameters .text: CcUnpinFileData: 3 parameters .text: CcPerformReadAhead: 1 parameters .text: CcSetDirtyInMask: 3 parameters .text: CcSetValidData: 2 parameters .text: CcAcquireByteRangeForWrite: 6 parameters .text: CcReleaseByteRangeFromWrite: 5 parameters … .data: KeFeatureBits .data: KeTimeSynchronization .data: KiBugCodeMessages .data: KiMaximumSearchCount .data: KiFindFirstSetRight .data: KiFindFirstSetLeft .data: KiFindLeftNibbleBitTable .data: KiProfileInterval .data: KiProfileIrql .data: ProfileCount .data: OpcodeIndex … Functions are listed with information about how many parameters they take, whereas variables have no parameter information. Any symbols that you want NTExport to export you must place in a file called exports.lst. The exports.lst file for Cacheman is listed below: CcDirtyPageThreshold CcDirtyPageTarget CcMaxDirtyWrite CcAvailablePagesThreshold CcFirstDelay CcIdleDelay CcCollisionDelay CcTargetCleanDelay MmSystemCacheWs MmWorkingSetReductionMaxCacheWs MmWorkingSetReductionMinCacheWs MmWorkingSetVolReductionMaxCacheWs MiDoPeriodicAgressiveTrimming MmPeriodicAgressiveTrimMinFree MmPeriodicAgressiveTrimMaxFree MmPeriodicAgressiveCacheWsMin You must be careful to type the names exactly as they are shown in the listing output because NTExport will not complain if it does not find an entry in the unexported symbols across Service Packs. Processing Now you are ready to process each Service Pack that you want Versions support for. If you are building a retail product you must process all released versions, but if you are just exploring possibilities or debugging you can just process the particular version you are running on your machine. NTExport requires you to tell it which Service Pack it is processing and where the .EXE and .DBG files are for the pack. In addition, you must tell it whether to process the uniprocessor or multiprocessor build of the pack. -m is used to tell NTExport to process NTKRNLMP (the multiprocessor build). Below are the commands that were entered to process Final Release (uniprocessor and multiprocessor) for Cacheman. Note that Final Release (e.g. no Service Pack) is specified as Service Pack 0. C:\>ntexport -sp0 -d \nt40 -e \nt40 NTExport V1.0 Copyright (C) 1997 Mark Russinovich and Bryce Cogswell http://www.ntinternals.com Creating unexported symbol information for SP 0, Uniprocessor Build. Processing symbol information...done. Producing export information...done. C:\>ntexport -sp0 -m -d \nt40 -e \nt40 NTExport V1.0 Copyright (C) 1997 Mark Russinovich and Bryce Cogswell http://www.ntinternals.com Creating unexported symbol information for SP 0, Multiprocessor Build. Processing symbol information...done. Producing export information...done. With each invocation NTExport generates two new files and updates two files that are global across invocations. The global files are ntfuncs.txt and ntvars.txt. With these files NTExport keeps track of what symbols have been exported and the ordering it has assigned to them. The two files unique to a Service Pack version are named funcSPxb.txt and varSPxb.txt, where x is the Service Pack number and b is either 'u' for a uniprocessor build or 'm' for a multiprocessor build. For example, the files that are generated after the two above NTExport commands are: ntfuncs.txt ntvars.txt varsp0u.txt funcsp0u.txt varsp0m.txt funcsp0m.txt All of these files are for NTExport's internal use. Generating a After processing the desired Service Pack versions, you will be Library ready to direct NTExport to generate a kernel-mode driver library. This is accomplished with the following command, executed in the directory containing all the intermediate files: C:\>ntexport -g NTExport V1.0 Copyright (C) 1997 Mark Russinovich and Bryce Cogswell http://www.ntinternals.com Reading export name tables...done. Reading export information for SP 0, Uniprocessor build...done. Reading export information for SP 0, Multiprocessor build...done. Reading export information for SP 1, Uniprocessor build...done. Reading export information for SP 1, Multiprocessor build...done. Generating export library source code...done. Build the EXPORT library using the NT DDK build utility. This command generates the following files: * Makefile - this is a standard NT build utiltity makefile * Sources - this is a standard NT build utility sources file that directs the build tool to create a library * export.c - this is the source code module that contains the code that dynamically exports the correct addresses for each symbol that the library will export. The determination is based on the Service Pack number and whether the build is uniprocessor or multiprocessor. The library uses the Registry to determine this at run-time. The library will be in \lib\i386\free or \lib\i386\checked depending on whether you are in the free or checked build environments. To actually build the export library use the NT DDK BUILD utility as you would for a regular kernel-mode driver. When the build is complete a library called export.lib will exist that can be linked with a driver that will use the exported variables or functions. If you want the library to have a different name than the default of export.lib, use the -n switch like this: C:\>ntexport -g -n explib This command directs NTExport to create a library called explib.lib. You can obtain a complete description of NTExport options by using the -? switch. Linking Your You are finally ready to use the functions and variables that the Driver with export library makes available. You must declare prototypes for The Library the functions you are using in your driver. Variables will be indirect - that is, you must dereference them to modify or read the actual values. For example, to read CcDirtyPageThreshold you would use a line like: dirtythreshold = *CcDirtyPageThreshold; You must also declare the variables as the appropriate external types. CcDirtyPageThreshold is a ULONG, so the correct declaration for it in your driver is: extern PULONG CcDirtyPageThreshold; Before using the library you must have your driver call ExportInit(). This function initializes the export library. If ExportInit returns STATUS_SUCCESS the library supports the Service Pack version that the driver is running on. If it returns STATUS_NOT_IMPLEMENTED, the Service Pack is not supported and your driver should not attempt to use the exported variables since they will not be valid. To link your driver with the library, add the following line to your driver's SOURCES file: TARGETLIBS=$(BASEDIR)\lib\*\$(DDKBUILDENV)\export.lib When a new Service Pack is released, an updated library and driver can be created within seconds just by processing the new .DBG and .EXE files and generating new code. The Cacheman program is an accompanying example of how to use NTExport libraries. It relies on several undocumented and unexported Cache Manager tuning parameters to modify the sizing and behavior of the system Cache. Conclusion The power that NTExport gives you to call on any function or and Warning access any variable or structure inside NT is unprecedented. The door to advanced system tuning utilties has been opened, and your driver will be able to use all the functionality that NT itself uses. The Cacheman example shows just how useful this power can be. The use of NTExport does not come without its price, though. This technique is inappropriate for general use or in commercial products. A driver that uses NTExport libraries will be tied to the Service Packs that the library was created for. While updates are easy and fast, users will have to obtain updates. In addition, since the variables and functions that NTExport makes available are officially undocumented, you should only use them when absolutely necessary. Microsoft has the right to change both their names and functionality without notice. ---------------------------------------------------------------------------- Download NTExport (36KB) [Image]