From:	CRDGW2::CRDGW2::MRGATE::"SMTP::CRVAX.SRI.COM::RELAY-INFO-VAX"  3-MAR-1992 23:02:46.48
To:	ARISIA::EVERHART
CC:	
Subj:	MWAIT V1.06, part 4 of 7

From:	RELAY-INFO-VAX@CRVAX.SRI.COM@SMTP@CRDGW2
To:	Everhart@Arisia@MRGATE

Received:  by crdgw1.ge.com (5.57/GE 1.123)
	 id AA19429; Tue, 3 Mar 92 22:13:32 EST
Message-Id:  <9203040313.AA19429@crdgw1.ge.com>
Received: From SUN2.NSFNET-RELAY.AC.UK by CRVAX.SRI.COM with TCP; Tue,  3 MAR 92 17:31:46 PST
Received: from ncdlab.ulcc.ac.uk by sun2.nsfnet-relay.ac.uk via JANET 
          with NIFTP id <3509-6@sun2.nsfnet-relay.ac.uk>;
          Tue, 3 Mar 1992 08:48:10 +0000
Date: Tue, 3 Mar 92 8:26 GMT
From: Nick de Smith <NICK@ncdlab.ulcc.ac.uk>
To: INFO-VAX <@nsfnet-relay.ac.uk:INFO-VAX@KL.SRI.com>
Subject: MWAIT V1.06, part 4 of 7

-+-+-+-+-+-+-+-+ START OF PART 4 -+-+-+-+-+-+-+-+
X;printf`09<CHMK vector base: !XL>, r11
X`09bisl`09#MWAIT_M_CHMK, MWAIT_L_FLAGS(r6) ; Say we found the CHMK vector
X`09brb`0940$
X27$:`09addl2`09#4, r11`09`09`09`09; No match, try next longword
X`09brb`0925$
X30$:`09printf`09<%MWAIT-W, CHMK vector not found>
X
X40$:`09movl`09LDRIMG$L_NONPAG_R_BASE(r10), r11
X45$:`09cmpl`09r11, r9`09`09`09`09; At end yet?
X`09bgtru`0950$`09`09`09`09; Branch if yes
X`09cmpl`09EXEC_CHECK, (r11)`09`09; Check that its the right vector
X`09bnequ`0947$
X`09cmpl`09EXEC_CHECK+4, 4(r11)
X`09bnequ`0947$
X`09cmpl`09EXEC_CHECK+8, 8(r11)
X`09bnequ`0947$
X`09movl`09r11, CMOD$AR_EXEC_DISPATCH_VECTOR
X;printf`09<CHME vector base: !XL>, r11
X`09bisl`09#MWAIT_M_CHME, MWAIT_L_FLAGS(r6) ; Say we found the CHME vector
X`09brb`09100$
X47$:`09addl2`09#4, r11`09`09`09`09; No match, try next longword
X`09brb`0945$
X50$:`09printf`09<%MWAIT-W, CHME vector not found>
X
X100$:`09rsb
X
X`09.disable`09local_block
X
X`09.end`09MWAIT
$ CALL UNPACK MWAIT.MAR;1 410399373
$ create 'f'
XMWAIT.MAR
XAuthor`09      :`09Nick de Smith`09NICK@NCDLAB.ULCC.AC.UK
XCreation date :`0909-Dec-91
X
XEdit`09Edit date`09By`09Why
X 06`0907-Feb-92`09NMdS`09More cosmetic changes.
X`09`09`09`09Modify PC resolution for RWAST processes to
X`09`09`09`09include all loaded device drivers.
X 05`0914-Jan-92`09NMdS`09Add in lots of detail on process and AST status,
X`09`09`09`09especially for JIB related waits. Some minor
X`09`09`09`09corrections and additions to the text. Make code
X`09`09`09`09work for all versions of VMS V5.x.
X 04`0909-Jan-92`09NMdS`09Add in finer detail for RWAST messages to
X`09`09`09`09save time with SDA.
X 03`0908-Jan-92`09NMdS`09Add in RWMBX example, messages and
X`09`09`09`09documentation. Add in more detail on general
X`09`09`09`09resource waits.
X 02`0923-Dec-91`09NMdS`09Add in extra detail to reflect changes in main
X`09`09`09`09code. Enhance messaging considerably. Update
X`09`09`09`09examples.
X 01`0909-Dec-91`09NMdS`09New. Generated from the VMS listings CDs and
X`09`09`09`09*much* other research.
X`09`09`09`09Also input from the Internals and Data
X`09`09`09`09Structures (V5.2) manual and from various DEC
X`09`09`09`09CSC/DECtel articles.
X
XThis version of MWAIT (1.06) uses no DEC copyright code.
X
XMWAIT is a utility that attempts to determine the reason for a process going
Xinto a wait state. Detail is given about LEF/CEF wait state, including the n
Vames
Xof common event blocks being used. The length of time that the process has b
Veen
Xin a wait state is also displayed, as is a detailed breakdown of MWAIT
Xreasons, in particular RWAST, RWMBX and job quota waits. You will need VMS V
V5.x
Xand CMEXEC to run MWAIT. MWAIT is written in MACRO32 and requires no layered
Xproducts.
X
XMWAIT is an organic piece of code, and is growing as new methods of examinin
Vg
Xthis problem are found. Any comments regarding the usefulness or otherwise o
Vf
XMWAIT would be appreciated. Comments regarding accuracy of reporting and
Xsuggestions for enhancements are especially welcome.
X
XMWAIT's documentation details all the known VMS wait states and in many case
Vs
Xsuggests methods to correct the problem (if, indeed, there is a problem).
X
XIf a process is in MWAIT, MWAIT checks the reason in PCB$L_EFWM. If this is
Xstandard (non-mutex) resource wait state, the reason is given. If its a mute
Vx
Xwait, the name of the mutex is determined and displayed. If the wait is job
Xquota related, the quotas are checked for exhaustion. If the detail of the M
VWAIT
Xstate cannot be determined, "MWAIT?" is displayed and the wait mask will
Xcontain the address of the unknown mutex.
X
XIf the state is RWAST, then MWAIT attempts to determine the system routine t
Vhat
Xcaused the wait, and displays all the associated causes. If the process head
Ver
Xof the target process is resident, the registers are displayed and MWAIT wil
Vl
Xattempt to resolve the PC as a system routine and will display a message to
Xenable you to precisely identify the cause of the problem. You may need to u
Vse
XSDA to get extra fine detail, but MWAIT does all the donkey work. MWAIT's PC
Xresolution attempts to locate executive code threads from system service and
Xbase image offsets by using the change mode vectors. This method may be (is)
Xdependent on VMS versions as the two offsets used may change. As a result, y
Vou
Xshould check with each release of VMS that MWAIT is giving sensible results.
V If
Xthis is a problem for you, contact me and ask. Note that MWAIT checks that t
Vhe
Xdata it is using 'looks like' the correct vectors, and will report if there
V is a
Xmis-match.
X
XIf the state is RWMBX, then MWAIT will attempt to determine the device causi
Vng
Xthe problem, together with the size of the write request that is blocking th
Ve
Xprocess and the device's buffer quotas and channel.
X
XIf the process is in a job pooled quota related wait, MWAIT will determine t
Vhe
Xquota that is causing the problem.
X
XMWAIT does not use $GETJPI etc. to return information on processes - it goes
Xstraight to the executive data structures without using any locking. This me
Vthod
Xensures that information can be returned whatever the state of the target
Xprocess, that it is 99.9% accurate, and gathered in an non-intrusive manner.
XThis also means that there is a finite (albeit very small) possibility of MW
VAIT
Xgetting an ACCVIO due to a data structure or pointer changing while being
Xinspected. This is *not* a problem - it will not harm your system so just re
V-run
XMWAIT.
X
X
XUsage
X-----
X`09$ @BUILD`09`09! Build MWAIT from sources
X
X`09$ MWAIT == "$dev:`5Bdir`5DMWAIT"
X`09$ MWAIT pid
Xor`09$ MWAIT
X`09_Target PID: pid
X`09...
X`09$
X
XIf the "pid" is specified as 0 or omitted, all processes are displayed.
X
XTypical MWAIT output from a normal process
X------------------------------------------
X
X`09$ MWAIT 20A0008E
X`09  Pid    User name    Process name    Prior  State  Port
X`0920A0008E SYSTEM       IPCACP          10/8   Hib    -Det- `20
X`09`09Wait mask: 7FFFFFFF, Cluster: 0, Time:    0 00:00:04.00
X`09`090: E0000001, 1: 00000000
X`09`092: 00000000 (`5BSYSTEM`5DIPCACP_FLAGS)
X`09`09Mode: Current: KERNEL, Previous: USER
X`09`09AST status: Active: (none), Enabled: KESU
X`09$!
X
XField`09`09Use
X-------------`09---------------
XPid`09`09Extended Process ID of the process being inspected
XUser name`09VMS username of process
XProcess name`09VMS process name of process
XPrior`09`09Current/Base priority of process
XState`09`09Process state - see below
XPort`09`09Control device for process (LTAn:/Det/Sub/Net/Bat etc)
XWait mask`09If the process state is LEF,LEFO or CEF, the wait mask
X`09`09has one bit SET for each event flag being waited for. If
X`09`09the text "(all EFNs)" appears, then ALL the EFNs in the mask
X`09`09must be set for the wait condition to be satisfied, else the
X`09`09setting of ANY of the EFNs will satisfy the condition.
X`09`09If the process is in an miscellaneous or JIB related wait state,
X`09`09then the wait mask contains detail of the state. The only case
X`09`09(excluding a LEF or CEF state) where the wait mask is useful is
X`09`09when the text "MWAIT?" is given for the state. In this case the
X`09`09wait mask contains the address of an unrecognised system mutex.
XCluster`09`09The event flag cluster number that the "wait mask" refers to.
X`09`09A value of "0" refers to event flags in the range 0..31, "1"
X`09`09refers to 32..63, "2" to 64..95 and "3" to 96..127. EFNs in
X`09`09the range 64..127 are found in "Common event blocks" - see
X`09`09below.
XTime`09`09VMS delta time since the process last went into a wait state.
X`09`09This time is reset after every process state transition.
X
X0: xxxxxxxx`09Contents of event flag cluster 0 (EFNs  0..31)
X1: xxxxxxxx`09   ''    ''  ''    ''     ''   1 (EFNs 32..63)
X2: xxxxxxxx (`5Buic`5Dname)`09Optional, only displayed when valid
X`09`09   ''    ''  ''    ''     ''   2 (EFNs 64..95)
X3: xxxxxxxx (`5Buic`5Dname)`09Optional, only displayed when valid
X`09`09   ''    ''  ''    ''     ''   3 (EFNs 96..127)
X`09`09The "(`5Buic`5Dname) field is only displayed for EFN clusters 2 and
X`09`093, and it reflectes the name of the associated common event flag
X`09`09block, together with its group qualification.
XMode`09`09The current and previous processor modes of the process.
XAST status`09A list of those AST levels that are enabled, and which are
X`09`09currently active. If at least one AST is waiting for delivery,
X`09`09the text "AST(s) waiting" is displayed. eg. from this you could
X`09`09determine if a process was locked in LEF at AST level, blocking`20
X`09`09other activity.
X
X
XTypical output from a process in RWAST
X--------------------------------------
X
X$ MWAIT 20A00452
X  Pid    User name    Process name    Prior  State  Port
X20A00452 NICK         NICK_1           6/4   RWAST  -Sub-
X`09Wait mask: 00000001, Cluster: 0, Time:    0 00:00:00.08
X`090: E0000001, 1: 00000000
X`09Mode: Current: KERNEL, Previous: USER
X`09AST status: Active: (none), Enabled: KESU
X`09Process resource wait is ENABLED.
X`09Process has no BIO left (BIOCNT zero).
X`09Process has 100. outstanding BIO operations (BIOCNT: 0., BIOLM: 100.)
X`09Process has 2. outstanding ASTS active (ASTCNT: 98. ASTLM: 100.)
X`09Process registers:
X`09`09R0:  00000001`09R1:  00000001`09R2:  80408F5A`09R3:  00000010
X`09`09R4:  80408F20`09R5:  8037C470`09R6:  7FFC9F90`09R7:  00000030
X`09`09R8:  80138148`09R9:  FFA00001`09R10: 00000030`09R11: 00000003
X`09`09AP:  7FF15928`09FP:  7FFE77E4`09PC:  801DE68E`09PSL: 00C00001
X`09Quota: R1 = quantity, R2 = address, R3 = width(bits).
X`09(Requested: 1. unit, Available: 0. units)
X`09Quota wait on: Buffered I/Os remaining count (BIOCNT: 0.)
X$
X
XThis example shows that the process is in RWAST because it has run out of qu
Vota
Xfor concurrent buffered I/O operations. The PC (801DE68E) has been identifie
Vd by
XMWAIT as lying in a VMS executive thread that checks for quota availability.
V The
Xtext says that R1 (1 unit) is the quantity required, R2 points to the locati
Von
Xwhere the available quota is stored, and R3 is the width in bits of the quot
Va
Xfield (16 bits in this example). MWAIT has identified the address in R2 as b
Veing
Xthe PCB$W_BIOCNT field for the process under inspection confirming that
Xexhausted BIOCNT was the cause of the RWAST. If no "Quota wait on..." text i
Vs
Xdisplayed, we would have to use SDA to determine the name of the quota. This
Xexample was generated by TEST0.C.
X
X
XTypical output from a process in RWMBX
X--------------------------------------
X
X$ MWAIT 20200534
X  Pid    User name    Process name    Prior  State  Port
X20200534 NICK         NICK_1           6/4   RWMBX  -Sub-
X`09Wait mask: 00000002, Cluster: 0, Time:    0 00:00:00.43
X`090: E0000000, 1: 00000000
X`09Mode: Current: USER, Previous: USER
X`09AST status: Active: U, Enabled: KESU, AST(s) waiting
X`09Process resource wait is ENABLED.
X`09Attempting to write 8. bytes to MBA2454: on channel 0060 (hex)
X`09Messages: 6., Quota (bytes): Initial = 50., Remaining = 2.
X
XThis process is in RSN$_MAILBOX (value "2", also in  the "wait mask" field).
V The
Xprocess was placed into RWMBX because it tried to write 8. bytes to mailbox
XMBA2454: which has 2. bytes of buffer space left. The mailbox's initial buff
Ver
Xquota was 50. bytes. There are currently 6 messages already in the mailbox
Xwaiting to be read (excluding the one this process is trying to write). The
Xwrite is being carried out on channel 60 (hex) in a user mode AST routine (t
Vhere
Xis a user mode AST active) and at least one AST is waiting for delivery (and
V is
Xcurrently blocked). This example was generated by TEST4.C.
X
X
XTypical output from processes in a Job Quota wait
X-------------------------------------------------
X
X$ MWAIT 202000CD
X  Pid    User name    Process name    Prior  State  Port
X202000CD NICK         NICK_1           6/4   JITQE  -Sub-
X`09Wait mask: 8097EB10, Cluster: 0, Time:    0 00:00:00.35
X`090: E0000000, 1: 00000000
X`09Mode: Current: USER, Previous: USER
X`09AST status: Active: (none), Enabled: KESU
X`09Job timer count exhausted (TQCNT: 0., TQLM: 200.)
X$
X
XThis process is in a job pooled quota wait for timer queue entries. Increase
V the
XUAF TQELM parameter or modify the code to prevent this happening. Note that
Xrunning out of a job pooled quota can severely effect the performance of all
Xother jobs in the job tree. The Wait mask contains the address of the JIB
Xfor the process under inspection. This example was generated by TEST1.C.
X
X
X$ MWAIT 202000E2
X  Pid    User name    Process name    Prior  State  Port
X202000E2 NICK         NICK_1           4/4   JIBYT  -Sub-
X`09Wait mask: 80D78190, Cluster: 0, Time:    0 00:01:49.44
X`090: E0000000, 1: 00000000
X`09Mode: Current: KERNEL, Previous: KERNEL
X`09AST status: Active: (none), Enabled: KESU
X`09Job buffered I/O count exhausted
X`09(Requested: 2080. bytes, Available: 1632. bytes, Quota: 39424.)
X
XThe process is waiting for buffered I/O byte count quota. The wait mask cont
Vains
Xthe address of the JIB for the process. The process has requested 2080. byte
Vs of
Xquota when there are only 1632. bytes left. The initial quota was 39424. byt
Ves.
XThe process will remain in this state until enough quota is available, or th
Ve
Xprocess is deleted. This example was generated by TEST5.C.
X
XNick de Smith`09NICK@NCDLAB.ULCC.AC.UK
X
X
X
X1. Table of standard wait states
X--------------------------------
X
XText`09Description
X-------`09-----------
X?????`09Unrecognised process state. Should never happen.
XColpg`09Collided page wait. More than 1 process has referenced the same page
X`09not in physical memory. The first such process goes into PFW (see below)
X`09and subsequent processes go into COLPG.
XMwait`09Mutual exclusion wait. The process is waiting for a depleted resourc
Ve,
X`09a job quota, or a locked mutex. The program decodes (to the best of its
X`09ability) the reason for the MWAIT. See elsewhere in this document.
XCef`09Common event flag wait. The process is waiting on for event flags in
X`09clusters 2 (64-95) or 3 (96-127). The program displays the name of any
X`09associated common event flag clusters, and their values. See LEF below.
XPfw`09Page fault wait. Process has referenced a page not in physical memory,
X`09and must wait until the page has been read in. Process deletion, AST
X`09delivery and a successful read of the page will place the process into
X`09COM or COMO.
+-+-+-+-+-+-+-+-  END  OF PART 4 +-+-+-+-+-+-+-+-

