From:	CRDGW2::CRDGW2::MRGATE::"SMTP::CUNYVM.CUNY.EDU::U0012%DGOGWDG5.BITNET" 30-AUG-1989 16:34
To:	MRGATE::"ARISIA::EVERHART"
Subj:	Pseudo tape driver - some writeup

Message-Id:  <8908302022.AA19520@crdgw1.ge.com>
Received: from DGOGWDG5.BITNET by CUNYVM.CUNY.EDU (IBM VM SMTP R1.2.1MX) with BSMTP id 5389; Wed, 30 Aug 89 16:20:46 EDT
Date:    Wed, 30 Aug 1989 22:18:04  +0200
From: "GWDGV1::MOELLER"  <U0012%DGOGWDG5.BITNET@CUNYVM.CUNY.EDU>
To: EVERHART@ARISIA.decnet
Subject: Pseudo tape driver - some writeup

Dear Mr. Everhart,

thank you for your prompt reply. This message started out almost as
promptly, but took me a long time to complete!

>       I've never had any troubles or problems with DEC with having used
>       VMS sources as references / starting points for VDdriver or FDdriver.
>       Also I'd much like to get a copy of your driver for local use, and
>       for the next vax sig tapes, if you're willing to share it.

I thought so, but wanted to have somebody make that explicit.

>       I found in my hacking that trying to run the ioc$movxxuser code at
>       lower than fork IPL generated all sorts of weird timing errors at
>       around the i/o termination points. They were a bitch to debug, even with
>       xdelta, and I recommend you steer clear of that can of worms. YOU may
>       not interrupt the operation, but VMS will.

I don't quite understand this remark. My idea is just that it might not
matter if IOC$MOVxxUSER, running in process context, is interrupted -
but I'm not sure ...

>          If you're willing to share your code, please email me a copy
>       ...
>       I should even be able to read/write 8mm tape by the end of the week
>       ...

Second remark first: we got an EXABYTE last week from SI. So YOU think they
can be exchanged (SI here would not confirm that!)?! In the text below,
there is another - important, I guess - remark concerning that EXABYTE!

Now for the main theme:

   I will be glad to share that beast (ZTDRIVER & friends);
however, I don't think it's fit for the general public
(there is neither a writeup [*] nor anything like a "user" interface -
actually I don't know for what purpose one can really use it!).
        [*] Note added in proof: Now, hours later, there is something, below!]

   I have tested ZTDRIVER on a VS2000 under VMS V4.7 and V5.1-1,
with the server running on a 8650 under VMS 4.7 only.
   No production use so far, in fact I mainly did it
        (a) as an exercise to port a driver from V4 to V5
        (b) to test VMS V5's great new magtape (BACKUP) features -
                we have tapes only on the 8650 (no TK50 at all)!

But as you have experience with such things yourself, you are
welcome to review what I have made up so far.

The directory looks like:
        ZTDEF.MAR;8               2/3
        ZTDRIVER.MAR;20          26/27
        ZTDRIVER.OPT;3            1/3
        ZTSERVER.MAR;11          21/21

        ZT1.C;18                 33/33    ???
        ZT1T.C;5                 13/15    ???

        ZT2.C;10                 47/48
        ZTNS.H;4                  3/3
        ZTNS2.C;13               19/21

        IODEF.H;1                17/18    ? these are made with (my) C_DEFS
        MTDEF.H;1                 3/3     ? from STARLET.MLB & LIB.MLB
        UCBDEF.H;1               17/18    ? --- see below ---

I will send the files NOT marked with '?' via VMS_SHARE following this message.
(if I'm lucky and there are no trailing 'v's as in this line: vvvvvvvvvvvvvvvvvv
- did you notice the info-vax etc. discussion last month?)

=============== A little description/technical note, made up right now: ========

   ZTDRIVER owes most of its size to my (esentially) copying of MTDRIVER's
FDT stuff, since I didn't know at the outset which I/O functions
are in actual use (MTAACP/RMS/BACKUP). Now I know - it's indeed only
the documented ones plus IO$_NOP, sometimes also represented as a SKIP 0 blocks
(if I remember correctly), with the ACP adding an apparently un-named
modifier bit 1@9.
   It handles a request in its STARTIO routine the following way:
copy pertinent data into its UCB extension, send that as a mailbox msg to
the server (which indicates its presence by storing its mailbox into the UCB).
It then WFIKPCH's where the timeout is not only very long, but the timeout
routine itself is a NOP (Same for CANCEL!).
At this point the server process is supposed to do just everything on its own,
except for modifying externally visible UCB fields. Currently, the server
need not go into kernel mode initially; only when it has figured out what
is actually to be done, it possibly calls IOC$MOVxxUSER to transfer data.
Finally the server into kernel mode, copies back its results into the
UCB extension, and does a fake 'interrupt' to wake the driver.
   When awake, the driver just updates the 'real' UCB fields from the data
it finds in the UCB extension and completes the I/O request.
   Currently this description is not completely correct, since there are
a few error paths where the driver decides on its own to reject a request;
I don't like them anymore.
   The driver starts out as OFFLINE, the server will set it ONLINE (when
storing its mailbox address) and turn it off again in its exit handler -
note that there is currently no EXEC mode exit handler!
        *** CAUTION: don't STOP the server, always $FORCEX it! ***
   Actually, the worst that can happen is that the driver gets stuck,
and as far as I know this should block just 1 process (and possibly its ACP -
MTAACP is not shared). The server initialization, is intended to clear this
condition in any case: it just overwrites the UCB extension fields that
may be left from an earlier instance and also triggers a pseudo-interrupt
(which will normally be ignored). So if the server dies, just re-start it!
        *** CAUTION: the server does not check if it is the only one!
                Once I had 3 servers running by mistake - just "force"d
                all of them (ZTA0 went offline) and started a new one
                (ZTA0 online again) - my first chance to experience
                VMS V5's new mount verification stuff ...

   Files: ZTDEF.MAR (defines message area / 'shared' UCB extension)
                must be made into ZT.MLB, since the .MAR use '.include "ZT"'
        ZTDRIVER.MAR is the driver (VMS 4.7 .. 5.1-1), to make:
                $ MAC ZTDRIVER
                $ LINK ZTDRIVER/OPT
        ZTSERVER.MAR encapsulated the server's kernel mode stuff
                in a few simple-minded subroutines.
                (needs just CMKRNL privilege to run, by the way).
        *** CAUTION: ZTSERVER and ZTDRIVER have to be generated and *used*
                *together* for a particular VMS version,
                there is no built-in version check!
                (I think the fact that both link with SYS.STB
                will make it impossible to run a wrong server)

   Now for the server: what could it do???
A first 'test' version (ZT1 + ZT1T) tried to make up a 'memory tape'.
I think such a thing is utterly useless, and as it stands, it can serve
as a rudimentary test device only since it won't switch reels.
[Just for fun: naturally I included EOT detection, but no way to simulate
an unloaded tape. The ACP bravely asks the operator, but silly old BACKUP
indeed initiates UNLOAD, then immediately tests if the tape is loaded;
if that succeeds - as in my case, and allegedly for some initial V5 TUDRIVER -
it happily continues without asking anyone!]
   If desired, I might include this program for completeness ...

   The second program (and the only one mentioned above) is a DECnet tape server
pair. The program serving the pseudo-tape is called ZT2
and the real server, which serves a real tape, is called ZTNS2.
   There is quite a bit of black magic involved in getting a real tape served,
the biggest problem being that there is nowhere a documentation for
the 'correct' behaviour of a tape driver - in fact different DEC drivers
do return different status codes on identical conditions. The second trouble
comes with hard errors: if the real-tape-server (ZTNS2, for short) receives
a media/hardware related error status, it does not know what exactly happened
and in particular, where the tape is positioned (and of course ZTNS2 ought
to be device-independent)!
   Here comes a horrible thing with VMS: apparently the author(s)
of MTAACP noted this problem, and so there was created a cell UCB$L_RECORD
where a magtape driver has to put the current block number (0 at BOT,
EOF marks are counted as blocks). I know that the ACP looks at this cell,
but normal programs can't, since it's not on the list of $GETDVI!
   Currently, ZTNS2 (intended to be a harmless, unprivileged program)
does not go into kernel mode to find out about UCB$L_RECORD, so it has to guess
and also return "tape position lost" status. There are actually 2 ways to do
just that, the first is to say so (SS$_TAPEPOSLOST) and the second is to
set a status bit MT$V_LOST (does anyone look at this status word, anyway?? -
there is a bit MT$V_EOF allegedly indicating "end of file mark passed",
MTDRIVER sets it from some hardware register, newer drivers apparently
never set it (?)). ZTNS2 has a conditional of which behaviour is preferred.

[Side note: the design choice to use the total block count in
        UCB$L_RECORD turns out to be almost disastrous,
        since no tape driver may ever use the omnipresent hardware function
        'SKIP FILEMARKS'. All requests of IO$_SKIPFILE must be converted
        into repeated 'SKIP BLOCKS' functions, so the driver can add
        (or subtract) the proper amount from UCB$L_RECORD.
        This almost forbids video tapes, since they can do a fast
        skip to filemarks ONLY, and at least the EXABYTE we recently got
        CANNOT physically skip blocks BACKWARD, only filemarks -
        it will emulate the function by skipping back to the previous
        file mark or to BOT, then skip forward at low speed,
        it may take HOURS to skip backward 1 block !!!!!!!!!!!!!!
        An "ancient" operating system like SPERRY/UNIVACs OS1100 knew that
        the 'current tape position' MUST be represented by
                <number of tapemarks> and
                <number of blocks since tapemark>
        which is safe enough since tapemarks do not tend to become data
        blocks (and vice versa) even on bad tapes.]


   I have to admit that all of the code has not yet been sufficiently tested,
because I was too lazy to implement "error insertion" in my memory tape,
and with real tapes you have to wait and see - and that takes time...!
   The 'protocol' used between ZT2 and ZTNS2 is strictly half-duplex:
ZT2 sends (sort of) I/O function, ZTNS2 finally replies with status & some more
information. In between, ZTNS2 may send at most a single 'DMA request':
   - on tape read, this is immediately followed by data (split into
        reasonably-sized packets) , upon which ZT2 replies
        (with a 'DMA status', always good so far)
   - on tape write, ZT2 replies with the data requested
I unnecessarily wrote ZTNS2 such that it would support full-duplex operation
of the DECnet link; on the other hand, ZT2 is completely synchronous.

   ZT2/ZTNS2 is not complete. Both programs conspire to ignore
most of the I/O function modifier bits, notably IO$M_INHRETRY;
they do not implement read backward (I don't even know in what direction
and where data would have to be placed in the user buffer);
DATACHECK is not end-to-end, but only done on the ZTNS2 end.
Also, IO$_SETMODE/CHAR is not propagated to ZTNS2 at all, and neither are the
real tape "characteristics" to ZT2 - this is truly a shortcoming in the
task-to-task protocol. The virtual ZT defaults to "TE16 at 1600bpi";
in fact this can be changed with IO$_SETCHAR without any influence on the
driver's behaviour.
   I did implement the IO$_WRITECHECK function from MTDRIVER,
which actually means "compare tape data to memory". Unfortunately,
no program uses it, so it's completely untested.

   Files:
        ZT1.C + ZT1T.C make up the memory tapes (ZT1 is the "driver" part,
                        ZT1T is just subroutines for in-core storage
                        of tape blocks and EOF marks).
        ZT2.C is the ZT-server for the task-to-task design
        *** CAUTION: ZT1.C *and* ZT2.C each carry their own definition
                        of the message area defined in ZTDEF.MAR ***
                Both ZT1 and ZT2 have an options to log all I/O operations;
                so far I had that always enabled.
        ZTNS2.C is the real-tape-server
                ZT1, ZT2 and ZTNS2 include:
        IODEF.H, MTDEF.H, UCBDEF.H which I created with my 'C_DEFS'
                procedure (recently re-posted on info-vax by Jerry Leichter).
                Note: IODEF.H from VAXC (VMS V4 version) won't do.
                By silly design, UCBDEF is currently *only* required to define
                a single bit position, UCB$M_VALID (this bit did *not* change
                from VMS V4 to V5 - UCBDEF changed a lot).
        ZTNS.H is included by ZT2.C and ZTNS2.C, it defines
                the task-to-task messages.

        To make ZT1:
                $ CC ZT1,ZT1T
                $ LINK ZT1,ZT1T,ZTSERVER        ! links with SYS
        To make ZT2 and ZTNS2:
                $ CC ZT2
                $ LINK ZT2,ZTSERVER             ! links with SYS
                $ CC ZTNS2
                $ LINK ZTNS2

   To use the programs,
        (a)     $ SYSGEN CONNECT ZTA0/noadapter
        *** BUG: ZTSERVER.MAR has _ZTA0 hard-coded. ***
        (b1)    $ run ZT1       ! from anywhere: terminal, batch, detached ...
        (c1)    when you are done, somehow stop ZT1 by $FORCEX,
                CTRL-Y, or STOP/QUE/ENTRY when in batch.
        (b2a)   on the ZT-side, make up a Network object, e.g. ZT2.COM,
                        like:   $ set proc/priv=cmkrnl
                                $ run ZT2
        (b2b)   on the real tape side:
                        $ define ZT_TAPE <any tape>
                        $ mount/foreign ZT_TAPE/multi_volume
                                               ^-- for V5, I guess
                                                (my tapes are still with V4)
                        $ define ZT_NETOBJECT "node::""task=ZT2/"
                                                     ^---------^ wierd (sorry),
                                                task spec starts with quotation
                                                mark and ends with slash.
                        $ run ZTNS2
        (c2)    when done, either kill ZT2 with $FORCEX (ZT-side),
                or stop ZTNS2 (by any means).
        *** BUG: ZT2 will *not* instantly notice that ZTNS2 has gone,
                but only on the next I/O.
                MOUNT/FOREIGN is ok to force an I/O (=> SS$_DEVOFFLINE).

===============================================================================


Enjoy,
        -wjm

Wolfgang J. Moeller, GWDG, D-3400 Goettingen, F.R.Germany | Disclaimer ...
Bitnet/Earn: U0012@DGOGWDG5         Phone: +49 551 201516 | No claim intended
