From:	SMTP%"RELAY-INFO-VAX@CRVAX.SRI.COM"  4-MAR-1994 10:30:48.90
To:	EVERHART
CC:	
Subj:	Modifying /TRACE and /DEBUG .EXE attributes (VAX & AXP)

Date: Wed, 2 Mar 1994 21:48:33 +0100
From: "GWDGV1::MOELLER" <moeller@gwdgv1.dnet.gwdg.de>
To: info-vax@sri.com
Subject: Modifying /TRACE and /DEBUG .EXE attributes (VAX & AXP)

Long ago, I posted SET_EXE.COM, a DCL program that would modify 
VAX/VMS executables to fix their /[no]TRACE and /[no]DEBUG attributes.
At that time, hardly anyone noticed, since there were so many
'recommended procedures' using $PATCH to do the same.

Modifying these attributes is most useful
	(a) if you want to LINK/DEBUG, but not have the program
		_start_ under control of the debugger
	(b) if you want to INSTALL a program that has not been
		linked /NOTRACE
	(c) [hacker's item censored :-]

Given that there's no $PATCH on AXP, the new version of SET_EXE (below),
which works on both VAX and AXP, and handles both executable formats
might find your interest (usage notes at top of the command file) ...

Wolfgang J. Moeller, Tel. +49 551 201516 or -510, <moeller@gwdgv1.dnet.gwdg.de> 
GWDG, D-37077 Goettingen, F.R.Germany            PSI%(0262)45050352008::MOELLER
Disclaimer: No claim intended! | <moeller@decus.decus.de>  <w.moeller@ieee.org> 

$! ................... Cut between dotted lines and save. ...................
$!...........................................................................
$! VAX/VMS archive file created by VMS_SHARE V06.10 7-FEB-1989.
$!
$! VMS_SHARE was written by James Gray (Gray:OSBUSouth@Xerox.COM) from
$! VMS_SHAR by Michael Bednarek (U3369429@ucsvc.dn.mu.oz.au).
$!
$! To unpack, simply save, concatinate all parts into one file and
$! execute (@) that file.
$!
$! This archive was created by user MOELLER
$! on  2-MAR-1994 09:45:55.44.
$!
$! It contains the following 1 file:
$!        SET_EXE.COM
$!
$!============================================================================
$ SET SYMBOL/SCOPE=( NOLOCAL, NOGLOBAL )
$ VERSION = F$GETSYI( "VERSION" )
$ if f$getsyi("cpu").gt.127 then goto version_ok
$ IF VERSION .GES "V4.4" THEN GOTO VERSION_OK
$ WRITE SYS$OUTPUT "You are running VMS ''VERSION'; ", -
    "VMS_SHARE V06.10 7-FEB-1989 requires VMS V4.4 or higher."
$ EXIT 44 ! SS$_ABORT
$VERSION_OK:
$ GOTO START
$!
$UNPACK_FILE:
$ WRITE SYS$OUTPUT "Creating ''FILE_IS'"
$ DEFINE/USER_MODE SYS$OUTPUT NL:
$ EDIT/TPU/COMMAND=SYS$INPUT/NODISPLAY/OUTPUT='FILE_IS'/NOSECTION -
    VMS_SHARE_DUMMY.DUMMY
b_part := CREATE_BUFFER( "{Part}", GET_INFO( COMMAND_LINE, "file_name" ) )
; s_file_spec := GET_INFO( COMMAND_LINE, "output_file" ); SET( OUTPUT_FILE
, b_part, s_file_spec ); b_errors := CREATE_BUFFER( "{Errors}" ); i_errors 
:= 0; pat_beg_1 := ANCHOR & "-+-+-+ Beginning"; pat_beg_2 := LINE_BEGIN 
& "+-+-+-+ Beginning"; pat_end := ANCHOR & "+-+-+-+-+ End"; POSITION
( BEGINNING_OF( b_part ) ); LOOP EXITIF SEARCH( SPAN( ' ' )@r_trail 
& LINE_END, FORWARD) = 0; POSITION( r_trail ); ERASE( r_trail ); ENDLOOP
; POSITION( BEGINNING_OF( b_part ) ); i_append_line := 0; LOOP EXITIF MARK
( NONE ) = END_OF( b_part ); s_x := ERASE_CHARACTER( 1 )
; IF s_x = '+' THEN r_skip := SEARCH( pat_beg_1, FORWARD, EXACT ); IF r_skip 
<> 0 THEN s_x := ''; MOVE_HORIZONTAL( -CURRENT_OFFSET ); ERASE_LINE; ENDIF
; ENDIF; IF s_x = '-' THEN r_skip := SEARCH( pat_end, FORWARD, EXACT )
; IF r_skip <> 0 THEN s_x := ''; MOVE_HORIZONTAL( -CURRENT_OFFSET ); m_skip 
:= MARK( NONE ); r_skip := SEARCH( pat_beg_2, FORWARD, EXACT ); IF r_skip 
<> 0 THEN POSITION( END_OF( r_skip ) ); MOVE_HORIZONTAL( -CURRENT_OFFSET )
; MOVE_VERTICAL( 1 ); MOVE_HORIZONTAL( -1 ); ELSE POSITION( END_OF( b_part ) 
); ENDIF; ERASE( CREATE_RANGE( m_skip, MARK( NONE ), NONE ) ); ENDIF; ENDIF
; IF s_x = 'V' THEN s_x := ''; IF i_append_line <> 0 THEN APPEND_LINE
; MOVE_HORIZONTAL( -CURRENT_OFFSET ); ENDIF; i_append_line := 1
; MOVE_VERTICAL( 1 ); ENDIF; IF s_x = 'X' THEN s_x := ''; IF i_append_line 
<> 0 THEN APPEND_LINE; MOVE_HORIZONTAL( -CURRENT_OFFSET ); ENDIF
; i_append_line := 0; MOVE_VERTICAL( 1 ); ENDIF; IF s_x <> '' THEN i_errors 
:= i_errors + 1; s_text := CURRENT_LINE; POSITION( b_errors ); COPY_TEXT
( "The following line could not be unpacked properly:" ); SPLIT_LINE
; COPY_TEXT( s_x ); COPY_TEXT( s_text ); POSITION( b_part ); MOVE_VERTICAL
( 1 ); ENDIF; ENDLOOP; POSITION( BEGINNING_OF( b_part ) ); LOOP r_x := SEARCH
( "`", FORWARD, EXACT ); EXITIF r_x = 0; POSITION( r_x ); ERASE_CHARACTER( 1 
); COPY_TEXT( ASCII( INT( ERASE_CHARACTER( 3 ) ) ) ); ENDLOOP
; IF i_errors = 0 THEN SET( NO_WRITE, b_errors, ON ); ELSE POSITION
( BEGINNING_OF( b_errors ) ); COPY_TEXT( FAO
( "The following !UL errors were detected while unpacking !AS", i_errors
, s_file_spec ) ); SPLIT_LINE; SET( OUTPUT_FILE, b_errors, "SYS$COMMAND" )
; ENDIF; EXIT; 
$ DELETE VMS_SHARE_DUMMY.DUMMY;*
$ CHECKSUM 'FILE_IS
$ WRITE SYS$OUTPUT " CHECKSUM ", -
  F$ELEMENT( CHECKSUM_IS .EQ. CHECKSUM$CHECKSUM, ",", "failed!!,passed." )
$ RETURN
$!
$START: 
$ FILE_IS = "SET_EXE.COM"
$ CHECKSUM_IS = 1426636937
$ COPY SYS$INPUT VMS_SHARE_DUMMY.DUMMY
X$! "patch" an executable to change the LINK /DEBUG and /TRACE settings
X$!
X$! usage:`009@SET_EXE image [/[no]DEBUG] [/[no]TRACE]
X$!
X$! w.j.m. 06-nov-1991
X$! mod 01-mar-1994 wjm: support AXP/VMS (1.5) executables
V$! mod 02-mar-1994 wjm: use IHA$ symbols for VAX; don't permit `123NOTRACE,DE
XBUG`125
X$!
X$! NOTE: this is *unsupported* stuff!
X$!*****
X$!
X$!* some definitions ...
X$!
X$ SYS$IMGSTA = %x7FFEDF68
X$ IHD$W_ACTIVOFF = 2
X$ IHD$L_LNKFLAGS = 32
X$ IHD$V_LNKDEBUG = 0
X$ IHA$L_TFRADR1 = 0
X$ IHA$L_TFRADR2 = 4
X$ IHA$L_TFRADR3 = 8
X$!
X$ e_SYS$IMGSTA_l = %x00000340`009! [???]
X$ e_SYS$IMGSTA_h = %xFFFFFFFF`009! [???]
X$ EIHD$L_ACTIVOFF =  16
X$ EIHD$L_LNKFLAGS =  80
X$ EIHD$V_LNKDEBUG =  0
X$ EIHD$L_MAJORID =  0
X$ EIHD$L_MINORID =  4
X$ EIHD$K_MAJORID =  3
X$ EIHD$K_MINORID =  0
X$ EIHA$L_TFRADR1 = 8
X$ EIHA$L_TFRADR1_H = 12
X$ EIHA$L_TFRADR2 = 16
X$ EIHA$L_TFRADR2_H = 20
X$ EIHA$L_TFRADR3 = 24
X$ EIHA$L_TFRADR3_H = 28
X$ EIHA$L_TFRADR4 = 32
X$ EIHA$L_TFRADR4_H = 36
X$!
X$ CLI$_MAXPARM = 229528 + 2`009! -E-MAXPARM, too many parameters ...
X$ CLI$_CONFQUAL = 231426`009! -E-CONFQUAL, conflicting qualifiers
X$ CLI$_INVQUAL = 231434 `009! -E-INVQUAL, invalid qualifier
X$ CLI$_REQPRMABS = 231442`009! -E-REQPRMABS, required parameter is absent
X$ SS$_BADIMGHDR = %x0044`009! -F-BADIMGHDR, bad image header
X$ SS$_NOSUCHFILE = %x0910 + 4`009! -F-NOSUCHFILE, no such file
X$!
X$ on warning then goto errex
X$!
X$!
X$!* parse the command line ...
X$!
X$ cmdline = f$edit(p1+" "+p2+" "+p3+" "+p4+" "+p5+" "+p6+" "+p7+" "+p8,-
X`009`009   "trim,compress,upcase")
V$ cmdline = f$extract(0,f$locate("!",cmdline),cmdline)`009! ... just to make
X sure
X$ cmdpar = cmdline
X$ cmdqual = ""
X$cmdparse_loop:
X$ cmdparse_qual = "/" + f$element(0," ",f$element(1,"/",cmdpar))
X$ if cmdparse_qual.eqs."//" then goto cmdparse_done
X$ cmdpar = cmdpar - cmdparse_qual
X$ cmdqual = cmdqual + cmdparse_qual
X$ goto cmdparse_loop
X$cmdparse_done:
X$ delete/symbol cmdparse_qual
X$ cmdpar = f$edit(cmdpar,"trim,compress")
X$!
X$ filename = f$element(0," ",cmdpar)
X$ if filename.eqs."" then return CLI$_REQPRMABS`009`009! too few parameters
X$ if f$element(1," ",cmdpar).nes." " then -
X`009return CLI$_MAXPARM`009`009`009`009! too many parameters
X$!
X$ q_trace = -1
X$ q_debug = -1
X$ iq = 0
X$q_loop:
X$ iq = iq + 1
X$ q = f$element(iq,"/",cmdqual)
X$ if q.eqs."/" then goto q_loop_end
X$ if q.eqs."" then return CLI$_INVQUAL
X$ if f$locate(q,"DEBUG").eq.0 then goto q_debug_on
X$ if f$locate(q,"TRACE").eq.0 then goto q_trace_on
X$ if f$length(q).lt.3 then return CLI$_INVQUAL
X$ if f$locate(q,"NOTRACE").eq.0 then goto q_trace_off
X$ if f$locate(q,"NODEBUG").eq.0 then goto q_debug_off
X$ return CLI$_INVQUAL
X$q_debug_on:
X$ q_debug = 1
X$ goto q_loop
X$q_trace_on:
X$ q_trace = 1
X$ goto q_loop
X$q_debug_off:
X$ q_debug = 0
X$ goto q_loop
X$q_trace_off:
X$ q_trace = 0
X$ goto q_loop
X$!
X$q_loop_end:
X$ if q_debug.eq.1.and.q_trace.eq.0 then return CLI$_CONFQUAL
X$ if q_debug.eq.1 then q_trace = 1`009! /DEBUG requires /TRACE
X$ if q_trace.eq.0 then q_debug = 0`009! /NOTRACE requires /NODEBUG
X$!
X$!
X$!* open file ...
X$!
X$ fs = f$parse(filename,".EXE")
X$ if fs.eqs."" then return SS$_NOSUCHFILE
X$!
X$ close/nolog SET_EXE_FILE
X$ open/read/write/share SET_EXE_FILE 'fs'
X$!
X$ if f$file(fs,"RFM").nes."FIX" .or.-
V     f$file(fs,"MRS").ne.512 then return SS$_BADIMGHDR`009! don't like this f
Xile
X$!
X$!
X$!* read and decode image header ...
X$!
X$ read SET_EXE_FILE b
X$!
X$ e_major = f$cvui(EIHD$L_MAJORID*8, 32, b)
X$ e_minor = f$cvui(EIHD$L_MINORID*8, 32, b)
X$ eihd = e_major.eq.EIHD$K_MAJORID .and. e_minor.eq.EIHD$K_MINORID
X$!
X$ if eihd
X$ then
X$`009write SYS$OUTPUT fs," assumed to be an AXP executable"
X$!
X$`009lnkdebug = f$cvui(EIHD$L_LNKFLAGS*8 + EIHD$V_LNKDEBUG, 1 ,b)
X$!
X$`009actoff = f$cvui(EIHD$L_ACTIVOFF*8,32,b)
X$`009tra1_l = f$cvui((actoff + EIHA$L_TFRADR1)*8, 32,b)
X$`009tra1_h = f$cvui((actoff + EIHA$L_TFRADR1_H)*8, 32,b)
X$`009tra2_l = f$cvui((actoff + EIHA$L_TFRADR2)*8, 32,b)
X$`009tra2_h = f$cvui((actoff + EIHA$L_TFRADR2_H)*8, 32,b)
X$`009tra3_l = f$cvui((actoff + EIHA$L_TFRADR3)*8, 32,b)
X$`009tra3_h = f$cvui((actoff + EIHA$L_TFRADR3_H)*8, 32,b)
X$`009tra4_l = f$cvui((actoff + EIHA$L_TFRADR4)*8, 32,b)
X$`009tra4_h = f$cvui((actoff + EIHA$L_TFRADR4_H)*8, 32,b)
X$!
X$`009lnktrace = tra1_l.eq.e_SYS$IMGSTA_l .and. tra1_h.eq.e_SYS$IMGSTA_h
X$ else
X$`009write SYS$OUTPUT fs," assumed to be a VAX executable"
X$!
X$`009lnkdebug = f$cvui(IHD$L_LNKFLAGS*8 + IHD$V_LNKDEBUG, 1 ,b)
X$!
X$`009actoff = f$cvui(IHD$W_ACTIVOFF*8,16,b)
X$`009tra1 = f$cvui((actoff + IHA$L_TFRADR1)*8, 32,b)
X$`009tra2 = f$cvui((actoff + IHA$L_TFRADR2)*8, 32,b)
X$`009tra3 = f$cvui((actoff + IHA$L_TFRADR3)*8, 32,b)
X$!
X$`009lnktrace = tra1.eq.SYS$IMGSTA
X$ endif
X$!
X$ if lnkdebug .and. .not.lnktrace then return SS$_BADIMGHDR`009! can't be
X$!
X$!
X$ chg = 0
X$ txt = "/TRACE setting: "
X$ txt = txt + f$element(lnktrace,"`124","OFF`124ON")
X$ if q_trace.ge.0 .and. q_trace.ne.lnktrace
X$ then
X$`009chg = 1
X$`009txt = txt + " => " + f$element(q_trace,"`124","OFF`124ON")
X$`009if lnktrace
X$`009then
X$`009`009if eihd
X$`009`009then
X$`009`009`009b[(actoff + EIHA$L_TFRADR1)*8,32] = tra2_l
X$`009`009`009b[(actoff + EIHA$L_TFRADR1_H)*8,32] = tra2_h
X$`009`009`009b[(actoff + EIHA$L_TFRADR2)*8,32] = tra3_l
X$`009`009`009b[(actoff + EIHA$L_TFRADR2_H)*8,32] = tra3_h
X$`009`009`009b[(actoff + EIHA$L_TFRADR3)*8,32] = tra4_l
X$`009`009`009b[(actoff + EIHA$L_TFRADR3_H)*8,32] = tra4_h
X$`009`009`009b[(actoff + EIHA$L_TFRADR4)*8,32] = 0
X$`009`009`009b[(actoff + EIHA$L_TFRADR4_H)*8,32] = 0
X$`009`009else
X$`009`009`009b[(actoff + IHA$L_TFRADR1)*8,32] = tra2
X$`009`009`009b[(actoff + IHA$L_TFRADR2)*8,32] = tra3
X$`009`009`009b[(actoff + IHA$L_TFRADR3)*8,32] = 0
X$`009`009endif
X$`009else
X$`009`009if eihd
X$`009`009then
X$`009`009`009if tra4_l.ne.0 .or. tra4_h.ne.0 then -
X`009`009`009`009return SS$_BADIMGHDR`009! too many trfadr's
X$`009`009`009b[(actoff + EIHA$L_TFRADR1)*8,32] = e_SYS$IMGSTA_l
X$`009`009`009b[(actoff + EIHA$L_TFRADR1_H)*8,32] = e_SYS$IMGSTA_h
X$`009`009`009b[(actoff + EIHA$L_TFRADR2)*8,32] = tra1_l
X$`009`009`009b[(actoff + EIHA$L_TFRADR2_H)*8,32] = tra1_h
X$`009`009`009b[(actoff + EIHA$L_TFRADR3)*8,32] = tra2_l
X$`009`009`009b[(actoff + EIHA$L_TFRADR3_H)*8,32] = tra2_h
X$`009`009`009b[(actoff + EIHA$L_TFRADR4)*8,32] = tra3_l
X$`009`009`009b[(actoff + EIHA$L_TFRADR4_H)*8,32] = tra3_h
X$`009`009else
X$`009`009`009if tra3.ne.0 then -`009! too many transfers addresses
X`009`009`009`009return SS$_BADIMGHDR
X$`009`009`009b[(actoff + IHA$L_TFRADR1)*8,32] = SYS$IMGSTA
X$`009`009`009b[(actoff + IHA$L_TFRADR2)*8,32] = tra1
X$`009`009`009b[(actoff + IHA$L_TFRADR3)*8,32] = tra2
X$`009`009endif
X$`009endif
X$ endif
X$ write SYS$OUTPUT txt
X$!
X$ txt = "/DEBUG setting: "
X$ txt = txt + f$element(lnkdebug,"`124","OFF`124ON")
X$ if q_debug.ge.0 .and. q_debug.ne.lnkdebug
X$ then
X$`009chg = 1
X$`009txt = txt + " => " + f$element(q_debug,"`124","OFF`124ON")
X$`009if eihd
X$`009then
X$`009`009b[EIHD$L_LNKFLAGS*8 + EIHD$V_LNKDEBUG, 1] = 1 - lnkdebug
X$`009else
X$`009`009b[IHD$L_LNKFLAGS*8 + IHD$V_LNKDEBUG, 1] = 1 - lnkdebug
X$`009endif
X$ endif
X$ write SYS$OUTPUT txt
X$!
X$!
X$!* update the file ...
X$!
X$ if chg then write/update/symbol SET_EXE_FILE b
X$!
X$ close SET_EXE_FILE
X$!
X$ exit
X$!
X$!
X$!* some error ...
X$!
X$errex:
X$ status = $status
X$ set noon
X$ close/nolog SET_EXE_FILE
X$ exit status.or.%x10000000
$ GOSUB UNPACK_FILE
$ EXIT

Wolfgang J. Moeller, Tel. +49 551 201516 or -510, <moeller@gwdgv1.dnet.gwdg.de> 
GWDG, D-37077 Goettingen, F.R.Germany            PSI%(0262)45050352008::MOELLER
Disclaimer: No claim intended! | <moeller@decus.decus.de>  <w.moeller@ieee.org> 

