124  INTRODUCTION  MANGLE §1

1* . Introduction. This program converts a file to a Pascal file. It was written by D. E. Knuth in September, 1981; a somewhat similar SAIL program had been developed in March, 1979. Since this program describes itself, a bootstrapping process involving hand-translation had to be used to get started.

For large files one should have a large memory, since keeps all the Pascal text in memory (in an abbreviated form). The program uses a few features of the local Pascal compiler that may need to be changed in other installations:

1) Case statements have a default.

2) Input-output routines may need to be adapted for use with a particular character set and/or for printing messages on the user’s terminal.

These features are also present in the Pascal version of TEX, where they are used in a similar (but more complex) way. System-dependent portions of can be identified by looking at the entries for ‘system dependencies’ in the index below.

The “banner line” defined here should be changed whenever is modified. For the Modula-2 versions of , we add a second decimal point and number to the main version

number, so we can differentiate between changes to the standard and changes local to the Modula-2 version.

define banner ≡ ´ ␣ ␣ ␣ ␣ ␣ ␣ ´

8* . The following parameters are set big enough to handle TEX, so they should be sufficient for most applications of . ⟨ Constants in the outer block 8* ⟩ ≡

buf size = 100; { maximum length of input line } max bytes = 45000; { 1/ww times the number of bytes in identifiers, strings, and module names; must

be less than 65536 } max toks = 50000;

{ 1/zz times the number of bytes in compressed Pascal code; must be less than 65536 } max names = 4000; { number of identifiers, strings, module names; must be less than 10240 } max texts = 2000; { number of replacement texts, must be less than 10240 } hash size = 353; { should be prime } longest name = 400; { module names shouldn’t be longer than this } line length = 72; { lines of Pascal output have at most this many characters } out buf size = 144; { length of output buffer, should be twice line length } stack size = 50; { number of simultaneous levels of macro expansion } max id length = 31; { long identifiers are chopped to this length, which must not exceed line length } unambig length = 20; { identifiers must be unique if chopped to this length }

This code is used in section 2.

§72  MANGLE  INTRODUCTION 125

72* . If the first byte of a token is less than ´ 200 , the token occupies a single byte. Otherwise we make a sixteen-bit token by combining two consecutive bytes a and b. If ´ 200 ≤ a < ´ 250 , then (a − ´ 200 ) × 2^{8} + b points to an identifier; if ´ 250 ≤ a < ´ 320 , then (a − ´ 250 ) × 2^{8} + b points to a module name; otherwise, i.e., if ´ 320 ≤ a < ´ 400 , then (a − ´ 320 ) × 2^{8} + b is the number of the module in which the current replacement text appears.

Codes less than ´ 200 are 7-bit ASCII codes that represent themselves. In particular, a single-character identifier like ‘x’ will be a one-byte token, while all longer identifiers will occupy two bytes.

Some of the 7-bit ASCII codes will not be present, however, so we can use them for special purposes. The following symbolic names are used:

param denotes insertion of a parameter. This occurs only in the replacement texts of parametric macros,

outside of single-quoted strings in those texts. begin comment denotes , which will become either or . end comment denotes , which will become either or . octal denotes the ´ that precedes an octal constant. hex denotes the that precedes a hexadecimal constant. check sum denotes the that denotes the string pool check sum. join denotes the concatenation of adjacent items with no space or line breaks allowed between them (the

operation of ). double dot denotes ‘ ’ in Pascal. verbatim denotes the that begins a verbatim Pascal string. It is also used for the end of the string. force line denotes the that forces a new line in the Pascal output. define param = 0 { ASCII null code will not appear } define verbatim = ´ 2 { extended ASCII alpha should not appear } define force line = ´ 3 { extended ASCII beta should not appear } define begin comment = ´ 11 { ASCII tab mark will not appear } define end comment = ´ 12 { ASCII line feed will not appear } define octal = ´ 14 { ASCII form feed will not appear } define hex = ´ 15 { ASCII carriage return will not appear } define double dot = ´ 40 { ASCII space will not appear except in strings } define check sum = 1 { this control code should not appear in normal text } define join = ´ 177 { ASCII delete will not appear }

115* . Please don’t ask how all of the following characters can actually get through outside of strings. It seems that and cannot actually occur at this point of the program, but they have been included just in case changes.

If is producing code for a Pascal compiler that uses ‘ ’ and ‘ ’ instead of square brackets (e.g., on machines with EBCDIC code), one should remove and from this list and put them into the preceding module in the appropriate way. Similarly, some compilers want ‘ ’ to be converted to ‘ ’. ⟨ Other printable characters 115* ⟩ ≡

, , # , $ , % , , , , , , , , , , , , , , , , , ,

, ` , , , ,

This code is used in section 113.

121* . Some Pascal compilers do not recognize comments in braces, so the comments must be delimited by ‘ ’ and ‘ ’. In such cases the statement ‘send out (misc, )’ that appears here should be replaced by ‘begin out contrib[1] ← ; out contrib[2] ← ; send out (str , 2); end’, and a similar change should be made to ‘send out (misc, )’.

Note: This is definitely the case with Modula-2, so the change must be made. define send left comment ≡

begin out contrib[1] ← ; out contrib[2] ← ; send out (str , 2); end

define send right comment ≡ begin out contrib[1] ← ; out contrib[2] ← ; send out (str , 2); end

⟨ Cases involving and 121* ⟩ ≡ begin comment : begin send left comment ; incr (brace level );

end; end comment : if brace level > 0 then

begin decr (brace level ); send right comment ;

end else err print (´ ␣ ␣ ´ );

module number : begin send left comment ;

if cur val < 0 then begin send out (misc, ); send val (−cur val ); end

else begin send val (cur val ); send out (misc, );

end; send right comment ; end;

This code is used in section 113.

142* . ⟨ Do special things when c = , , , ; return at end 142* ⟩ ≡

if c = then begin c ← buffer [loc]; if (c ≠ ␣ ) ∧ (c ≠ tab mark ) ∧ (c ≠ ) ∧ (c ≠ ) ∧ (c ≠ ) then incr (loc) else begin err print (´ ␣ ␣ ␣ ␣ ´ ); decr (loc); return;

end end

else if (c = ) ∧ (buffer [loc] ≠ ) then incr (loc)

else if (c = ) ∧ (buffer [loc] = ) then begin incr (bal ); incr (loc); end

else if (c = ) ∧ (buffer [loc] = ) then begin incr (loc); if bal = 0 then return; decr (bal ); end

This code is used in section 141.

145* . At the top level, get next is a multi-way switch based on the next character in the input buffer. A new module code is inserted at the very end of the input file. function get next : eight bits ; { produces the next input token }

label restart , done, found ; var c: eight bits ; { the current character }

d: eight bits ; { the next character }

j, k: 0 . . longest name; { indices into mod text } begin restart : if loc > limit then

begin get line;

if input has ended then

begin c ← new module; goto found ; end;

end; c ← buffer [loc]; incr (loc); if scanning hex then ⟨ Go to found if c is a hexadecimal digit, otherwise set scanning hex ← false 146 ⟩;

case c of

, up to( ), , up to( ): ⟨ Get an identifier 148 ⟩; : ⟨ Get a preprocessed string 149 ⟩; : ⟨ Get control code and possible module name 150 ⟩; ⟨ Compress two-symbol combinations like ‘ ’ 147* ⟩ ␣ , tab mark : goto restart ; { ignore spaces and tabs }

othercases do nothing

endcases; found : debug if trouble shooting then debug help; gubed

get next ← c; end;

147* . Note that the following code substitutes and for the respective combinations ‘ ’ and ‘ ’. Explicit braces should be used for TEX comments in Pascal text.

define compress (#) ≡

begin if loc ≤ limit then begin c ← #; incr (loc); end; end

⟨ Compress two-symbol combinations like ‘ ’ 147* ⟩ ≡ : if buffer [loc] = then compress (double dot )

else if buffer [loc] = then compress ( ); : if buffer [loc] = then compress (left arrow ); : if buffer [loc] = then compress (equivalence sign); : if buffer [loc] = then compress (greater or equal ); : if buffer [loc] = then compress (less or equal )

else if buffer [loc] = then compress (not equal ); : if buffer [loc] = then

begin incr (loc); skip comment ; goto restart ; end

else if buffer [loc] = then compress ( );

This code is used in section 145*.

128  INDEX  MANGLE §189

189* . Index. Here is a cross-reference table for the processor. All modules in which an identifier is used are listed with that identifier, except that reserved words are indexed only when they appear in format definitions, and the appearances of identifiers in module names are not indexed. Underlined entries correspond to where the identifier was declared. Error messages and a few other things like “ASCII code” are indexed here too.

The following sections were changed by the change file: 1, 8, 72, 115, 121, 142, 145, 147, 189.