                                                9 September 2005.  SMS.

                     Wget 1.9.1 for VMS (1.9.1e)
                     ===========================

------------------------------------------------------------------------

      Disclaimer
      ----------

   No claims of any sort are made herein.  The main Wget developers seem
indifferent to this effort.  Any suggestions for improving the code in
the VMS-specific sections are welcome, but low expectations are more
realistic than high ones.

------------------------------------------------------------------------

      Description
      -----------

   This note accompanies a quick VMS port of Wget version 1.9.1, a GNU
free network utility to retrieve files from the World Wide Web using
HTTP and FTP.  The most obvious features seem to work, but many have not
been tested.

   Built and tested (slightly) in these environments:

OpenVMS Alpha V7.3-2, Compaq C V6.5-001, TCPIP V5.4 - ECO 5

OpenVMS IA64 V8.2, HP C S7.1-013, TCPIP V5.5.  (Woo-hoo.)

OpenVMS VAX V7.3, Compaq C V6.4-005, TCPIP V5.1.

------------------------------------------------------------------------

      News
      ----

   Version 1.9.1e  2005-09-09
   --------------------------

- The default User-Agent string on VMS now includes the host
  architecture and VMS version.  For example, "Wget/1.9.1e (VMS Alpha
  V7.3-2)".

- Fixed a problem on VMS with an ODS5 destination file system when a
  URL-derived file name contained an invalid character ("?" or "*").  A
  typical complaint might look like:
     Cannot write to `www.foraddift.non/aspx/index.asp?sid=463e'
     (file specification syntax error).
  Now, "?" is converted to "!", and "*" to "#".

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

   Version 1.9.1d  2005-06-21
   --------------------------

- Replaced the remaining instances where ".orig" was appended to a file
  name.  Now (for VMS), "_orig" is used everywhere (except in the
  documentation).

- Disabled (for VMS) the code which appended ".nnn" (decimal "nnn") to
  an output file name to avoid overwriting an existing file.  The file
  name is now used unchanged, and VMS file versioning is expected to
  prevent overwriting.  (The multi-dot file names made this way caused
  I/O errors on ODS2 file systems.)

- Changed to do translation of CR-LF line endings to local line endings
  for ASCII FTP transfers.

- Changed (on VMS) to use fixed-512 format for binary FTP files.  Added
  the new "--ftp-stmlf" option ("ftpstmlf on" command) to let the user
  restore the old behavior, where Stream_LF was used for both ASCII and
  binary.

- Added (on VMS) a new open callback function which senses (where
  supported) the process RMS_DEFAULT values for file extend quantity
  (deq), multi-block count (mbc), and multi-buffer count (mbf), and sets
  the FAB/RAB parameters accordingly.  The default deq is now much
  larger than before (16384, was none), and the default mbc is now 127
  (was none), speeding creation of a large output file.  Explicitly set
  RMS_DEFAULT values override built-in defaults.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

   Version 1.9.1c  2004-12-15
   --------------------------

- Added https (SSL) support, changing DESCRIP.MMS and VMS-WGET.COM, and
  adding WGET_SSL.OPT and WGET_SSL_HP.OPT.  Limited testing was done on
  Alpha, using HP SSL V1.1-B and OpenSSL version 0.9.7e.  The build
  procedure automatically enables SSL support if it can find one of the
  SSL header files.  The details are explained in comments in
  VMS-WGET.COM, including its new command-line options, HPSSL and NOSSL.

- Changed CONNECT.C to fix some problems with MultiNet TCP/IP software.

   - A build problem related to header files (netdb.h) when compiled
     with MultiNet.  Typical symptom:

         %CC-E-REDEFSTRUCT, In this declaration, the struct "hostent" is
         redefined. at line number 178 in module NETDB of text library
         SYS$COMMON:[SYSLIB]DECC$RTLDEF.TLB;1

   - A run-time problem with an FTP URL. Typical symptom: 

         accept: bad address

- Changed FTP.C to fix various problems with time-stamping ("-N",
  "--timestamping") when using an FTP URL.  Some of the misbehavior was
  specific to a VMS FTP server, but not all.  The typical result was a
  file with the modification date (and, in some cases, the creation
  date) left at the date of the download, rather than set to match the
  date on the server.  Some misleading diagnostic messages also have
  been corrected.

- Changed FTP-LS.C to fix a typographical error (unclosed comment).

- Changed NETRC.C to fix a potential build problem on non-VMS systems.
  Adding some VMS-specific code introduced some improper statement
  ordering, which may generate complaints (or failure) with some C
  compilers.

- Changed VMS.C to fix a problem with the utime() function (supplied
  with Wget) when the user specified a non-UNIX file name with the Wget
  "-O" option.  Typical symptom: 

     utime(home_sms:[sms.wget]t.h): error 0

  As the message might suggest, the file date-time was not being set
  correctly.  With this fix, utime() should work with either a UNIX-like
  or VMS-like file specification, and it should also set errno properly
  when it fails, enabling a more informative error message.

- New MMS/MMK description files and associated command procedures enable
  automatic source dependency generation.  Changed: DESCRIP.MMS and
  VMS-WGET.COM.  New: DESCRIP_DEPS.MMS, DESCRIP_MKDEPS.MMS,
  DESCRIP_SRC.MMS, COLLECT_DEPS.COM, and MOD_DEP.COM.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

   Version 1.9.1b  2004-09-28
   --------------------------

- Changed CONFIG.H_VMS to fix a build problem with C RTL before VMS
  V7.3-1 with C Runtime ECO V3.0 on Alpha:
     %LINK-W-NUDFSYMS, 2 undefined symbols:
     %LINK-I-UDFSYM,         DECC$GXSNPRINTF
     %LINK-I-UDFSYM,         DECC$GXVSNPRINTF
     [...]

- Changed VMS.C:

   - Fixed a build problem with C RTL before V7.2:
        %CC-E-UNDECLARED, In this statement, "DVI$C_ACP_F11V5" is not
        declared. at line number 262 in file
        disk:[xxx.WGET-1_9_1A_VMS.SRC]VMS.C;1

   - Fixed a build problem with C RTL before VMS V7.3 with C Runtime ECO
     2 on Alpha: 
        %CC-I-IMPLICITFUNC, In this statement, the identifier
        "decc$feature_get_index" is implicitly declared as a function.
        at line number 155 in file disk:[xxx.WGET-1_9_1A_VMS.SRC]VMS.C;1

        %CC-I-IMPLICITFUNC, In this statement, the identifier
        "decc$feature_get_value" is implicitly declared as a function.
        at line number 159 in file disk:[xxx.WGET-1_9_1A_VMS.SRC]VMS.C;1

        %CC-I-IMPLICITFUNC, In this statement, the identifier
        "decc$feature_set_value" is implicitly declared as a function.
        at line number 169 in file disk:[xxx.WGET-1_9_1A_VMS.SRC]VMS.C;1

     and the related: 

        %LINK-W-NUDFSYMS, 3 undefined symbols:
        %LINK-I-UDFSYM,         DECC$FEATURE_GET_INDEX
        %LINK-I-UDFSYM,         DECC$FEATURE_GET_VALUE
        %LINK-I-UDFSYM,         DECC$FEATURE_SET_VALUE
        [...]

   - Fixed a build problem with Multinet involving use of both the
     Multinet-specific and generic netdb.h header files:

        %CC-E-REDEFSTRUCT, In this declaration, the struct "hostent" is
        redefined. at line number 178 in module NETDB of text library
        SYS$COMMON:[SYSLIB]DECC$RTLDEF.TLB;1

   - Fixed a typographical error (unclosed comment) which kept the C RTL
     feature DECC$EFS_CHARSET from being enabled as intended. This
     caused a run-time problem on non-VAX systems with some ODS5
     extended file specifications with multiple dots in file names,
     typically when "-m" ("--mirror"), "-O" ("--output-document"), or
     "-r" ("--recursive") was used: 

        hostname/test/a.B.c: file specification syntax error
     or
        test/a.B.c: non-translatable vms error code: 0x186D4
        rms-f-syn, file specification syntax error

   - Fixed a spurious diagnostic message when doing HTTP and
     complex FTP transfers: 

        utime().  sys$parse() = 00010001.

- Changed FTP.C to fix a variety of problems encountered when using a
  VMS FTP server.  These problems (not seen in Wget 1.5.3h) were caused
  mostly by inappropriate code added (in the main Wget development
  stream) to "help" deal with a VMS FTP server.  Removing this code and
  reforming the procedure used for getting to relative directories seems
  to have fixed the problems.  As before, a single-slash URL
  ("ftp://host/one/two/name.type") will be treated as specifying a
  relative directory ("one/two"), while a double-slash URL
  ("ftp://host//one/two/name.type") will be treated as specifying an
  absolute directory ("/one/two").  As usual, the VMS FTP server will
  attempt to puzzle out whether "/one/two" should really be interpreted
  as "one:[two]", but it's better equipped to do it than Wget is.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

   Version 1.9.1a  2004-04-28
   --------------------------

- Changes to VMS-WGET.COM since Wget 1.5.3h:

   - A separate LIST command-line option now controls compiler listing
     files.  The DBG option no longer implies compiler listing files. 
     The LIST option also generates a link map with cross-reference.

   - The DBG option may actually work now.

   - Added support for IA64 architecture.

- Changes to DESCRIP.MMS since Wget 1.5.3h:

   - Compiler listing files (.LIS) and link maps (.MAP) are now put into
     the system-specific directory with the corresponding object files
     (.OBJ), namely, [.SRC.ALPHA], [.SRC.IA64], or [.SRC.VAX]. 
     Previously, they were put into the source directory ([.SRC])
     itself.

   - CLEAN target now deletes any .LIS and .MAP files.

   - New HELP target creates a Wget help library.

   - The OS_TYPE macro is now defined, including the VMS version.

   - No more complaints like "%MMS-W-MBREDEFILL, Illegal attempt to
     redefine macro 'CFLAGS'" on old versions of MMS.

   - A separate LIST macro now controls compiler listing files.  The
     DBG macro no longer implies compiler listing files.

   - New support for IA64 architecture.

- Built-in initialization of some DECC$* run-time options on non-VAX
  systems.  Currently this behavior is compiled in, so any changes must
  be made in the VMS.C source file.  These changes have no effect on VAX
  (or older Alpha) systems where the OS and C run-time support is
  absent.

   - Command-line case preservation.  Wget for VMS enables the following
     DECC$* run-time option:
        DECC$ARGV_PARSE_STYLE
     With extended command parsing enabled ("SET PROCESS
     /PARSE_STYLE = EXTENDED"), command-line case is preserved.  This
     obviates quoting case-sensitive option switches, URLs, and other
     parameters.

   - Extended file names on ODS5 file systems.  Wget for VMS enables the
     following DECC$* run-time options:
        DECC$EFS_CASE_PRESERVE
        DECC$EFS_CHARSET
     Thus, if an ODS5 file system is used as the destination for
     downloaded files, multiple dots will be permitted and case will be
     preserved in file names.  If an ODS2 file system is used, the same
     character substitutions will be made as in Wget 1.5.3h, namely,
     "." -> "_" (where needed), "~" -> "-", and "@" -> "$".

------------------------------------------------------------------------

      Changes to Source Files
      -----------------------


   Files new for VMS (all in [.SRC], except [.DOC]WGET.HLP, and
[]VMS_NOTES.TXT):

      COLLECT_DEPS.COM   Dependency collector for DESCRIP_MKDEPS.MMS.

      CONFIG.H_VMS       VMS-specific CONFIG.H.

      DESCRIP.MMS        Main MMK or MMS description ("make") file.

      DESCRIP_DEPS.MMS   Source dependency description file.

      DESCRIP_MKDEPS.MMS Source dependency generator description file.

      DESCRIP_SRC.MMS    Source list description file.

      VMS-WGET.COM       DCL procedure to build the executable from the
                         source, using MMK or MMS (and DESCRIP*.MMS).

      VMS.C              Various VMS-specific code: DEC C run-time
                         initialization, ODS2-ODS5 file system
                         detection, ODS2 file name adjustment, file open
                         callback function for better RMS parameters,
                         "utime()" replacement function, and for VMS
                         before V7.0, "getpwuid()" and "localtime_r()".

      VMS.H              VMS-specific header file.  For VMS C RTL before
                         V7.3, replacement for <utime.h>.  For VMS C RTL
                         before V7.0, replacement for <pwd.h>. 
                         Declarations of replacement data
                         types/structures and function prototypes for
                         VMS.C.

      VMS_IP.H           IP software type-specific "#include"
                         directives.  Used by HOST.C.

      VMS_NOTES.TXT      These notes.

      WGET.HLP           Simple help file.

      WGET.OPT           LINK options file.

      WGET_MULTINET.OPT  LINK options file supplement for Multinet.

      WGET_SSL.OPT       LINK options file supplement for OpenSSL.

      WGET_SSL_HP.OPT    LINK options file supplement for HP SSL.


   Files modified for VMS:

      CONNECT.C         Added code for VMS header files.  Changed some
                        variable names to avoid conflicts.

      CONVERT.C         Changed to use "_orig" instead of ".orig" as the
                        file name suffix used for the "-K"
                        ("--backup-converted") option.

      COOKIES.C         Fixed %CC-I-QUESTCOMPARE1 with a type cast.
                        Fixed %CC-W-LONGEXTERN by changing 33-character
                        symbol cookie_jar_generate_cookie_header to
                        cookie_jar_generate_cookie_hdr.

      COOKIES.H         Changed cookie_jar_generate_cookie_header to
                        cookie_jar_generate_cookie_hdr (COOKIES.C).

      FTP.C             Changed to eliminate a spurious -debug message
                        ("Unrecognized permissions for <directory>").
                        Changed to eliminate spurious complaints when
                        recursive downloads hit an empty directory.
                        Changed to fix various problems dealing with a
                        VMS FTP server.
                        Changed to fix various problems with FTP and
                        time-stamping ("-N").
                        Changed to distinguish between ASCII and binary
                        FTP transfers for proper local line endings.
                        Added (for VMS) VMS-specific file open options.

      FTP-LS.C          Fixed %CC-W-OUTTYPELEN from attempt to fprintf()
                        an int (u->port) using a "%hu" conversion
                        specifier.  It now uses "%d".
                        Changed to open the ".listing" file in text
                        mode, not binary.
                        Changed VMS FTP DIR-parsing code to tolerate
                        more variability (leading blank lines).
                        Added support for the environment variable
                        "WGET_TIMEZONE_DIFFERENTIAL" for file time
                        adjustment (as in Wget 1.5.3h for VMS).

      HOST.C            Added code for VMS header files.

      HOST.H            Added code for VMS header files.
      
      HTTP.C            Fixed multiple %CC-I-QUESTCOMPARE1 with type
                        casts.
                        Changed cookie_jar_generate_cookie_header to
                        cookie_jar_generate_cookie_hdr (COOKIES.C).
                        Added code to try using the logical name
                        "SYS$TIMEZONE_DIFFERENTIAL" for UTC-to-local
                        time conversions if "gmtime()" fails.
                        Added code to use SYS$TIMEZONE_DAYLIGHT_SAVING
                        in some local-UTC time conversions.
                        Changed to use an OS-specific file name suffix,
                        "_orig" for VMS, rather than a hard-coded
                        ".orig".
                        Added (for VMS) VMS-specific file open options.

      INIT.C            Changed to use "SYS$LOGIN:.wgetrc" as the name
                        of the initialization file on VMS.
                        Changed to open ".wgetrc" in text mode, not
                        binary.
                        Added (for VMS) "--ftp-stmlf" option ("ftpstmlf
                        on" command to force Stream_LF format (instead
                        of fixed-512) for binary FTP output files.

      MAIN.C            Changed the program identification debug message
                        ("built on OS_TYPE" instead of "on OS_TYPE").
                        Changed to set ODS5 flag if the download
                        destination is on an ODS5 file system.
                        Added (for VMS) "--ftp-stmlf" option ("ftpstmlf
                        on" command to force Stream_LF format (instead
                        of fixed-512) for binary FTP output files.

      NETRC.C           Changed to use "SYS$LOGIN:.netrc" as the name of
                        the user name and password file on VMS.

      OPTIONS.H         Added (for VMS) "--ftp-stmlf" option ("ftpstmlf
                        on" command to force Stream_LF format (instead
                        of fixed-512) for binary FTP output files.

      RETR.C            Changed to adjust CR-LF line endings in ASCII
                        FTP data to proper local line endings.
                        Changed to disable (for VMS) a useless fflush()
                        which was slowing file I/O.

      RETR.H            Changed to accomodate changes for FTP to
                        distinguish between ASCII and binary FTP
                        transfers for proper local ASCII line endings.

      URL.C             Added code to replace unacceptable dots (".")
                        (and other invalid characters) with underscores
                        ("_"), tildes ("~") with hyphens ("-"), and at
                        signs ("@") with dollar signs ("$"),
                        respectively, in URL-derived file specifications
                        on ODS2 file systems.

      UTILS.C           Added VMS-specific code to work around the lack
                        of "fork()".
                        Fixed multiple %CC-I-QUESTCOMPARE1 with type
                        casts.
                        Changed to assist ODS2 name reversion.
                        Changed to use a macro, UNIQ_SEP, instead of a
                        hard-coded ".", as the separator character when
                        adding a decimal "nnn" suffix to a file name to
                        make it unique.  Not defining UNIQ_SEP, as on
                        VMS, now disables adding the suffix.  (See
                        WGET.H.)

      VERSION.C         Changed "1.9.1" to "1.9.1d".

      WGET.H            Added a definition of an OS-specific file name
                        suffix, ORIG_SFX ("_orig", for VMS), to be used
                        instead of a hard-coded ".orig".  (See
                        CONVERT.C, HTTP.C.)
                        Added a definition of an OS-specific separator
                        character, UNIQ_SEP, for the decimal "nnn"
                        version number appended to a file name to make
                        it unique.  (See UTILS.C.)

------------------------------------------------------------------------

      Home source URL
      ---------------

http://www.gnu.org/directory/wget.html
http://ftp.gnu.org/gnu/wget/
ftp://ftp.gnu.org/gnu/wget/

------------------------------------------------------------------------

      Instructions
      ------------

   Extract the files from the distribution kit.  The kit includes
objects and executables for Alpha, IA64, and VAX, in the corresponding
host-specific subdirectories.  These builds were done with "DASHD" and
without SSL support.

   The command procedure VMS-WGET.COM attempts to determine the CPU
type, the IP package, and SSL availability.  It defines the appropriate
macros for MMK or MMS, and then builds the appropriate executable from
the source:

      SET DEFAULT [.WGET-1_9_1D.SRC]
      @ VMS-WGET CLEAN ! Deletes all object, list, map, and executable
                       ! files for the current CPU type.
      @ VMS-WGET       ! Copies CONFIG.H_VMS to CONFIG.H, compiles the
                       ! source files, and links the executable,
                       ! WGET.EXE.  Add "DASHD" to this command line to
                       ! enable the Wget "-d" command-line option.  Add
                       ! "DBG" to this command line to build a debug
                       ! executable.  Add "LIST" if compiler listing and
                       ! link map files are desired.
                       ! The distributed kit was built with "DASHD".
      @ VMS-WGET HELP  ! Creates a help library.  (Optional.  See
                       ! below.)

   Set the symbol for the foreign command:

      WGET :== $ actual_device:[actual.directory]WGET.EXE

   A very basic VMS HELP file is supplied in [.DOC]WGET.HLP.  To be
used, it must be added to a HELP library, either an existing library or
one just for Wget.  As mentioned above, a new Wget help library may be
created using the command, "@ VMS-WGET HELP".

   When a new HELP library is created, a logical name "HLP$LIBRARY_n"
may be defined to allow the HELP utility to find it.  See "HELP
/USERLIBRARY" for details.

   Try to read the original documentation to learn how to use Wget.  The
principal differences in this VMS adaptation are:

   "-b" ("background") has no effect.  (It uses "fork()".)

   Excess dots are replaced by underscores in URL-derived file
specifications on ODS2 file systems.  For example, on a UNIX system, the
URL "ftp://ftp.anywhere.org/x.y.z/a.b.c" could produce the local file
"ftp.anywhere.org/x.y.z/a.b.c".  With this VMS adaptation, the local
ODS2 file would be "[.FTP_ANYWHERE_ORG.X_Y_Z]A_B.C".  On an ODS5 file
system, the file would be [.ftp^.anywhere^.org.x^.y^.z]a^.b.c".

------------------------------------------------------------------------

   The accompanying source kit suffers from storage on an ODS2 file
system, which does not preserve case or allow multiple dots in a file
name.  Building this kit should work on VMS, but it may be expected to
fail on other systems.  To use this kit on a non-VMS system, the files
may need to be renamed to restore their original mixed-case and/or
multi-dot names.  The UNIX shell script "vms_name_fix.sh" (or something
like it) should do the job.

------------------------------------------------------------------------

   Steven M. Schweda               (+1) 651-699-9818  (voice)
   382 South Warwick Street        sms@antinode.org
   Saint Paul  MN  55105-2547

