From:	MERC::"uunet!CRVAX.SRI.COM!RELAY-INFO-VAX"  1-MAY-1992 20:42:40.39
To:	INFO-VAX@SRI.COM
CC:	
Subj:	Re: Notes Compression

    Since I couldn't find an address, I'm sending this to the list.
    The following came from the Bboard list, and is a life-saver!  Local
    mods are the min-note value (originally 5), the submission date routine
    and some of the file purging and listing commands.  I apologize for the
    size of this thing ...

    	Mary

                            __ __ __ __ __ __
 Mary T. Chadbourne, Bitnet Postmaster/Techrep    Maricopa  CGC   EMC   GC 
 the Maricopa Colleges, Phoenix, Arizona USA      GWC       LIB   MC    PC 
 1+ (602) 996-9366                                PVC       RIO   SCC   SMC
 BITNET: chadbour@maricopa.bitnet        INTERNET:  chadbourne@maricopa.edu  
                            -- -- -- -- -- --


$ ! BMC 24-Feb-1990
$ !	Initial version
$ ! BMC 07-Mar-1990
$ !	Fix actually doing the CONVERT. We need a .FDL without any ALLOCATION
$ !		clauses.
$ !	Purge only on date. Don't use size as a limiting factor.
$ ! BMC 22-Apr-1990 (Geez, it takes forever to get things done... (sigh))
$ !	Added resubmit and only-run-on-day-selected code
$ !	PURGE of Notes files works now. Needs SYSPRV when running, though...
$ !
$ sver = 'f$verify(0)
$ !
$ ! Set up our defaults. This is (most likely) the section that you'll be
$ ! changing to customize for your site.
$ !
$ expire_days = 30			! 60 days seems reasonable to me...
$! run_on_day = "Sunday"			! Only purge on Sundays
$ run_on_time = "01:30"			! Sunday _mornings_, that is...
$ never_purge = ":X-INFO-BBOARD:X-UNKNOWN:NEAX:ASTRA:X-MAC-L:"
$ archive_Notes = "F"			! EXTRACT the Notes before deleting?
$ min_note = 2				! Only purge in 2-*
$ !
$ ! Find out where we're located and move there
$ !
$ spec = f$environment("PROCEDURE")
$ home_spec = f$element(0, "]", spec)+"]"	! It looks wierd, but it works.
$ set default 'home_spec'
$ !
$ ! Resubmit ourselves to run next Sunday (after figuring out where we are,
$ !  what we are, etc.
$ !
$ com_log_file = home_spec+f$parse(spec,,,"NAME","SYNTAX_ONLY")+".LOG"
$ purge/log/keep=3 'com_log_file
$! ***
$ 	! Modified to run weekly
$	! early Sunday morning  MTC 02-JUL-1990
$ !
$ dat = f$time()		
$ day=f$cvtime(dat,,"WEEKDAY")	
$ if day .eqs. "Sunday" then dlt = "6"
$ if day .EQS. "Saturday" then dlt = "7"	! standard interval
$ if day .eqs. "Friday" then dlt = "1"		!assume not two in a row
$ if day .eqs. "Thursday" then dlt = "2"	! otherwise, just bump delta
$ if day .eqs. "Wednesday" then dlt = "3"
$ if day .eqs. "Tuesday" then dlt = "4"
$ if day .eqs. "Monday" then dlt = "5"
$do_sub:
$ write sys$output "   do_sub DLT=''DLT'"
$ submit 'spec /noprint/log='com_log_file/restart/after="tomorrow+''dlt'-''run_on_time'"/queue=mail$batch/restart
$ !
$ ! Now check to see if we should actually keep going for today.
$ !
$! only_run_on_day = f$edit(run_on_day, "LOWERCASE")
$! cur_day = f$edit(f$cvtime(f$time(),, "WEEKDAY"), "LOWERCASE")
$! if only_run_on_day .eqs. cur_day then goto Run_today	! Winner...
$! write sys$output "Only going to run on ''run_on_day'."
$! write sys$output "That's not today, so we're not going to run now."
$ set noverify
$ goto Run_today
$ ! exit
$ !
$Run_today:
$ !write sys$output "   Run_today   "
$ !
$ ! Let's figure out the data that we need to know from what they gave us
$ !
$ never_purge_len = f$length(never_purge)
$ expire_date = f$cvtime("TODAY-''expire_days'-","COMPARISON")
$ !
$ ! First make sure that we're on the right node...
$ !
$ lib_spec = f$trnlnm("notes$library")
$ notes_node = f$parse(lib_spec,,,"NODE","SYNTAX_ONLY")
$ if notes_node .eqs. f$trnlnm("SYS$NODE") then goto Start
$ if notes_node .eqs. "" then goto start      ! no node in definition 7-19-90 MTC
$ write sys$output "%NOTPURG-F-WRONGNODE You must be on the NOTES$LIBRARY node."
$ exit
$ !
$ ! Time to start this thing going...
$ !
$Start:
$! write sys$output "   Start   "
$ dir/date=all/size   notes$library:X-*.note;
$ lib_spec = lib_spec-notes_node
$ file_spec = lib_spec+"X-*.NOTE"
$ !
$ ! Now let's start looking at files
$ !
$ open/write temp notes_list.out
$ close temp
$ !
$ did_one = "F"
$ !
$Top_of_file_search:
$ write sys$output "   Top_of_file_search   "
$ next_file = f$search(file_spec)
$ if next_file .eqs. "" then goto Past_file_search
$ fname = f$parse(next_file,,,"NAME","SYNTAX_ONLY")
$ if f$locate(":"+fname+":", never_purge) .lt. never_purge_len then -
	goto Top_of_file_search
$ did_one = "T"
$ open/write scan_it notes_scan.com
$ write scan_it "DEFINE/USER SYS$OUTPUT notes_temp.tmp"
$ write scan_it "$ NOTES/NONOTEBOOK "+fname
$ write scan_it "dir ''min_note'-l"
$ write scan_it "exit"
$ write scan_it "$ EXIT"
$ close scan_it
$ @notes_scan
$! dir/date/size notes_scan.com
$ delete notes_scan.com;
$ !
$ ! Now let's parse the output
$ !
$ open/append/share=read clean_up notes_list.out
$ write clean_up "F "+fname
$ WRITE SYS$OUTPUT "F ''fname' "
$ open/read dir_in notes_temp.tmp
$Dir_scan_start:
$ !write sys$output "   Dir_scan_start   "
$ read dir_in line
$ if f$element(1, " ", line) .nes. "Topic" then goto Dir_scan_start
$ !
$ ! Now the next line is useful...
$ !
$ Initial = "E "			! Assume it's expired...
$Dir_scan_real:
$ !write sys$output "   Dir_scan_real   "
$ read/end=Dir_scan_Done dir_in line
$ line = f$edit(line, "COMPRESS")
$ if line .eqs. "" then goto Dir_scan_real
$ if f$element(1, " ", line) .eqs. "End" then goto Dir_scan_Done
$ if f$integer(f$element(1, " ", line)) .lt. Min_Note then goto Dir_scan_real
$ topic_date = f$cvtime(f$element(3, " ", line), "COMPARISON")
$ if topic_date .ges. expire_date then Initial = "U "
$ write clean_up Initial+f$element(1, " ", line)+" "+f$element(4, " ", line)+ -
	" "+f$element(3, " ", line)
$ goto Dir_scan_real
$ !
$Dir_scan_Done:
$ !write sys$output "   Dir_scan_Done   "
$ write clean_up ""
$ close clean_up
$ close dir_in
$ !dir/date/size notes_temp.tmp
$ delete notes_temp.tmp;
$ !
$ goto Top_of_file_search
$ !
$Past_file_search:
$ write sys$output "   Past_file_search   "
$ !
$ if did_one then goto Parse_Initial
$ write sys$output "%NOTPURG-F-NONETOPURGE No files to purge"
$ delete notes_list.out;
$ goto exit_purge
$ !
$ ! Now, first parse pass of the file...
$ !
$Parse_Initial:
$! write sys$output "   Parse_Initial   "
$ did_one = "F"
$ open/read file_list notes_list.out
$ open/write file_purge_list notes_purge.ctl
$Next_list_record:
$ read/end=List_scan_Done file_list line
$ record_command = "Parse_"+f$element(0, " ", line)
$ goto 'record_command'
$Parse_F:
$! write sys$output "   Parse_F   "
$ filename = f$element(1, " ", line)
$ expired = 0
$ unexpired = 0
$ open/write expire_list expired.tmp
$ open/write unexpire_list unexpired.tmp
$ goto Next_list_record
$Parse_E:
$! write sys$output "   Parse_E   "
$ expired = expired + 1
$ write expire_list line
$ goto Next_list_record
$Parse_U:
$! write sys$output "   Parse_U   "
$ unexpired = unexpired + 1
$ write unexpire_list line
$ goto Next_list_record
$Parse_:
$! write sys$output "   Parse_   "
$ close expire_list
$ close unexpire_list
$ if expired .le. 0 then goto File_list_indiv_clean
$ !
$File_list_copy:
$! write sys$output "   File_list_copy   "
$ did_one = "T"
$ write file_purge_list "F "+filename
$ open/read expire_list expired.tmp
$File_list_copy_1:
$! write sys$output "   File_list_copy_1   "
$ read expire_list line
$ write file_purge_list line
$ expired = expired - 1
$ if expired .gt. 0 then goto File_list_copy_1
$ write file_purge_list " "
$ close expire_list
$File_list_indiv_clean:
$! write sys$output "   File_list_indiv_clean   "
$! dir/date/size *.tmp
$! delete expired.tmp;
$! delete unexpired.tmp;
$ goto Next_list_record
$ !
$List_scan_Done:
$! write sys$output "   List_scan_Done   "
$ close file_list
$ close file_purge_list
$! dir/date/size notes_list.out
$ delete notes_list.out;
$ if did_one then goto Purge_build
$ write sys$output "%NOTPURG-F-NONETOPURGE No files to purge"
$ dir/date/size notes_purge.ctl
$ delete notes_purge.ct*
$goto  exit_purge
$ !
$ ! Now we start building the actual delete command file (Oy!)
$ !
$Purge_build:
$ write sys$output "   Purge_build   "
$ !
$ open/write/share=read purge_cmd notes_purge_execute.com
$ open/read purge_list notes_purge.ctl
$Next_purge_record:
$ read/end=Purge_build_Done purge_list line
$ record_command = "Purge_"+f$element(0, " ", line)
$ goto 'record_command'
$Purge_F:
$! write sys$output "Purge_f "
$ fname = f$element(1, " ", line)
$ write purge_cmd "$ WRITE SYS$OUTPUT ""Deleting topics from ''fname'"""
$ write purge_cmd "$ NOTES/NONOTEBOOK "+fname
$ write purge_cmd "SET MODERATOR"
$ goto Next_purge_record
$Purge_E:
$! write sys$output "Purge_E":
$ topic = f$element(1, " ", line)
$ if archive_Notes then write purge_cmd "EXTRACT/APPEND ''fname'."+ -
	f$element(3, " ", line)+" ''topic'.*"
$ replies = f$integer(f$element(2, " ", line))
$Purge_E_1:
$! write sys$output "Purge_E_1"
$ write purge_cmd "DELETE/NOCONFIRM NOTE ''topic'."+f$string(replies)
$ replies = replies - 1
$ if replies .ge. 0 then goto Purge_E_1
$ goto Next_purge_record
$Purge_:
$! write sys$output "Purge_ "
$ write purge_cmd "EXIT"
$ full_filename = lib_spec+fname+".NOTE"
$ write purge_cmd "$ CONVERT/FDL=NOTES_PURGE.FDL "+full_filename+" "+-
	full_filename
$ write purge_cmd "$ SET FILE/PROT=(S:RWED) "+full_filename+";-1"
$ write purge_cmd "$ PURGE "+full_filename
$ goto Next_purge_record
$ !
$Purge_build_Done:
$ write sys$output "   Purge_build_Done   "
$ write purge_cmd "$ EXIT"
$ close purge_cmd
$ close purge_list
$ dir/date/size notes_purge.ctl
$ !
$ Jnet_node = f$edit(f$trnlnm("JAN_LOCAL_NODE"), "TRIM")
$ BBoard_name = f$edit(f$trnlnm("BBOARD_HOOK_NAME"), "TRIM")
$ write sys$output "Suspend Bboard "
$ send 'BBoard_name'@'Jnet_node' suspend
$ !
$ oldpriv = f$setprv("WORLD")
$ pid_context = ""
$ !
$Find_Notes_PID:
$ pid = f$pid(pid_context)
$ if pid .eqs. "" then goto Ready_to_run        ! didn't find a Notes process
$ if f$getjpi(pid, "MODE") .nes. "NETWORK" then goto Find_Notes_PID
$ if f$extract(0, 6, f$getjpi(pid, "PRCNAM")) .nes. "NOTES$" then -
	goto Find_Notes_PID
$ !
$ ! Now we've got the PID, next loop until it's not serving anyone
$ !
$Wait_until_clear:
$ prcnam = f$getjpi(pid, "PRCNAM")
$ num = f$integer(f$element(0, "*", f$element(1, "_", prcnam)))
$ if num .eq. 0 then goto Ready_to_run
$ wait 00:00:15 			! 15 seconds should be good
$ goto Wait_until_clear
$ !
$ ! Finally, we're clear to go for it
$ !
$Ready_to_run:
$ write sys$output "  Ready_to_run   "
$ dir/date/size notes_purge_execute.com
$ x = f$setprv(oldpriv)
$ @notes_purge_execute.com
$ write sys$output "Resume Bboard "
$ send 'BBoard_name'@'Jnet_node' resume
$ dir/date/size notes_purge_execute.com
$ delete/BEFORE=TODAY notes_purge_execute.com;*,notes_purge.CTL;*,*.SAV;*,*.tmp;*
$ exit_purge:
$ dir/date=all/size notes$library:X-*.note;
$ exit

    <NOTES_PURGE.FDL >

SYSTEM
	SOURCE                  VAX/VMS

FILE
	BEST_TRY_CONTIGUOUS     yes
	BUCKET_SIZE             3
	CLUSTER_SIZE            2
	CONTIGUOUS              no
	EXTENSION               15
	FILE_MONITORING         no
	GLOBAL_BUFFER_COUNT     0
	ORGANIZATION            indexed
	PROTECTION              (system:RWE, owner:RWE, group:RW, world:RW)

RECORD
	BLOCK_SPAN              yes
	CARRIAGE_CONTROL        none
	FORMAT                  variable
	SIZE                    1024

AREA 0
	BEST_TRY_CONTIGUOUS     yes
	BUCKET_SIZE             3
	EXTENSION               15

AREA 1
	BEST_TRY_CONTIGUOUS     yes
	BUCKET_SIZE             3
	EXTENSION               3

AREA 2
	BEST_TRY_CONTIGUOUS     yes
	BUCKET_SIZE             3
	EXTENSION               12

KEY 0
	CHANGES                 no
	DATA_KEY_COMPRESSION    no
	DATA_RECORD_COMPRESSION yes
	DATA_AREA               0
	DATA_FILL               100
	DUPLICATES              no
	INDEX_AREA              1
	INDEX_COMPRESSION       no
	INDEX_FILL              100
	LEVEL1_INDEX_AREA       1
	NAME                    ""
	NULL_KEY                no
	PROLOG                  3
	SEG0_LENGTH             4
	SEG0_POSITION           0
	TYPE                    bin4

KEY 1
	CHANGES                 yes
	DATA_KEY_COMPRESSION    no
	DATA_AREA               2
	DATA_FILL               100
	DUPLICATES              yes
	INDEX_AREA              2
	INDEX_COMPRESSION       no
	INDEX_FILL              100
	LEVEL1_INDEX_AREA       2
	NAME                    ""
	NULL_KEY                no
	SEG0_LENGTH             4
	SEG0_POSITION           72
	TYPE                    bin4

KEY 2
	CHANGES                 yes
	DATA_KEY_COMPRESSION    yes
	DATA_AREA               2
	DATA_FILL               100
	DUPLICATES              yes
	INDEX_AREA              2
	INDEX_COMPRESSION       yes
	INDEX_FILL              100
	LEVEL1_INDEX_AREA       2
	NAME                    ""
	NULL_KEY                yes
	NULL_VALUE              0
	SEG0_LENGTH             64
	SEG0_POSITION           4
	TYPE                    string

                 <<< SYS1:[NOTES$LIBRARY]X-INFO-VAX.NOTE;38 >>>
                          -< Info-VAX Mailing List  >-
================================================================================
Note 109.145        Info-VAX Postings for 04/26/92 - 05/02/92         145 of 158
DIST::INFOVAX "GONE MAD, WILL BE BACK LATER <stu925" 28 lines  30-APR-1992 22:07
                            -< Notes Compression? >-
--------------------------------------------------------------------------------
Hi,
 
I am having a problem.  When deleting notes off a noteboard, the size of the
Note file does not decrease.  Is there a way to fix this (somehow I thought I
saw a post about it a while ago, but as I didn't use Notes at the time, I
didn't think to grab it... but I could be wrong.)
 
                                Thanks
                                   -Medvitz
 
 
 
 
-----------------------------------------------------------------------
Dave Medvitz, CCG
Grove City College
 
        "Excuse me... What does God need with a starship?
                                        -Capt James T Kirk
 
        "Where are we going?"
        "Where they went."
        "What if they went nowhere?"
        "Then this will be your big chance to get away from it all"
                                        -Kirk & McCoy
 
        "I'm always in trouble, Lady Polgara, I wouldn't
                know how to act if I weren't"


