archie/prospero/patches/LOG

1 April, 1994 -- swa merging in mitra patches.

10Mar93-18Nov93T+.patch -- ignored; generated by mitra but not in the
                        important list.  I reviewed it just to remind
                        myself what had changed and to set a context for 
                        this work.

Mitra wasn't kidding -- it really was 20,000 lines of difference.  A
lot of that was .PS files though, thank goodness.

Note: BCN has checked out a few source files; may be future problems.


Returned to work on April 5, at 8:45 pm.

10Mar94.newpsite.patch:  an existing ISI patch.  Ignored.

10Mar94.bug.patch:

 flushed include/p_open_tcp.h (gone from FILES).

 reordered include/ardp.h; all included
	[done]

 fixed mitra's concerns about dirsrv_v1.c.

bug1.patch:

	p_initialize() change makes sense.
	regex bug makes sense.
	merging it in.

debug.patch:
	debugging code looks ok.
	getting rid of some weird debugging code was fine.

purify.patch: done

purify1.patch: done

pfailure.patch:
	maybe pointless, as mitra said.  Added it with one minor
change: now active only when DEBUG_PFAILURE is defined, not when
NDEBUG is not defined.
        [done]
       
pfailure1.patch: [done]                

debugNwarn.patch: 

	I solved the ctype.h problem with isascii() in a
	different way, since isascii() isn't a part of the POSIX
	standard.  I just wrote an isascii() macro for the three
	places Prospero uses it.  So mitra's revised ctype.h didn't
	get included.


	lib/pfs/strtok.c is vestigial; the only place it's used is in
	the WAIS code, which should use the standard library version
	of strtok().  I'm removing lib/pfs/strtok.c from the
	distribution.

	other such functions that are now standardly available on new
	UNIX releases should be removed from the Prospero distribution
	as well, or at least moved to the misc directory.
        lib/pfs/strspn.c and others.

        I'd already fixed the -Wshadow `warnings with prototypes in
        include/ardp.h and include/pfs.h, but in a slightly different
        way than you did; six of one, half a dozen of the other.  I
        left it my way out of inertia; your solution was good too.

mitraopt.patch:
        my addition of support for GENERATION will change a bit of this.
        See the top level Makefile.  Note that I haven't tested this
        GENERATION macro.

        can you explain to me some time what you use the 'echobinfiles' to
        actually do?

        I modified .gdbinit to include most of your stuff.
        
        I added your definitions of 'tags' to the master makefile.

`       rest I did not include.

mitraopt1.patch:
        fixed the installdirs: target in the top-level Makefile that this fixes.
        see above comment about 'generation'
        Rest I did not include.

mitraopt2.patch:        
        did not include.

thread.patch:
        I fixed the bogus code you pointed out in dsrobject_v6.c and dsdir.c
        I did not include your program user/pfsinit, since I felt that
        it was not of widely general utility, except as an example.

        
---went to sleep at 4:15 am, 4/6/94---
12:23 pm, 4/6/94 -- back on

	finishing threads.patch:
	
	the patch includes modifications that I already made to my
sources and later applied to yours, but this doesn't affect you since
those patches are present in my source in any case.
	(lib/psrv/dsdir.c, lib/psrv/dsrfinfo.c, a couple of others).

thread1.patch:
        
        a nice series of patches.  many thanks.
        Adding the close() operations in case of error return was great.

        I am not entirely convinced of the validity of the changes to
the dispatching loop in dirsrv().  I have been looking over the
solaris manual pages for select() and friends; it's not clear to me
that select() is guaranteed not to block. I hope to be able to discuss
this with you.  

        [done]

doc.patch:

I did not include pandora.todolist in the prealpha, because that is an
internal document for Pandora and ISI to use, not a general part of
the release.  It's in my sources though and I'm being mindful of it.

I added documentation to the doc/system_calls file so that it woudln't
confuse a new user.

        [done]

aolwait.patch:
        did not include change to ARDP_DEFAULT_TIMEOUT from 4 to 6 in
include/ardp.h; this should be a special change you make in mitraopts,
I think. 

        I similarly did not include your changes to include/plog.h to
change the L_WTTIME_THRESHOLD, L_SYSTIME_THRESHOLD, and
L_COMP_SVC_THRESHOLD, and others that can be configured in
<pserver.h>.  I added documentation to include/pserver.h and
include/plog.h to make it clearer to the users that these defaults
can be overridden more elegantly by editing pserver.h.

Please let me know if you think we should change any of the defaults
in include/ardp.h or include/plog.h for general purpose use; I'm open
to suggestions.


	I did not make the changes in this patch to ardp_pr_actv.c or
ardp_send.c, because the comment in ardp_pr_actv.c file indicated that
it was a temporary change to test a particular theory.  Please let me
know if you want them to be a permanent part of the distribution.

	After reading through this patch and thinking about it, I did
not merge any of it into our sources.  We may want to discuss it if
there are things you feel are of general use.  I was uncomfortable
adding ARDP_WAIT_TOO_LONG to the standard ARDP library error codes
without further discussion with you.


	
solaris.patch:
	I changed ARDP_MY_WINDOW_SZ for Solaris from 9 to 6 in the
master sources; please let me know if this was correct.
        I am wondering if -lmalloc (which I see in the compilation
arguments in server/Makefile) is always necessary on SOLARIS, or
whether it's just a special-purpose thing that you're doing.  Please
let me know.
        None of the rest of this patch seemed like something that
should go into the master sources.

pand05.patch:
        unused.


6:46 pm: done


4/11/94, 

4:45 pm:  Done with pfs_threads.h.  All patches complete. 

Summary of work:
   /nfs/gost/build/10Mar94.patched/patches/done:
   10Mar94.bug.patch         10Mar94.doc.patch         10Mar94.purify1.patch 
   10Mar94.bug1.patch        10Mar94.pfailure.patch    10Mar94.thread.patch 
   10Mar94.debug.patch       10Mar94.pfailure1.patch   10Mar94.thread1.patch 
   10Mar94.debugNwarn.patch  10Mar94.purify.patch      pfs_threads.h 

   /nfs/gost/build/10Mar94.patched/patches/half-done:
   10Mar94.mitraopt.patch  

   /nfs/gost/build/10Mar94.patched/patches/ignored:
   10Mar93-18Nov93T+.patch      10Mar94.mitraopt2.patch 
   10Mar93-18Nov93T+.patch.~1~  10Mar94.newpsite.patch 
   10Mar94.aolwait.patch        10Mar94.pand05.patch 
   10Mar94.mitraopt1.patch      10Mar94.solaris.patch 


Issue: Our overriding stdlib.h has the difficulty that 'putenv'
doesn't appear in the include files.  Neither POSIX nor ANSI C have a
standard environment setting function.  

Later cleanup:

ctype.h: see above.
stdlib.h:
	I #ifdef'd this with SOLARIS, renamed it solaris_stdlib.h

renamed fout_of_memory to p__fout_of_memory.  Renamed is_out_of_memory
to p__is_out_of_memory.  Renamed finternal_error() to p__finternal_error().


Moved re_comp_exec to lib/pfs/re_comp_exec.c.  Renamed it to p__re_comp_exec.c.

changed dirsrv_internal_error_handler() back, since the test against
the out_of_memory variable seemed incorrect to me.  Please talk to me
if you think this change should go into the release.

Fixed problems with fprintf() in lib/psrv/plog.c

Looked at the pcompat library again, significant restructuring:
	app/Makefile.pcompat and app/Makefile.nopcompat are now both just 
	app/Makefile. 

	lib/pcompat/Makefile.pcompat and
        lib/pcompat/Makefile.nopcompat are  now both just
        lib/pcompat/Makefile.  


in list_macros.h, please explain this comment:

/* someone else uses the above code, they just don't have it in a macro!*/

---
ietftypes is now installed just in P_BINARIES by default, not in /usr/local/lib
---

April 29, 1994:

Started installing a new batch of MITRA patches, completed on 27 April 1994.

purify2.patch -- done

thread.patch:

this was modified from the first time I applied thread.patch.  This
time, pfsinit and pfsinit+ were no longer in the patch file.  That
seemed appropriate to me.  IGNORED

thread2.patch:

internal_err.c has changed in such a way that the changes this patch made to
internal_err.c are no longer useful..


I had already made some of the changes to lib/psrv/dsrfinfo.c, but in
a slightly different way.

thread3.patch: done

May 1, 1994

thread4.patch: stuff for vcache.  looks good.  Some of the function
argument renaming things (done for the sake of -Wshadow) were things
I'd also done in a different way.  DONE.


doc1.patch: DONE.

aolwait.patch:

I had to be careful with this one. I assume you did not intend to put the changes to plog.h into general distribution.

I also did not understand the changes you made to ardp_pr_actv.c and
therefore I did not apply them.

I also did not understand why you turned off the retry count.

In general, I do not understand the ARDP_WAIT_TOO_LONG error and how
it fits into our general interface.  As it stands, it seems
AOL-specific enough that I'm uncomfortable incorporating it as it
stands into our sources; I would be glad to discuss some general
mechanism we could use to accomplish the same end.  One is clearly
needed to solve this problem (swa: Discuss with BCN)

I think the change (to call ardp_rwait when there's a queue) is good.
I think we need a general hook in the ARDP code so that you can set
such a function from a higher level; that section of the ARDP code is
not general enough (for instance, it currently contains archie-specific code.)

(NOT APPLIED)

dnscache.patch:

I am not comfortable with configuring the behavior of ARDP library
stuff in <pserver.h>.

a) sometimes people compile just the Prospero clients, not the server.
This makes the whole ARDP library dependent upon <PSERVER.H>

b) ARDP.H needs its own separate configuration file so that it can be
separated from the rest of Prospero.

This change has been incorporated into the sources; we should change
the configuration in the future.

c) your change to lib/pfs/myhost.c was one I'd made in a
slightly different way.

DONE

dnscache1.patch:  DONE.  Good patch.

dircache.patch:

This is a note to myself to reconcile include/pserver.h with the
master include/pserver.h.dist.  This patch changed pserver.h.  In the
future, changes to the distribution pserver.h should be made to
include/pserver.h.dist.  I will have to implement this change to our
configuration proceedings.

PATCH DONE

memleaks.patch:  GOOD PATCH.  DONE

ackall.patch:  This is not listed in apply.patches.  I assume that it
should not be applied. In any case, the change to lib/ardp/ardp_accept.c should not be inside an #ifdef PFS_THREADS.

NOT APPLIED.

solaris.patch:

This patch has configuration changes that should only be configured
under solaris interleaved with things that should be in the general
distribution (e.g., the comments on what you need to configure to use
SOLARIS).  I have used judgement to make the changes that seemed to me
to be general.   

We need a more general way of modifying <ardp.h>

PARTLY APPLIED


solaris1.patch, solaris2.patch:
pfsinit doesn't striike me as generally useful.

        NOT APPLIED


All patches through April 27th now applied.

Now to compile (without PFS_THREADS):

Prospero wouldn't compile with PFS_THREADS turned off until I #if 0'd out
the special code in <pfs_threads.h> that redefined locked_fopen()
in case of threadless compilation.

That code should be reinstated, for efficiency, but only when we add
appropriate #ifdefs to comment out the whole file locking set of code
when not multi-threaded.



I have a new version of include/pfs_threads.h that is compatible with yours.

-- 

I made a significant change to lib/pfs/pmap_cache.c: instead of
EXECVCACHE being the special option, the special option is now called
INCREASE_CLIENT_EXECUTABLE_SIZE_DO_NOT_EXEC_VCACHE.  It is heavily
commented in include/psite.h, with an explanation of the circumstances
under which one might enable it.  The default is to definitely not
have it enabled.

If you would like the vcache library to stay a part of the release,
then that libpvcache code will need a great deal of work, work that we
don't have the time to put into it.  All of the external identifiers
in libpvache() will have to start with a p__vcache, except for a few
strictly defined interfaces.   Otherwise, anybody in the outside
linking with that library will encounter problems.

Also, the interface to that library should be documented somewhere
(just a half-page explaining how you interact with it and how you link
with it).  Also, the dependencies problems will need to be fixed.
Right now, it's really awkward to use and to explain to others how to
use it; thus my reluctance to include it as a regular part of the
distribution.

Also, it's not clear to me that that code doesn't memory leak; if it
doesn't, please change the comment at the head of vcache.c to explain
that we've been using it commercially and have avoided leaks by that
means.

I am leaving the code in right now, but in an unsupported status,
because of the above-mentioned problems with it.


--
The client-side caching in the VCACHE library is not something I'm
comfortable putting into the release right now.  The 
 Also, the caching strategy needs to be
documented in the library manual.  Also, a policy for cleaning old
objects out of the cache and keeping it from taking over the disk
space needs to appear.


--

Another issue is that comments in our code should be appropriately
spelled.  It's full of "dont" and "doesnt" and "wont" and "couldnt"
---
I made many changes to the configuration stuff.

I have cut out your 'bintar' and 'echobinfiles' dependencies; if you need
them back we can talk about it.
--

I have gotten rid of  your test dependencies in user/vcache/Makefile.
---
I got rid of the 'pure' dependency in the top-level and server makefiles.   The code is here if you want to use it for yourself:

in top Makefile:

##
## This target is used only when using PURIFY.
## Ignore it for normal work.
## This only debugs the server for now.  Needs to be modified for your needs.
## Mitraism.

pure:
	for i in server ; do \
            ( cd $$i; \
            echo "Making in $$i"; \
	    $(MAKE) $(PASTHRU) pure 2>&1 | $(SED-SCRIPT) ) ;\
	done

in Makefile.config:
##
## PURIFY
## 
## Purify is a developer's tool used to check for memory leaks and other error
## conditions.
## If you have never heard of it or if you don't use it, just don't worry 
## about it.

# If you use Purify, you can specify a command syntax for it
# Make a purified dirsrv with 'make pure'.
PURIFY = purify


# PURE_LIBS is usually LIBS , except on Solaris when it excludes thread
# No idea why Mitra needed -lmalloc.
# PURE_LIBS = -lsocket -lnsl
PURE_LIBS = $(LIBS) -lmalloc 

# This is for PURIFY
# Note that problems with threads may have to be debugged with -g, not 
# with -ggdb3. 

PURE_CFLAGS  = -g $(GCCWARN) $(GCCTRY)


in server/MAkefile:
## Used when using the special PURIFY package.
## It is not clear to me why this target is needed, but mitra uses it.
## It may be yanked out of the full release.

pure:	pure_dirsrv

pure_dirsrv: $(DIRSRV_OBJECTS) ${SRV_LIBS_DEPENDENCIES} 
	${PURIFY} ${CC} ${F_CCL} -o $@ $(DIRSRV_OBJECTS) \
$(SRV_LIBS) $(PURE_LIBS) 

--- 
gopher_gw_diag_mutexes() and gopher_gw_init_mutexes now called
from DIRSRV,, not lib/psrv/psrv_mutexes.c.  This eliminates
ordering/dependency problems.

-- 
another practice that should be eliminated is that of declaring
subfunctions inside master files; they should always be in include files.

-- 

It is not clear to me why VLDEBUGBEGIN and VLDEBUGEND should be left
in the sources.  Ditto CHECK_MEM(). Is there any reason why they are
still useful?  If not generally useful, they should be removed before
the release.