1 April, 1994 -- swa merging in mitra patches. 10Mar93-18Nov93T+.patch -- ignored; generated by mitra but not in the important list. I reviewed it just to remind myself what had changed and to set a context for this work. Mitra wasn't kidding -- it really was 20,000 lines of difference. A lot of that was .PS files though, thank goodness. Note: BCN has checked out a few source files; may be future problems. Returned to work on April 5, at 8:45 pm. 10Mar94.newpsite.patch: an existing ISI patch. Ignored. 10Mar94.bug.patch: flushed include/p_open_tcp.h (gone from FILES). reordered include/ardp.h; all included [done] fixed mitra's concerns about dirsrv_v1.c. bug1.patch: p_initialize() change makes sense. regex bug makes sense. merging it in. debug.patch: debugging code looks ok. getting rid of some weird debugging code was fine. purify.patch: done purify1.patch: done pfailure.patch: maybe pointless, as mitra said. Added it with one minor change: now active only when DEBUG_PFAILURE is defined, not when NDEBUG is not defined. [done] pfailure1.patch: [done] debugNwarn.patch: I solved the ctype.h problem with isascii() in a different way, since isascii() isn't a part of the POSIX standard. I just wrote an isascii() macro for the three places Prospero uses it. So mitra's revised ctype.h didn't get included. lib/pfs/strtok.c is vestigial; the only place it's used is in the WAIS code, which should use the standard library version of strtok(). I'm removing lib/pfs/strtok.c from the distribution. other such functions that are now standardly available on new UNIX releases should be removed from the Prospero distribution as well, or at least moved to the misc directory. lib/pfs/strspn.c and others. I'd already fixed the -Wshadow `warnings with prototypes in include/ardp.h and include/pfs.h, but in a slightly different way than you did; six of one, half a dozen of the other. I left it my way out of inertia; your solution was good too. mitraopt.patch: my addition of support for GENERATION will change a bit of this. See the top level Makefile. Note that I haven't tested this GENERATION macro. can you explain to me some time what you use the 'echobinfiles' to actually do? I modified .gdbinit to include most of your stuff. I added your definitions of 'tags' to the master makefile. ` rest I did not include. mitraopt1.patch: fixed the installdirs: target in the top-level Makefile that this fixes. see above comment about 'generation' Rest I did not include. mitraopt2.patch: did not include. thread.patch: I fixed the bogus code you pointed out in dsrobject_v6.c and dsdir.c I did not include your program user/pfsinit, since I felt that it was not of widely general utility, except as an example. ---went to sleep at 4:15 am, 4/6/94--- 12:23 pm, 4/6/94 -- back on finishing threads.patch: the patch includes modifications that I already made to my sources and later applied to yours, but this doesn't affect you since those patches are present in my source in any case. (lib/psrv/dsdir.c, lib/psrv/dsrfinfo.c, a couple of others). thread1.patch: a nice series of patches. many thanks. Adding the close() operations in case of error return was great. I am not entirely convinced of the validity of the changes to the dispatching loop in dirsrv(). I have been looking over the solaris manual pages for select() and friends; it's not clear to me that select() is guaranteed not to block. I hope to be able to discuss this with you. [done] doc.patch: I did not include pandora.todolist in the prealpha, because that is an internal document for Pandora and ISI to use, not a general part of the release. It's in my sources though and I'm being mindful of it. I added documentation to the doc/system_calls file so that it woudln't confuse a new user. [done] aolwait.patch: did not include change to ARDP_DEFAULT_TIMEOUT from 4 to 6 in include/ardp.h; this should be a special change you make in mitraopts, I think. I similarly did not include your changes to include/plog.h to change the L_WTTIME_THRESHOLD, L_SYSTIME_THRESHOLD, and L_COMP_SVC_THRESHOLD, and others that can be configured in . I added documentation to include/pserver.h and include/plog.h to make it clearer to the users that these defaults can be overridden more elegantly by editing pserver.h. Please let me know if you think we should change any of the defaults in include/ardp.h or include/plog.h for general purpose use; I'm open to suggestions. I did not make the changes in this patch to ardp_pr_actv.c or ardp_send.c, because the comment in ardp_pr_actv.c file indicated that it was a temporary change to test a particular theory. Please let me know if you want them to be a permanent part of the distribution. After reading through this patch and thinking about it, I did not merge any of it into our sources. We may want to discuss it if there are things you feel are of general use. I was uncomfortable adding ARDP_WAIT_TOO_LONG to the standard ARDP library error codes without further discussion with you. solaris.patch: I changed ARDP_MY_WINDOW_SZ for Solaris from 9 to 6 in the master sources; please let me know if this was correct. I am wondering if -lmalloc (which I see in the compilation arguments in server/Makefile) is always necessary on SOLARIS, or whether it's just a special-purpose thing that you're doing. Please let me know. None of the rest of this patch seemed like something that should go into the master sources. pand05.patch: unused. 6:46 pm: done 4/11/94, 4:45 pm: Done with pfs_threads.h. All patches complete. Summary of work: /nfs/gost/build/10Mar94.patched/patches/done: 10Mar94.bug.patch 10Mar94.doc.patch 10Mar94.purify1.patch 10Mar94.bug1.patch 10Mar94.pfailure.patch 10Mar94.thread.patch 10Mar94.debug.patch 10Mar94.pfailure1.patch 10Mar94.thread1.patch 10Mar94.debugNwarn.patch 10Mar94.purify.patch pfs_threads.h /nfs/gost/build/10Mar94.patched/patches/half-done: 10Mar94.mitraopt.patch /nfs/gost/build/10Mar94.patched/patches/ignored: 10Mar93-18Nov93T+.patch 10Mar94.mitraopt2.patch 10Mar93-18Nov93T+.patch.~1~ 10Mar94.newpsite.patch 10Mar94.aolwait.patch 10Mar94.pand05.patch 10Mar94.mitraopt1.patch 10Mar94.solaris.patch Issue: Our overriding stdlib.h has the difficulty that 'putenv' doesn't appear in the include files. Neither POSIX nor ANSI C have a standard environment setting function. Later cleanup: ctype.h: see above. stdlib.h: I #ifdef'd this with SOLARIS, renamed it solaris_stdlib.h renamed fout_of_memory to p__fout_of_memory. Renamed is_out_of_memory to p__is_out_of_memory. Renamed finternal_error() to p__finternal_error(). Moved re_comp_exec to lib/pfs/re_comp_exec.c. Renamed it to p__re_comp_exec.c. changed dirsrv_internal_error_handler() back, since the test against the out_of_memory variable seemed incorrect to me. Please talk to me if you think this change should go into the release. Fixed problems with fprintf() in lib/psrv/plog.c Looked at the pcompat library again, significant restructuring: app/Makefile.pcompat and app/Makefile.nopcompat are now both just app/Makefile. lib/pcompat/Makefile.pcompat and lib/pcompat/Makefile.nopcompat are now both just lib/pcompat/Makefile. in list_macros.h, please explain this comment: /* someone else uses the above code, they just don't have it in a macro!*/ --- ietftypes is now installed just in P_BINARIES by default, not in /usr/local/lib --- April 29, 1994: Started installing a new batch of MITRA patches, completed on 27 April 1994. purify2.patch -- done thread.patch: this was modified from the first time I applied thread.patch. This time, pfsinit and pfsinit+ were no longer in the patch file. That seemed appropriate to me. IGNORED thread2.patch: internal_err.c has changed in such a way that the changes this patch made to internal_err.c are no longer useful.. I had already made some of the changes to lib/psrv/dsrfinfo.c, but in a slightly different way. thread3.patch: done May 1, 1994 thread4.patch: stuff for vcache. looks good. Some of the function argument renaming things (done for the sake of -Wshadow) were things I'd also done in a different way. DONE. doc1.patch: DONE. aolwait.patch: I had to be careful with this one. I assume you did not intend to put the changes to plog.h into general distribution. I also did not understand the changes you made to ardp_pr_actv.c and therefore I did not apply them. I also did not understand why you turned off the retry count. In general, I do not understand the ARDP_WAIT_TOO_LONG error and how it fits into our general interface. As it stands, it seems AOL-specific enough that I'm uncomfortable incorporating it as it stands into our sources; I would be glad to discuss some general mechanism we could use to accomplish the same end. One is clearly needed to solve this problem (swa: Discuss with BCN) I think the change (to call ardp_rwait when there's a queue) is good. I think we need a general hook in the ARDP code so that you can set such a function from a higher level; that section of the ARDP code is not general enough (for instance, it currently contains archie-specific code.) (NOT APPLIED) dnscache.patch: I am not comfortable with configuring the behavior of ARDP library stuff in . a) sometimes people compile just the Prospero clients, not the server. This makes the whole ARDP library dependent upon b) ARDP.H needs its own separate configuration file so that it can be separated from the rest of Prospero. This change has been incorporated into the sources; we should change the configuration in the future. c) your change to lib/pfs/myhost.c was one I'd made in a slightly different way. DONE dnscache1.patch: DONE. Good patch. dircache.patch: This is a note to myself to reconcile include/pserver.h with the master include/pserver.h.dist. This patch changed pserver.h. In the future, changes to the distribution pserver.h should be made to include/pserver.h.dist. I will have to implement this change to our configuration proceedings. PATCH DONE memleaks.patch: GOOD PATCH. DONE ackall.patch: This is not listed in apply.patches. I assume that it should not be applied. In any case, the change to lib/ardp/ardp_accept.c should not be inside an #ifdef PFS_THREADS. NOT APPLIED. solaris.patch: This patch has configuration changes that should only be configured under solaris interleaved with things that should be in the general distribution (e.g., the comments on what you need to configure to use SOLARIS). I have used judgement to make the changes that seemed to me to be general. We need a more general way of modifying PARTLY APPLIED solaris1.patch, solaris2.patch: pfsinit doesn't striike me as generally useful. NOT APPLIED All patches through April 27th now applied. Now to compile (without PFS_THREADS): Prospero wouldn't compile with PFS_THREADS turned off until I #if 0'd out the special code in that redefined locked_fopen() in case of threadless compilation. That code should be reinstated, for efficiency, but only when we add appropriate #ifdefs to comment out the whole file locking set of code when not multi-threaded. I have a new version of include/pfs_threads.h that is compatible with yours. -- I made a significant change to lib/pfs/pmap_cache.c: instead of EXECVCACHE being the special option, the special option is now called INCREASE_CLIENT_EXECUTABLE_SIZE_DO_NOT_EXEC_VCACHE. It is heavily commented in include/psite.h, with an explanation of the circumstances under which one might enable it. The default is to definitely not have it enabled. If you would like the vcache library to stay a part of the release, then that libpvcache code will need a great deal of work, work that we don't have the time to put into it. All of the external identifiers in libpvache() will have to start with a p__vcache, except for a few strictly defined interfaces. Otherwise, anybody in the outside linking with that library will encounter problems. Also, the interface to that library should be documented somewhere (just a half-page explaining how you interact with it and how you link with it). Also, the dependencies problems will need to be fixed. Right now, it's really awkward to use and to explain to others how to use it; thus my reluctance to include it as a regular part of the distribution. Also, it's not clear to me that that code doesn't memory leak; if it doesn't, please change the comment at the head of vcache.c to explain that we've been using it commercially and have avoided leaks by that means. I am leaving the code in right now, but in an unsupported status, because of the above-mentioned problems with it. -- The client-side caching in the VCACHE library is not something I'm comfortable putting into the release right now. The Also, the caching strategy needs to be documented in the library manual. Also, a policy for cleaning old objects out of the cache and keeping it from taking over the disk space needs to appear. -- Another issue is that comments in our code should be appropriately spelled. It's full of "dont" and "doesnt" and "wont" and "couldnt" --- I made many changes to the configuration stuff. I have cut out your 'bintar' and 'echobinfiles' dependencies; if you need them back we can talk about it. -- I have gotten rid of your test dependencies in user/vcache/Makefile. --- I got rid of the 'pure' dependency in the top-level and server makefiles. The code is here if you want to use it for yourself: in top Makefile: ## ## This target is used only when using PURIFY. ## Ignore it for normal work. ## This only debugs the server for now. Needs to be modified for your needs. ## Mitraism. pure: for i in server ; do \ ( cd $$i; \ echo "Making in $$i"; \ $(MAKE) $(PASTHRU) pure 2>&1 | $(SED-SCRIPT) ) ;\ done in Makefile.config: ## ## PURIFY ## ## Purify is a developer's tool used to check for memory leaks and other error ## conditions. ## If you have never heard of it or if you don't use it, just don't worry ## about it. # If you use Purify, you can specify a command syntax for it # Make a purified dirsrv with 'make pure'. PURIFY = purify # PURE_LIBS is usually LIBS , except on Solaris when it excludes thread # No idea why Mitra needed -lmalloc. # PURE_LIBS = -lsocket -lnsl PURE_LIBS = $(LIBS) -lmalloc # This is for PURIFY # Note that problems with threads may have to be debugged with -g, not # with -ggdb3. PURE_CFLAGS = -g $(GCCWARN) $(GCCTRY) in server/MAkefile: ## Used when using the special PURIFY package. ## It is not clear to me why this target is needed, but mitra uses it. ## It may be yanked out of the full release. pure: pure_dirsrv pure_dirsrv: $(DIRSRV_OBJECTS) ${SRV_LIBS_DEPENDENCIES} ${PURIFY} ${CC} ${F_CCL} -o $@ $(DIRSRV_OBJECTS) \ $(SRV_LIBS) $(PURE_LIBS) --- gopher_gw_diag_mutexes() and gopher_gw_init_mutexes now called from DIRSRV,, not lib/psrv/psrv_mutexes.c. This eliminates ordering/dependency problems. -- another practice that should be eliminated is that of declaring subfunctions inside master files; they should always be in include files. -- It is not clear to me why VLDEBUGBEGIN and VLDEBUGEND should be left in the sources. Ditto CHECK_MEM(). Is there any reason why they are still useful? If not generally useful, they should be removed before the release.