git.openafs.org Git - openafs.git/log

auth: plug auth realms memory leaks

The function _afsconf_FreeRealms, called by afsconf_CloseInternal, leaks
two afsconf_realms structures.

The function _afsconf_LoadRealms also leaks those two structures when it
fails.

These memory leaks were discovered with valgrind.

Change-Id: I1436ce21609951bc3433b6c91221cc45e78881bc
Reviewed-on: https://gerrit.openafs.org/13395
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>

Add the CellServDB pathname to the afsconf_dir

The determination of the CellServDB pathname is platform-dependent.
However, error reporting in the current code base assumes the CellServDB
location is platform-independent.

Add the pathname of the CellServDB file to the configuration directory
structure and set the new cellservDB member when opening the
configuration. Use this value when checking if the CellServDB has
changed and update the callers to use the cellservDB member when
reporting errors about the CellServDB file.

Change-Id: I5a3393fb9d4ae3c637d5a0d773598115314bfe1c
Reviewed-on: https://gerrit.openafs.org/13408
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

afs: Do not ignore errors in afs_CacheFetchProc

afs_CacheFetchProc currently has a section of code that looks like
this pseudocode:

    if (!code) do {
        while (length > 0) {
            code = read_from_rx();
            if (code) {
                break;
            }
            code = write_to_cache();
            if (code) {
                break;
            }
        }
        code = 0;
    } while (moredata);
    return code;

When we encounter an error when reading from rx or writing to the
cache, we break out of the current loop to stop processing and return
an error. But there are _two_ loops in this section of the code, so
what we actually do is break out of the inner loop, set 'code' to 0,
and then usually return (since 'moredata' is usually never set).

This means that when we encounter an unexpected error either from the
net or disk (or the memcache layer), we ignore the error and return
success. This means that we'll store a subset of the relevant chunk's
data to disk, and flag that chunk as complete and valid for the
relevant DV. If the error occurred before we wrote anything to disk,
this means we'll store an empty chunk and flag it as valid. The chunk
will be flagged as valid forever, serving invalid data, until the
cache chunk is evicted or manually kicked out. This can result in
files and directories appearing blank or truncated to applications
until the bad chunk is removed.

Possibly the most common way to encounter this issue is when using a
disk cache, and the underlying disk partition is full, resulting in an
unexpected ENOSPC error. Theoretically this can be seen from an
unexpected error from Rx, but we would have to see a short read from
Rx without the Rx call being aborted. If the call was aborted, we'd
get an error from the call to rx_EndCall() later on.

To fix this, change all of these 'break's into 'goto done's, to be
more explicit about where we are jumping to. Convert all of the
'break's in this function in the same way, to make the code flow more
consistent and easier to follow. Remove the 'if () do' on a single
line, since it makes it a little harder to see from a casual glance
that there are two nested loops here.

This problem appears to have been introduced in commit 61ae8792 (Unite
CacheFetchProcs and add abstraction calls), included in OpenAFS
1.5.62.

Change-Id: Ib965a526604e629dc5401fa0f1e335ce61b31b30
Reviewed-on: https://gerrit.openafs.org/13428
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Linux_5.0: replaced current_kernel_time with ktime_get_coarse_real_ts64

In Kernel commit fb7fcc96a86cfaef0f6dcc0665516aa68611e736 the
current_kernel_time/current_kernel_time64 functions where renamed
and the calling was standardized.

According to the Linux Documentation/core-api/timekeeping.rst
ktime_get_coarse_real_ts64 is the direct replacement for
current_kernel_time64. Because of year 2038 issues, there is no
replacement for current_kernel_time.

Updated code that used current_kernel_time to use new name and calling
convention.

Updated autoconf test that sets IATTR_TAKES_64BIT_TIME as well.

Change-Id: I607bdcf6f023425975e5bb747e0e780b3d2a7ce5
Reviewed-on: https://gerrit.openafs.org/13434
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Linux_5.0: replace do_gettimeofday with ktime_get_real_ts64

In Kernel commit e4b92b108c6cd6b311e4b6e85d6a87a34599a6e3 the
do_gettimeofday function was removed.

According to the Linux Documentation/core-api/timekeeping.rst
ktime_get_real_ts64 is the direct replacement for do_gettimeofday

Updated the macro osi_GetTime to use ktime_get_real_ts64 if it is
available.

Change-Id: I7fcd49958de83a6a040e40bd310a228247c481b2
Reviewed-on: https://gerrit.openafs.org/13433
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

LINUX: correct include for ktime_get_coarse_real_ts64()

The include for the ktime_get_coarse_real_ts64() autoconf test is
incorrect; ktime_get_coarse_real_ts64() has always been in linux/ktime.h
(via #include timekeeping.h), not linux/time.h.

This autoconf test still ran correctly because the OpenAFS build was
inadvertently picking up ktime.h via the default autoconf include path.
Therefore, this commit is needed only to provide documentation and
clarity to future maintainers.

Introduced as a cut-n-paste error (from the current_kernel_time test)
with commit 3c454b39d04f4886536267c211171dae30dc0344 for Linux 4.20.

Change-Id: I994b03a1700330756216c7feab0121c82d0f3ee4
Reviewed-on: https://gerrit.openafs.org/13437
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Linux_5.0: Use super_block flags instead of Mount flags when filling sb

In Kernel commit e262e32d6bde0f77fb0c95d977482fc872c51996
the mount flags (MS_) were moved from uapi/linux/fs.h to
uapi/linux/mount.h. This caused a compile failure in
src/afs/LINUX/osi_vfsops.c

The Linux documentation in uapi/linux/mount.h indicates that the MS_
(mount) flags should only be used when calling sys_mount and filesystems
should use the SB_ (super_block) equivalent.

src/afs/LINUX/osi_vfsops.c utilized the mount flag MS_NOATIME while
filling the super_block. Changed to use SB_NOATIME (which has the same
numeric value as MS_NOATIME) if available.

Change-Id: I2b2199de566fbadd45e857b37d24ce63002c7736
Reviewed-on: https://gerrit.openafs.org/13432
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

vol: remove empty directories left by vos zap -force

The vos zap -force command does not remove the directories associated
with the volume in question (AFS_NAMEI_ENV). When the vos zap -force
command is executed, the volume server goes through the /vicep*/AFSIDat
directories and removes the files associated with the volume id received
as an argument. Unfortunately, the volume server does not remove the
directories associated with this volume. As a result, empty directories
are left behind.

To fix this problem, remove the empty directories left behind when vos
zap -force is executed.

Change-Id: I56fd52918223f87e424121bac6a086d7b0a46284
Reviewed-on: https://gerrit.openafs.org/12879
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

roken: Use srcdir for roken-post.h

roken-post.h is a source file, not a generated file in the objdir.
Specify $(srcdir) so we can work with objdir builds.

Change-Id: I1d00ba1f28bea99770c2af56890fbf22ee764820
Reviewed-on: https://gerrit.openafs.org/13387
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Redhat: correct path to kernel module in dkms.config

This fix corrects some annoying error and warning messages during
dkms install or uninstall.

Install:
DKMS: build completed.

openafs:
Running module version sanity check.
ERROR: modinfo: could not open /lib/modules/2.6.32-754.6.3.el6.x
86_64/weak-updates/openafs.ko: No such file or directory
- Original module
   - No original module exists within this kernel
- Installation
   - Installing to /lib/modules/2.6.32-754.6.3.el6.x86_64/extra/
Adding any weak-modules
WARNING: Can't read module /lib/modules/2.6.32-754.6.3.el6.x86_6
4/weak-updates/openafs.ko: No such file or directory
egrep: /lib/modules/2.6.32-754.6.3.el6.x86_64//weak-updates/open
afs.ko: No such file or directory

Remove
Status: Before uninstall, this module version was ACTIVE on this
kernel.
Removing any linked weak-modules
rmdir: failed to remove `.': Invalid argument
WARNING: Can't read module /lib/modules/2.6.32-754.6.3.el6.x86_6
4/weak-updates/openafs.ko: No such file or directory
egrep: /lib/modules/2.6.32-754.6.3.el6.x86_64//weak-updates/open
afs.ko: No such file or directory

openafs.ko:
- Uninstallation
   - Deleting from:/lib/modules/2.6.32-754.6.3.el6.x86_64/extra/
- Original module
   - No original module was found for this module on this kernel
   - Use the dkms install command to reinstall any previous
   module version.

Background:

Commit 1c96127e37c0ec41c7a30ea3e4aa68f3cc8a24f6 standardized the
location where the openafs.ko module is installed (from
/kernel/3rdparty to /extra/).  The RPM Spec file was not updated to
build the dkms.conf file with the corrected location.

From the documentation for dkms

DEST_MODULE_LOCATION is ignored on Fedora Core 6 and higher, Red Hat
Enterprise Linux 5 and higher, Novell SuSE Linux Enterprise Server 10
and higher, Novell SuSE Linux 10.0 and higher, and Ubuntu.  Instead,
the proper distribution-specific directory is used.

However the DEST_MODULE_LOCATION is still used saving and restoring old
copies of the module.

The NO_WEAK_MODULES parameter prevents dkms from creating a symlink into
weak-updates directory, which can lead to broken symlinks when
dkms-openafs is removed.  The weak modules facility was designed to
eliminate the need to rebuild kernel modules when kernel upgrades occur
and relies on the symbols within the kABI.  Openafs uses symbols that
are outside the kABI, and therefor is not a candidate for a weak module.

Change-Id: I52a332036056a359a57a3ab34d56781c896a2eea
Reviewed-on: https://gerrit.openafs.org/13404
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Marcio Brito Barbosa <mbarbosa@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

build: declare test targets as phony

Modern versions `make` will not build the 'test' target since a
directory exists with the same name.

    $ grep -C1 '^test:' Makefile
    test:
        cd test; $(MAKE)

    $ make test
    make: 'test' is up to date.

Declare these targets as .PHONY to force make to build the test programs
even when the 'test' directory is present. Also use '&&' to concatenate
commands instead ';' to avoid running the second command when the first
fails.

Change-Id: Id561d7610f80b87b59c632801fa0a4b216feb42d
Reviewed-on: https://gerrit.openafs.org/13419
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

lwp: Avoid freeing 'stackmemory' on AIX32

Commit 55013a11 (lwp: Fix possible memory leak from scan-build) added
some free() calls to some otherwise-leaked memory. However, one of
these calls frees the 'stackmemory' pointer, which on AIX32 is not a
pointer from malloc/calloc, but calculated from reserveFromStack().

To avoid corrupting the heap, skip this free call on AIX32. This
commit adds another #ifdef to avoid this, which is unfortunate, but
this is also how the free is avoided in the existing code for
Free_PCB().

Change-Id: I6c4518f810e56c362ee744f250747fe8fc765b13
Reviewed-on: https://gerrit.openafs.org/13426
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

rx: remove rx_atomic bitops

The rx_atomic bitops were introduced with commit
1839cdbe268f4b19ac8e81ae78548f5c78e0c641 ("rx: atomic bit ops").

The last (only) reference to them was recently removed with commit
5ced6025b9f11fadbdf2e092bf40cc87499ed277 ("rx: Convert rxinit_status to
rx_IsRunning()").

Remove the now unreferenced bitops.  This commit is comprised of partial
or complete reverts of the following commits:

ae4ad509d35 rx: fix rx_atomic warnings under Solaris (partial)
c16423ec4e6 rx: fix atomics on darwin (partial)
9dc6dd9858a rx: Fix AIX test_and_set_bit (complete)
1839cdbe268 rx: atomic bit ops (complete)

Note: The rx_atomic bitops for Linux systems are known to be broken due
to incorrect casting of rx_atomic_t into the unsigned long operand
expected by the native Linux bitops.  The failure modes include silent
overruns on little-endian and incorrect results on big-endian.  Do not
merely revert this commit in order to bring these bitops back into the
tree.

Change-Id: I6b63519f63d370ccc8df816b4388487909c17dcd
Reviewed-on: https://gerrit.openafs.org/13390
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

rx: Statically check rx_statisticsAtomic size

Currently, rx_GetStatistics assumes that struct rx_statistics and
rx_statisticsAtomic have the same size (we just memcpy between them).
However, this is never checked, and rx_statistics contains many 'int'
fields where rx_statisticsAtomic has rx_atomic_t fields.

If these are not the same size, our rx stats will silently break, so
add a static assert to make sure they are the same size.

Change-Id: I889867f4a85530c30dd15d32d1822144ea128a95
Reviewed-on: https://gerrit.openafs.org/13414
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Revert "rx: fix rx_atomic warnings under Solaris"

This reverts commit ae4ad509d35aab73936a1999410bd80bcd711393. While
that commit did fix the mentioned warnings on Solaris, it also changed
the size of rx_atomic_t. Our code in rx_stats.c assumes that an
rx_atomic_t is 4-bytes wide, and so changing the size of rx_atomic_t
broke our reporting for stats in the 'rx_stats' structure.

To fix this, revert that commit. This reintroduces the mentioned
warnings, but those warnings are reported for our atomic bit-op
functions, which are unused and will be removed by another commit.

Change-Id: Ie3e72cc06690d9f8de79e8f0274ea51079004c38
Reviewed-on: https://gerrit.openafs.org/13415
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

auth: update the auth test programs

Fix build errors for the auth test programs.

Close the configuration directory before exiting the testcellconf
program so we can check for leaks.

Add a call to afsconf_GetExtendedCellInfo to the testcellconf test
program.

Use libcmd to parse the testcellconf command line options.

Add the -reload option to testcellconf to perform an optional reload
test. The user must have file permissions to touch the CellServDB to
perform the reload test.

Change-Id: I1cb4cacf9a15ccf7066fb32bfe5f5d03ef64bfd7
Reviewed-on: https://gerrit.openafs.org/13394
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>

afs: avoid afs_GetDownDSlot panic on afs_WriteDCache failure

If afs_GetDownDSlot() finds insuffcient free slots in the
afs_freeDSList, it will walk the afs_DLRU attempting to flush and free
eligible dcaches.  However, if an error occurs during the flush to
CacheItems (afs_WriteDCache()), e.g., -EINTR, afs_GetDownDSlot() will
assert.

However, a panic in this case is overkill, since afs_GetDownDSlot() is a
best-effort attempt to free dslots.  The caller (afs_UFSGetDSlot()) will
allocate more dcaches if needed.

Instead:
- Refactor afs_GetDownDSlot() by moving the QRemove() call to after the
afs_WriteDCache logic, so it accompanies the logic that puts the dcache
back on the freelist.  This is safe because we hold the afs_xdcache W
lock for the duration of the routine.
- If afs_WriteDCache() returns an error, return early and let the caller
handle any recovery.

Change-Id: Ifd0d56120095c9792998ff935776bbd339a76c8a
Reviewed-on: https://gerrit.openafs.org/13364
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

vos: restore status information to 'vos status'

Commit d3eaa39da3693bba708fa2fa951568009e929550 'rx: Make the rx_call
structure private' created accessors for several rx_call members.
However, it simply #ifdef'd out the packet counters and timestamps
reported by 'vos status' (AFSVol_Monitor). This is a regression for the
1.8.x 'vos status' command.

Instead, supply an accessor so 'vos status' can again be used to monitor
the progress of certain volume operations.

FIXES 134856

Change-Id: I91f5831b21f128bd8e86db63387a454c9e57bcdf
Reviewed-on: https://gerrit.openafs.org/13400
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>

afs: Reword "cache is full" messages

Currently, there are multiple different areas in the code that log a
message that look like this, when we encounter an ENOSPC error when
writing to the cache:

*** Cache partition is FULL - Decrease cachesize!!! ***

The message is a bit unclear, and doesn't even mention AFS at all.
Reword the message to try to explain a little more what's happening.

Also, since we log the same message in several different places, move
them all to a common function, called afs_WarnENOSPC, so we only need
to change the message in one place.

Change-Id: If1c259bd22a382ff56ed29326aa20c86389d06bc
Reviewed-on: https://gerrit.openafs.org/13410
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

afs: remove dead code afs_osi_SetTime

afs_osi_SetTime() has been dead code since -settime support was removed
with commit 1d9888be486198868983048eeffabdfef5afa94b 'Remove
-settime/RXAFS_GetTime client support'.

Remove the dead code.

No functional change is incurred by this commit.

Change-Id: Ie5559325b4c98d7e0786c75ae6507ab9c2c47376
Reviewed-on: https://gerrit.openafs.org/13393
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>

Linux 4.20: do_settimeofday is gone

With Linux commit 976516404ff3fab2a8caa8bd6f5efc1437fed0b8 'y2038:
remove unused time interfaces', do_settimeofday() is gone.

However, OpenAFS only calls do_settimeofday() from afs_osi_SetTime(),
which has been dead code since -settime support was removed from afsd
with commit 1d9888be486198868983048eeffabdfef5afa94b 'Remove
-settime/RXAFS_GetTime client support'.

Instead of fixing afs_osi_SetTime() to use a current Linux API, remove
it as dead code.

No functional change is incurred by this commit. However, this change
is required in order to build OpenAFS on Linux 4.20.

Change-Id: I74913deb249de66b0da71539f2596c971f0fd99a
Reviewed-on: https://gerrit.openafs.org/13392
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>

Linux 4.20: current_kernel_time is gone

With Linux commit 976516404ff3fab2a8caa8bd6f5efc1437fed0b8 'y2038:
remove unused time interfaces' (4.20-rc1), current_kernel_time() has
been removed.

Many y2038-compliant time APIs were introduced with Linux commit
fb7fcc96a86cfaef0f6dcc0665516aa68611e736 'timekeeping: Standardize on
ktime_get_*() naming' (4.18).  According to
Documentation/core-api/timekeeping.rst, a suitable replacement for:

  struct timespec current_kernel_time(void)

would be:

  void ktime_get_coarse_real_ts64(struct timespec64 *ts))

Add an autoconf test and equivalent logic to deal.

Change-Id: I4ff622ad40cc6d398267276d13493d819b877350
Reviewed-on: https://gerrit.openafs.org/13391
Tested-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

ubik: calloc ubik_dbase

Instead of using malloc and initializing various fields to 0, allocate
our ubik_dbase using calloc, to more easily ensure all fields are
initialized.

Change-Id: I5c2f345a82a2eb73d53ffc3e1b0fa408af6a8311
Reviewed-on: https://gerrit.openafs.org/13363
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

viced: fix typo in help for option -unsafe-nosalvage

Change-Id: I4e72533747250cee1b7d8c091c63c78948be6c28
Reviewed-on: https://gerrit.openafs.org/13367
Reviewed-by: Stephan Wiesand <stephan.wiesand@desy.de>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

viced: correct option parsing for -vlru*, -novbc

Commit a5effd9f1011aa319fdf432c67aec604053b8656 "viced: Use libcmd for
command line options" modernized the option parsing for (da)fileserver,
but introduced a few errors for the following options:

-vlruthresh <nn>
-vlruinterval <nn>
-vlrumax <nn>
-novbc

Correct the errors.

Change-Id: If57dfabaa8d4e456b63d47694d288bd8c4235ad2
Reviewed-on: https://gerrit.openafs.org/13365
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

budb: Remove db.lock

Ever since commit dc8f18d6 (Protect ubik cache accesses), the 'lock'
field in struct memoryDB has been unused. Remove it from the struct
definition.

Change-Id: I90131421ae2e2322debf4249e7464126480832d1
Reviewed-on: https://gerrit.openafs.org/13362
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

ubik: Remove version_cond

Several areas in the code do something like this whenever the database
version is changed:

    #ifdef AFS_PTHREAD_ENV
        opr_cv_broadcast(&ubik_dbase->version_cond);
    #else
        LWP_NoYieldSignal(&ubik_dbase->version);
    #endif

However, ever since commit 3fae4ea1 (ubik: remove unused code),
nothing in the tree waits for this condvar, so it currently doesn't do
anything. Remove this unneeded code.

Change-Id: I6903ed89f9dcee2ce154be8883d656d297c97902
Reviewed-on: https://gerrit.openafs.org/13361
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Remove one more automake VERSION reference

The configure summary was still referencing the old automake-specific
VERSION var. Use the autoconf PACKAGE_VERSION var instead, so this
actually shows our version.

Change-Id: I18007935d0235931f1d2e023abddee7356e8ac2d
Reviewed-on: https://gerrit.openafs.org/13360
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

autoconf: remove unnecessary mkdir during configure

Remove an uneeded mkdir command to create the JAVA/libjafs object
directory, since this directory is automatically created by the
config.status when generating the JAVA/libjafs/Makefile.

Change-Id: Ib02a38c5c23790cb07e5c2433fd4870e8763c3a3
Reviewed-on: https://gerrit.openafs.org/12994
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

autoconf: remove spurious no-op

Change-Id: I27242481dc3039f6776deb89e31793deee7f2840
Reviewed-on: https://gerrit.openafs.org/12993
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

autoconf: fix pio checks name

The autoconf macro to perform the positional i/o checks was misnamed as
hpux checks (since there happens to be a specific check for hpux at the
top of the macro). Change the macro name and m4 file name to be more
accurately named.

Change-Id: Ib85728fbfe67930cb5f9f1f0e34f7aa1195fdfc6
Reviewed-on: https://gerrit.openafs.org/12992
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

vol: avoid query for parent id when deleting disk header

When a DAFS volume server removes a volume disk header file (V*.vol),
the volume server invokes an fssync command to have the file server
delete the Volume Group Cache (VGC) entry corresponding to the volume id
and the parent id of the removed volume header.

The volume parent id is unknown to the volume server when removing a
volume disk header on behalf of a "vos zap -force" operation. In this
case, the volume server issues a fssync query to attempt look up to the
parent id from the file server's VGC. If this fssync query fails for
some reason, volume server is unable to delete the VGC entry for the
deleted volume header. The volume server logs an error and vos zap
reports a undocumented error code.

One common way this can be encountered is to issue a "vos zap -force" on
a file server that has just been restarted. In this case, the VGC may
not be fully populated yet, so the volume server is not able to look up
the parent id of the given volume.

With this commit, relax the requirement for the parent id when deleting
VGC entries. A placeholder of 0 is used to mean any parent id for the
given volume id.

This obviates the need to query for the parent id when performing a "vos
zap -force", and allows the volume server to remove any VGC entries
associated with the volume id being zapped.

Change-Id: Iee8647902d93a3c992fca4c4f3880a3393f0b95f
Reviewed-on: https://gerrit.openafs.org/12839
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

Remove automake autoconf vars

Commit 4706854f (autoconf: updates and cleanup) removed our invocation
of AM_INIT_AUTOMAKE, which defines the output variables PACKAGE and
VERSION. Several files in our build system are still referencing
@PACKAGE@ and @VERSION@, though, leaving them un-substituted. This
most easily is seen as the AFSVersion version string remaining as
"@VERSION@" when the tree is built without git, but it also affects
some packaging in the tree.

Remove references to @VERSION@ and @PACKAGE@, replacing them with
their autoconf equivalents @PACKAGE_VERSION@ and @PACKAGE_TARNAME@.

Change-Id: I6c6a09a46c4af4259009a4a60cfdaee63d6258c2
Reviewed-on: https://gerrit.openafs.org/13357
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

afs: Remove afs_xosi

Since OpenAFS 1.0, all platforms in libafs have a lock called
afs_xosi, which is acquired and released around calls like VOP_GETATTR
on cache files. However, this lock doesn't appear to protect anything;
on all platforms, the code that runs while the lock is held uses only
calls VOP_GETATTR and accesses local variables (aside from
afs_osi_cred, which we use similarly in many other places). The
purpose of the lock has never been documented, and is not mentioned at
all in the afs_rwlocks text file.

The comment by the afs_xosi lock declaration suggests that the lock
was originally introduced to protect access to 'tvattr', which perhaps
was a global variable in the past. All uses of 'tvattr' are local now,
though, so protecting access to it doesn't make any sense.

So, remove afs_xosi, to remove the unnecessary serialization of
VOP_GETATTR calls.

Change-Id: Ib3764600ae0155057361418c86b49a3507bdcd94
Reviewed-on: https://gerrit.openafs.org/13350
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

afs: Free 'addrs' array

Currently, 3 places in libafs allocate an 'addrs' array in a very
similar way to loop through our list of servers:
ForceAllNewConnections(), afs_LoopServers(), and PCallBackAddr(). Of
these, only afs_LoopServers actually frees the array.
ForceAllNewConnections and PCallBackAddr leak the memory, but these
are only hit from infrequent pioctls that can only be run by root, so
the impact is small.

Fix ForceAllNewConnections and PCallBackAddr to free the array.

Change-Id: Ic348e29cefa7c41cbcb30f738f943e8d022a97f0
Reviewed-on: https://gerrit.openafs.org/13355
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

macos: packaging support for MacOS X 10.14

This commit introduces the new set of changes / files required to
successfully create the dmg installer on OS X 10.14 "Mojave".

Change-Id: Ia1238b454350777bbfbf3dfd2be0c6c523348928
Reviewed-on: https://gerrit.openafs.org/13349
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

macos: add support for MacOS 10.14

This commit introduces the new set of changes / files required to
successfully build the OpenAFS source code on OS X 10.14 "Mojave".

Change-Id: Ib7cbd531ad6db3340d59e76abdecbe75886a4d5c
Reviewed-on: https://gerrit.openafs.org/13348
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

auth: check if argument of afsconf_Close* is null

Currently, we do not check if the argument of afsconf_Close /
afsconf_CloseInternal is equal to null. In order to avoid a possible
segmentation fault, add the checks.

Change-Id: I45635ad2d735505637072867edb7ff17da3c671a
Reviewed-on: https://gerrit.openafs.org/13352
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>

afs: make sure to call afs_Analyze after afs_Conn

The afs_Conn function is used to pick a connection for a given RPC. The
RPC is normally wrapped within a do-while loop which calls afs_Analyze
to handle the RPC code and manage the server connection references.
Among other things, afs_Analyze can mark the server as down, blacklist
idle servers, etc.

There are some special cases in which we break out of this do-while loop
early, by putting the connection reference given by afs_Conn and then
jumping out of the loop.

In these cases, be sure to call afs_Analyze to put the server connection
we got from afs_Conn, and to handle the RPC return code, possibly
marking the server as down or blacklisted.

Change-Id: Ic2c43f20d153376b93d79bbb5145914f8e478957
Reviewed-on: https://gerrit.openafs.org/13288
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

DARWIN: replace macro exported by automake

Commit 4706854f57043c8393baa922dd1974176e110a19 removed automake
references from the source tree. As a result, VERSION (exported by
AM_INIT_AUTOMAKE and obtained from Autoconf's AC_INIT macro) is not
available anymore. Unfortunately, a reference to this macro can be found
in src/afs/DARWIN/osi_module.c. Consequently, builds on OS X fail with
the following message:

osi_module.c:144:32: error: use of undeclared identifier 'VERSION'

To fix this problem, replace VERSION by PACKAGE_VERSION (defined by
AC_INIT).

Change-Id: Ib3821d79c4cddd59c399985762e13dec755d8642
Reviewed-on: https://gerrit.openafs.org/13354
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

ubik: do not reuse the offset variable for the sync site address

The ubik SendFile function performs a sanity check of the host address
before proceeding with the file transfer. Currently this check reuses
the file offset local variable to hold the value of the sync site
address, a 32-bit IPv4 address. Not only is this confusing, but also
causes a signed/unsigned type mismatch when comparing host addresses.
Instead of being so stingy with local variables, declare a new local
variable of the correct type to hold the value of the sync site address.

This separation is also a prerequisite for supporting larger address
types in the future.

Change-Id: I116fe210f418e6914afeff37c44d30bf795e2413
Reviewed-on: https://gerrit.openafs.org/13351
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

vlserver: Remove sascnvldb

"sascnvldb" appears to be a variant of cnvldb that was used to convert
vldb database blobs from even older versions than what cnvldb handles.
However, it has never been built by default (some makefile rules
reference the program, but it's never built unless the user explicitly
runs 'make sascnvldb'), and it currently cannot build due to a variety
of compiler errors.

Remove the dead code.

Change-Id: I5692d2cd058aa4ae9222ce25721001aabcca5eb7
Reviewed-on: https://gerrit.openafs.org/13345
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

fsint: remove dead code

The last references to these objects were removed with commit
3828c257ae33306bbdd3c6db9381601fe5b1b110
"dead-code-and-prototyes-20060214".

A few mentions of CBS and BBS are left in the documentation as
historical references:
- doc/man-pages/pod1/rxgen.pod
- src/kauth/AuthServer.mss

Change-Id: Ia24eef7bb1509ff10d11de5c51e688e27f69417a
Reviewed-on: https://gerrit.openafs.org/13324
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

cmd: improve help for programs without subcommands

Some programs do not have subcommands (other than the standard "help",
and "version" subcommands).  The cmd library provides the "noopcode"
mechanism for new subcommand-less programs, but older programs take
advantage of the optional "initcmd" token to simulate subcommand-less
programs.  The "initcmd" token is optional to run the command, however
it is required to display the command help.

For example, running the xstat_cm_test program without any options gives
a syntax error:

    $ xstat_cm_test
    xstat_cm_test: Missing required parameter '-cmname'
    ...

Retrying with -help (or help, -h, --help), gives the rather unhelpful
output:

    $ xstat_cm_test -help
    xstat_cm_test: Commands are:
    apropos         search by help text
    help            get help on commands
    initcmd         initialize the program

It is not obvious to the user how to get the command usage for the
program, nor that the initcmd subcommand to "initialize the program" is
actually is a placeholder to run the program.

Instead, display the command usage when help is requested and initcmd is
the only defined subcommand for a program.

For example:

    $ xstat_cm_test -help
    Usage: src/xstat/xstat_cm_test [initcmd]
             -cmname <Cache Manager name(s) to monitor>+
             -collID <Collection(s) to fetch>+ [-onceonly]
             [-frequency <poll frequency, in seconds>]
             [-period <data collection time, in minutes>] [-debug] [-help]
    Where: -onceonly  Collect results exactly once, then quit
           -debug     turn on debugging output

The libcmd library now supports an "noopcode", which should used for
future subcommand-less programs, but converting old programs to remove
the initcmd opcode could break scripts which actually specify the
optional initcmd token.

This commit adds a new libcmd flag called CMD_IMPLICIT which is used to
denote built-in subcommands such as "version" and "help".

Change-Id: Iee9cb2761254543f74166e5c240685f85b6915b6
Reviewed-on: https://gerrit.openafs.org/10983
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Avoid format truncation warnings

With gcc 7.3, we start getting several warnings like the following:

vutil.c: In function ‘VWalkVolumeHeaders’:
vutil.c:860:34: error: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 63 [-Werror=format-truncation=]
snprintf(name, VMAXPATHLEN, "%s" OS_DIRSEP "%s", partpath, dentry->d_name);

Most or all of these truncations should be okay, but increase the size
of the relevant buffers so we can build with warning checking turned
on.

Change-Id: Iac62d6fcfa46f523c34bf1b0ebc2770d3d67c174
Reviewed-on: https://gerrit.openafs.org/13274
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

vlserver: Remove 'register' argument

Commit 4a531cb7 (death to register) removed the 'register' declaration
from variables/arguments. But commit 3bf03502 (vlserver: Add a struct
for trans-specific data) accidentally added one back in at around the
same time, probably due to a rebase/merge mistake.

Take the 'register' declaration back out.

Change-Id: I73f206a57ab6b97195771e39556d2b0064be4cf3
Reviewed-on: https://gerrit.openafs.org/13346
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

CellServDB update 14 May 2018

Update all three copies in the tree, and the rpm specfile.

Change-Id: I572ff4e39ab757128f0082a4f447565e94b8dee5
Reviewed-on: https://gerrit.openafs.org/13134
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

Log binding ip address and port during startup

Many daemons currently have the ability to bind to a specific ip
address using the -rxbind parameter. The behavior can be a little
unintuitive, however, since we only bind to the ip address we find via
NetInfo/NetRestrict processing, and only if we end up with a single ip
address. Since that processing involves examining the set of ip
addresses available, this can have confusing results if, for instance,
a daemon starts up while an administrator is changing the local ip
configuration.

If a daemon binds to a different ip address than the administrator
expects, this can be very confusing, especially since for most daemons
we don't log our bound ip anywhere. To help alleviate this, change the
startup code for all of our daemons to log what ip we are trying to
bind to (or "0.0.0.0" if none), along with our local port.

Change-Id: I18d3647c4d134177a0a17c6a64583d444558a9f6
Reviewed-on: https://gerrit.openafs.org/13272
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

fsprobe: add fsprobe_Wait function

Move the lwp code to wait in the fsprobe applications down to the
fsprobe library.

This is a non-functional change in anticipation of converting the
fsprobe library and programs to pthreads.

Change-Id: I2972b13e2e3eeb691c64c91b0640bbc97e7d0b21
Reviewed-on: https://gerrit.openafs.org/12747
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

xstat: add xstat_*_Wait functions

Add the xstat_cm_Wait and xstat_fs_Wait functions and move the code to
wait for the xstat data collection to complete from the applications
down to the xstat library.

This is a non-functional change in anticipation of converting the xstat
library and programs to pthreads.

Change-Id: Ifd1d6bcda618c89b4ce46e1e64f33b0b30a89a72
Reviewed-on: https://gerrit.openafs.org/12746
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

rx: Convert rxinit_status to rx_IsRunning()

Currently, all rx code examines the atomic rxinit_status to determine
if rx is running (that is, if rx_InitHost has been called, and
rx_Finalize/shutdown_rx hasn't been called). This is used in rx.c to
see if we're redundantly calling our setup/teardown functions, and
outside of rx.c in a couple of places to see if rx-related resources
have been initialized.

The usage of rxinit_status is a little confusing, since setting bit 0
indicates that rx is not running, and clearing bit 0 indicates rx is
running. Since using rxinit_status requires atomic functions, this
makes code checking or setting rxinit_status a little verbose, and it
can be hard to see what it is checking for. (For example, does
'if (!rx_atomic_test_and_clear_bit(&rxinit_status, 0))' succeed when
rx running, or when rx is not running?)

The current usage of rxinit_status in rx_InitHost also does not handle
initialization errors correctly. rx_InitHost clears rxinit_status near
the beginning of the function, but does not set rxinit_status if an
error is encountered. This means that any code that checks
rxinit_status (such as another rx_InitHost call) will think that rx
was initialized successfully, but various resources aren't actually
setup. This can cause segfaults and other errors as the code tries to
actually use rx.

This can easily be seen in bosserver, if bosserver is started up while
the local host/port is in use by someone else. bosserver will try to
rx_InitHost, which will fail, and then we'll try to rx_InitHost again,
which will immediately succeed without doing any init. We then
segfault quickly afterwards as we try to use unitialized rx resources.

To fix all of this, refactor code using rxinit_status to use a new
function, called rx_IsRunning(), to make it a little clearer what
we're checking for. We also re-introduce the LOCK_RX_INIT locks to
prevent functions like rx_InitHost and rx_Finalize from running in
parallel.

Note that non-init/shutdown code (such as rx_upcall or rx_GetIFInfo)
does not need to wait for LOCK_RX_INIT to check if rx is running or
not. These functions only care if rx is currently setup enough to be
used, so we can immediately return a 'yes' or 'no' answer. That is, if
rx_InitHost is in the middle of running, rx_IsRunning returns 0, since
some resouces may not be fully initialized.

Change-Id: Ia14a6a725c9662b9db0adef48c33b48a93ffe051
Reviewed-on: https://gerrit.openafs.org/12761
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

SOLARIS: Fix libafs $(KOBJ) parallel make race

Currently, our COMPDIRS make rule for SOLARIS libafs builds looks like
this:

    ${COMPDIRS} ${INSTDIRS} ${DESTDIRS}:
        for t in $(KOBJ) ; do
            # set some variables ; \
            cd $$t ; \
            $(MAKE) $@_libafs || exit $$? ; \
            cd ../ ;\
        done

And Makefile.common has this:

    all: setup $(COMPDIRS)

Where the 'setup' rule creates the $(KOBJ) dirs and sets up some
symlinks.

For parallel builds, this means that our commands in the ${COMPDIRS}
target can be running in parallel with the 'setup' target, and so our
$(KOBJ) dirs may not exist by the time we try to 'cd $$t'.

For single-KOBJ platforms this actually largely works, since the 'cd'
will fail, but then the subsequent 'make' will run (just in the wrong
dir), but this can cause us to wastefully re-compile the same source
files (and cause some possibly confusing error messages). For
platforms with multiple KOBJs, this causes obvious problems, since we
don't cd into each KOBJ dir.

To solve this, just have the ${COMPDIRS}/etc rule depend on setup, so
we know that 'setup' has finished running. Also change our way of
'cd'ing into each KOBJ dir to actually cause the rule to fail, to make
any errors here more obvious and consistent.

Change-Id: Id2e662f36ef47a6182716728167b2da4713893c6
Reviewed-on: https://gerrit.openafs.org/13344
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

SOLARIS: Fix platforms for KOBJ definition

Currently, we define KOBJ to "MODLOAD32 MODLOAD64" for the following
platforms:

<all -sun4x_510 sun4x_511 sunx86_511>

Which doesn't make any sense, since "all" includes sun4x_511 and
sunx86_511. The previous commits that modified this line, e4c2810f
(Remove support for Solaris pre-8) and c6a22d67 (SOLARIS: Do not build
x86 kernel module on 5.11), clearly meant to change the platforms
sun4x_511 and sunx86_511 to use the KOBJ on the next line, but omitted
the leading "-" for the platform.

This doesn't break anything, since the Makefile on these platforms
expands to:

KOBJ = MODLOAD32 MODLOAD64
KOBJ = MODLOAD64

So the first KOBJ line is effectively ignored. It's confusing, though,
so fix this line so these platforms only get one KOBJ definition.

Change-Id: Idea9fdee4ac5883428748c2a5fdfa9707406436a
Reviewed-on: https://gerrit.openafs.org/13343
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Run ctfconvert/ctfmerge for all objects

Commit 88cb536f (autoconf: detect ctf-tools and add ctf to libafs)
introduced running ctfconvert and ctfmerge for libafs on Solaris, but
didn't add any CTF data for userspace code. This commit causes the
same commands to be run for every binary that we build (if the ctf
tools are available).

To accomplish this, also refactor how we run ctfconvert and ctfmerge.
The approach in commit 88cb536f would require us to modify the
makefile rule for every executable to run RUN_CTFCONVERT and
RUN_CTFMERGE, which is somewhat impractical. So instead in this
commit, we modify all of our *_CCRULE and *_LDRULE variables to wrap
the compiler invocation with the new CC_WRAPPER script. This means our
*RULE variables change from something like this:

FOO_CCRULE = $(RUN_CC) $(CC) $(XXX_FLAGS) -o $@

to something like this:

FOO_CCRULE = $(RUN_CC) $(CC_WRAPPER) $(CC) $(XXX_FLAGS) -o $@

CC_WRAPPER expands to the script src/config/cc-wrapper, which just
runs ctfconvert or ctfmerge on the relevant files after the
compiler/linker runs. If the CTF tools are not configured, CC_WRAPPER
expands to nothing, to limit our impact on other platforms.

This commit was developed in collaboration with
mbarbosa@sinenomine.net.

Change-Id: Id19ba9d739edc68f01c2db7d5caa20758ec8144a
Reviewed-on: https://gerrit.openafs.org/13308
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Call rx_InitHost once during daemon startup

Currently, a few daemons calls rx_InitHost in different places, and
under different conditions. For example, vlserver calls rx_InitHost
only when we -rxbind to a specific ip address, and then also makes an
additional rx_Init call. Other daemons always call rx_InitHost, or
just call rx_InitHost sometimes and don't make an extra rx_Init call.

To try to make the various daemons behave a little more consistently,
change the startup code to always call rx_InitHost, and to only call
it once. Note that rx_InitHost is the same as calling rx_Init with
INADDR_ANY as the ip address, and calling rx_Init* after a previous
rx_Init* call is effectively a no-op.

Change-Id: Ifd15175349a7b4695e684ca82deb8a8af5063073
Reviewed-on: https://gerrit.openafs.org/13271
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

pthread.m4: Add missing 'test' to conditional

Commit c5def62d (autoconf: update pthread checks) accidentally omitted
a 'test' in one of the conditionals. This causes an ugly error message
during configure:

checking for pthread_attr_init in -lpthread... yes
./configure[31043]: x-lpthread: not found [No such file or directory]

Replace the missing 'test'.

Change-Id: I28b82594e43a4ab42a5eb9fcc78e0ce8c5517d8b
Reviewed-on: https://gerrit.openafs.org/13342
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

ubik: remove unused code

ubik_GetVersion and ubik_WaitVersion have been unused since at least
OpenAFS 1.0. Remove them.

No functional change should be incurred by this commit.

Change-Id: Iee6952f35d8c34e9f05a4e6011f5795f7222fb08
Reviewed-on: https://gerrit.openafs.org/13325
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Remove alpha_dux/alpha_osf references

Several files were still referencing the alpha_dux* and alpha_osf*
sysnames. The code for these platforms has been removed, so get rid of
this cruft.

Change-Id: I042fcc29be322bf557829974242553bb6d5b2be4
Reviewed-on: https://gerrit.openafs.org/13339
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

libafs: Remove .i Makefile rules

Makefile.common.in defines a suffix rule to generate .i files from .c
files, but we never actually need to do this. The rule originates from
before OpenAFS 1.0, which also did not use the rule. Remove the
unused definitions.

Change-Id: I057b2aca7d17e3e85e93d886a65c954e8d9d708f
Reviewed-on: https://gerrit.openafs.org/13338
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

volser: ensure GCTrans transaction walk remains valid

Commit bc56f5cc97a982ee29219e6f258b372dbfe1a020 ("volser: Delete
timed-out temporary volumes") introduced new logic to GCTrans().
Unfortunately, part of this logic temporarily drops VTRANS_LOCK in order
to call VPurgeVolume().  While this lock is dropped, other volser_trans
may be added or deleted from the allTrans list.  Therefore, GCTrans
should not trust the next pointer (nt = tt->next) which was obtained
before the lock was dropped.

One symptom observed in the field was a segfault while examining
tt->volume.  Neither tt nor volume were valid any longer, since tt had
been set from a stale nt at the top of the loop.

To repair, improve, and clarify this logic:
- Refactor so nt is assigned correctly and as late as possible.
- Add comments to explain the placement of the assigns to future
maintainers.

Change-Id: Ibd3a504bddd3622730aa349576341e20f2f27836
Reviewed-on: https://gerrit.openafs.org/13286
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

volser: add more logs for failures during restore

In the current version of the volserver, some failures during volume
restores are not logged. In order to help debugging, this commit
introduces extra logs for possible failures during this process, so we
guarantee that an error at any point during the restore causes a
message to be logged.

Change-Id: I3647155aeb3f10316d9d7fecb5b126efc909f7b4
Reviewed-on: https://gerrit.openafs.org/13252
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

afsmonitor: remove unused LWP_WaitProcess

Remove the unimplemented once-only flag and the unused LWP_WaitProcess
call.

Change-Id: Idec5815f6f20019b9be4b973794d8b05cea7f6c9
Reviewed-on: https://gerrit.openafs.org/12745
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

volser: combine GCTrans conditional clauses

In preparation for a future commit, combine two conditional clauses in
GCTrans().

No functional change should be incurred by this commit.

Change-Id: Ib08d5b83dd26327124fe0119e6e5f459adc5f78a
Reviewed-on: https://gerrit.openafs.org/13303
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

ubik: positional io for db reads and writes

The ubik library was written before positional i/o was available and
issues an lseek system call for each database file read and write. This
change converts the ubik database accesses to use positional i/o on
platforms where pread and pwrite are available, in order to reduce
system call load.

The new inline uphys_pread and uphys_pwrite functions are supplied on
platforms which do not supply pread and pwrite. These functions fall
back to non-positional i/o. If these symbols are present in the database
server binary then the server process will continue to call lseek before
each read and write access of the database file.

This change does not affect the whole-file database synchronization done
by ubik during database recovery (via the DISK_SendFile and DISK_GetFile
RPCs), which still uses non-positional i/o. However, that code does not
share file descriptors with the phys.c code, so there is no possibility
of mixing positional and non-positional i/o on the same FDs.

Change-Id: I28accd24f7f27b5e8a4f1dd0e3e08bab033c16e0
Reviewed-on: https://gerrit.openafs.org/12272
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Marcio Brito Barbosa <mbarbosa@sinenomine.net>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

volser: warn if older version of volume is restored

Volume restores work by overwriting vnodes with the data in the given
volume dump. If we restore a partial incremental dump from an older
version of the volume, this generally results in a partly-corrupted
volume, since directory vnodes may contain references that don't exist
in the current version of the volume (or are supposed to be in a
different directory).

Currently, the volserver does not prevent restoring older volume data
to a volume, and this doesn't necessarily always result in corrupted
data (for instance, if we are restoring a full volume dump over an
existing volume). But restoring old volume data seems more likely to
be a mistake, since reverting a volume back to an old version, even
without corrupting data, is a strange thing to do and may cause
problems with our methods of cache consistency.

So, log a warning when this happens, so if this is a mistake, it
doesn't happen silently. But we still do not prevent this action, since
it's possible something could be doing this intentionally. We detect
this just by checking if the updateDate in the given header is older
than the current updateDate for the volume on disk.

Note: Restoring a full dump file (-overwrite f) will not result in
corrupted data. In this scenario, the restore operation removes the
volume on disk first (if present). After that, the dump file is
restored. In this case, we do not log anything (the volume is not
corrupted).

Change-Id: Iac55cc8bb1406ca6af9a5e43e7d37c6bfa889e91
Reviewed-on: https://gerrit.openafs.org/13251
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

update: convert upserver and client from LWP to pthreads

Build the upserver and the upclient with phreads instead of LWP
and convert the IOMRG sleeps in the client to regular sleeps.

Change-Id: I183765ef180f34d38b87a13ec49f16f4a60afcc8
Reviewed-on: https://gerrit.openafs.org/12754
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Correct some redundant if() clauses

A few if() conditions currently contain redundant syntax, due to typos.
Fix the conditions to actually check different things, according to what
the author probably originally intended. (via cppcheck)

Change-Id: I7e46217e1f84fe65677ada345d227f31f1988fe6
Reviewed-on: https://gerrit.openafs.org/13157
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

ubik: remove unnecessary lseeks in uphys_open

The ubik database file access layer has a file descriptor cache to avoid
reopening the database file on each file access. However, the file
offset is reset with lseek on each and every use of the cached file
descriptor, and the file offset is set twice when reading or writing
data records.

This change removes unnecessary and duplicate lseek system calls to
reduce the system call load.

Change-Id: I460b226d81e4eb64dc87918175acab495aa698cd
Reviewed-on: https://gerrit.openafs.org/12271
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Marcio Brito Barbosa <mbarbosa@sinenomine.net>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

klog.krb5 -lifetime is not implemented

The klog.krb5 -lifetime option was copied from earlier versions of log
and klog, which had the ability to set the krb4 token lifetime. However,
the -lifetime option is not feasible the krb5 version, and so is not
implemented in klog.krb5.

Update the klog.krb5 man page to document the -lifetime option has no
effect. Remove the code which unnecessarily checks the unused klog.krb5
-lifetime command line argument.

The unused lifetime variable was discovered by Pat Riehecky using the
clang scan-build static analyzer.

Change-Id: I5f459ec46eaff87a69ccdf7de386a671d0944a5a
Reviewed-on: https://gerrit.openafs.org/13309
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: PatRiehecky <jcpunk@gmail.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

util: add defines for ktime never and now values

Add preprocessor symbolic names for ktime values representing never and
right now. The names are intended to be consistent with the ktime date
never value definition. This commit does not make any functional
change.

Change-Id: Ia6735b585e50aeb018481f76552fbb4f607b8529
Reviewed-on: https://gerrit.openafs.org/13310
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

ubik: Buffer log writes with stdio

Currently, when we write ubik i/o operations to the db log, we tend to
issue several syscalls involving small writes and fstat()s. This is
because each "log" operation involves at least one write, and each log
operation tends to be pretty small.

Each logged operation hitting disk separately is unnecessary, since
the db log does not need to hit the disk at all until we are ready to
commit the transaction. So to reduce the number of syscalls when
writing to the db, change our log writes to be buffered in memory
(using stdio calls). This also avoids needing to fstat() the
underlying log file, since we open the underlying file in append-only
mode, since we only ever append to (and truncate) the log file.

To implement this, we introduce a new 'buffered_append' phys
operation, to explicitly separate our buffered and non-buffered
operations, to try to avoid any bugs from mixing buffered and
non-buffered i/o. This new operation is only used for the db log.

Change-Id: I5596117c6c71ab7c2d552f71b0ef038f387e358a
Reviewed-on: https://gerrit.openafs.org/13070
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Joe Gorse <jhgorse@gmail.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Reviewed-by: Marcio Brito Barbosa <mbarbosa@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>

autoconf: Use `uname -p` instead of $HOST_CPU for ctf tools

Currently, we check if the ctf tools are present searching for them in
a few directories. One of these directories (/opt/onbld/bin/$HOST_CPU) looks at
the $HOST_CPU variable, which on x86 can be 'x86_64' or 'i386', but the only
valid directories for the onbld tools are 'i386' and 'sparc'. So instead of
$HOST_CPU, just use $(uname -p), which is only ever 'i386' on x86, and 'sparc'
on sparc.

[adeason@sinenomine.net: reword commit message]

Change-Id: I972cf1cc0dda81f5ee454b14ddbe2830c82c838d
Reviewed-on: https://gerrit.openafs.org/13275
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

doc: the last partition name is /vicepiu

The last valid partition name supported by OpenAFS is /vicepiu, not
/vicepiv. Update the docs and man pages to say so.

Change-Id: I6e1cce775d332d76f605a26f16502c651461994b
Reviewed-on: https://gerrit.openafs.org/13177
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

tests: partition name to id function tests

Add unit tests for the utility functions to convert between partition
names and partition ids.

Change-Id: I4b12f9d611cb9f3ce49909cda5cbcedd3e6c3d10
Reviewed-on: https://gerrit.openafs.org/13176
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>

Add rxgk_crypto_rfc3961.c

rxgk wrappers around an external crypto library, in this case, our
in-tree rfc3961 library. Primitives for encryption/decryption and
MIC/VerifyMIC, ways to generate and free rxgk_key objects, etc..

Change-Id: I7525086043baf54f5c3019b3f5ab3495760c4236
Reviewed-on: https://gerrit.openafs.org/10565
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>

Remove DUX/OSF code

Remove code for DUX/OSF platforms. DUX code was removed from the
libafs client in commit 392dcf67 ("Complete removal of DUX client
code") and the alpha_dux* param files were removed in dc4d9d64 ("afs:
Remove AFS_BOZONLOCK_ENV"). This code has always been disabled since
those commits, so remove any code referencing AFS_DUX*_ENV,
AFS_OSF_ENV, and related symbols.

Change-Id: I3787b83c80a48e53fe214fdecf9a9ac0b63d390c
Reviewed-on: https://gerrit.openafs.org/13260
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>

venus: fix memory leak

In GetPrefCmd, when we request server prefs from the kernel and our
output buffer is not big enough, pioctl() will return E2BIG and we
allocate more memory and try again. However, if the size of the output
buffer reaches 16k bytes and this space is still not enough (or if
pioctl fails and errno != E2BIG), we return without releasing the
memory that was previously allocated.

To fix this problem, free our output buffer when this happens.

Change-Id: Ib34cb12629528ddf2a763386f0ac5494eb8be695
Reviewed-on: https://gerrit.openafs.org/12293
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

rx: reset packet header userStatus field on reuse

OpenAFS Rx fails to set the rx packet header userStatus field for most
packets sent other than type RX_PACKET_TYPE_ACK.  If the userStatus
field is not set, its value will be random garbage based upon the
prior use of the memory allocated to the rx_packet.

This change explicitly sets the userStatus field to zero for all
DATA and Special packet types.

Background
----------

OpenAFS Rx allocates a pool of rx_packet structures that are reused
for both incoming and outgoing Rx packets throughout the lifetime
of the process (or kernel module).

The rx packet header field userStatus is set by rxi_Send() to
rx_call.localStatus.  rxi_Send() is called from both rxi_SendAck()
when sending RX_PACKET_TYPE_ACK packets and from rxi_SendSpecial()
when called with a non-NULL call structure (RX_PACKET_TYPE_BUSY,
RX_PACKET_TYPE_ACKALL, or RX_PACKET_TYPE_ABORT).  rx_call.localStatus
defaults to zero and can be modified by the application calling
rx_SetLocalStatus().

The userStatus field is neither set nor reset when sending
RX_PACKET_TYPE_DATA packets and all packets sent without a call
structure.  When allocated packets are reused in these cases, the
value of the userStatus leaks from the prior packet use.  The
userStatus field is expected to be zero unless intentionally set by
the application protocol to another value.

The AFS3 suite of rx services uses the rx_header.userStatus field
only in the RXAFS service and only as part of the definition
for RXAFS_StoreData and RXAFS_StoreData64 RPCs.  The StoreData RPCs
use the rx_header.userStatus field as an out-of-band communication
mechanism that permits the fileserver to signal to the cache manager
when the RXAFS_StoreData[64] has been assigned to an application
worker (thread) and the worker has acquired all of the required locks
and other resources necessary to complete the RPC.  This signal can be
sent before all of the application data has been received.  The cache
manager reads the userStatus value via rx_GetRemoteStatus().  When
bit-0 of the remote status value equals one and CSafeStore mode is
disabled, the cache manager can wakeup any threads blocked waiting for
the store operation to complete.

Cache managers that perform a workload heavy in RXAFS_StoreData[64] RPCs
will end up with an increasing percentage of packets in which the
userStatus field is one instead of zero.

Fileservers processing a workload heavy in RXAFS_StoreData[64] RPCs
will likewise end up with an increasing percentage of packets in which
the userStatus field is one instead of zero.

Cache managers and Fileservers will therefore send DATA and call free
special packets with a non-zero userStatus field to peer services
(RXAFS, RXAFSCB, VL, PR).

The failure to reset the userStatus field has not been a problem in
the past because only the OpenAFS cache manager has ever queried the
userStatus via rx_GetRemoteStatus() and only when issuing
RXAFS_StoreData[64] RPCs.

Failure to correct this flaw interferes with future use of the userStatus
field in yet to be registered AFS3 RPCs and existing non-AFS3 services
that make use of the userStatus when sending data to a service.

Change-Id: I32c0bba93b8e5197036d92168956b6e2a95406fb
FIXES: 134554
Reviewed-on: https://gerrit.openafs.org/13165
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

budb: SBUDB_FindLatestDump should check result of FillDumpEntry

FillDumpEntry may return an error, but FindLatestDump doesn't check its
result. Therefore, SBUDB_FindLatestDump may return invalid results.

Instead, check the return code from FillDumpEntry and abort the call if
it fails.

Change-Id: If0b44ba2a12a76511129d77110ef669b00780ff0
Reviewed-on: https://gerrit.openafs.org/13312
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

butc: repair build error

Commit c43169fd36348783b1a5a55c5bb05317e86eef82 introduced a build error
by invoking TLog with an extraneous set of internal parentheses.

Remove the offending parentheses.

Change-Id: Ibc52501b01ecbe9f86262566446d63e66486272f
Reviewed-on: https://gerrit.openafs.org/13311
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

Fix typos in audit format strings

Commit 9ebff4c6caa8b499d999cfd515d4d45eb3179769 introduced audit
framework support for several butc-related data types, but had
a typo ('$d' for '%d') in a couple of places, that was not reported
by compiler format-string checking. Fix the typo to properly print
all the auditable data.

Change-Id: Ibefa9f8f1c0567bc6fe606327af26fcb0dbeadba

OPENAFS-SA-2018-001 backup: use authenticated connection to butc

Use the standard routine to pick a client security object, instead of
always assuming rxnull. Respect -localauth as well as being able to
use the current user's tokens, but also provide a -nobutcauth argument
to fall back to the historical rxnull behavior (but only for the connections
to butc; vldb and budb connections are not affected).

Change-Id: Ibf8ebe5521bee8d0f7162527e26bc5541d07910d

OPENAFS-SA-2018-001 butc: require authenticated connections with -localauth

The butc -localauth option is available to use the cell-wide key to
authenticate to the vlserver and buserver, which in normal deployments
will require incoming connections to be authenticated as a superuser.
In such cases, the cell-wide key is also available for use in
authenticating incoming connections to the butc, which would otherwise
have been completely unauthenticated.

Because of the security hazards of allowing unauthenticaed inbound
RPCs, especially ones that manipulate backup information and are allowed
to initiate outboud RPCs authenticated as the superuser, default to
not allowing unauthenticated inbound RPCs at all. Provide an opt-out
command-line argument for deployments that require this functionality
and have configured their network environment (firewall/etc.) appropriately.

Change-Id: Ia6349757a4c6d59d1853df1a844e210d32c14feb

OPENAFS-SA-2018-001 Add auditing to butc server RPC implementations

Make the actual implementations into helper functions, with the RPC
stubs calling the helpers and doing the auditing on the results, akin
to most other server programs in the tree. This relies on support for
some additional types having been added to the audit framework.

Change-Id: Ic872d6dfc7854fa28bd3dc2277e92c7919d0d0c0

OPENAFS-SA-2018-001 audit: support butc types

Add support for several complex butc types to enable butc auditing.

Change-Id: I6aedd933cf5330cda40aae6f33827ae65409df32

OPENAFS-SA-2018-001 butc: remove dummy osi_audit() routine

This local stub was present in the original IBM import and is unused.
It will conflict with the real audit code once we start adding auditing
to the TC_ RPCs, so remove it now.

Change-Id: I3e74e01464af122f245c3b0fe8f3985e422d13b4

OPENAFS-SA-2018-003 rxgen: prevent unbounded input arrays

RPCs with unbounded arrays as inputs are susceptible to remote
denial-of-service (DOS) attacks. A malicious client may submit an RPC
request with an arbitrarily large array, forcing the server to expend
large amounts of network bandwidth, cpu cycles, and heap memory to
unmarshal the input.

Instead, issue an error message and stop rxgen when it detects an RPC
defined with an unbounded input array. Thus we will detect the problem
at build time and prevent any future unbounded input arrays.

Change-Id: Ib110f817ed1c8132ea2549025876a5200c728fab

OPENAFS-SA-2018-003 volser: prevent unbounded input to various AFSVol* RPCs

Several AFSVol* RPCs are defined with an unbounded XDR "string" as
input.

RPCs with unbounded arrays as inputs are susceptible to remote
denial-of-service (DOS) attacks. A malicious client may submit an
AFSVol* request with an arbitrarily large string, forcing the volserver
to expend large amounts of network bandwidth, cpu cycles, and heap
memory to unmarshal the input.

Instead, give each input "string" an appropriate size.
Volume names are inherently capped to 32 octets (including trailing NUL)
by the protocol, but there is less clearly a hard limit on partition names.
The Vol_PartitionInfo{,64} functions accept a partition name as input and
also return a partition name in the output structure; the output values
have wire-protocol limits, so larger values could not be retrieved by clients,
but for denial-of-service purposes, a more generic PATH_MAX-like value seems
appropriate. We have several varying sources of such a limit in the tree, but
pick 4k as the least-restrictive.

[kaduk@mit.edu: use a larger limit for pathnames and expand on PATH_MAX in
commit message]

Change-Id: Iea4b24d1bb3570d4c422dd0c3247cd38cdbf4bab

OPENAFS-SA-2018-003 volser: prevent unbounded input to AFSVolForwardMultiple

AFSVolForwardMultiple is defined with an input parameter that is defined
to XDR as an unbounded array of replica structs:
typedef replica manyDests<>;

RPCs with unbounded arrays as inputs are susceptible to remote
denial-of-service (DOS) attacks. A malicious client may submit an
AFSVolForwardMultiple request with an arbitrarily large array, forcing
the volserver to expend large amounts of network bandwidth, cpu cycles,
and heap memory to unmarshal the input.

Even though AFSVolForwardMultiple requires superuser authorization, this
attack is exploitable by non-authorized actors because XDR unmarshalling
happens long before any authorization checks can occur.

Add a bounding constant (NMAXNSERVERS 13) to the manyDests input array.
This constant is derived from the current OpenAFS vldb implementation, which
is limited to 13 replica sites for a given volume by the layout (size) of the
serverNumber, serverPartition, and serverFlags fields.

[kaduk@mit.edu: explain why this constant is used]

Change-Id: Id12c6a7da4894ec490691eb8791dcd3574baa416

OPENAFS-SA-2018-003 budb: prevent unbounded input to BUDB_SaveText

BUDB_SaveText is defined with an input parameter that is defined to XDR
as an unbounded array of chars:
   typedef char charListT<>;

RPCs with unbounded arrays as inputs are susceptible to remote
denial-of-service (DOS) attacks.  A malicious client may submit a
BUDB_SaveText request with an arbitrarily large array, forcing the budb
server to expend large amounts of network bandwidth, cpu cycles, and
heap memory to unmarshal the input.

Modify the XDR definition of charListT so it is bounded.  This typedef
is shared (as an OUT parameter) by BUDB_GetText and BUDB_DumpDB, but
fortunately all in-tree callers of the client routines specify the same
maximum length of 1024.

Note: However, SBUDB_SaveText server implementation seems to allow for up to
BLOCK_DATA_SIZE (2040) = BLOCKSIZE (2048) - sizeof(struct blockHeader)
(8), and it's unknown if any out-of-tree callers exist.  Since we do not need a
tight bound in order to avoid the DoS, use a somewhat higher maximum of
4096 bytes to leave a safety margin.

[kaduk@mit.edu: bump the margin to 4096; adjust commit message to match]

Change-Id: Ic3fe2758a9c97ed02c6e6d05f0de0865959b5b04

OPENAFS-SA-2018-003 vlserver: prevent unbounded input to VL_RegisterAddrs

VL_RegisterAddrs is defined with an input argument of type bulkaddrs,
which is defined to XDR as an unbounded array of afs_uint32 (IPv4 addresses):
  typedef afs_uint32 bulkaddrs<>

The <> with no value instructs rxgen to build client and server stubs
that allow for a maximum size of "~0u" or 0xFFFFFFFF.

Ostensibly the bulkaddrs array is unbounded to allow it to be shared
among VL_RegisterAddrs, VL_GetAddrs, and VL_GetAddrsU.  The VL_GetAddrs*
RPCs use bulkaddrs as an output array with a maximum size of MAXSERVERID
(254). VL_RegisterAddrss uses bulkaddrs as an input array, with a
nominal size of VL_MAXIPADDRS_PERMH (16).

However, RPCs with unbounded array inputs are susceptible to remote
denial-of-service attacks.  That is, a malicious client may send a
VL_RegisterAddrs request with an arbitrarily long array, forcing the
vlserver to expend large amounts of network bandwidth, cpu cycles, and
heap memory to unmarshal the argument.  Even though VL_RegisterAddrs
requires superuser authorization, this attack is exploitable by
non-authorized actors because XDR unmarshalling happens long before any
authorization checks can occur.

Because all uses of the type that our implementation support have fixed
bounds on valid data (whether input or output), apply an arbitrary
implementation limit (larger than any valid structure would be), to
prevent this class of attacks in the XDR decoder.

[kaduk@mit.edu: limit the bulkaddrs type instead of introducing a new type]

Change-Id: Ibcc962ccc46aec7552b86d1d9fda7cc14310bc03

OPENAFS-SA-2018-002 butc: Initialize OUT scalar value

In STC_ReadLabel, the interaction with the tape device is
synchronous, so there is no need to allocate a task ID for status
monitoring. However, we do need to initialize the output value,
to avoid writing stack garbage on the wire.

Change-Id: Id2066e1fe95fa1de02577dfd844697b1ae770f30

OPENAFS-SA-2018-002 ubik: prevent VOTE_Debug, VOTE_XDebug information leak

VOTE_Debug and VOTE_XDebug (udebug) both leave a single field
uninitialized if there is no current transaction. This leaks the memory
contents of the ubik server over the wire.

struct ubik_debug
- 4 bytes in member writeTrans

In common code to both RPCs, ensure that writeTrans is always
initialized.

[kaduk@mit.edu: switch to memset]

Change-Id: I91184b4ed0c159982a883ebaa9634406400eae93

OPENAFS-SA-2018-002 kaserver: prevent KAM_ListEntry information leak

KAM_ListEntry (kas list) does not initialize its output correctly. It
leaks kaserver memory contents over the wire:

struct kaindex
- up to 64 bytes for member name
- up to 64 bytes for member instance

Initialize the buffer.

[kaduk@mit.edu: move initialization to top of server routine]

Change-Id: I5cc430fc996e7e89d38a384d092b9d4fad248fa4

OPENAFS-SA-2018-002 butc: prevent TC_DumpStatus, TC_ScanStatus information leaks

TC_ScanStatus (backup status) and TC_GetStatus (internal backup status
watcher) do not initialize their output buffers. They leak memory
contents over the wire:

struct tciStatusS
- up to 64 bytes in member taskName (TC_MAXNAMELEN 64)
- up to 64 bytes in member volumeName "

Initialize the buffers.

[kaduk@mit.edu: move initialization to top of server routines]

Change-Id: I0337d233e1dced56e351ed00471c9738fcd3b9db

OPENAFS-SA-2018-002 butc: prevent TC_ReadLabel information leak

TC_ReadLabel (backup readlabel) does not initialize its output buffer
completely. It leaks butc memory contents over the wire:

struct tc_tapeLabel
- up to 32 bytes from member afsname (TC_MAXTAPELEN 32)
- up to 32 bytes from member pname (TC_MAXTAPELEN 32)

Initialize the buffer.

[kaduk@mit.edu: move initialization to the RPC stub]

Change-Id: I30f4aa32801791913b397a58c36c86c019dc51ef

OPENAFS-SA-2018-002 budb: prevent BUDB_* information leaks

The following budb RPCs do not initialize their output correctly.
This leaks buserver memory contents over the wire:

BUDB_FindLatestDump (backup dump)
BUDB_FindDump (backup volrestore, diskrestore, volsetrestore)
BUDB_GetDumps (backup dumpinfo)
BUDB_FindLastTape (backup dump)

struct budb_dumpEntry
- up to 32 bytes in member volumeSetName
- up to 256 bytes in member dumpPath
- up to 32 bytes in member name
- up to 32 bytes in member tape.tapeServer
- up to 32 bytes in member tape.format
- up to 256 bytes in member dumper.name
- up to 128 bytes in member dumper.instance
- up to 256 bytes in member dumper.cell

Initialize the buffer in common routine FillDumpEntry.

Change-Id: Ic057a6c906ce2acd39e0e4ea0a0ba1e100bba3e9

OPENAFS-SA-2018-002 afs: prevent RXAFSCB_TellMeAboutYourself information leak

RXAFSCB_TellMeAboutYourself does not completely initialize its output
buffers. This leaks kernel memory over the wire:

struct interfaceAddr
Unix cache manager (libafs)
- up to 124 bytes in array addr_in ((AFS_MAX_INTERFACE_ADDR 32 * 4) - 4))
- up to 124 bytes in array subnetmask "
- up to 124 bytes in array mtu "

Windows cache manager
- 64 bytes in array addr_in ((AFS_MAX_INTERFACE_ADDR 32 - CM_MAXINTERFACE_ADDR 16)* 4)
- 64 bytes in array subnetmask "
- 64 bytes in array mtu "

The following implementations of SRXAFSCB_TellMeAboutYourself are not susceptible:
- fsprobe
- libafscp
- xstat_fs_test

Initialize the buffer.

Change-Id: I2ef868dd9269db7004a21cf913b6787948357d10

OPENAFS-SA-2018-002 afs: prevent RXAFSCB_GetLock information leak

RXAFSCB_GetLock (cmdebug) does not correctly initialize its output.
This leaks kernel memory over the wire:

struct AFSDBLock
- up to 14 bytes for member name (16 - '<cellname>\0')

Initialize the buffer.

Change-Id: I4c5c8d67816c51645c0db44dc8f19b1b27c02757

OPENAFS-SA-2018-002 ptserver: prevent PR_ListEntries information leak

PR_ListEntries (pts listentries) does not properly initialize its output
buffers. This leaks ptserver memory over the wire:

struct prlistentries
- up to 62 bytes for each entry name (PR_MAXNAMELEN 64 - 'a\0')

Initialize the buffer, and remove the now redundant memset for the
reserved fields.

Change-Id: I29d70c7e4dd567b8b046037f29f71911b8a0593f