dafs: preattach should wait for exclusive states

In rare circumstances an FSYNC_VOL_ON operation may fail silently,
leaving the volume in its previous state.  The only clue is a FileLog
message "volume <nnnn> not in quiescent state".

This is caused by a race condition in the volume package: an
FSYNC_VOL_ON operation is attempting to preattach a volume
(in VPreAttachVolumeByVp_r()) at the same time a fileserver RPC
(e.g. FetchStatus) is detaching the volume (in VReleaseVolumeHandles_r())
at the conclusion of attach2() logic.

The fix calls VWaitExclusiveState_r() before calling
VPreAttachVolumeByVp_r().

Change-Id: Ib66859381d29311fda3e08984dcb740eadafb340
Reviewed-on: http://gerrit.openafs.org/8814
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Derrick Brashear <shadow@your-file-system.com>

diff --git a/src/vol/volume.c b/src/vol/volume.c
index fe49fbb..384f6de 100644 (file)
--- a/src/vol/volume.c
+++ b/src/vol/volume.c
@@ -2161,11 +2161,26 @@ VPreAttachVolumeById_r(Error * ec,
        return NULL;
     }
 
+    /* ensure that any vp we pass to VPreAttachVolumeByVp_r
+     * is NOT in exclusive state.
+     */
+ retry:
     vp = VLookupVolume_r(ec, volumeId, NULL);
+
     if (*ec) {
        return NULL;
     }
 
+    if (vp && VIsExclusiveState(V_attachState(vp))) {
+       VCreateReservation_r(vp);
+       VWaitExclusiveState_r(vp);
+       VCancelReservation_r(vp);
+       vp = NULL;
+       goto retry;    /* look up volume again */
+    }
+
+    /* vp == NULL or vp not exclusive both OK */
+
     return VPreAttachVolumeByVp_r(ec, partp, vp, volumeId);
 }
 
@@ -2181,6 +2196,8 @@ VPreAttachVolumeById_r(Error * ec,
  *
  * @pre VOL_LOCK is held.
  *
+ * @pre vp (if specified) must not be in exclusive state.
+ *
  * @warning Returned volume object pointer does not have to
  *          equal the pointer passed in as argument vp.  There
  *          are potential race conditions which can result in
@@ -2205,6 +2222,11 @@ VPreAttachVolumeByVp_r(Error * ec,
 
     *ec = 0;
 
+    /* don't proceed unless it's safe */
+    if (vp) {
+       opr_Assert(!VIsExclusiveState(V_attachState(vp)));
+    }
+
     /* check to see if pre-attach already happened */
     if (vp &&
        (V_attachState(vp) != VOL_STATE_UNATTACHED) &&