Windows: Add caching to cm_GetAddrsU Cache the results of VL_GetAddrsU queries and reuse the results for subsequent calls when possible. Change-Id: I7e2b086ec311208a46439588bc820a1929d2b2b9 Reviewed-on: http://gerrit.openafs.org/10764 Tested-by: Jeffrey Altman <jaltman@your-file-system.com> Reviewed-by: Derrick Brashear <shadow@your-file-system.com> Reviewed-by: Jeffrey Altman <jaltman@your-file-system.com>
Windows: reverse order of EACCESS and USER locks The user lock is obtain while holding the eaccess lock. Reflect it in the hierarchy. Change-Id: I3aac945287415cd3babbe52f9fdeb93ab4d729bd Reviewed-on: http://gerrit.openafs.org/7247 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Chas Williams - CONTRACTOR <chas@cmf.nrl.navy.mil> Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com> Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Windows: Add per object per user EACCES caching If a cache manager is told by a file server that the user does not have permission to fetch status for an object, the cache manager must avoid requesting a fetch status a second time for that object for the same user. Doing so risks triggering the rx call abort throttling which can have a significant impact on end user usability of the Explorer Shell and other applications. The cache manager cannot make a decision on whether or not to issue an RXAFS_FetchStatus RPC based upon the type of the object because the type is unknown to the cache manager. A file server will succeed a FetchStatus request when the parent directory ACL grants lookup permission if the object in question is the directory or is a symlink/mountpoint. Only file objects require read/write permissions to obtain status information. The rx call abort throttling is broken is many ways and must be avoided. Call aborts are tracked by call channel and occur whenever ten call aborts are issued on the same call channel in a row regardless of the amount of time that has elapsed. The EACCES cache works by storing EACCES events by the FID and User for which the event occurred, when it occurred and the FID of the parent directory. By definition, the parent FID of a volume root directory is itself. Entries are removed from the cache under the following circumstances: 1. When the parent FID's callback expires or is replaced. 2. When the parent FID's cm_scache object is recycled. 3. When the user's tokens expire or are replaced. Entries are not removed when the FID's cm_scache object is recycled. This patchset also implements correct behavior if the VLF_DFSFILESET flag is set on a volume. Change-Id: I69507601f9872c9544e52a1d5e01064fa42efb81 Reviewed-on: http://gerrit.openafs.org/6996 Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com> Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
rx: RX_CALL_IDLE and RX_CALL_BUSY Allocate new Rx error codes for Idle and Busy calls but do not send these errors on the wire. They are only intended for local use. RX_CALL_IDLE is an indication to an application that requests it that the rx peer is maintaining an open call channel but has not sent any actual data for the length of the registered idle dead timeout. RX_CALL_BUSY is an indication to an application that requests it that the rx peer believes the selected call channel is in use by a pre-existing call. When either RX_CALL_IDLE or RX_CALL_BUSY are assigned as the call error and an abort must be sent to the rx peer, the errors are translated to RX_CALL_TIMEOUT. This is necessary because it is not possible to add new Rx error values in a method that is safe for peers that are not expecting them. This patchset also documents which Rx errors defined in rx.h are used on the wire and which are not. The Unix and Windows cache managers are updated to build with these new error codes. Change-Id: Ib236f27b88d503c68134534bb069e12dd83537d8 Reviewed-on: http://gerrit.openafs.org/6128 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com> Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Windows: create scache->redirMx to reduce contention Relying on the cm_scache_t.rw lock to protect the cm_scache_t.redirQueue* results in a large amount of contention between processing extent requests and releases from the afs redirector and the threads attempting to read from or write data to the file server. There is no reason why the same lock must be used. Allocate a dedicated mutex to protect the queue. By placing the new mutex after the buf_globalLock in the locking hierarchy it permits the lock acquisition logic for extent processing to be simplified further reducing cm_scache_t.rw lock transitions. Change-Id: Id2ded86c1f3757a2f1071c8cf39f2fbc6bcfcfaa Reviewed-on: http://gerrit.openafs.org/6053 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com> Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Windows: fix locking hierarchy in service The smb username lock and the daemon global lock can be requested while the scache dirlock is held if there are no free buffers and the service is forced to claw back extents from the redirector. Adjust the locking hierarchy accordingly. Change-Id: I85387a16ca580d678af45f3931aa5e81fe0a0f2c Reviewed-on: http://gerrit.openafs.org/6000 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com> Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Windows: Redirector interface for afsd_service.exe Over the last three years the afsd_service sources have been gradually separated into distinct layers for the SMB server and the AFS cache. The eventual goal of this work was to permit the addition of alternative interfaces to the cache manager in parallel. This patchset implements the first alternative interface, a reverse ioctl model that communicates with a native IFS redirector driver. The driver will be submitted in a subsequent patchset. Although it is possible to run afsd_service with both the SMB and RDR interfaces active at the same time. In practice it is somewhat impractical because it destroys the uniformity of the \\AFS name space. The RDR loads at boot time and claims all of \\AFS. The SMB interface if active at the same time must use the old \\%HOSTNAME%-AFS. As implemented, if the RDR interface is functional the SMB interface is not started. Only if the RDR interface fails will the SMB interface be activated. The afsd_service.exe maintains all of its primary responsibilities for communicating with the AFS servers, processing callbacks, enforcing permissions, handling afs path ioctls, Windows RPC service simulation, and object management. The biggest change is in the cm_buf_t management. Data is exchanged with the RDR by passing control over cm_buf_t->data buffers in the form of Windows File Extents. This avoids data copies across a communication channel which significantly improves performance at a substantial complexity cost. Credential management is switched from a Windows username binding to a GUID binding where the GUIDs represent authentication groups that are managed by the RDR. This patchset includes additional changes to support integrated logon in conjunction with the RDR. In particular, adding support for authentication groups. Change-Id: I7135489421c67a429ec3b2acd4c8ae08b8329f6d Reviewed-on: http://gerrit.openafs.org/5432 Tested-by: BuildBot <buildbot@rampaginggeek.com> Tested-by: Rod Widdowson <rdw@steadingsoftware.com> Reviewed-by: Jeffrey Altman <jaltman@openafs.org> Tested-by: Jeffrey Altman <jaltman@openafs.org>
Window: breakout CM error codes into separate header Change-Id: I67be608c6cb153904fa2ca8c5ad6cbc7943064e5 Reviewed-on: http://gerrit.openafs.org/5052 Reviewed-by: Derrick Brashear <shadow@dementia.org> Reviewed-by: Jeffrey Altman <jaltman@openafs.org> Tested-by: Jeffrey Altman <jaltman@openafs.org>
Windows: handle rx busy call channel Register an error code for rx busy call channel detection. Force a retry whenever CM_RX_BUSY_CALL_CHANNEL is received by cm_Analyze(). Log the event to both the internal trace log and the Windows Event Log along with the server address. Change-Id: I196fb99d38bb89f57f296fd1b60d2a7f17fec80c Reviewed-on: http://gerrit.openafs.org/4183 Reviewed-by: Derrick Brashear <shadow@dementia.org> Reviewed-by: Jeffrey Altman <jaltman@openafs.org> Tested-by: Jeffrey Altman <jaltman@openafs.org>
Windows: cleanup preprocessor definition namespace Do not use reserved preprocessor symbol names. Instead use OPENAFS_<PATH>_<HEADER>_H formatted names where <PATH> is the subdirectory path from src/ in which the header file originates in the repository. Change-Id: I998d7feeddeb9660f3fc514e2ba752c54e402a24 Reviewed-on: http://gerrit.openafs.org/3599 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Derrick Brashear <shadow@dementia.org> Reviewed-by: Jeffrey Altman <jaltman@openafs.org> Tested-by: Jeffrey Altman <jaltman@openafs.org>
Windows: Kill AFS_LARGEFILES preprocessor symbol All builds define AFS_LARGEFILES so kill the symbol and discard the !AFS_LARGEFILES source code. Change-Id: I36a2131e30b24d3d1a8f37f5629795bdd92c6b27 Reviewed-on: http://gerrit.openafs.org/2910 Reviewed-by: Jeffrey Altman <jaltman@openafs.org> Tested-by: Jeffrey Altman <jaltman@openafs.org>
death to trailing whitespace if we're gonna clean up... Change-Id: I5ab03f29468577b62dacab41a67eadfd8c43f812 Reviewed-on: http://gerrit.openafs.org/2463 Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Windows: Revise SMB QuerySecurityInfo for MS10-020 MS10-020 (http://support.microsoft.com/kb/980232) has caused many problems for implementors of SMB 1.0 servers and applications that call GetFileSecurity() without checking the return code to determine if the call succeeded. The gist of the vulnerability was that the SMB redirector would pass any buffer it received to the application regardless of whether or not it was valid. MS10-020 protects the applications by strictly validating the SMB response data structure and the data in the security descriptor that is returned. The problem for SMB 1.0 server implementors is that there have been at least three different protocol descriptions for NT_TRANSACT_QUERY_SECURITY_DESC published over the last decade and all of them are incomplete. Therefore, just about no one but Microsoft has an SMB 1.0 server implementation that produces the exact out that they are expecting to validate. The end result is that in an attempt to protect applications from crashing due to invalid input being passed in directly caused dozens of applications to crash by not returning any security descriptor data at all. Even when the applications didn't crash they might not have been able to save their data. Cisco WAAS and NetApp DataOnTap systems were most adversely affected and they have had CIFS protocol licenses for many many years. To fix OpenAFS here is what needed to be done: 1. Instead of returning a security descriptor that gives ownership to the NUL SID, give it to the Everyone SID and set the flag that states that everyone has full access. 2. Validate the input parameters. In particular, check to ensure that the SMB file descriptor is valid and the file has not been deleted. 3. Enforce the maximum output data and parameter counts. 4. Handle buffer overflow and buffertoosmall conditions in the manner that Microsoft expects them to be handled. In particular, note that the parameter data which is returned in the SMB Data Region is not counted in the Data Count. Even if MaxData is 0, we can still return parameters values as long as MaxParm is large enough. LICENSE MIT Change-Id: I95034bc6f24a282decc507edcffb93bc58b986be Reviewed-on: http://gerrit.openafs.org/2110 Tested-by: Jeffrey Altman <jaltman@openafs.org> Reviewed-by: Derrick Brashear <shadow@dementia.org> Reviewed-by: Asanka Herath <asanka@secure-endpoints.com> Reviewed-by: Jeffrey Altman <jaltman@openafs.org>
Windows: Monitor requests and gather diagnostics before a timeout This patch monitors SMB requests that are being serviced and automatically enables logging if a request takes longer than one minute to complete. If the requst hasn't completed by the two minute mark, the code generates a minidump. Once a minimump is generated, no more minidumps will be produced for another 5 minutes. SMB monitoring can be enabled/disabled using the new registry parameter 'SMBRequestMonitor.' Change-Id: I5aae22f6bfa635cec4a803089b483698641080eb Reviewed-on: http://gerrit.openafs.org/1632 Reviewed-by: Asanka Herath <asanka@secure-endpoints.com> Tested-by: Asanka Herath <asanka@secure-endpoints.com> Reviewed-by: Derrick Brashear <shadow@dementia.org> Reviewed-by: Jeffrey Altman <jaltman@openafs.org> Tested-by: Jeffrey Altman <jaltman@openafs.org>
Windows: Implement SRVSVC and WKSSVC RPC interfaces Windows uses RPC over SMB to communicate with file servers for administrative tasks including enumeration and queries of file server shares. This patch implements support for RPC over SMB and partially implements the SRVSVC and WKSSVC RPC interfaces. LICENSE MIT Reviewed-on: http://gerrit.openafs.org/301 Tested-by: Jeffrey Altman <jaltman@openafs.org> Reviewed-by: Jeffrey Altman <jaltman@openafs.org>
windows-cellservdb-lookup-20090525 LICENSE MIT Add a new Registry distribution method for CellServDB info. The CellServDB registry schema is as follows: HKLM\SOFTWARE\OpenAFS\Client\CellServDB\[cellname]\ "LinkedCell" REG_SZ "[cellname]" "Description" REG_SZ "[comment]" "ForceDNS" DWORD {0,1} HKLM\SOFTWARE\OpenAFS\Client\CellServDB\[cellname]\[servername]\ "HostName" REG_SZ "[hostname]" "IPv4Address" REG_SZ "[address]" "IPv6Address" REG_SZ "[address]" <future> "Comment" REG_SZ "[comment]" "Rank" DWORD "0..65535" "Clone" DWORD "{0,1}" <future - server only> "vlserver" DWORD "7003" <future> "ptserver" DWORD ... <future> ForceDNS is implied non-zero if there are no [servername] keys under the [cellname] key. Otherwise, ForceDNS is zero. If [servername] keys are specified and none of them evaluate to a valid server configuration, the return code is success. This prevents failover to the CellServDB file or DNS. Registry distributed info takes precedence over the CellServDB file. Registry support has been added to both the Windows specific cm_config interface and the auth/cellconfig interface utilized by aklog, the services, and the vast majority of support commands. Enhance the DNS lookup for Cell vlserver info to support ranking info which is used with _vlserver._udp SRV record lookups when AFSDB records are not present. Priorities become ranks.
windows-smb_dir_watch_lock-20081003 LICENSE MIT re-order the lock hierarchyfor smb_Dir_Watch_Lock so that the lock does not have to be dropped when sending notifications within smb_NotifyChange().
windows-local-dir-updates-20080916 LICENSE MIT When a local directory update occurs to the directory pages stored in the dcache, there is no guarrantee that the same modifications will be made to the cached pages as are being made by the file server. In the situation that one or more of the cached pages are recycled, it is not permissible to obtain the missing pages from the file server without first invalidating the rest of the cached pages. This is necessary to prevent mixing of incompatible data representations. Define a new flag CM_SCACHEFLAG_LOCAL which is used to indicate that dcache entries were locally modified even though they are not dirty. As the previous code could have corrupted the contents of the dcache, bump CM_CONFIG_DATA_VERSION in order to force the rebuilding of the cache. Add error CM_ERROR_NOTINCACHE to indicate that a requested directory page is not present in the cache and will not be created on the fly. Prefetch all dcache entries for directories and ensure that a consistent set is being used.
windows-misc-20080822 LICENSE MIT 1. In multi-threaded applications deadlocking is always a problem. Deadlock avoidance requires a strict adherence to a documented hierarchy. The lock hierarchy for OAFW is described in a file called locks.txt. There are two problems. First, some of the locks are not included in locks.txt. Second, it is nearly impossible given the depth of function calls for any programmer to identify all of the locks that are held at any given time a function is called. This patch implements a new locking order verification mechanism. Each lock is assigned a lock level at initialization. Each thread maintains a queue of held locks. Each time a lock is acquired the queue is checked to ensure that no locks with a higher level than the requested lock has already been acquired. If a violation occurs, the service panics. 2. When the service panics ensure that a minidump will always be generated. 3. Remove unused lock cm_bufGetMutex. 4. The lock order verifier identified approximately a dozen lock order violations that are corrected. 5. A race condition within the function path cm_GetSCache() -> cm_GetNewSCache() -> cm_RecycleSCache() permitted a cm_scache_t object to be issued simultaneously to two threads. This would eventually result in a panic due to the resulting under count. 6. Fix interpretation of the empty string as the ioctl path to mean the current directory. "fs lsm", "symlink list", etc. now return a "not a ..." error instead of "does not exist". 7. Add SMB_STRF_SRCNULTERM flag to smb_ParseStringXXX functions to indicate that the input string is a nul terminated string. Assign it when input strings are nul terminated. 8. The CIFS protocol specification for handling NT_TRANSACT_CREATE does not match the observed behavior. The 'nameLength' is specified in bytes not in characters. Fix the implementation to match. 9. The cm_HaveAccessRights() attempt at deadlock avoidance by calling lock_TryRead() on the parent directory cm_scache_t rw-lock does not avoid the deadlock. Avoid the deadlock by enforcing the lock order of lowest vnode first. Then remove the infinite loop avoidance in cm_SyncOp() that was returning an unwarranted access denied error.
windows-large-files-20080728 LICENSE MIT as part of the restructuring of code to separate the smb layer from the cm layer, large file support was broken. Define AFS_LARGEFILES in the right place so that it will be used in the cm.