8 Rx provides data collections for remote debugging and troubleshooting using UDP
9 packets. This document provides details on the protocol, data formats, and
10 the data format versions.
16 A simple request/response protocol is used to request this information
17 from an Rx instance. Request and response packets contain an Rx header but
18 only a subset of the header fields are used, since the debugging packages are
19 not part of the Rx RPC protocol.
21 The protocol is simple. A client sends an Rx DEBUG (8) packet to an
22 address:port of an active Rx instance. This request contains an arbitrary
23 request number in the callNumber field of the Rx header (reused here since
24 DEBUG packets are never used in RPCs). The payload of the request is simply a
25 pair 32 bit integers in network byte order. The first integer indicates the
26 which data collection type is requested. The second integer indicates which
27 record number of the data type requested, for data types which have multiple
28 records, such as the rx connections and rx peers. The request packet must have
29 the CLIENT-INITIATED flag set in the Rx header.
31 Rx responds with a single Rx DEBUG (8) packet, the payload of which contains
32 the data record for the type and index requested. The callNumber in the Rx
33 header contains the same number as the value of the request, allowing the
34 client to match responses to requests. The response DEBUG packet does not
35 contain the request type and index parameters.
37 The first 32-bits, in network byte order, of the response payload indicate
40 * 0xFFFFFFFF (-1) index is out of range
41 * 0xFFFFFFF8 (-8) unknown request type
47 OpenAFS defines 5 types of data collections which may be
50 1 GETSTATS Basic Rx statistics (struct rx_debugStats)
51 2 GETCONN Active connections [indexed] (struct rx_debugConn)
52 3 GETALLCONN All connections [indexed] (struct rx_debugConn)
53 4 RXSTATS Detailed Rx statistics (struct rx_statistics)
54 5 GETPEER Rx peer info [indexed] (struct rx_peerDebug)
56 The format of the response data for each type is given below. XDR is
57 not used. All integers are in network byte order.
59 In a typical exchange, a client will request the "basic Rx stats" data first.
60 This contains a data layout version number (detailed in the next section).
62 Types GETCONN (2), GETALLCONN (3), and GETPEER (5), are array-like data
63 collections. The index field is used to retrieve each record, one per packet.
64 The first record is index 0. The client may request each record, starting with
65 zero, and incremented by one on each request packet, until the Rx service
66 returns -1 (out of range). No provisions are made for locking the data
67 collections between requests, as this is intended only to be a debugging
71 Data Collection Versions
72 ========================
74 Every Rx service has a single byte wide debugging version id, which is set at
75 build time. This version id allows clients to properly interpret the response
76 data formats for the various data types. The version id is present in the
77 basic Rx statistics (type 1) response data.
79 The first usable version is 'L', which was present in early Transarc/IBM AFS.
80 The first version in OpenAFS was 'Q', and versions after 'Q' are OpenAFS
81 specific extensions. The current version for OpenAFS is 'S'.
83 Historically, the version id has been incremented when a new debug data type is
84 added or changed. The version history is summarized in the following table:
86 'L' - Earliest usable version
87 - GETSTATS (1) supported
88 - GETCONNS (2) supported (with obsolete format rx_debugConn_vL)
89 - Added connection object security stats (rx_securityObjectStats) to GETCONNS (2)
92 'M' - Added GETALLCONN (3) data type
93 - Added RXSTATS (4) data type
96 'N' - Added calls waiting for a thread count (nWaiting) to GETSTATS (1)
99 'O' - Added number of idle threads count (idleThreads) to GETSTATS (1)
102 'P' - Added cbuf packet allocation failure counts (receiveCbufPktAllocFailures
103 and sendCbufPktAllocFailures) to RXSTATS (4)
106 'Q' - Added GETPEER (5) data type
110 (?) - Added number of busy aborts sent (nBusies) to RXSTATS (4)
111 - rxdebug was not changed to display this new count
114 'R' - Added total calls which waited for a thread (nWaited) to GETSTATS (1)
115 - OpenAFS 1.5.0 (devel)
116 - OpenAFS 1.6.0 (stable)
118 'S' - Added total packets allocated (nPackets) to GETSTATS (1)
119 - OpenAFS 1.5.53 (devel)
120 - OpenAFS 1.6.0 (stable)
124 Debug Request Parameters
125 ========================
127 The payload of DEBUG request packets is two 32 bit integers
128 in network byte order.
132 afs_int32 type; /* requested type; range 1..5 */
133 afs_int32 index; /* record number: 0 .. n */
136 The index field should be set to 0 when type is GETSTAT (1) and RXSTATS (4).
143 GETSTATS returns basic Rx performance statistics and the overall debug
146 struct rx_debugStats {
147 afs_int32 nFreePackets;
148 afs_int32 packetReclaims;
149 afs_int32 callsExecuted;
150 char waitingForPackets;
154 afs_int32 nWaiting; /* Version 'N': number of calls waiting for a thread */
155 afs_int32 idleThreads; /* Version 'O': number of server threads that are idle */
156 afs_int32 nWaited; /* Version 'R': total calls waited */
157 afs_int32 nPackets; /* Version 'S': total packets allocated */
162 GETCONN (2) and GETALLCONN (3)
163 ==============================
165 GETCONN (2) returns an active connection information record, for the
168 GETALLCONN (3) returns a connection information record, active or not,
169 for the given index. The GETALLCONN (3) data type was added in
172 The data format is the same for GETCONN (2) and GETALLCONN (3), and is
175 struct rx_debugConn {
179 afs_int32 callNumber[RX_MAXCALLS];
185 char sparec[3]; /* force correct alignment */
186 char callState[RX_MAXCALLS];
187 char callMode[RX_MAXCALLS];
188 char callFlags[RX_MAXCALLS];
189 char callOther[RX_MAXCALLS];
190 /* old style getconn stops here */
191 struct rx_securityObjectStats secStats;
197 Note: the char 'flags' member is no longer able to represent all possible values in the
198 rx_connection 'flags' member, after the latter was enlarged from u_char to afs_uint32.
200 An obsolete layout, which exhibited a problem with data alignment, was used in
201 Version 'L'. This is defined as:
203 struct rx_debugConn_vL {
207 afs_int32 callNumber[RX_MAXCALLS];
213 char callState[RX_MAXCALLS];
214 char callMode[RX_MAXCALLS];
215 char callFlags[RX_MAXCALLS];
216 char callOther[RX_MAXCALLS];
217 /* old style getconn stops here */
218 struct rx_securityObjectStats secStats;
219 afs_int32 sparel[10];
223 The layout of the secStats field is as follows:
225 struct rx_securityObjectStats {
226 char type; /* 0:unk 1:null,2:vab 3:kad */
228 char sparec[10]; /* force correct alignment */
229 afs_int32 flags; /* 1=>unalloc, 2=>auth, 4=>expired */
231 afs_uint32 packetsReceived;
232 afs_uint32 packetsSent;
233 afs_uint32 bytesReceived;
234 afs_uint32 bytesSent;
244 RXSTATS (4) returns general rx statistics. Every member of the returned
245 structure is a 32 bit integer in network byte order. The assumption is made
246 sizeof(int) is equal to sizeof(afs_int32).
248 The RXSTATS (4) data type was added in Version 'M'.
251 struct rx_statistics { /* General rx statistics */
252 int packetRequests; /* Number of packet allocation requests */
253 int receivePktAllocFailures;
254 int sendPktAllocFailures;
255 int specialPktAllocFailures;
256 int socketGreedy; /* Whether SO_GREEDY succeeded */
257 int bogusPacketOnRead; /* Number of inappropriately short packets received */
258 int bogusHost; /* Host address from bogus packets */
259 int noPacketOnRead; /* Number of read packets attempted when there was actually no packet to read off the wire */
260 int noPacketBuffersOnRead; /* Number of dropped data packets due to lack of packet buffers */
261 int selects; /* Number of selects waiting for packet or timeout */
262 int sendSelects; /* Number of selects forced when sending packet */
263 int packetsRead[RX_N_PACKET_TYPES]; /* Total number of packets read, per type */
264 int dataPacketsRead; /* Number of unique data packets read off the wire */
265 int ackPacketsRead; /* Number of ack packets read */
266 int dupPacketsRead; /* Number of duplicate data packets read */
267 int spuriousPacketsRead; /* Number of inappropriate data packets */
268 int packetsSent[RX_N_PACKET_TYPES]; /* Number of rxi_Sends: packets sent over the wire, per type */
269 int ackPacketsSent; /* Number of acks sent */
270 int pingPacketsSent; /* Total number of ping packets sent */
271 int abortPacketsSent; /* Total number of aborts */
272 int busyPacketsSent; /* Total number of busies sent received */
273 int dataPacketsSent; /* Number of unique data packets sent */
274 int dataPacketsReSent; /* Number of retransmissions */
275 int dataPacketsPushed; /* Number of retransmissions pushed early by a NACK */
276 int ignoreAckedPacket; /* Number of packets with acked flag, on rxi_Start */
277 struct clock totalRtt; /* Total round trip time measured (use to compute average) */
278 struct clock minRtt; /* Minimum round trip time measured */
279 struct clock maxRtt; /* Maximum round trip time measured */
280 int nRttSamples; /* Total number of round trip samples */
281 int nServerConns; /* Total number of server connections */
282 int nClientConns; /* Total number of client connections */
283 int nPeerStructs; /* Total number of peer structures */
284 int nCallStructs; /* Total number of call structures allocated */
285 int nFreeCallStructs; /* Total number of previously allocated free call structures */
287 afs_int32 fatalErrors;
288 int ignorePacketDally; /* packets dropped because call is in dally state */
289 int receiveCbufPktAllocFailures; /* Version 'P': receive cbuf packet alloc failures */
290 int sendCbufPktAllocFailures; /* Version 'P': send cbuf packet alloc failures */
291 int nBusies; /* Version 'R': number of busy aborts sent */
299 GETPEER (5) returns a peer information record, for the given index.
301 struct rx_debugPeer {
309 struct clock burstWait;
312 struct clock timeout;
315 afs_int32 inPacketSkew;
316 afs_int32 outPacketSkew;
320 u_short maxDgramPackets;
321 u_short ifDgramPackets;
324 u_short nDgramPackets;
326 afs_hyper_t bytesSent;
327 afs_hyper_t bytesReceived;
328 afs_int32 sparel[10];