Administration Guide


[Return to Library] [Contents] [Previous Topic] [Bottom of Topic] [Next Topic] [Index]


Managing Server Encryption Keys

This chapter explains how to maintain your cell's server encryption keys, which are vital for secure communications in AFS.


Summary of Instructions

This chapter explains how to perform the following tasks by using the indicated commands:

Add a new server encryption key bos addkey and kas setpassword
Inspect key checksums in the Authentication Database kas examine
Inspect key checksums in the KeyFile bos listkeys
Remove an old server encryption key bos removekey

About Server Encryption Keys

An encryption key is a string of octal numbers used to encrypt and decrypt packets of information. In AFS, a server encryption key is the key used to protect information being transferred between AFS server processes and between them and their clients. A server encryption key is essentially a password for a server process and like a user password is stored in the Authentication Database.

Maintaining your cell's server encryption keys properly is the most basic way to protect the information in your AFS filespace from access by unauthorized users.

Keys and Mutual Authentication: A Review

Server encryption keys play a central role in the mutual authentication between client and server processes in AFS. For a more detailed description of mutual authentication, see A More Detailed Look at Mutual Authentication.

When a client wants to contact an AFS server, it first contacts the Ticket Granting Service (TGS) module of the Authentication Server. After verifying the client's identity (based indirectly on the password of the human user whom the client represents), the TGS gives the client a server ticket. This ticket is encrypted with the server's encryption key. (The TGS also invents a second encryption key, called the session key, to be used only for a single episode of communication between server and client. The server ticket and session key, together with other pieces of information, are collectively referred to as a token.)

The client cannot read the server ticket or token because it does not know the server encryption key. However, the client sends it to the AFS server along with service requests, because the ticket proves to the AFS server processes that it has already authenticated with the TGS. AFS servers trust the TGS to grant tickets only to valid clients. The fact that the client possesses a ticket encrypted with the server's encryption key proves to the server that the client is valid. On the other hand, the client assumes that only a genuine AFS server knows the server encryption key needed to decrypt the ticket. The server's ability to decrypt the ticket and understand its contents proves to the client that the server is legitimate.

Maintaining AFS Server Encryption Keys

As you maintain your cell's server encryption keys, keep the following in mind.


Displaying Server Encryption Keys

To display the server encryption keys in the /usr/afs/etc/KeyFile file on any file server machine, use the bos listkeys command. Use the kas examine command to display the key in the Authentication Database's afs entry.

By default the commands do not display the actual string of octal digits that constitute a key, but rather a checksum, a decimal number derived by encrypting a constant with the key. This prevents unauthorized users from easily accessing the actual key, which they can then use to falsify or eavesdrop on protected communications. The bos listkeys and kas examine commands generate the same checksum for a given key, so displaying checksums rather than actual keys is generally sufficient. If you suspect that the keys differ in a way that the checksums are not revealing, then you are probably experiencing authentication problems throughout your cell. The easiest solution is to create a new server encryption key following the instructions in Adding Server Encryption Keys or Handling Server Encryption Key Emergencies. Another common reason to issue the bos listkeys command is to display the key version numbers currently in use, in preparation for choosing the next one; here, the checksum is sufficient because the key itself is irrelevant.

If it is important to display the actual octal digits, include the -showkey argument to both the bos listkeys and kas examine commands.

To display the KeyFile file

  1. Verify that you are authenticated as a user listed in the /usr/afs/etc/UserList file. If necessary, issue the bos listusers command, which is fully described in To display the users in the UserList file.
       % bos listusers <machine name>
    

  2. Issue the bos listkeys command to display the contents of one machine's /usr/afs/etc/KeyFile file.
       % bos listkeys <machine name> [-showkey]
    

    where

    listk
    Is the shortest acceptable abbreviation of listkeys.

    machine name
    Names a file server machine. In the normal case, it is acceptable to name any machine, because correct cell functioning requires that the KeyFile file be the same on all of them.

    -showkey
    Displays the octal digits that constitute each key.

In the following example, the output displays a checksum for each server encryption key rather than the actual octal digits. The penultimate line indicates when an administrator last changed the file, and the final line confirms that the output is complete.

   % bos listkeys fs1.abc.com
   key 0 has cksum 972037177
   key 1 has cksum 2825165022
   Keys last changed on Wed Jan 13 11:20:29 1999. 
   All done.

To display the afs key from the Authentication Database

  1. Issue the kas examine command to display the afs entry in the Authentication Database.

    The Authentication Server performs its own authentication rather than accepting your existing AFS token. By default, it authenticates your local (UNIX) identity, which possibly does not correspond to an AFS-privileged administrator. Include the -admin argument to name an identity that has the ADMIN flag on its Authentication Database entry. To verify that an entry has the flag, issue the kas examine command as described in To check if the ADMIN flag is set.

       % kas examine afs [-showkey]  \
                     -admin  <admin principal to use for authentication>  
       Administrator's (admin_user) password: admin_password 
    

    where

    e
    Is the shortest acceptable abbreviation of examine.

    afs
    Designates the afs entry.

    -showkey
    Displays the octal digits that constitute the key.

    -admin
    Names an administrative account with the ADMIN flag on its Authentication Database entry, such as admin. The password prompt echoes it as admin_user. Enter the appropriate password as admin_password.

In the following example, the admin user displays the afs entry without using the -showkey flag. The second line shows the key version number in parentheses and the key's checksum. The line that begins with the string last mod reports the date on which the indicated administrator changed the key. There is no necessary relationship between this date and the date reported by the bos listkeys command, because the latter date changes for any type of change to the KeyFile file, not just a key addition. For a description of the other lines in the output from the kas examine command, see its reference page in the IBM AFS Administration Reference.

   % kas examine afs  -admin admin
   Administrator's (admin) password: admin_password 
   User data for afs
    key (1) cksum is 2825165022, last cpw: no date
    password will never expire.
    An unlimited number of unsuccessful authentications is permitted.
    entry expires on never. Max ticket lifetime 100.00 hours.
    last mod on Wed Jan 13 11:21:36 1999 by admin
    permit password reuse

Adding Server Encryption Keys

As noted, AFS records server encryption keys in two separate places:

  1. In the file /usr/afs/etc/KeyFile on the local disk of each server machine, for use by the AFS server processes running on the machine

  2. In the afs entry in the Authentication Database, for use by the Ticket Granting Service (TGS) when creating tokens

To ensure that server processes and the TGS share the same AFS server encryption key, execute all the steps in this section without interruption.

The following instructions include a step in which you restart the database server processes (the Authentication, Backup, Protection, and Volume Location Server processes) on all database server machines. As a database server process starts, it reads in the server encryption key that has the highest key version number in the KeyFile file and uses it to protect the messages that it sends for synchronizing the database and maintaining quorum. It uses the same key throughout its lifetime, which can be for an extended period, even if you remove the key from the KeyFile file. However, if one of the peer database server processes restarts and the others do not, quorum and database synchronization break down because the processes are no longer using the same key: the restarted process is using the key that currently has the highest key version number, and the other processes are still using the key they read in when they originally started. To avoid this problem, it is safest to restart all of the database server processes when adding a new key.

After adding a new key, you can remove obsolete keys from the KeyFile file to prevent it from becoming cluttered. However, you must take care not to remove keys that client or server processes are still using. For discussion and instructions, see Removing Server Encryption Keys.

To add a new server encryption key

  1. Verify that you are authenticated as a user listed in the /usr/afs/etc/UserList file. If necessary, issue the bos listusers command, which is fully described in To display the users in the UserList file.
       % bos listusers <machine name>
    

  2. Issue the bos listkeys command to display the key version numbers that are already in use, as a first step in choosing the key version number for the new key.
       % bos listkeys <machine name>
    

    where

    listk
    Is the shortest acceptable abbreviation of listkeys.

    machine name
    Names any file server machine.

  3. Choose a key version number for the new key, based on the output from Step 2 and the following requirements:

  4. Issue the bos addkey command to create a new AFS server encryption key in the KeyFile file.

    If you run the United States edition of AFS and use the Update Server to distribute the contents of the system control machine's /usr/afs/etc directory, substitute the system control machine for the machine name argument. (If you have forgotten which machine is the system control machine, see To locate the system control machine.)

    If you run the international edition of AFS or do not use the Update Server, repeat the bos addkey command, substituting each server machine in your cell for the machine name argument in turn.

    To avoid visible echoing of the string that corresponds to the new key, omit the -key argument from the command line; instead enter the string at the prompts that appear when you omit it, as shown in the following syntax specification.

       % bos addkey  -server <machine name> -kvno <key version number>
       input key: afs_password
       Retype input key: afs_password
    

    where

    addk
    Is the shortest acceptable abbreviation of addkey.

    -server
    Names the cell's system control machine if you are using the Update Server, or each server machine in turn if you are not.

    -kvno
    Specifies the new key's key version number as an integer from the range 0 (zero) through 255.

    Remember the number. You need to use it again in Step 6. If you are using the international edition of AFS, be sure to type the same number each time you issue this command.

    afs_password
    Is a character string similar to a user password, of any length from one to about 1,000 characters. To improve security, include nonalphabetic characters and make the string as long as is practical (you need to type it only in this step and in Step 6). If you are using the international edition of AFS, be sure to type the same string each time you issue this command.

    Do not enter an octal string directly. The BOS Server scrambles the character string into an octal string appropriate for use as an encryption key before recording it in the KeyFile file.

  5. If you are using the Update Server, wait for a few minutes while the Update Server distributes the new KeyFile file to all server machines. The maximum necessary waiting period is the largest value provided for the -t argument to the upclientetc process's initialization command used on any of the server machines; the default time is five minutes.

    To be certain that all machines have the same KeyFile file, issue the bos listkeys command for every file server machine and verify that the checksum for the new key is the same on all machines.

       % bos listkeys <machine name>
    

    If you are not using the Update Server, try to complete Step 4 within five minutes.

  6. Issue the kas setpassword command to enter the same key in the afs entry in the Authentication Database.

    The Authentication Server performs its own authentication rather than accepting your existing AFS token. By default, it authenticates your local (UNIX) identity, which possibly does not correspond to an AFS-privileged administrator. Include the -admin argument to name an identity that has the ADMIN flag on its Authentication Database entry. To verify that an entry has the flag, issue the kas examine command as described in To check if the ADMIN flag is set.

       % kas setpassword -name afs -kvno <kvno>  \
                         -admin  <admin principal to use for authentication>  
       Administrator's (admin_user) password: admin_password 
       new_password: afs_password
       Verifying, please re-enter new_password: afs_password
    

    where

    sp
    Is an acceptable alias for setpassword (setp is the shortest acceptable abbreviation).

    -name afs
    Creates the new key in the afs entry.

    -kvno
    Specifies the same key version number as in Step 4.

    -admin
    Names an administrative account with the ADMIN flag on its Authentication Database entry, such as admin. The password prompt echoes it as admin_user. Enter the appropriate password as admin_password.

    afs_password
    Is the same character string you entered in Step 4.

  7. (Optional.) If you want to verify that the keys you just created in the KeyFile file and the Authentication Database afs entry are identical and have the same key version number, follow the instructions in Displaying Server Encryption Keys.

  8. Issue the bos restart command to restart the database server processes on all database server machines. This forces them to start using the key in the KeyFile file that currently has the highest key version number.

    Repeat this command in quick succession for each database server machine, starting with the machine that has the lowest IP address.

       % bos restart  <machine name> buserver kaserver ptserver vlserver
    

    where

    res
    Is the shortest acceptable abbreviation of restart.

    machine name
    Names each database server machine in turn.

    buserver kaserver ptserver vlserver
    Designates the Backup Server, Authentication Server, Protection Server, and Volume Location (VL) Server, respectively.

Removing Server Encryption Keys

You can periodically remove old keys from the /usr/afs/etc/KeyFile file to keep it to a reasonable size. To avoid disturbing cell functioning, do not remove an old key until all tokens sealed with the key and held by users or client processes have expired. After adding a new key, wait to remove old keys at least as long as the longest token lifetime you use in your cell. For Authentication Database user entries created under AFS version 3.1 or higher, the default token lifetime is 25 hours; for entries created under AFS version 3.0, it is 100 hours.

There is no command for removing the key from the afs entry in the Authentication Database, because the key field in that entry must never be empty. Use the kas setpassword command to replace the afs key, but only as part of the complete procedure detailed in To add a new server encryption key.

Never remove from the KeyFile file the key that is currently in the afs entry in the Authentication Database. AFS server processes become unable to decrypt the tickets that clients present to them.

To remove a key from the KeyFile file

  1. Verify that you are authenticated as a user listed in the /usr/afs/etc/UserList file. If necessary, issue the bos listusers command, which is fully described in To display the users in the UserList file.
       % bos listusers <machine name>
    

  2. Issue the bos listkeys command to display the key version number of each key you want to remove. The output also reveals whether it has been at least 25 hours since a new key was placed in the KeyFile file. For complete instructions for the bos listkeys command, see To display the KeyFile file.
       % bos listkeys <machine name>
    

  3. Issue the kas examine command to verify that the key currently in the Authentication Database's afs entry does not have the same key version number as any of the keys you are removing from the KeyFile file. For detailed instructions for the kas examine command, see To display the afs key from the Authentication Database.
       % kas examine afs  -admin <admin principal to use for authentication>  
       Administrator's (admin_user) password: admin_password 
    

  4. Issue the bos removekey command to remove one or more server encryption keys from the KeyFile file.

    If you run the United States edition of AFS and use the Update Server to distribute the contents of the system control machine's /usr/afs/etc directory, substitute the system control machine for the machine name argument. (If you have forgotten which machine is the system control machine, see To locate the system control machine.)

    If you run the international edition of AFS or do not use the Update Server, repeat the bos removekey command, substituting each server machine in your cell for the machine name argument in turn.

       % bos removekey <machine name> <key version number>
    

    where

    removek
    Is the shortest acceptable abbreviation of removekey.

    machine name
    Names the cell's system control machine if you are using the Update Server, or each server machine in turn if you are not.

    key version number
    Specifies the key version number of each key to remove.

Handling Server Encryption Key Emergencies

In rare circumstances, the AFS server processes can become unable to decrypt the server tickets that clients or peer server processes are presenting. Activity in your cell can come to a halt, because the server processes believe that the tickets are forged or expired, and refuse to execute any actions. This can happen on one machine or several; the effect is more serious when more machines are involved.

One common cause of server encryption key problems is that the client's ticket is encrypted with a key that the server process does not know. Usually this means that the /usr/afs/etc/KeyFile on the server machine does not include the key in the afs Authentication Database entry, which the Authentication Server's Ticket Granting Service (TGS) module is using to encrypt server tickets.

Another possibility is that the KeyFile files on different machines do not contain the same keys. In this case, communications among server processes themselves become impossible. For instance, AFS's replicated database mechanism (Ubik) breaks down if the instances of a database server process on the different database server machines are not using the same key.

The appearance of the following error message when you direct a bos command to a file server machine in the local cell is one possible symptom of server encryption key mismatch. (Note, however, that you can also get this message if you forget to include the -cell argument when directing the bos command to a file server machine in a foreign cell.)

   bos: failed to contact host's bosserver (security object was passed a bad ticket).

The solution to server encryption key emergencies is to put a new AFS server encryption key in both the Authentication Database and the KeyFile file on every server machine, so that the TGS and all server processes again share the same key.

Handling key emergencies requires some unusual actions. The reasons for these actions are explained in the following sections; the actual procedures appear in the subsequent instructions.

Prevent Mutual Authentication

It is necessary to prevent the server processes from trying to mutually authenticate with you as you deal with a key emergency, because they possibly cannot decrypt your token. When you do not mutually authenticate, the server processes assign you the identity anonymous. To prevent mutual authentication, use the unlog command to discard your tokens and include the -noauth flag on every command where it is available.

Disable Authorization Checking by Hand

Because the server processes recognize you as the user anonymous when you do not mutually authenticate, you must turn off authorization checking. Only with authorization checking disabled do the server processes allow the anonymous user to perform privileged actions such as key creation.

In an emergency, disable authorization checking by creating the file /usr/afs/local/NoAuth by hand. In normal circumstances, use the bos setauth command instead.

Work Quickly on Each Machine

Disabling authorization checking is a serious security exposure, because server processes on the affected machine perform any action for anyone. Disable authorization checking only for as long as necessary, completing all steps in an uninterrupted session and as quickly as possible.

Work at the Console

Working at the console of each server machine on which you disable authorization checking ensures that no one else logs onto the console while you are working there. It does not prevent others from connecting to the machine remotely (using the telnet program, for example), which is why it is important to work quickly. The only way to ensure complete security is to disable network traffic, which is not a viable option in many environments. You can improve security in general by limiting the number of people who can connect remotely to your server machines at any time, as recommended in Improving Security in Your Cell.

Change Individual KeyFile Files

If you use the Update Server to distribute the contents of the /usr/afs/etc directory, an emergency is the only time when it is appropriate to change the KeyFile file on individual machines instead. Updating each machine's file is necessary because mismatched keys can prevent the system control machine's upserver process from mutually authenticating with upclientetc processes on other server machines, in which case the upserver process refuses to distribute its KeyFile file to them.

Even if it appears that the Update Server is working correctly, the only way to verify that is to change the key on the system control machine and wait the standard delay period to see if the upclientetc processes retrieve the key. During an emergency, it does not usually make sense to wait the standard delay period. It is more efficient simply to update the file on each server machine separately. Also, even if the Update Server can distribute the file correctly, other processes can have trouble because of mismatched keys. The following instructions add the new key file on the system control machine first. If the Update Server is working, then it is distributing the same change as you are making on each server machine individually.

If your cell does not use the Update Server, or uses the international edition of AFS, you always change keys on server machines individually. The following instructions are also appropriate for you.

Two Component Procedures

There are two subprocedures used frequently in the following instructions: disabling authorization checking and reenabling it. For the sake of clarity, the procedures are detailed here; the instructions refer to them as necessary.

Disabling Authorization Checking in an Emergency

  1. Become the local superuser root on the machine, if you are not already, by issuing the su command.
       % su root
       Password: root_password
    

  2. Create the file /usr/afs/local/NoAuth to disable authorization checking.
       # touch /usr/afs/local/NoAuth
    

  3. Discard your tokens, in case they were sealed with an incompatible key, which can prevent some commands from executing.
       # unlog
    

Reenabling Authorization Checking in an Emergency

  1. Become the local superuser root on the machine, if you are not already, by issuing the su command.
       % su root
       Password: root_password
    

  2. Remove the /usr/afs/local/NoAuth file.
       # rm /usr/afs/local/NoAuth
    

  3. Authenticate as an administrative identity that belongs to the system:administrators group and is listed in the /usr/afs/etc/UserList file.
       # klog <admin_user>
       Password: <admin_password>
    

  4. If appropriate, log out from the console (or close the remote connection you are using), after issuing the unlog command to destroy your tokens.

To create a new server encryption key in emergencies

  1. On the system control machine, disable authorization checking as instructed in Disabling Authorization Checking in an Emergency.

  2. Issue the bos listkeys command to display the key version numbers already in use in the KeyFile file, as a first step in choosing the new key's key version number.
       # bos listkeys <machine name> -noauth
    

    where

    listk
    Is the shortest acceptable abbreviation of listkeys.

    machine name
    Specifies a file server machine.

    -noauth
    Bypasses mutual authentication with the BOS Server. Include it in case the key emergency is preventing successful mutual authentication.

  3. Choose a key version number for the new key, based on what you learned in Step 2 plus the following requirements:

  4. On the system control machine, issue the bos addkey command to create a new AFS server encryption key in the KeyFile file.
       # bos addkey <machine name> -kvno <key version number> -noauth
       input key: afs_password
       Retype input key: afs_password
    

    where

    addk
    Is the shortest acceptable abbreviation of addkey.

    machine name
    Names the file server machine on which to define the new key in the KeyFile file.

    -kvno
    Specifies the key version number you chose in Step 3, an integer in the range 0 (zero) through 255. You must specify the same number in Steps 7, 8, and 13.

    -noauth
    Bypasses mutual authentication with the BOS Server. Include it in case the key emergency is preventing successful mutual authentication.

    afs_password
    Is a character string similar to a user password, of any length from one to about 1,000 characters. To improve security, make the string as long as is practical, and include nonalphabetic characters.

    Do not type an octal string directly. The BOS Server scrambles the character string into an octal string appropriate for use as an encryption key before recording it in the KeyFile file.

    Remember the string. You need to use it again in Steps 7, 8, and 13.

  5. On every database server machine in your cell (other than the system control machine), disable authorization checking as instructed in Disabling Authorization Checking in an Emergency. Do not repeat the procedure on the system control machine, if it is a database server machine, because you already disabled authorization checking in Step 1. (If you need to learn which machines are database server machines, use the bos listhosts command as described in To locate database server machines.)

  6. Wait at least 90 seconds after finishing Step 5, to allow each of the database server processes (the Authentication, Backup, Protection and Volume Location Servers) to finish electing a new sync site. Then issue the udebug command to verify that the election worked properly. Issue the following commands, substituting each database server machine's name for server machine in turn. Include the system control machine if it is a database server machine.
       # udebug <server machine> buserver
       # udebug <server machine> kaserver
       # udebug <server machine> ptserver
       # udebug <server machine> vlserver
    

    For each process, the output from all of the database server machines must agree on which one is the sync site for the process. It is not, however, necessary that the same machine serves as the sync site for each of the four processes. For each process, the output from only one machine must include the following string:

       I am sync site ...
    

    The output on the other machines instead includes the following line

       I am not sync site
    

    and a subsequent line that begins with the string Sync host and specifies the IP address of the machine claiming to be the sync site.

    If the output does not meet these requirements or seems abnormal in another way, contact AFS Product Support for assistance.

  7. On every database server machine in your cell (other than the system control machine), issue the bos addkey command described in Step 4. Be sure to use the same values for afs_password and kvno as you used in that step.

  8. Issue the kas setpassword command to define the new key in the Authentication Database's afs entry. It must match the key you created in Step 4 and Step 7.
       # kas setpassword  -name afs  -kvno <key version number> -noauth
       new_password: afs_password
       Verifying, please re-enter new_password: afs_password
    

    where

    sp
    Is an acceptable alias for setpassword (setp is the shortest acceptable abbreviation).

    -kvno
    Is the same key version number you specified in Step 4.

    afs_password
    Is the same character string you specified as afs_password in Step 4. It does not echo visibly.

  9. On every database server machine in your cell (including the system control machine if it is a database server machine), reenable authorization checking as instructed in Reenabling Authorization Checking in an Emergency. If the system control machine is not a database server machine, do not perform this procedure until Step 11.

  10. Repeat Step 6 to verify that each database server process has properly elected a sync site after being restarted in Step 9.

  11. On the system control machine (if it is not a database server machine), reenable authorization checking as instructed in Reenabling Authorization Checking in an Emergency. If it is a database server machine, you already performed the procedure in Step 9.

  12. On all remaining (simple) file server machines, disable authorization checking as instructed in Disabling Authorization Checking in an Emergency.

  13. On all remaining (simple) file server machines, issue the bos addkey command described in Step 4. Be sure to use the same values for afs_password and kvno as you used in that step.

  14. On all remaining (simple) file server machines, reenable authorization checking as instructed in Reenabling Authorization Checking in an Emergency.

[Return to Library] [Contents] [Previous Topic] [Top of Topic] [Next Topic] [Index]



© IBM Corporation 2000. All Rights Reserved