RPC Programs and Procedures(NFS) details

■ Requirement : RPC Programs and Procedures(NFS) details
■ OS Environment : Linux[RHEL 5]
■ Application: rpc
■ Resolution : 

RPC Programs and Procedures(RPC request Message) :

The RPC call message has three unsigned integer fields -- remote
program number, remote program version number, and remote procedure
number -- which uniquely identify the procedure to be called.
Program numbers are administered by a central authority
(rpc@sun.com). Once implementors have a program number, they can
implement their remote program; the first implementation would most
likely have the version number 1. Because most new protocols evolve,
a version field of the call message identifies which version of the
protocol the caller is using. Version numbers enable support of both
old and new protocols through the same server process.

The procedure number identifies the procedure to be called. These
numbers are documented in the specific program's protocol
specification. For example, a file service's protocol specification
may state that its procedure number 5 is "read" and procedure number
12 is "write".

Just as remote program protocols may change over several versions,
the actual RPC message protocol could also change. Therefore, the
call message also has in it the RPC version number, which is always
equal to two for the version of RPC described here.

check : http://kmaiti.blogspot.com/2011/03/how-rpc-works.html

RPC Reply Message :

The reply message to a request message has enough information to
distinguish the following error conditions:

(1) The remote implementation of RPC does not support protocol
version 2. The lowest and highest supported RPC version numbers
are returned.

(2) The remote program is not available on the remote system.

(3) The remote program does not support the requested version
number. The lowest and highest supported remote program version
numbers are returned.

(4) The requested procedure number does not exist. (This is
usually a client side protocol or programming error.)

(5) The parameters to the remote procedure appear to be garbage
from the server's point of view. (Again, this is usually caused
by a disagreement about the protocol between client and service.)

check : http://kmaiti.blogspot.com/2011/03/how-rpc-works.html

Program Number Assignment :

Program numbers are given out in groups of hexadecimal 20000000
(decimal 536870912) according to the following chart:

0 - 1fffffff defined by rpc@sun.com
20000000 - 3fffffff defined by user
40000000 - 5fffffff transient
60000000 - 7fffffff reserved
80000000 - 9fffffff reserved
a0000000 - bfffffff reserved
c0000000 - dfffffff reserved
e0000000 - ffffffff reserved

>> The first group is a range of numbers administered by rpc@sun.com and
should be identical for all sites.
>> The second range is for applications peculiar to a particular site. This range is intended
primarily for debugging new programs. When a site develops an
application that might be of general interest, that application
should be given an assigned number in the first range. Application
developers may apply for blocks of RPC program numbers in the first
range by sending electronic mail to "rpc@sun.com".
>>The third group is for applications that generate program numbers dynamically. The
final groups are reserved for future use, and should not be used.

You can capture the tcpdump and analysis it by wireshark.

nfs server daemons:

rpc.portmap
rpc.mountd, rpc.nfsd
rpc.statd, rpc.lockd (if necessary), and rpc.rquotad

nfs client daemons :

portmap, lockd, and statd

check : rpcinfo -p

URL : http://kmaiti.blogspot.com/2011/03/how-nfs-works.html

How NFS works

How RPC works?

■ Requirement : How RPC works?
■ OS Environment : Linux[RHEL 5]
■ Application: rpc
■ Resolution: 
     
        A means of communication between two tasks running on separate machines linked by a LAN. One machine can request a service, typically computation, from the other, by executing a high level request known as a language procedure call.

I have attached pictures one by one to get an idea :

How to configure NFSv4 with kerberos on linux?

■ Requirement : How to configure NFSv4 with kerberos on linux
■ OS Environment : Linux[RHEL, Centos]
■ Application: nfsv4, kerberos

• kerberos(i.e NFSv4) server : RHEL 5.5, 64 bit arch
• NFS client : RHEL 4, 64bit arch
• krb5-workstation-1.6.1-55.el5
• pam_krb5-2.2.14-18.el5
• krb5-libs-1.6.1-55.el5
• krb5-server-1.6.1-55.el5
• krb5-libs-1.6.1-55.el5
• pam_krb5-2.2.14-18.el5
• cyrus-sasl-gssapi-2.1.22-5.el5_4.3
• krb5-server-1.6.1-55.el

■ Prerequisites :

 server : vm12.gsslab.pnq.example.com and
client : vm217.gsslab.pnq.example.com

---------
1. Time Synchronization: All machines that will participate in kerberos authentication must have a reliable, synchronized time source. Most large orgainization offer their own time sources. You can use the RHEL configuration tool system-config-time to set this up. So, time of both the server and clients will be same.

2. Hostnames : All hosts must have their hostname set to the fully qualified hostname as reported by DNS. Both forward and reverse mapping must work properly. Like :

[root@vm12 data]# hostname
vm12.gsslab.pnq.example.com
[root@vm12 data]# host vm12.gsslab.pnq.example.com
vm12.gsslab.pnq.example.com has address 10.65.211.12
[root@vm12 data]#

The host may be referenced by a CNAME, but the official host name (as reported by hostname) must be an ‘A’ record. This is important; if you don’t have this setup properly then some things will work, while other things will fail mysteriously. If the host name does not match the reverse DNS lookup, Kerberos authentication will fail.

3. Packages :

On client machine : Make it sure that following packages are installed on the client machines.

-------
krb5-libs
krb5-workstation
pam_krb5
cyrus-sasl-gssapi
-------

You can execute the following command to install the packages :

#up2date krb5-libs krb5-workstation pam_krb5 cyrus-sasl-gssapi

On server machine : Make it sure that following package is installed :

--------
krb5-server
--------

You can execute the following command to install it.

#yum -y install krb5-server


4. You should have all of your machines registered in DNS under one (or more) domains. In our case, our machines are in the "gsslab.pnq.example.com" domain. We also authenticate our supercomputers which are in the "vm12.gsslab.pnq.example.com" domain.

You need to choose a kerberos realm. A kerberos realm is completely different from a DNS domain, but in most cases you will want to use the same name. By convention, kerberos realms are all upper case. The kerberos realm is "GSSLAB.PNQ.EXAMPLE.COM". This realm serves both the gsslab.pnq.example.com and the vm12.gsslab.pnq.example.com DNS domains.

Server setup :--

5. Configuring Kerberos on the Server :

A) There are a number of files that have to be manually edited on the server :

Edit /etc/krb5.conf

The stock version of this file will have EXAMPLE.COM or example.com everywhere you want to put your own realm or domain name. The two sections in question are libdefaults and domain_realm. The other sections do not need to be changed. In libdefaults, enter your own kerberos realm name. You may want to set the clock skew to a lower value (provided you are synchronizing time with ntp). The file will look like :

---------
[root@vm12 ~]#cat /etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults]
default_realm = GSSLAB.PNQ.EXAMPLE.COM
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
forwardable = yes

[realms]
GSSLAB.PNQ.EXAMPLE.COM = {
kdc = vm12.gsslab.pnq.example.com:88
admin_server = vm12.gsslab.pnq.example.com:749
default_domain = gsslab.pnq.example.com
}

[domain_realm]
.gsslab.pnq.example.com= GSSLAB.PNQ.EXAMPLE.COM
gsslab.pnq.example.com = GSSLAB.PNQ.EXAMPLE.COM

[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}
[root@vm12 ~]#
---------

B) Edit /var/kerberos/krb5kdc/kdc.conf

In this file, only the realms section needs to be modified. It is important to change the key types as well. I can confirm that the setting below work perfectly in our environment. You may want to decide on appropriate values for the maximum life of each ticket, and for how long each ticket can be renewed. Reasonable values are 1 day and 1 week but your needs will vary. The values here are the absolute maximum that the KDC will issue. Each principal has its own maximum as well. File will look like :

------
[root@vm12 ~]# cat /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
v4_mode = nopreauth
kdc_tcp_ports = 88

[realms]
GSSLAB.PNQ.EXAMPLE.COM = {
#master_key_type = des3-hmac-sha1
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal des-cbc-crc:v4 des-cbc-crc:afs3
}
[root@vm12 ~]#
------

C) Edit /var/kerberos/krb5kdc/kadm5.acl :

This file determines who can modify the kerberos database. You need to change the realm. File will look like :

-------
[root@vm12 ~]# cat /var/kerberos/krb5kdc/kadm5.acl
*/admin@GSSLAB.PNQ.EXAMPLE.COM *
[root@vm12 ~]#
-------

D) Make sure /etc/gssapi_mech.conf looks like :

-----
[root@vm12 ~]# cat /etc/gssapi_mech.conf
# library initialization function
# ================================ ==========================
# The MIT K5 gssapi library, use special function for initialization.
libgssapi_krb5.so.2 mechglue_internal_krb5_init
#
[root@vm12 ~]#
-----

E) Create the Kerberos database :

Execute the following command :

------
[root@vm12 ~]# kdb5_util -r GSSLAB.PNQ.EXAMPLE.COM create -s
------

This will prompt you for a password. You will only have to enter this password when you initially configure a slave KDC, so choose something large and random and store it in a secure place. Really, you may only have to enter this once more, so make it secure.

F) Add the first Administrative User :

I do administration as root, so the first user I add is root/admin. The default realm is appended automatically, so the command to use is as follows :

------
[root@vm12 ~]# kadmin.local -q "addprinc root/admin"
------

Enter a password when prompted. You will need this password every time you administer the database.

G) At this point it is necessary to enable and start the kerberos services :

-----
[root@vm12 ~]#chkconfig kadmin on
[root@vm12 ~]#service kadmin start
[root@vm12 ~]#chkconfig krb5kdc on
[root@vm12 ~]#service krb5kdc start
-----

To test if everything is working, execute "kadmin" or "kadmin.local". By default, the current user appended with ‘/admin’ is used as the principle.

[root@vm12 ~]# kadmin
Authenticating as principal root/admin@GSSLAB.PNQ.EXAMPLE.COM with password.
Password for root/admin@GSSLAB.PNQ.EXAMPLE.COM: //Please enter admin password
kadmin: listprincs
K/M@GSSLAB.PNQ.EXAMPLE.COM
host/vm12.gsslab.pnq.example.com@GSSLAB.PNQ.EXAMPLE.COM
host/vm217.gsslab.pnq.example.com@GSSLAB.PNQ.EXAMPLE.COM
kadmin/admin@GSSLAB.PNQ.EXAMPLE.COM
kadmin/changepw@GSSLAB.PNQ.EXAMPLE.COM
kadmin/history@GSSLAB.PNQ.EXAMPLE.COM
kadmin/vm12.gsslab.pnq.example.com@GSSLAB.PNQ.EXAMPLE.COM
kmaiti@GSSLAB.PNQ.EXAMPLE.COM
krbtgt/GSSLAB.PNQ.EXAMPLE.COM@GSSLAB.PNQ.EXAMPLE.COM
nc@GSSLAB.PNQ.EXAMPLE.COM
nfs/vm12.gsslab.pnq.example.com@GSSLAB.PNQ.EXAMPLE.COM
nfs/vm217.gsslab.pnq.example.com@GSSLAB.PNQ.EXAMPLE.COM
root/admin@GSSLAB.PNQ.EXAMPLE.COM
kadmin:

The additional principles have been created by the tool. They are required so leave them be.

H) Create a Host Principal for the KDC :

Now you will want to create a host principal for the KDC. This is required for replication (see below). You also need to add this principal to the local key table.

-----
[root@vm12 ~]# kadmin
Authenticating as principal root/admin@GSSLAB.PNQ.EXAMPLE.COM with password.
Password for root/admin@GSSLAB.PNQ.EXAMPLE.COM:
kadmin: addprinc -randkey host/vm12.gsslab.pnq.example.com //Execute this command. Don't forget to replace the hostname.
kadmin: ktadd host/vm12.gsslab.pnq.example.com // Adding key for the host.
-----

I) Setup the default Policy :

-----
[root@vm12 ~]# kadmin
Authenticating as principal root/admin@GSSLAB.PNQ.EXAMPLE.COM with password.
Password for root/admin@GSSLAB.PNQ.EXAMPLE.COM:
kadmin: add_policy -maxlife 180days -minlife 2days -minlength 8 -minclasses 3 -history 10 default //Execute this line.
-----

You can also add other policies and apply different policies to different principals. You can also change the Maximum Renewal Time.

K) Creating Kerberos Principals for client :

Run kadmin on the server and create the following principals. Replace vm217.gsslab.pnq.example.com with the fully qualified name of the client machine.

-----
[root@vm12 ~]# kadmin
Authenticating as principal root/admin@GSSLAB.PNQ.EXAMPLE.COM with password.
Password for root/admin@GSSLAB.PNQ.EXAMPLE.COM:
kadmin: addprinc -randkey nfs/vm217.gsslab.pnq.example.com
-----


L) Generate key in the keytab file for the admin and this will be saved in /var/kerberos/krb5kdc/kadm5.keytab since this has been mentioned in /var/kerberos/krb5kdc/kdc.conf. Use following commands :

-----
[root@vm217 ~]# mv /etc/krb5.keytab /etc/krb5.keytab.bk
[root@vm217 ~]# kadmin
Authenticating as principal root/admin@GSSLAB.PNQ.EXAMPLE.COM with password.
Password for root/admin@GSSLAB.PNQ.EXAMPLE.COM:
kadmin: ktadd -k /var/kerberos/krb5kdc/kadm5.keytab kadmin/kadmin //Execute these commands
kadmin: ktadd -k /var/kerberos/krb5kdc/kadm5.keytab kadmin/changepw
-----

M) Make it sure that ports 88 and 749 has opened at the firewall. Restart the firewall, kadmin and krb5kdc services.


Client Setup :--

a). Copy Files :

Copy the file /etc/krb5.conf from server(Kerberos server) to client machine.

b). Make it sure that ports 88 and 749 has opened at the firewall. Restart the firewall.
c). Create Kerberos Principals :

Execute "kadmin" command on the client konsole. Add the principal to the keytab file as follows for NFS.

-----
[root@vm217 ~]# kadmin
Authenticating as principal root/admin@GSSLAB.PNQ.EXAMPLE.COM with password.
Password for root/admin@GSSLAB.PNQ.EXAMPLE.COM:
kadmin: ktadd -e des-cbc-crc:normal nfs/vm217.gsslab.pnq.example.com
-----

6) Configuring kerberos for NFSv4(Assuming that NFSv4 has been installed on the server ) :-

On the kerberos(i.e NFSv4) server :-

A). Create the necessary entries in /etc/exports. First, create an NFSv4 mount point. I would suggest /export. Next bind the real path to the NFSv4 mount point. In this example, we want to export the /data directory. We create /export/data for NFSv4 and mount /data there.

-----
[root@vm12 /]#mkdir -m 1777 /export
[root@vm12 /]#mkdir /export/data
[root@vm12 /]#mount -n --bind /data /export/data
-----

B) Add the following lines in the /etc/exports file :

-----
/export gss/krb5(sync,rw,fsid=0,insecure,no_subtree_check,anonuid=65534,anongid=65534)
/export/data gss/krb5(sync,rw,nohide,insecure,no_subtree_check,anonuid=65534,anongid=65534)
-----

B) Modify /etc/idmapd.conf and it'll look like :

-----
[root@vm12 /]# cat /etc/idmapd.conf
[General]

Verbosity = 0
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
Domain = gsslab.pnq.example.com

[Mapping]

Nobody-User = nfsnobody
Nobody-Group = nfsnobody

[Translation]
Method = nsswitch
[root@vm12 /]#
-----

C) Make the value of SECURE_NFS to yes in /etc/sysconfig/nfs :

To enable secure NFS, you must add the following line to /etc/sysconfig/nfs

-----
SECURE_NFS=yes
-----

That's it.

Testing :

On client machine issue the following command to mount the exporting directory of the server :

------
#mount -t nfs4 -o sec=krb5 vm12.gsslab.pnq.example.com:/ /mnt
------

Now create the files inside the /data directory on the server and view the files inside /mnt directory on the client machine.

Note : NFS daemons looks like on the server :

------
[root@vm12 /]# service nfs status
rpc.svcgssd (pid 8974) is running...
rpc.mountd (pid 8994) is running...
nfsd (pid 8991 8990 8989 8988 8987 8986 8985 8984) is running...
rpc.rquotad (pid 8979) is running...
[root@vm12 /]
------


Try :)