/
nfs-krb5-setup.txt
395 lines (304 loc) · 14.6 KB
/
nfs-krb5-setup.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
How to setup kerberized NFS on FreeBSD (particularily NFSv4)
Setting up Kerberized NFS (more correctly: NFS using RPCSEC_GSS authentication
via the kerberos5 mechanism) on FreeBSD:
For the example:
nfs-server.my.domain - The FQDN of the NFS server
nfs-client.my.domain - The FQDN of the NFS client
MY.REALM - The default Kerberos realm
ricktst - A user assigned uid 502, with a Kerberos
principal name ricktst@MY.REALM
/export - The single file system being exported to the client
In the GSSAPI there are two kinds of principal names,
User and Host Based.
These names are translated to Kerberos principal names for Kerberos
mechanism, which is the only mechanism used.
A User principal name normally
refers to a user and looks like:
ricktst <-- which becomes ricktst@MY.REALM in KerberosV
It normally has a TGT in a credentials cache file (/tmp/krb5cc_502)
created by kinit or during the user's login process.
A Host Based principal name also has the fully qualified host domain name (FQDN)
in it (so it can only be used on that host) and looks like:
nfs@nfs-server.my.domain <-- which becomes
nfs/nfs-server.my.domain@MY.REALM in KerberosV
It normally has credentials in a keytab file created on the KDC
and copied to the machine's /etc/krb5.keytab via some secure mechanism.
(These are sometimes referred to as a machine credentials, or in
Kerberos, a service principal. Note that the NFS client can
(mis)use a service principal as a client machine credential,
referred to as a host based initiator credential, below.)
(Since this principal will only work on the system called
nfs-server.my.domain, the damage done if the keytab file
is compromised on nfs-server.my.domain it is limited to that system.)
These principals have the advantage that their credentials
do not expire, because they can be refreshed indefinitely.
For the NFS server, the following principal will need to be added to
the KDC and a keytab entry for it will need to be generated and copied
to /etc/krb5.keytab on the NFS server (the first component must be "nfs"):
nfs/nfs-server.my.domain@MY.REALM
Section 14.5 of the FreeBSD handbook does a nice job of describing
how to set up the Heimdal kdc that is distributed with FreeBSD and
the use of kadmin to acquire the above keytab entry.
(However, note that "service kdc start" is used to start the kdc
and not what the handbook currently says.)
The following should work for Heimdal and MIT KDCs, but does not
work for Windows AD KDCs.
Once this is done, the credential may be tested by doing:
# kinit -k nfs/nfs-server.my.domain
on the NFS server as su/root.
For the NFS client, the same is usually done to create a host based
initiator credential, although the first component does not need to
be "nfs" and is usually "host".
For example, create the principal and a keytab entry for:
host/nfs-client.my.domain@MY.REALM
on the KDC and then copy the keytab to /etc/krb5.keytab on nfs-client and
test it via:
# kinit -k host/nfs-client.my.domain
on the client, running as su/root.
(There is a way to do mounts without this host based initiator credential.
See below.)
For RPCSEC_GSS, there are 3 types of service which, for KerberosV are:
krb5 - Use KerberosV for user authentication, but only protect the
RPC header from compromise.
krb5i - Use KerberosV for user authentication, but also use excrypted
checksums on the RPC data to protect against "man in the middle"
attacks involving replacement of the RPC data.
krb5p - Use KerberosV for user authentication, but also encrypt
the RPC data, so that it isn't on the wire in clear text.
Since Kerberized NFS does not use uid/gids, these do not need to
be uniform across the client(s) and server(s). However, uniform
user/group name databases across all client(s) and server(s)
is strongly recommended.
Example server setup (after confirming the above Kerberos config):
Create a /etc/exports file with the following lines in it:
/export -sec=krb5:krb5i:krb5p -network 192.168.1.0 -mask 255.255.255.0
V4: /export -sec=krb5:krb5i:krb5p -network 192.168.1.0 -mask 255.255.255.0
This allows all the client(s) to use any of the three
RPCSEC_GSS types of service. The client(s) are limited
to the local 192.168.1.0 subnet.
Add the following lines to your /etc/rc.conf:
nfsuserd_enable="YES"
gssd_enable="YES"
nfs_server_enable="YES"
nfsv4_server_enable="YES"
mountd_enable="YES"
If you are running FreeBSD 12.3 or 13.0 (and maybe others?), you should
apply this simple patch to your /etc/rc.d/gssd file.
(All client(s) and server(s).)
--- etc/rc.d/gssd.old 2022-02-13 07:53:46.456616000 -0800
+++ etc/rc.d/gssd 2022-02-13 13:34:01.662048000 -0800
@@ -4,7 +4,8 @@
#
# PROVIDE: gssd
-# REQUIRE: root mountcritlocal
+# REQUIRE: root mountcritlocal NETWORKING kdc
+# BEFORE: mountcritremote
# KEYWORD: nojail shutdown
. /etc/rc.subr
Then reboot the server and it should be ready to handle Kerberized NFS mounts.
(If you are restarting the daemons without rebooting, you must stop both
the gssd and nfsd and then start the gssd before nfsd.)
Example client setup (after confirming the above Kerberos config):
Add the following lines to your /etc/rc.conf:
nfsuserd_enable="YES"
gssd_enable="YES"
# -h is only needed for the host based initiator credential mounts
gssd_flags="-h"
nfs_client_enable="YES"
nfscbd_enable="YES"
Apply the above patch to /etc/rc.d/gssd, as required.
If your DNS is misconfigured, there may be a long delay
of over 25 seconds during the mount attempt. Unless you
have a kernel with the following patch, the 25 second
timeout on the kernel upcall to the gssd(8) will happen
and the gssd(8) daemon will terminate. A similar delay will
occur when a user's TGT has expired. (The GSSAPI library
seems to attempt to contact a DNS service even if /etc/nsswitch.conf
specifies "files" for the hosts line, if there is a /etc/resolv.conf
file on the system.)
If you run into this problem, you can:
- Delete the /etc/resolv.conf file.
or
- Fix DNS so that it works.
or
- Apply this patch to your kernel and rebuild it from sources.
(After applying the patch, the mount can take 30 seconds, but
it will succeed and the gssd will not get terminated.)
https://people.freebsd.org/~rmacklem/gssdtimeout.patch
Reboot and the client should be ready to do a Kerberized NFS mount.
There are two ways the mount can be done.
To do a mount using a host based initiator credential, as su/root:
(This is the one where the client has a host/nfs-client.my.domain@MY.REALM
principal entry in its /etc/krb5.keytab file.)
# mount -t nfs -o nfsv4,sec=krb5,gssname=host nfs-server.my.domain:/ /export
Since an entry for the principal host/nfs-client.my.domain@MY.REALM is
being used for the client's machine credential, this mount should continue
to work until unmounted. It does require that the client machine have
a fixed, well known DNS host name (nfs-client.my.domain for this example).
To do a mount without a host based initiator credential:
Then, after rebooting, log in as "root" and:
# kinit ricktst
- Input ricktst's Kerberos password at the prompt.
# klist
- Should show a valid TGT for "ricktst".
# mount -t nfs -o nfsv4,sec=krb5 nfs-server.my.domain:/ /export
So long as there is a valid TGT for any principal in the KDC in root's
credential cache, this mount should work until that TGT expires.
It is possible to do this kind of mount as non-root.
First, as root, do:
# sysctl vfs.usermount=1
Then, login as "ricktst" and:
$ kinit
- Input ricktst's Kerberos password at the prompt.
$ klist
- Should show a valid TGT for "ricktst".
$ mount -t nfs -o nfsv4,sec=krb5 nfs-server.my.domain:/ /export
- Note that for this to work, /export must be owned by "ricktst".
Also, "umount /export" might take 1 minute, since it must time out
the Mount protocol Unmount RPC, if you have a NFSv4 only server.
These mounts will stop working when the TGT expires. However, it does not
require the client to have a fixed, well known DNS host name (a laptop
using WIFI, for example) nor an entry in /etc/krb5.keytab.
If you, for some reason, choose to switch between these two types of
Kerberized mounts, you will need to do:
# umount /export
# kdestroy
before doing the other style of mount, so that there is no "cruft" in
the credentials cache.
Once the mount is done, users will need to have valid TGTs to access files
on the /export mount.
Here are some gotchas to be aware of:
- The time of day clocks for all systems must be synchronized to within the
clock skew specified in /etc/krb5.conf.
- KerberosV principal names are case sensitive, although DNS names are not.
The simplest way to avoid grief is to use all lower case characters
in your DNS host names and all upper case characters for your
KerberosV REALM name.
- The host name resolver functions must return the fully
qualified host name i.e. nfs-server.my.domain and not nfs-server.
If you are using /etc/hosts, put the fully qualified name first, like:
192.168.1.12 nfs-server.my.domain nfs-server
If the mount is not working:
Add the "-v" command line option to gssd by adding this line to /etc/rc.conf
and rebooting (both server and client):
gssd_flags="-v"
The gssd daemon should now log information wherever your /etc/syslog.conf
directs daemon.info. If there are large negative numbers in the
logged messages, they are Kerberos errors that can be found in
/usr/include/krb5_err.h.
If you can access it, the KDC should log information that may be useful.
Thats about all I can think of w.r.t. diagnostics.
Good luck with it, rick
--- This is an email that Peter Eriksson posted with good information.
Copied here with his permission.
Weve been using NFSv4 with Kerberos from Linux clients here for many years
(with Solaris-based NFS servers and MIT Kerberos) and lately using FreeBSD
as the NFS server OS (in an Microsoft AD Kerberos environment).
There are a few differences in behaviour (from a Solaris perspective),
mainly regarding the pseudo NFSv4 filesystem but not something that
can't be handled. In the process of moving to FreeBSD based servers
I've also been testing lots of different clients in order to make sure
everything works as expected.
Here are some notes:
General stuff:
1. Have a _kerberos.my.zone.com DNS TXT record containing the Kerberos realm
(nice to have)
2. Have a _nfsv4domain.my.zone.com DNS TXT record containing the NFSv4 domain
(not all clients use it, but its nice to have)
* FreeBSD server (with ZFS filesystems), things below /export is NFS-shared
as (for example) server:/staff/user1
1. /etc/exports (we _only_ allow sec=krb<various flavours> for home directories):
V4: /export -sec=krb5:krb5i:krb5p
Or (on a server also serving some (read-only) sec=sys filesystems below /exports)
V4: /export -sec=krb5:krb5i:krb5p:sys
2. /etc/zfs/exports (generated from set sharenfs=xxx on the ZFS filesystems)
Home-server:
/export/staff -sec=krb5:krb5i:krb5p
/export/staff/user1 -sec=krb5:krb5i:krb5p
/export/staff/user2 -sec=krb5:krb5i:krb5p
/export/students -sec=krb5:krb5i:krb5p
/export/students/user3 -sec=krb5:krb5i:krb5p
Software-server:
/export/db/icons -sec=sys -ro
/export/old/ifm/solaris-x86 -sec=krb5:krb5i:krb5p:sys -ro
/export/old/ifm/windows-86 -sec=krb5:krb5i:krb5p:sys -ro
/export/software -sec=krb5:krb5i:krb5p -ro
3. Make sure you have host/FQDN@REALM and nfs/FQDN@REALM Kerberos principals
in /etc/krb5.keytab and that FQDN is listed in /etc/hosts like:
1.2.3.4 foo.bar.com foo
4. rc.conf (we have a lot of users in our AD so we have to use a large number
for usermax, replace liu.se with your NFSv4 domain for nfsuserd_flags)
gssd_enable="YES"
nfs_server_enable="YES"
nfsv4_server_enable="YES"
nfscbd_enable="YES"
mountd_enable="YES"
nfsuserd_enable="YES"
nfsuserd_flags="-manage-gids -domain liu.se -usertimeout 10 -usermax 100000 16"
5. Make sure you use NTPD so the clock is correct.
* All clients (Solaris 10, OmniOS, MacOS 10.12-10.14, FreeBSD 11.0-11.2,
CentOS 7, Debian 9, Ubuntu 17-18 tested):
1. Make sure FQDN is in /etc/hosts
2. Make sure you use NTPD so the clock is correct.
3. Have a host/FQDN@REALM Kerberos host principal in /etc/krb5.keytab
(nfs or root is not needed for NFS-mounting to work)
4. We use a fairly default /etc/krb5.conf, sort of like:
[libdefaults]
default_realm = REALM
dns_lookup_realm = true
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
default_ccache_name = KEYRING:persistent:%{uid}
KEYRING probably only works on Linux and there are some problems with
KEYRING in Debian & Ubuntu since not everything in it supports it due to
them using Heimdal instead of MIT (like for smbclient) but it mostly works.
Works fine in CentOS 7 though - in general CentOS 7 feels more
enterprise-ready than Debian & Ubuntu. The old classic FILE-ccaches
should work fine though.
For mounting we use the automounter and a "executable map
(perl script) that looks up records in DNS (Hesiod-style) since the built-in
Hesiod support in most automounters is a bit.. lacking. Works quite well.
You can find the scripts we use here:
http://www.grebo.net/~peter/nfs
(The dns-update scripts use data from an SQL database so probably isn't
directly usable to anybody else. We use the same SQL database to populate
a locally developed BerkeleyDB-based NSS-database on each FreeBSD server
in order to speed things up since AD/LDAP-looks with ~90k users and silly
amounts of AD groups takes forever, even with cacheing).
Some Linux-specific stuff:
Packages needed:
CentOS:
- nfs-utils
- libnfsidmap
- nfs4-acl-tools
- autofs
Debian:
- keyutils
- nfs-kernel-server # rpc.idmapd needs this due to a bug in Debian
Ubuntu:
- keyutils
Other nice-to have packages:
- hesiod
- autofs-hesiod
Some settings to check for:
/etc/default/nfs-common:
NEED_IDMAPD=yes
NEED_GSSD=yes
/etc/idmapd.conf (replace liu.se with your NFSv4 domain):
Domain=liu.se
/etc/request-key.d/id_resolver.conf (should be there already if using a
modern Linux and you've added the packages above):
create id_resolver * * /usr/sbin/nfsidmap %k %d
MacOS:
Basically require the latest - 10.14 (Mojave) - for things to work smoothly.
In theory 10.12 & 10.13 should work but there is some bug in them that causes
the OS to panic when you try to use NFS & Kerberos. 10.11 and earlier doesn't
support good enough encryption for us. But with 10.14 you just need
to get a Kerberos ticket and then you can mount things just fine.
/etc/nfs.conf should contain (replace liu.se with your NFSv4 domain):
nfs.client.default_nfs4domain=liu.se
(There are a lot of problems you can run into with Microsofts AD
implementation of Kerberos too that we've had to be fighting with,
but that's a whole other topic)
- Peter