Data Domain CIFS and NFS Troubleshooting SG PDF

Welcome to the Data Domain CIFS and NFS Troubleshooting Course.
Copyright 2015 EMC Corporation. All Rights Reserved. Published in the USA. EMC believes the information in this
publication is accurate as of its publication date. The information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS
OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY
DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software
license. The trademarks, logos, and service marks (collectively "Trademarks") appearing in this publication are the
property of EMC Corporation and other parties. Nothing contained in this publication should be construed as granting
any license or right to use any Trademark without the prior written permission of the party that owns the Trademark.
EMC, EMC AccessAnywhere Access Logix, AdvantEdge, AlphaStor, AppSync ApplicationXtender, ArchiveXtender,
Atmos, Authentica, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, Avamar,
Bus-Tech, Captiva, Catalog Solution, C-Clip, Celerra, Celerra Replicator, Centera, CenterStage, CentraStar, EMC
CertTracker. CIO Connect, ClaimPack, ClaimsEditor, Claralert ,cLARiiON, ClientPak, CloudArray, Codebook
Correlation Technology, Common Information Model, Compuset, Compute Anywhere, Configuration Intelligence,
Configuresoft, Connectrix, Constellation Computing, EMC ControlCenter, CopyCross, CopyPoint, CX, DataBridge , Data
Protection Suite. Data Protection Advisor, DBClassify, DD Boost, Dantz, DatabaseXtender, Data Domain, Direct Matrix
Architecture, DiskXtender, DiskXtender 2000, DLS ECO, Document Sciences, Documentum, DR Anywhere, ECS,
elnput, E-Lab, Elastic Cloud Storage, EmailXaminer, EmailXtender , EMC Centera, EMC ControlCenter, EMC LifeLine,
EMCTV, Enginuity, EPFM. eRoom, Event Explorer, FAST, FarPoint, FirstPass, FLARE, FormWare, Geosynchrony, Global
File Virtualization, Graphic Visualization, Greenplum, HighRoad, HomeBase, Illuminator , InfoArchive, InfoMover,
Infoscape, Infra, InputAccel, InputAccel Express, Invista, Ionix, ISIS,Kazeon, EMC LifeLine, Mainframe Appliance for
Storage, Mainframe Data Library, Max Retriever, MCx, MediaStor , Metro, MetroPoint, MirrorView, Multi-Band
Deduplication,Navisphere, Netstorage, NetWorker, nLayers, EMC OnCourse, OnAlert, OpenScale, Petrocloud,
PixTools, Powerlink, PowerPath, PowerSnap, ProSphere, ProtectEverywhere, ProtectPoint, EMC Proven, EMC Proven
Professional, QuickScan, RAPIDPath, EMC RecoverPoint, Rainfinity, RepliCare, RepliStor, ResourcePak, Retrospect,
RSA, the RSA logo, SafeLine, SAN Advisor, SAN Copy, SAN Manager, ScaleIO Smarts, EMC Snap, SnapImage,
SnapSure, SnapView, SourceOne, SRDF, EMC Storage Administrator, StorageScope, SupportMate, SymmAPI,
SymmEnabler, Symmetrix, Symmetrix DMX, Symmetrix VMAX, TimeFinder, TwinStrata, UltraFlex, UltraPoint,
UltraScale, Unisphere, Universal Data Consistency, Vblock, Velocity, Viewlets, ViPR, Virtual Matrix, Virtual Matrix
Architecture, Virtual Provisioning, Virtualize Everything, Compromise Nothing, Virtuent, VMAX, VMAXe, VNX, VNXe,
Voyence, VPLEX, VSAM-Assist, VSAM I/O PLUS, VSET, VSPEX, Watch4net, WebXtender, xPression, xPresso, Xtrem,
XtremCache, XtremSF, XtremSW, XtremIO, YottaYotta, Zero-Friction Enterprise Storage.
Revision Date: 10/2015
Revision Number: MR-7WN-DDCNT.5520.4.0
Copyright 2015 EMC Corporation. All rights reserved. Data Domain CIFS and NFS Troubleshooting 1
Course Overview
Description
This EMC Education Services course provides the student with a detailed look at the concepts,
requirements, and procedures for troubleshooting CIFS and NFS protocol-related issues on an EMC Data
Domain system.
Audience
This course provides valuable knowledge and skill for those whose responsibilities include configuring or
troubleshooting CIFS or NFS access to Data Domain systems.
Objectives
Upon completion of this course, you will be able to:
Perform CIFS Troubleshooting
Perform NFS Troubleshooting
Course Content
This training contains content restricted to EMC Corporation employees and partners.
Restricted content is marked with the DeepDive icon on the slide.
Copyright 2015 EMC Corporation. All rights reserved. Data Domain System Monitoring and Logging 3
MODULE - Troubleshooting CIFS
Upon completion of this module, you will be able to:
Describe CIFS
List possible CIFS problem areas
Troubleshoot CIFS authentication
Troubleshoot CIFS shares
Perform file access troubleshooting
LESSON - Describing CIFS
This lesson covers the following topics:
CIFS Overview
Major CIFS Components
Workgroup Authentication Overview
Active Directory Authentication Overview
CIFS UDP and TCP communication ports
DD OS CIFS Implementation
CIFS Overview
The Common Internet File System (CIFS) allows computers to use remote disks and files as if they were
directly attached to the local machine. CIFS was formerly known as Server Message Block (SMB), but the
name was changed to CIFS. Even so, the underlying protocol is still called SMB.
SMB was initially developed by IBM and later expanded by Microsoft. It is continuously being improved
and new features and functionality are added to support new operating systems and applications.
CIFS is the native file sharing protocol used by all versions of Windows. Early versions of CIFS used
NetBIOS as the transport. Eventually, NetBIOS over TCP (NBT) was supported. The most recent versions
of CIFS support SMB directly over TCP without the use of NetBIOS.
CIFS uses other protocols to perform supporting functions. For example, CIFS uses the Domain Name
System (DNS) to locate network devices. It uses Kerberos to perform authentication. It uses an
implementation of the Lightweight Directory Access Protocol (LDAP) to manage resources. CIFS requires
the CIFS servers and Domain Controllers to be in sync. Devices can be manually synchronized or be
synchronized using a Network Time Protocol (NTP) server.
As you can see on the screen, SMB version 1 was released with windows 9X, 2000, XP, and 2003. SMB
version 2.X was released with windows Vista, 7, and 2008. And SMB version 3.0 was released with
Windows 8 and 2012.
References
Server Message Block (SMB)
NetBIOS
Domain Name System (DNS)
Network Time Protocol (NTP)
Kerberos
Lightweight Directory Access Protocol (LDAP)
The major components of CIFS are the client, the server, and the domain controller.
Clients
CIFS clients are devices that access services provided by other device.
Servers
CIFS servers are devices that provide services to the client devices. These services can include file
services, print services, fax services or some other type of service.
Domain Controllers
Domain controllers are devices that centralize network management. They typically provide directory and
authentication services, but can also be used to provide other services such as domain name services.
References
Client Server Model
Domain Controller
Kerberos
Windows Domain
Organizational Unit
Workgroup
There are two types of CIFS authentication schemes supported on Data Domain systems, Workgroup and Active
Directory.
Peer-to-peer
Workgroup authentication uses a peer-to-peer model. This means that any network device can act as a server and
share resources with other devices on the network. Devices can also act as clients by using the shared resources of
servers. This means that a single computer can act as a client or a server, depending on whether it uses or provides
services.
Workgroups provide logical grouping
A CIFS workgroup is a logical grouping of devices to make them easier to find on the network. Devices can be
grouped based upon function, organization, or some other scheme. For example, all the print servers may be in the
workgroup called PRINTERS, and all the file servers may be in a workgroup called FILES. Or all devices associated
with the engineering division can be in the ENGINEERING work group.
Access and Authentication
Each server is required to manage access and authentication for every user. When a server shares a resource, the
server must identify which users have access to the resource and the system administrator must configure the
username and password for each user on the server. This can be very difficult to manage in a large organization.
The account used by the computer operator to access resources on the server does not have to be the same as the
account used to log into their local client computer.
This means that the operator on CLIENT can log into SERVER as USER-01 or USER-02 provided SERVER has an
account for both users. If SERVER only has an account for USER-02, then the operator on CLIENT may log into
CLIENT as USER-01 and into SERVER as USER-02. This distributed access and authentication mechanism is similar to
how we typically log into different websites.
Advantages
An advantage to workgroup authentication is that it is simple to setup. Sharing resources in a small business or in a
home is relatively easy to configure and requires no other resources than the computers already in use.
Workgroup authentication requires very little infrastructure and specialized knowledge to implement.
Disadvantages
Disadvantages to workgroup authentication include the lack of central management. As the network grows, each
device has to be configured separately. This makes it difficult, if not impossible, to keep devices in sync.
References
Client Server Model
Workgroup
Active Directory is a database application that runs on a domain controller. The Active Directory is
responsible for securing the network by granting or denying permissions to resources based upon the
location, user, or group. When a request is made to access a resource, the server queries the Active
Directory using the LDAP protocol to learn the level of access that can be granted. This means that each
server no longer has to maintain its own user database, but now leverages the user database on the
active directory.
Authentication to all resources (including the network, printers, and file shares) depends upon the user's
rights.
The Active Directory responds to requests that are transmitted from other devices to the domain
controller. These requests are sent to the Active Directory using the Lightweight Directory Access
Protocol (LDAP).
An advantage to Active Directory authentication is that it provides a central site to manage users
permissions. User accounts are not needed on each of the servers.
Another advantage to Active Directory authentication is that users can be logically separated by domains
(realms) and groups. This means an enterprise can designate some resources as only being available to
the "ENGINEERING" domain and others only available to the "CORP" domain. By the same token, the
group "HR_EMEA" can have permission to resources that are restricted to "HR_US."
Active Directory provides more security across a corporation because its underlying authentication
technologies (such as Kerberos) are more robust than those used by Workgroup Authentication.
A disadvantage to Active Directory authentication is that the Domain Controller and the Active Directory
application must be available in order for permission to be granted. If these components fail, or if they
become inaccessible, authentication cannot be performed.
Another disadvantage is that Active Directory Authentication is complex and requires many components
be in sync in order for it to operate. This complexity requires specialized knowledge to configure and
maintain. An organization needs to grow to a sufficient size before the implementation of Active
Directory authentication becomes a requirement.
References
Active Directory
Lightweight Directory Access Protocol (LDAP)
Naming conventions in Active Directory for computers, domains, sites, and OUs
CIFS UDP and TCP Communications Ports
The TCP and UDP ports listed on the screen must be accessible and unfiltered by firewalls in order for
CIFS to work properly.
On the screen, you see that:
Port 53 services the Domain Name System (DNS).
Ports 88, 464, and 543 support Kerberos. Data Domain systems do not implement the Kerberos klogin
service at port 543, however, it may be implemented by other servers.
Port 123 services the Network Time Protocol.
Port 135 supports Client Server Communications. Data Domain systems do not implement this
service.
Ports 137, 138, and 139 support NetBIOS communications.
Port 389 services the Lightweight Directory Access Protocol (LDAP).
Port 445 supports the Active Directory Service as well as SMB file sharing.
References
http://msdn.microsoft.com/en-us/library/cc959833.aspx
Up to version 4.7, the Data Domain operating system (DD OS) used a native Samba implementation to
provide CIFS services. Wikipedia states that Samba is "a free software re-implementation of the
SMB/CIFS networking protocol."
Starting with 4.8, DD OS used a Hybrid Solution with Likewise providing authentication services and
Samba providing data services.
SMB versions 2.x and 3.x are not supported by the DD OS implementation. Only SMB version 1 is
currently supported.
DD OS supports windows 2000 and newer clients however, non-Windows clients are not officially
supported - though they may work.
DD OS allows a maximum of 600 CIFS connections. The number of connections allowed by DD OS is
based the amount of memory installed on the Data Domain system.
References
Samba
LESSON SUMMARY - Describing CIFS
This lesson covered the following topics:
CIFS Overview
CIFS UDP and TCP communication ports
Knowledge Check
LESSON - CIFS Troubleshooting Overview
Potential Problem Areas
High-level CIFS Troubleshooting Steps
Troubleshooting Network Connectivity
Troubleshooting CIFS Client
CIFS Capture and Display Filters
There are many areas that must be explored when diagnosing CIFS issues. These include:
Network connectivity
CIFS Client Configuration.
CIFS Server (Data Domain system) Configuration
Authentication Configuration
Files System Configuration
Share settings and
File access permissions
We'll be exploring each of these areas for the remainder of this module.
High-level CIFS Troubleshooting Steps - 01
When troubleshooting CIFS issues, it is important to gather potentially relevant information as quickly as
possible. The first request should be for the date and time of the CIFS event that initiated the call. Try to
get as close to the minute as possible.
Next, retrieve the auto support file if it is available or request a copy from the caller.
Also, request the support bundle so that its contents are available as soon as needed.
When requesting the support bundle, check to see if there are recent core dumps in the /ddvar/core
directory. Remember, core dump files are not included in the support bundle.
Request any error messages or log files generated by the application, client, or other involved device.
Check the Data Domain system for active or recent alerts using the alerts show CLI command.
As part of the initial troubleshooting process, review the log files to locate clues about the nature of the
issue. The screen provides a list of pertinent log files.
Logs /ddvar/log/debug/cifs
# log view debug/cifs/cifs.log
# log view debug/cifs/clients.log
# log view debug/cifs/join_domain.log
# log view debug/cifs/kinit.log
# log view debug/cifs/log.smbd
# log view debug/cifs/smbd.log
Logs /ddvar/log/debug
# log view debug/messages.engineering
# log view debug/ddfs.info
DNS
Check the DNS configuration. Use the net show domainname and net show dns CLI commands.
Time
Request the date, time, and timezone for the DC, Data Domain system, and client workstation. This
should be done simultaneously on all systems if at all possible so you can compare the settings on the
various systems. The commands for different systems are shown on the screen.
Finally, request any other information that you think may be relevant to the case.
If these steps fail to provide a starting point for troubleshooting, the systematic troubleshooting steps
outlined in this training should be used to help locate the source of the problem.
OS Date / Time Timezone

Windows C:>date/t C:>timedate.cpl
C:>time/t
Linux $date $date
DD OS #systemshowdate #configshowtimezone
DNS
You can go into BASH mode to check the Data Domain system's resolv.conf file. The more command
provides screen-by-screen output.
Troubleshooting network connectivity includes the ability for the CIFS client, Data Domain system,
domain controller, domain name service, and network time protocol servers to communicate with one
another to support the CIFS architecture.
When troubleshooting network connectivity, you'll need to use the standard processes outlined in the EMC
Data Domain Network Troubleshooting course. These include troubleshooting the:
Networking hardware
IP address configuration
Routing configuration
DNS configuration
Firewall configuration
It is important that you eliminate network connectivity as a problem as quickly as possible.
If using Active Directory authentication, the Data Domain system must be able to communicate with the
Domain Controller and the Network Time Protocol server.
Remember, ensure that CIFS related packets are not blocked by any firewall.
Troubleshooting the CIFS Client
If you have determined that the network is not the problem, troubleshoot the CIFS client
configuration by reviewing any CIFS-related configuration parameters. You may compare the client
configuration to a system that is currently working.
If applicable, review the integration guide for the application. Many integration guides are available,
including those listed in the student guide. You can use the link in the notes section of the student
guide to help you locate these documents.
https://support.emc.com/search/?product_id=9012&resource=DOC_LIB&AlloftheseWrds=cifs%20inte
gration&SearchWithin=true&adv=y
Review that the client is targeting the correct server name or IP address. Review the correct share
name is being used.
It is important that you use the full network path with the server name and share name.
If the share is configured to be hidden, it will not be visible if you attempt to browse the list of share
by only using the server name in the path.
Use a fully qualified domain name just in case the server's domain is not included in the DNS suffix
search list.
Integration Guides
Data Domain and IBM DB2 v9.7 and Later with CIFS and NFS Integration Guide
Symantec Backup Exec 2010 CIFS and VTL Integration Guide
Symantec Backup Exec 2010, 2010 R2, and 2010 R3.
Symantec Backup Exec 2012 CIFS and VTL Integration Guide
Innovation Upstream Reservoir 3.7.2 and Later CIFS, NFS, and VTL Integration Guide
NetWorker 8.0 with Data Domain CIFS, NFS, and VTL Integration Guide
vRanger Integration Guide 5.0.0.19238 and 5.2.0.22058 (CIFS and NFS)
vRanger 5.0 to 6.0.1 for CIFS and NFS Integration Guide
LaserVault Backup 2.10.70 CIFS Integration Guide
Atempo Time Navigator 4.2 Integration Guide CIFS, NFS, VTL
BridgeHead HT Backup 4.x Integration Guide for CIFS, NFS, and VTL
Veeam 4.0 Backup and Replication for VMware ESX Server with CIFS Integration Guide
HP Data Protector 6.2 Integration Guide
Data Domain and VMware Data Recovery Integration Guide
Data Domain with Microsoft SharePoint 2013 Integration Guide
Data Domain and IBM InfoSphere Optim Data Growth Solution 9.x Integration Guide
DataTrust Solutions vBRM Integration Guide
Data Domain and DataGlobal dg hyparchive Integration Guide
Integrating CIFS and NFS Backup with EMC Data Domain Archiver
Troubleshooting CIFS Client (Continued)
When troubleshooting workgroup authentication, make sure to verify that the username and password
are correct.
IPv6 CIFS Guidelines - UNC Address Format
Addresses formatted with Windows Universal Naming Convention format (\\serverName\shareName)
cannot use standard IPv6 addresses because the colon and percent sign are considered to be invalid
characters. This requires that an alternate form of the IPv6 address be used.
For UNC addresses, the IPv6 address must be mapped as follows:
First, the colons are changed to dashes.
Next, the percent sign is changed to a lowercase letter s.
Finally, the .ipv6-literal.net domain name is appended to the address. This effectively turns the IPv6
address into a domain name.
Microsoft has acquired the .ipv6-literal.net second-level domain name to support this type of IPv6
mapping.
Microsoft operating systems do not attempt to resolve addresses with this suffix through an external DNS,
but resolve these address internally - and essentially turn the modified address back into its original form.
On the screen is an example of how an address is mapped. The original and modified addresses are
shown.
Also on the screen is an example of the net use command:
c:>netusez:\\2001db81.ipv6literal.net\sharename
The website - http://ipv6-literal.com - may be used to translate IPv6 addresses into the UNC formats as
well as many others.
References
http://ipv6-literal.com
Copyright 2015 EMC Corporation. All rights reserved. EMCNFS

Data Domain CIFS and DD OS 5.5.1 Differences 23
Troubleshooting
Troubleshooting CIFS Server CLI
In a Data Domain CIFS environment, the Data Domain system acts as a CIFS Server. Therefore, the CIFS
service must be enabled. To verify the CIFS service is enabled, use the cifs status CLI command.
You can enable the CIFS service by using the cifs enable CLI command.
Troubleshooting the CIFS Server using the GUI
You can use the web-based GUI to verify the CIFS service is enabled.
First, select "Data Management" from the top menu.
Next, select "CIFS" from the sub menu.
Information about the CIFS service is displayed near the top of the page.
You will see the status of the CIFS server which is shown as either enabled or disabled.
If needed, you can enable the CIFS service by selecting the Enable button.
The status of the CIFS service should change and the button should now allow the servce to be
disabled.
CIFS Capture Filter
When using Wireshark or tcpdump, the capture can be limited to CIFS related traffic by using the filter
shown on the screen. This causes the capture utility to disregard all packets except for those that are
going to or coming from a TCP or UDP port that has been identified as being related to CIFS.
host<ddsystemip>and(tcp port53ortcp port88ortcp port135ortcp port137ortcp
port138ortcp port139ortcp port389ortcp port445ortcp port464ortcp port543or
udp port53orudp port88orudp port123orudp port137orudp port138orudp port139or
udp port389)
To use this filter with Wireshark, copy it from the student guide and paste it into the Wireshark capture
filter field and provide the IP address for the Data Domain system.
CIFS Capture Filters (Continued)
The net tcpdump CLI command does not allow the use of capture filters, but the se tcpdump CLI
command does. On the screen is an example of the filter being used in conjunction with the se tcpdump
CLI command. Notice that the filter is enclosed by single quotes and is at the end of the command string.
setcpdumpi anys0w/ddvar/traces/cifscapture01.cap'host10.and(tcp port53ortcp
port88ortcp port135ortcp port137ortcp port138ortcp port139ortcp port389ortcp
port445ortcp port464ortcp port543orudp port53orudp port88orudp port123orudp
port137orudp port138orudp port139orudp port389)'
CIFS UDP and TCP Display Filters
In Wireshark, the capture filters and display filters are defined using different (but similar) syntaxes. The
display of a network trace may be limited to CIFS related traffic, by using the filter shown on the screen.
Substitute the IP address of the target Data Domain system for the <dd-system-ip> string, and cut and
paste the filter directly into the Wireshark display filter field.
ip.addr ==<ddsystemip>&&(tcp.port ==53||tcp.port ==88||tcp.port ==135||tcp.port
==137||tcp.port ==138||tcp.port ==139||tcp.port ==389||tcp.port ==445||tcp.port
==464||tcp.port ==543||udp.port ==53||udp.port ==88||udp.port ==123||udp.port
==137||udp.port ==138||udp.port ==139||udp.port ==389)
LESSON SUMMARY - CIFS Troubleshooting Overview
This lesson covered the following topics :
High-level CIFS Troubleshooting Steps
Troubleshooting CIFS Client
Troubleshooting the CIFS Server
CIFS Capture and Display Filters
Knowledge Check
LESSON - Troubleshooting Workgroup Authentication
This lesson discusses the following topics:
Reviewing CIFS Workgroup Authentication
Configuring CIFS Workgroup Authentication
CIFS Workgroup Connection Walkthrough
Reviewing CIFS Workgroup Authentication using CLI
To review the CIFS Workgroup Authentication Configuration, use the cifs show config CLI command. This
command displays the authentication mode - either Workgroup or Active Directory - as well as the
parameters associated with the configured authentication method.
The Workgroup name represents a logical grouping of devices that participate in a peer-to-peer Microsoft
network.
The Windows Internet Name Service (WINS) server is similar to the IP Domain Name Service (DNS).
Whereas DNS provides the IP address based upon the IP name, WINS provides the IP address based
upon the NetBIOS name.
The NetBIOS (NB) host name allows the system to be addressed by NetBIOS clients. The IP host name
will be used if an NB host name is not explicitly configured. The "net show hostname" CLI command
provides you with the configured IP hostname.
References
About.com Workgroup
http://compnetworking.about.com/cs/design/g/bldef_workgroup.htm
Workgroup (computer networking)
http://en.wikipedia.org/wiki/Workgroup_(computer_networking)
NetBIOS Name Resolution
http://technet.microsoft.com/en-us/library/cc958811.aspx
Windows Internet Name Service (WINS)
http://en.wikipedia.org/wiki/WINS_Server
Reviewing CIFS Authentication - GUI
Follow this process to review the CIFS Authentication configuration using the GUI:
After selecting the target Data Domain system in the web-based GUI:
1. Select the "Data Management" menu item.
2. Select the "CIFS" sub menu item.
3. Select the "Configuration" tab.
4. View the configuration tab in the bottom portion of the screen.
Configuring CIFS Workgroup Authentication - CLI
Use this process to configure the Data Domain system for CIFS workgroup authentication:
1. First, set the authentication mode to workgroup and provide the workgroup name using the "cifs set
authentication workgroup" CLI command. The workgroup name may be up to 15 characters long.
2. Next, add accounts for the users that will access this device. These accounts can be used by the
computers hosting backup applications. You will be prompted to configure a password for the users.
3. The NB Hostname defaults to the first part of the IP host name ending at the first dot. If necessary,
the NB Hostname may be changed from its default value by using the "cifs set nb-hostname" CLI
command. Do not change the NB hostname from default values unless there is a specific need to do
so. The NB hostname may be up to 15 characters long and can contain only alphanumeric , hyphen (-
), and underscore characters (_). Use the "net show hostname" CLI command to review the
configured IP host name.
4. Finally, if needed, configure the WINS server using the "cifs set wins-server" CLI command. This
command requires you to specify an IP address.
Reference
About.com Workgroup
Configuring CIFS Workgroup Authentication using the web-based GUI
After selecting the target Data Domain system from the left side of the web-based GUI, follow these steps
to configure CIFS Workgroup Authentication:
1. First, select the "Data Management" menu item.
2. Next, select the "CIFS" sub menu item.
3. Now, select the "Configuration" tab.
4. Then, select the "Configure Authentication" button. The "Configure Authentication" panel is
displayed.
5. Select "Workgroup" from the mode pull down box.
6. Next, select the "General tab".
7. Now, select the "Use Default" checkbox if you wish to use the default workgroup name which is
unspecified (blank). Unselect the "Use Default" checkbox if you wish to enter configure a Workgroup
name.
8. Next, select the "Advanced Tab."
9. Now, select the "Use Default" server name if you wish to have the NB host name be derived from the
IP host name. If you wish for the NB host name to be statically configured, you must unselect the
checkbox and enter the name you prefer.
10. Finally, select "Ok" to accept the configuration changes and return to the CIFS configuration screen.
If needed, the WINS server must be configured using the CLI. WINS server configuration is not available
in the GUI.
Configuring CIFS Workgroup Authentication using the web-based GUI (Continued)
After the CIFS workgroup authentication is configured, verify the CIFS user has been added to the
system. If the CIFS user does not exist, add them now. Follow these steps to verify or configure the
CIFS user:
1. After selecting the target Data Domain system from the left side of the web-based GUI, select the
"System Settings" menu item.
2. Select the "Access Management" sub menu item.
3. Select the "Local Users" tab.
4. Examine the list of user names and determine if the CIFS user is already configured.
5. If the CIFS user is not configured, select the create button. The "Create User" configuration panel is
displayed.
6. Select the "General" tab.
7. Enter the user name, password, and then verify the password.
8. Select the role for the user from the "Role" drop-down menu. The "data-access " role should be
sufficient for the CIFS user as they will not be required to administer the Data Domain system, only
access the shared directories.
9. Select the "Advanced Tab."
10. If required, configure the "Password Aging Policy."
11. Next, enter the date for the account to be disabled, if required.
12. Finally, select the "OK" button to accept the configuration changes and return to the User
configuration screen.
Copyright 2015 EMC Corporation. All rights reserved. Data Domain CIFS and NFS Troubleshooting
Workgroup initial authentication is a multi-step process
1. First, the client connects to CIFS server
2. Next the client negotiates the SMB dialect
3. Then, the client negotiates the authentication method
4. Now, the client connects to the named pipe
5. Finally, the client connects to shared resource
The next few slides covers these steps in more detail.
Client Connects to Server
1. To initiate a workgroup authentication connection, the CIFS client verifies the target device is listening
on TCP port 445 by sending a TCP SYN packet. If the target device is not listening on this port, the
connection attempt times out or is refused.
2. If the server is listening on this port, it acknowledges the client by sending a SYN ACK packet from
port 445. This is the standard response to a SYN request.
3. Finally, the CIFS client completes the 3-way TCP handshake by sending a TCP ACK to the target device
on TCP port 445.
Client Negotiates SMB Dialect
1. The next thing the CIFS client must do is to negotiate the SMB dialect by sending an SMB Negotiate
Protocol request to the server targeting TCP port 445. There are several versions of the SMB protocol
and the client and server must agree on the version that will be used. The SMB Negotiate Protocol
Request is used for this purpose. This packet provides the server with a list of the versions of the
SMB protocol supported by the client. On screen is an example of the type of information provided to
the server.
2. Next , the CIFS Server sends a negotiate protocol response. In the response is a version of SMB that
the server selects from the list of SMB versions supported by the client.
Client Negotiates Authentication Method
The next step in the process is to negotiate the authentication method that will be used. Through the
years, the SMB protocol has made continuous improvements to its authentications schemes as market
conditions change and technology improves. Because there are several authentication schemes, clients
and servers need to agree which method to use.
1. First, the client sends a Session Setup Andx Request to tell the server the types of authentication it
supports.
2. The server responds by sending a challenge to the client. The challenge is a random number
generated by the server.
3. Next, the client sends a Session Setup AndX Response to the server. To generate the response, the
client uses the random number provided by the server against the password provided by the operator
and sends the result to the server along with its domain name, user name, host name, and other
information.
4. The server performs the same calculation against the password in its database and then compares its
result with the result provided by the client. The server then sends a Session Setup AndX Response
packet to the client to let it know if the client has successfully authenticated.
The Data Domain system does not distinguish between an incorrect user name or an incorrect password.
A trace of a transaction with a Data Domain system shows that the system responds with the SMB error
STATUS_LOGON_FAILURE. All STATUS_LOGON _FAILURES are interpreted by the client as bad password,
even if the user does not exist on the server.
Client Connects to Named Pipe
The named pipe service (IPC$) enables the client to issue Remote Procedure Calls (RPCs) to the server.
Among other things, the CIFS client uses the named pipe mechanism to:
List all shares
List all users
List files within a share
Stop/Start services
1. First , the client attempts to connect to the named pipe by using the Tree Connect AndX Request
packet along with the path: \\<cifs-server>\IPC$
2. The server send a success or failure message to the client using the Tree Connect Andx Response
packet.
Client Connects to Share
The client has now successfully gained access to the server. The next thing to do is to actually gain
access to the shared resource. Access to resources may be restricted based upon the user or the name
or IP address of the client workstation. The client follows this process to determine if the user may access
the share:
1. The client requests access to resource by using a Tree Connect Andx Request packet along with the
target path (\\<cifs-server>\<share-name>).
2. Server verifies access based upon the rights configured with the sharename. The server sends the
client its response using a Tree Connect Andx Response packet.
LESSON SUMMARY - Troubleshooting Workgroup Authentication
Reviewing CIFS Workgroup Authentication
Configuring CIFS Workgroup Authentication
Knowledge Check
LESSON - Troubleshooting Active Directory Authentication
Preparing for CIFS Active Directory Authentication
Reviewing CIFS Active Directory Authentication Configuration
Configuring Kerberos / AD Authentication
There is some information you need to gather as you prepare to configure Active Directory authentication
on the Data Domain system. This information includes:
The domain name to which the Data Domain system will be a member. The domain name is also
referred to as the realm name.
If you going to use manual entries, you need the DNS name or IP address of the closest domain
controller. You don't need this information if you are going to configure the Data Domain system to
automatically locate a domain controller. When the Data Domain system is configured to automatically
locate the Domain Controller (DC), it uses the first DC to respond to its query.
You'll need the user name for the account that the Data Domain system will use to interact with the
Domain Controller.
This account should be configured to allow the Data Domain system to read and write to the Active
Directory to provide updated information on shared resources or changes in permissions and users on
the Data Domain system.
You will also need the password for the Data Domain system's account.
If the Active Directory and has been configured correctly, this information should enable the Data Domain
system to "Join" the domain.
References
181329 : Using CIFS "Set Authentication Active-Directory" Command
https://support.emc.com/kb/181329
181313 : Joining a Data Domain System to a Windows Domain
180810 : CIFS Content Browsing Page
Reviewing CIFS Active Directory Authentication using CLI
To review the CIFS Authentication Configuration, use the cifs show config CLI command. This command
displays the authentication mode - either Workgroup or Active Directory - as well as the parameters
associated with the configured authentication method.
The Mode field should be set to Active-Directory, not Workgroup.
The Realm field is the same as the Domain Name. Verify that it is configured as expected.
The Domain Controllers field shows the DCs currently being used by the Data Domain system.
Verify the Windows Internet Name Service (WINS) server field. WINS is similar to the IP Domain
Name Service (DNS). DNS provides the IP address based upon the IP name and WINS provides the IP
address based upon the NetBIOS name.
Verify the NetBIOS (NB) Hostname field is configured correctly. The NB hostname allows the system to
be addressed by NetBIOS clients.
The default NB name is the first 15 characters of the IP host name. Use the net show hostname CLI
command to view the configured IP hostname.
References
About.com Workgroup
Reviewing CIFS Authentication - GUI
Follow this process to review the CIFS Authentication configuration using the GUI:
After selecting the target Data Domain system in the web-based GUI:
1. Select the "Data Management" menu item.
2. Select the "CIFS" sub menu item.
3. Select the "Configuration" tab.
4. View the configuration tab in the bottom portion of the screen.
Configuring CIFS Active Directory Authentication - CLI
After gathering the information needed to configure Active Directory authentication, follow these steps:
1. First, set the authentication options.
A. If you are using the Window DC to provide Kerberos authentication for your NFS clients, use the
authentication kerberos CLI command.
B. If NFS Clients do not use the Windows DC for Kerberos Authentication, use the "cifs set
authentication active-directory" CLI command. This command requires the realm (domain name)
and IP addresses of the domain controllers as arguments. If you wish for the Data Domain
system to discover the IP addresses of the domain controllers, substitute an asterisks (*) for the
domain controller's IP address.
2. After invoking the command to set the authentication options, the Data Domain system disables CIFS,
prompts you for the user name and password, and attempts to "Join" the domain by locating and
authenticating with the Active Directory on the Domain Controller. The Data Domain system will let
you know if this step is successful or not.
3. Next, If required, you should set the NetBIOS host name using the "cifs set nb-hostname" CLI
command. Usually this step is not needed.
4. Also, if there are devices in your network that require the services of a WINS server to interact with
the Data Domain system, you should configure the WINS server IP address using the "cifs set wins-
server" CLI command. In most environments, this step is unnecessary.
Configuring CIFS IPv6 AD Authentication - CLI
Starting with DD OS 5.5.1, IPv6 addresses are allowed with the cifs set authentication CLI command.
There is an example on the screen.
Configuring CIFS AD Authentication With NFS Support - GUI
to configure Kerberos Authentication support for CIFS and NFS clients using the web-based GUI:
1. First, select the "System Settings" menu item.
2. Next, select the "Access Management" sub menu item.
3. Now, select the "Authentication" tab.
4. Review the "Active Directory" settings.
5. Select "Configure" to change the configuration. The Active Directory / Kerberos settings panel is
displayed.
6. Next, select the "Windows / Active Directory" option from the screen.
7. Now, select "Next".
8. Enter the Realm or Domain name and the account credentials required for the Data Domain system to
join the domain.
9. Select "Next."
10. Next, configure the "CIFS server name."
11. Now, select to automatically assign or manually configure the IP addresses for the Domain
Controllers. If you choose to manually configure the DC's IP addresses, enter them at this time.
12. Next, choose whether to use the default Organizational Unit "Computers" or to specify another.
13. Select "Next".
14. Review the summary configuration information.
15. Select "Finish" accept the configuration.
References
Wikipedia - Organizational Unit
http://en.wikipedia.org/wiki/Organizational_Unit
Configuring CIFS AD Authentication Without NFS Support - GUI
to configure CIFS Active Directory Authentication using the web-based GUI:
1. First, select the "Data Management" menu item.
2. Next, select the "CIFS" sub menu item.
3. Now, select the "Configuration" tab.
4. Select the "Configure Authentication" button. The "Configure Authentication" panel is displayed.
5. Next, select "Active Directory" from the mode pull down box.
6. Now, select the "General tab".
7. Enter the Realm or Domain name in the input box.
8. Enter the account credentials required for the Data Domain system to join the domain.
9. Select the "Advanced Tab."
10. Next, select the "Use Default" server name if you wish to have the NB host name be derived from the
IP host name. If you wish for the NB host name to be manually configured, you must unselect the
checkbox and enter a name.
11. Now, select to automatically or manually configure the IP addresses for the Domain Controllers. If
you choose to manually configure the DC's IP addresses, enter them at this time.
12. Next, choose whether to use the default Organizational Unit "Computers" or to specify another by
deselecting the "Use Default Computers" checkbox. Enter the name of the "Organizational Unit" to
which the Data Domain system will belong.
13. Finally, select "Ok" to accept the configuration changes and return to the CIFS configuration screen.
If needed, the WINS server must be configured using the CLI. WINS server configuration is not available
in the GUI.
References
Wikipedia - Organizational Unit
http://en.wikipedia.org/wiki/Organizational_Unit
LESSON SUMMARY - Troubleshooting Active Directory Authentication
Reviewing CIFS Active Directory Authentication Configuration
Configuring Kerberos / AD Authentication
Knowledge Check
LESSON - Troubleshooting Join Domain Issues
Describing the term "Join Domain Issue"
Describing the Join Domain Transaction Flow
Listing the reasons for using the command line to configure Active Directory Authentication
Addressing Domain Controller Not Found issues
Addressing other common problems such as: Time skew, Invalid user, invalid password, and AD
configuration
Describing Join Domain Issues
The Data Domain system must join the domain (realm) in to interact with the Active Directory. For this
to happen, the Data Domain system must be able to:
Locate the Domain Controller (DC). This requires that the Data Domain system be able to resolve the
Domain name for the controller and to be able to transmit data using the required UDP and TCP ports.
Log into the DC. This means the Data Domain system must have an account on the DC.
Read and Write to the Organizational Unit (OU) on the Active Directory (AD). This means that the OU
on the AD needs to allow the Data Domain system to manipulate the contents.
Authenticate using Kerberos
Keep time in sync with DC.
Join Domain - Transaction Flow
The follow high-level steps outlines the individual transactions executed when a Data Domain system
joins a domain.
1. First, the Data Domain system looks up the DNS name for the DC.
2. If the last step was successful, the Data Domain system sends a Connectionless Lightweight Directory
Access Protocol (CLDAP) query to the DC to determine if the DC is running LDAP. CLDAP uses UDP
instead of TCP, so it takes less time to setup and execute the query.
3. Once the Data Domain system determines the DC is running LDAP, the Data Domain system initiates
a Kerberos (KRB) ticket exchange.
4. Next, the Data Domain system sends an LDAP query to the DC to determine if the OU exists and if the
Data Domain system object already exist in the OU.
5. If the Data Domain system object does not exist in the OU, the Data Domain system will create the
object, provided it has sufficient permissions to do so.
Reference
Connection-less Lightweight X.500 Directory Access Protocol
https://tools.ietf.org/html/rfc1798
Kerberos: The Network Authentication Protocol
http://web.mit.edu/kerberos/
Join Domain - Packet Trace
On the screen is a walkthrough of a capture of a packet trace for a Data Domain system joining a domain.
This trace should help you in understanding how the Data Domain system joins the domain, and to
understand what to look for when assisting customers.
Capture the Trace
Before you can examine a trace you must first capture it. There are a number of ways to accomplish this,
but the easiest is to use the net tcpdump capture CLI command. The net tcpdump capture CLI
command places the capture in the /ddvar/traces directory on the Data Domain system. You may also
capture a trace using Wireshark on the Domain Controller.
The trace can be reviewed using tshark (from Wireshark) or ethereal.
DNS Lookup
The first thing that happens is the Data Domain system requests the IP address of the DC from the DNS.
You can mimic this transaction with the net lookup command.
CLDAP Authentication
The next transaction is the CLDAP authentication. These packets are used to determine if DC is running
LDAP.
Kerberos Authentication
The next transaction is the Kerberos authentication request and response.
Reference
TCPDUMP Man Page
http://www.tcpdump.org/manpages/tcpdump.1.html
Tethereal Man Page
http://www.linuxcommand.org/man_pages/tethereal1.html
Join Domain - Packet Trace (SE Mode Tools)
The tcpdump utility is also available in SE mode and in Bash mode. In these modes, it offers more
options if needed.
Saving the tcpdump to the CIFS directory ensures the capture will be included in any support bundles
generated afterwards.
You may review the trace on the Data Domain system using the tethereal utility in SE-mode or bash-
mode.
The dash lowercase r option (-r) identifies the capture file to read.
The dash uppercase R option (-R) identifies the types of packets to display. As you can see, SMB,
LDAP, DNS, CLDAP, and Kerberos packets are to be displayed.
Join Domain - Packet Trace (Continued)
Now you'll see the SMB negotiation.
The next step is to setup an SMB session with the DC.
Following that, the Data Domain system connects to the IPC share on the DC.
Now, connect to the lsass service which handles the LDAP protocol.
References
Local Security Authority Subsystem Service (LSASS)
http://msdn.microsoft.com/en-us/library/aa939478(v=winembedded.5).aspx
Security Subsystem Architecture
Understanding LDAP Security Processing
http://blogs.technet.com/b/askds/archive/2009/09/21/understanding-ldap-security-processing.aspx
About IPC$
http://support.microsoft.com/kb/314984
http://msdn.microsoft.com/en-us/library/windows/desktop/aa365574(v=vs.85).aspx
The next section shows the Data Domain system sending a Query to the DC using LDAP to obtain the
base.
Finally, the Data Domain system attempts to add itself to the directory on the DC. If an entry exists, the
DC sends an entryAlreadyExists message.
When working with CIFS-related join-domain network traces, verify the packets follow the steps for
joining a Windows domain.
If certain steps keep repeating, they need to be investigated.
Join Domain- Use the Command Line
When configuring Active Directory authentication, you should use the CLI because of the feedback it
provides, the ready access to log files, and the troubleshooting tools available to help you diagnose
issues.
Join domain issues quite often manifest themselves when the system administrator first configures Active
Directory authentication. When using the "cifs set authentication active-directory " command, the Data
Domain system often provides feedback to indicate the possible cause of the problem as well as some
steps that can be taken to fix the issue.
On the screen is the error message that is displayed when the system cannot find the domain controller.
The message reminds the system administrator to verify the DC can be found through the DNS and that
port 389 is not blocked by the firewall.
Not only can join domain issues occur during the initial configuration period, but also when the Data
Domain system attempts to periodically resync with other network components.
Notes
Error from /ddr/var/log/debug/messages. engineering
sms: NOTICE: cifs_join: result_status [3] result_string [Failed to lookup the domain controller for given
domain. Check that the domain name is correctly entered. Check that your DNS server is reachable, and
that your system is configured to use DNS in nsswitch. Check that port 389 UDP is not blocked by your
firewall.]
Error from /ddr/var/log/debug/cifs/join_domain.log
20140507130333:ERROR:Lsass Error [CENTERROR_DOMAINJOIN_UNRESOLVED_DOMAIN_NAME]
Join Domain - DC Not Found
If the Data Domain system cannot find the DC, a message will be entered in the "messages.engineering "
log file with the text "Failed to lookup the domain controller for given domain."
Windows Client Environment Verification
You can find out more about the domain by querying a client that is supposed to be in the same domain
and has successfully logged into the network. Using the set command in a command window provides
you with a list of environment variables including the computer name, the login server (which is the same
as the domain controller), the user DNS domain name (which is the domain name in FQDN format), the
domain name, and the user name. Compare this information to what you are using to configure Active
Directory on the Data Domain system and ensure any differences are understood.
Join Domain - DC Not Found (Continued)
Windows Client DNS Verification
Next check the operation of the DNS from the windows client.
First, get a list of domain controller IP addresses for the realm by using the nslookup "<realm>"
command. Specify the realm in fully qualified domain name format. If you've been given an IP address
instead of a DNS name for the DC, make sure the IP address is listed in the output.
If you've been given the DNS name of DC, lookup the IP address by using the "nslookup <dc_FQDN>"
command on the windows workstation. When the IP address returns, check to see if it is on the list of
DCs for that domain.
You can retrieve the name DC by using the "nslookup <dc_ip>" command on the windows client.
Finally, you may wish to confirm the DDR's DNS entries by using the "nslookup" command with the Data
Domain system host name and IP address as arguments.
Data Domain system DNS Verification
You can verify the DDR's access to the realm's DNS entry by using the "net lookup" CLI command. Verify
the Data Domain system can obtain the realm information from the DNS by using the "net lookup"
command. Specify the realm name in fully qualified DNS name (FQDN) format.
Obtain the IP address of the DC by invoking the "net lookup" CLI command with the DC's fully qualified
DNS name as the target.
Obtain the DC's fully qualified DNS name by invoking the "net lookup" CLI command with the DC's IP
address as the target.
Connectivity Verification and Latency Assessment
Use the ping command to verify connectivity and latency. From the windows client, ping the Data Domain
system and the DC. From the Data Domain system, ping the DC.
Note, the ping command may fail if ICMP ECHO packets are filtered by firewalls in the network or if the
target device is configured to ignore these packets. Do not assume a ping failure indicates that the
device is down or there is no connectivity. A failure with PING only means that PING was not able to
verify connectivity and more investigation is required.
References
Wikipedia Environment Variables
http://en.wikipedia.org/wiki/Environment_variables
181313 : Joining a Data Domain System to a Windows Domain
Microsoft Support - using nslookup
If a CIFS client cannot map to a Data Domain System in an environment with multiple domain controllers,
the joining of the Data Domain System to an active directory domain may have failed. Out of sync domain
controllers can cause the problem. Look in the Data Domain System log files for messages similar to
"Preauthentication" failed or "Client not found in Kerberos database."
The workaround is to re-join the Data Domain System to a domain using a single IP address in the CIFS
command on the Data Domain System.
Join Domain - DC Not Found (deep dive)
You can get the name of a domain controller (and a lot of other useful information) by entering bash
mode and executing the "lw-get-dc-name" command. An example of this output is available in the
student guide.
This command is part of the likewise suite and is found in the /opt/likewise/bin directory on the Data
Domain system.
If this command is successful, it returns the name of the domain controller. Ping the controller to verify
accessibility.
If the "lw-get-dc-name" command is unable to obtain information about the realm , it returns an error.
You can workaround this issue by using the IP address of the DC instead of the DC's FQDN.
The fields returned are named using Hungarian notation. The names listed are prepended with dw, w,
psz, and puc to indicate the type of variable in source code. They are (d)ouble (w)ord, (w)ord, (p)ointer
to (s)tring ending in (z)ero, and (p)ointer to (u)nsigned (c)har. I'm telling you this so you don't waste
time looking up the meaning yourself.
References
Likewise Enterprise Installation and Administration Guide (pdf)
http://one.emc.com/clearspace/docs/DOC-97020
Likewise Open Installation and Administration Guide
http://one.emc.com/clearspace/docs/DOC-97021
Hungarian notation.
http://www.cse.iitk.ac.in/users/dsrkg/cs245/html/Guide.htm
Example Output from Successful Completion of the "lw-get-dc-name" Command
lwgetdcnamecorp.emc.com
PrintingLWNET_DC_INFOfields:
===============================
dwDomainControllerAddressType =23
dwFlags =12796
dwVersion =5
wLMToken =65535
wNTToken =65535
pszDomainControllerName =corpcascv1.corp.emc.com
pszDomainControllerAddress =137.69.224.15
pucDomainGUID(hex)=E907841FE1F668498DC68E3BAE948F1D
pszNetBIOSDomainName =CORP
pszFullyQualifiedDomainName =corp.emc.com
pszDnsForestName =emcroot.emc.com
pszDCSiteName =CorpUSCASantaClara1
pszClientSiteName =CorpUSCASantaClara1
pszNetBIOSHostName =CORPCASCV1
pszUserName =<EMPTY>
Join Domain - Time Difference
There are some time sync requirements in order for active directory authentication to work. These
requirements are that the time on the Data Domain system and DC be within 5 minutes and that the Data
Domain system and DC be in the same time zone.
You can verify the Data Domain system's time and timezone configuration by using the system show
date and config show timezone CLI commands. If needed, you can set the date and time zone with
the system set date and config set timezone CLI commands.
If the you are using a Network Time Protocol (NTP) server, use the ntp status and ntp show config CLI
commands to verify the NTP settings on the Data Domain system.
Remember that NTP uses port 123. Communication between the Domain Controllers, Data Domain
systems, and NTP servers should not be blocked by firewalls.
Join Domain - Verify DC Time in Bash
You can verify the time on the Domain Controller from the Data Domain system by using the lw-get-dc-
time BASH command.
This command is located in the /opt/likewise/bin/directory.
Join Domain - Invalid User
If one of the following error messages appears in the messages.engineering or join_domain.log file after
executing the "cifs set authentication active-directory" CLI command, the user name may not be
configured correctly on the DC:
Failed to join the domain with the following error message: *** Permission denied
kinit(v5): Preauthentication failed while getting initial credentials
The user account is invalid or the password is incorrect for the given username
The user does not exist or syntax (domain\administrator) is incorrect.
Verify the user name exists in target domain by checking the DC. Log in with the Down Level Logon
Name (domain\username) format using the domain prefix if the user is a regular domain user. Employ
the User Principal Name (UPN) login style if the user is a trusted domain user.
Remember, user names are not case sensitive while passwords are.
References
User Name Formats
http://msdn.microsoft.com/en-us/library/windows/desktop/aa380525(v=vs.85).aspx
Join Domain - Incorrect Password
Error messages in the messages.engineering or join_domain.log can identify that the problem lies with
the password employed by the user.
Join Domain - User Unauthorized
A "permission denied" error message can be displayed after executing the cifs set authentication
active-directory CLI command or written to the messages.engineering or join_domain.log. This may
indicate that the Data Domain system user name does have permission to write to the Organizational Unit
(OU) configured on the Data Domain system.
There are a few things you an try to address this issue:
First, verify the Data Domain system has permission to write to the OU by looking at the Domain
Controller.
Next, you can try logging into the DC from a Windows client and verify the Data Domain user is able to
write a new object to the OU. If a new section is not available, it indicates the Data Domain system
user does not have permissions.
You can also configure the Data Domain system to use the credentials of a user whose access is
confirmed to work.
Join Domain - Authentication Issues
User authentication issues will manifest themselves by presenting an error when the "cifs config
authentication active-directory" CLI command is executed or by writing the error to the clients.log or
the cifs.log files.
If you suspect an authentication error, verify the cifs feature is enabled and running by using the cifs
status CLI command.
Next, you can check the configuration of the "idmap-type" and "ntfs-acls" options. The "idmap-type"
option should be set to "rid, "none", or not set. The "ntfs-acls" option should be "enabled" or not set.
Now, verify the NB-hostname is set and that the name has a maximum of 15 characters.
Finally, you should:
Verify the authentication type with the "cifs show config" CLI command.
View the users types with the "user show list" CLI command.
Verify the Data Domain system user configured on the system with the "cifs troubleshooting
user-list" CLI command.
View the details of the Data Domain system cifs user with the "cifs troubleshooting user" CLI
command.
Join Domain - Authentication Issues (Continued)
You can also go into SE-mode to verify the status of the processes used for authentication.
Execute the se ps -A SE-mode command and look for lsassd, lwiod, and netlogond processes.
Diagnosing AD Config
When diagnosing the AD, verify the NetBIOS name matches the first part of the DNS name. Also, if
strong security policies are set on the AD, make sure that anonymous access to the IPC$ share is allowed
and that the NETLOGON pipe is enabled.
LESSON SUMMARY- Troubleshooting Join Domain Issues
Describing the term "Join Domain Issue"
Describing the Join Domain Transaction Flow
Listing the reasons for using the command line to configure Active Directory Authentication
Address Domain Controller Not Found issues
Addressing other common problems such as: Time skew, Invalid user, invalid password, and AD
configuration
Knowledge Check
LESSON - CIFS Authentication Troubleshooting
Using trusted domain accounts
The loss of the Data Domain system machine account
Kerberos ticket decryption error
IDMAP issues
SSH to Data Domain system not working
Authentication - Trusted Domain Accounts
When troubleshooting the authentication of trusted domain accounts, verify the authentication mode for
the Data Domain system using the "cifs show config" CLI command.
You should also determine which authentication method is in use as well as the user's account location.
Determine if the user has an account that is configured on the local Data Domain system, or is the user's
account is part of the same domain as the Data Domain system, or is the user's account is part of a
trusted domain.
Authentication - Trusted Domain Accounts (Continued)
If necessary, use the se ps -A SE-mode command to verify that the winbindd or lsassd processes are
running.
The winbindd process is used by Samba and the lsassd process is used by Likewise.
Authentication - Trusted Domain Accounts (Continued)
If the user is from a trusted domain, use the "cifs option show" CLI command to verify the CIFS
allowtrusteddomains option is enabled on the Data Domain system. If it is not enabled, you can use the
"cifs option set allowtrusteddomains enabled" CLI to enable the option.
Review the client log file on Data Domain system. The client log file is named using the IP address of the
client with the .log extension (192.168.50.99.log).
This file may indicate that there is a memory allocation error or the client connection failed because of too
many sessions. The client log file is located at /ddvar/log/debug/cifs.
#cifsoptionshow
CurrentlySetOptions:
"idmaptype"issetto"rid"
"ntfsacls"issetto"enabled"
KnownUsefulOptions:
allowtrusteddomains enabled|disabled(default:enabled)
loglevel [010](default:1)
maxxmit[1638465536](default:65536)
restrictanonymousenabled|disabled(default:disabled)
smbdmemlimit[524288001073741824](default:209715200)
ntfsacls enabled|disabled(default:enabled)
idmaptypenone|rid(default:rid)
organizationalunite.g."Computers/Servers/ddrunits"(default:Computers)
dd admingroup1e.g."domain\group2"(default:"DomainAdmins")
dd admingroup2e.g."domain\group1"(default:None)
dd usergroup1e.g."domain\group1"(default:None)
dd usergroup2e.g."domain\group2"(default:None)
dd backupoperatorgroup1e.g."domain\group1"(default:None)
dd backupoperatorgroup2e.g."domain\group2"(default:None)
Authentication - Loss of Data Domain system Machine Account
If the Data Domain system has lost its machine account on the Active Directory, check for errors in the
"winbindd.log". Use the "wbinfo" client command to confirm that the Data Domain system is disconnected
from the domain. Rejoin the Data Domain system to the domain.
Authentication - Kerberos Ticket Decryption Error
This error is caused when the Windows client attempts to use a stale Kerberos ticket. This can happen
when the Data Domain System is rebooted, or re-joins a domain, and obtains a new Kerberos ticket. The
existing ticket on the Windows client will then be stale and not work.
This can also happen if the Data Domain system changes its user account password on the domain
controller. This change in passwords can render the ticket on the client as invalid.
This error can also be caused when there is a discrepancy of greater than 5 minutes between system
times on the client, Data Domain system, or domain controller.
You can work around this issue in one of the following ways:
Logoff and log back on to the client. This causes the client to obtain a new ticket from the Kerberos
server.
Use the klist utility to list and purge stale Kerberos tickets from the Windows client.
The klist utility was part of the Windows Resource Kit Tools for Windows 2003, and is available for
other Windows platforms.
Type klist at the command prompt to see if it is installed on your windows client.
Reboot the windows client if the problem persists.
Example
C:>klist
CurrentLogonId is0:0xFFFFF
CachedTickets:(1)
#0>Client:user@CORP.EXAMPLE.COM
Server:krb5/CORP.EXAMPLE.COM@CORP.EXAMPLE.COM
KerbTicket EncryptionType:AES256CTSHMACSHA196
TicketFlags0x40e00000>forwardable renewableinitialpre_authent
StartTime:9/17/201510:42:41(local)
EndTime:9/17/201520:42:41(local)
RenewTime:9/24/201510:42:41(local)
SessionKeyType:AES256CTSHMACSHA196
Join Domain - Kerberos Ticket Decryption Error (Continued)
If needed, you can empty the Kerberos cache on the Data Domain system by using the "lw-ad-cache --
delete-all" bash command. This command is located in the /opt/likewise/bin directory.
Authentication - IDMAP Issue
The IDMAP facility is the part of the CIFs process that maps UNIX User Identifiers (UIDs) and Group
Identifiers (GIDs) to Windows security identifiers (SIDs).
In a multi-protocol environment, there is a potential for IDMAP issues where the system is not able to
associate the user UIDs, GIDs, and SIDs with one another.
In earlier versions of DD OS, this type of error created a Relative Identifier (RID) error message in the
log.winbindd-idmap or log.wb* files. These files may be found in the /ddvar/log/debug/cifs directory on
Use the log list debug CLI command to verify the existence of these log files. Use the log view CLI
command to display the log files.
Reference
Chapter 14. Identity Mapping (IDMAP)
https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/idmapper.html
ID Man Page
http://ss64.com/bash/id.html
Authentication - IDMAP Issue
If you suspect the IDMAP facility is having issues, examine the various IDs associated use by executing
the cifs troubleshooting user CLI command on the Data Domain system.
The cifs troubleshooting CLI command provides the UID, GID, SID, user name, and group names
associated with the user.
An example of the output from this command is shown on the screen.
#cifstroubleshootinguser<username>
Userddsys\sysadmin
UserID100
SID515213273096021316007606720610370761200
Groupddsys\DDusers
GroupID60001800
iduid=100(ddsys\sysadmin)
group=60001800(ddsys\DDUsers)
groups=60001800(ddsys\DDusers)
Authentication - IDMAP Issue (Continued)
If the cifs troubleshooting user CLI command does not provide adequate information, or is not
available on the Data Domain system, the id <username> bash command can be used.
This command allows you to directly interrogate the IDMAP facility on the Data Domain system. The
output of this command should provide the various UNIX IDs associated with the user.
On screen is an example of the output of this command.
!#idsysadmin
id=100(sysadmin)
gid=50(admin)
groups=50(admin),60001800(ddsys\DDUsers)
Authentication - IDMAP Issues (Continued)
If the issue involves a connected Windows client, verify the user's Windows user's SID by executing the
whoami program on the Windows client. An example of the output from this command is shown on the
screen.
C:>whoami /user
USERINFORMATION

UserNameSID
===========================================================
corp\bkup01S152185424539819725790413622881271234567
Another program, the Windows Management Instrumentation Command-line (WMIC), also provides the
SID. Unfortunately, this Windows utility takes longer to execute than the whoami program. To display
the user's SID using the wmic, use the syntax shown on the screen:
C:>wmic useraccount wherename='%username%'getsid

SID
S152185424539819725790413622881271234567
Reference
Get SID of user
http://www.windows-commandline.com/get-sid-of-user/
WMIC - Take Command-line Control over WMI
http://technet.microsoft.com/en-us/library/bb742610.aspx
http://msdn.microsoft.com/en-us/library/aa394531(v=vs.85).aspx
Authentication - SSH to Data Domain system not Working
If the customer would like to SSH into DD system using windows login, check CIFS authentication is
enabled for admin access. Then review that IDMAP is working for the user.
If the customer is unable to create an SSH connection to the Data Domain system from a Windows client,
verify that CIFS authentication is enabled for adminaccess by executing the "adminaccess
authentication show" CLI command.
#adminaccess authenticationshow
CIFSauthentication:disabled
If CIFS authentication is disabled, enable it with the "adminaccess authentication add cifs" CLI
command.
#adminaccess authenticationaddcifs
CIFSauthentication:enabled
Review that the UIDs is working for the user account:
#cifstroubleshootingusersysadmin
Userbluewhale\sysadmin
UserID100
SID515213273096021316007606720610370761200
Groupbluewhale\DDusers
GroupID60001800
iduid=100(bluewhale\sysadmin)group=60001800(bluewhale\DDUsers)
groups=60001800(bluewhale\DDusers)
LESSON SUMMARY - CIFS Authentication Troubleshooting
During this lesson the following topics were covered:
Using trusted domain accounts
The loss of the Data Domain system machine account
Kerberos ticket decryption error
IDMAP issues
SSH to Data Domain system not working
Knowledge Check
LESSON - Share and File Access Control Troubleshooting
Troubleshooting Share-Level Access Control Issues
Describing and Troubleshooting SMB Signing
Troubleshooting File-level Access Control Issues
Share User Authorization
The following is a high-level overview of how CIFS authentication and authorization works:
1. First, the client establishes a connection to the file server - or Data Domain system in this case- and
the file server passes information about itself back to the client.
2. Now, the server determines if the user exists by looking in its local user database, or by sending a
query to the Active Directory. If the user exists, the Data Domain system requests the Windows SID
for every group associated with the user.
3. The next step is to authenticate the user by determining if the user provided the correct password or
other credentials. This can be done by interrogating the Active Directory or by looking at the local
user accounts on the Data Domain system. There will be a message in the clients.log file to indicate
success or failure.
4. Now, the Windows client connects to the IPC share on the Data Domain system. If the Windows client
fails to connect to the IPC share, it assumes that the Data Domain system is not configured as a CIFS
server.
5. If the connection to the IPC share is successful, the Windows client verifies the share exists by
attempting to mount the share with an SMB Tree Connect to the Data Domain system.
6. Next, the Data Domain system verifies that the user is allowed to access the share.
7. Finally, the Data Domain system verifies that the machine (also known as the client) is allowed access
to share.
Share User Access - Administrative Restrictions
Sometimes the user cannot access the Windows shared folder because of intentional or unintentional
administrative restrictions. The restrictions can be:
Client access restrictions which identify the hosts from which users are allowed to access the shared
folder.
Group access restrictions which identify the groups whose members are allowed access to the shared
folder.
User access restrictions which identify the users who are allowed access to the shared folder.
Host Access Restrictions
Use the "cifs share show <sharename>" CLI command to see if the shared folder has client restrictions
applied. When creating client restrictions, be aware that all clients are restricted by default when the
share is created. You can specify which clients will be allowed to access the shared folder when you
create the folder, or you can modify the client list as a subsequent step.
If you choose to restrict access to particular clients, you must provide the IP address or the DNS name of
individual stations. This means that you cannot restrict access to a group of clients with a single entry.
If you use a DNS host name in the client list, the Data Domain system must be able to resolve that host
name when that client attempts to access the shared folder.
User / Group Restricted Access
Access may be restricted to certain users or groups. As long as the user access list is empty, all users are
allowed access to the shared folder. If there is any entry in the user access list, then access is restricted
to the users on the list. Do not enter an asterisks (*) to indicate "all users." Doing so causes the Data
Domain system to only allow a user with that name to be allowed to link to the shared folder.
Also, remember that users and groups must be entered in domain\username format. If there is a space in
the user name or group, enter the name in quotes. If a group name is specified, verify the Data Domain
system is able to find the user in the group by using the "cifs troubleshooting user
<domain>\\<username>" CLI command. Note the two backslashes used in the "cifs troubleshooting user
<domain>\\<username>" CLI command. Also, verify the group is discoverable to the Data Domain
system by using the "cifs troubleshooting list-groups" CLI command.
Share User Access - Verifying User
To verify a user is discoverable by the Data Domain system, execute the "cifs troubleshooting user" CLI
command with the domain and username arguments. Make sure to put in two backslashes as separators
between the two elements.
Share User Access - Testing User Login (Deep Dive)
To test user access from the Data Domain system, use the "smbclient" bash command. The -U argument
is followed by the domain and username. These arguments are then followed by the target server name
and share name. Note that the slash is used as a delimiter in this syntax instead of the more familiar
backslash.
In the example on the screen, we are testing the user BKUPSRVR03 from the BACUPNET domain's ability
to access the backup share on the localhost, which is the Data Domain system from which this command
is being executed.
References
smbclient man page
https://www.samba.org/samba/docs/man/manpages/smbclient.1.html
Share User Access - Verifying Share Config
When troubleshooting share configuration issues, first check whether the share exists using the cifs share
show CLI command.
Next, check that the share name points to an existing directory.
Review that the client access is correct.
Check for any error messages on the Windows client.
A system error 53 indicates the client cannot find the share.
A system error 59 indicates an unexpected network error occurred.
Verify the user's access has not been restricted by Windows Security Settings (ACL).
Share User Access - Verifying Share Config (Continued)
If the problem still persists, check for authentication issues. Make sure the user has privileges to the
shared folder. Also make sure the password is correct.
SMB Signing - Description
SMB Signing is a feature that allows communications using SMB to be digitally signed at the packet level.
Digitally signing the packets enables the recipient of the packets to confirm their point of origination and
their authenticity. This security mechanism in the SMB protocol helps avoid issues like tampering of
packets and "man in the middle" attacks.
SMB Signing States
There are currently three states for SMB Signing - enabled, disabled, and required. The enabled state
makes SMB Signing available if required by the connected device. The required state informs the other
connected device that SMB Signing must be used. The disabled state means that the device will not
support SMB signing, even if the connected device requires it. SMB connections will fail if one device
requires SMB signing, and the other device has SMB Signing disabled.
Data Domain system Support
Support for SMB Signing started with DD OS version 5.2.4. SMB Signing is unsupported before DD OS
5.2.4.
SMB Signing - Managing Windows Clients
Windows clients use different mechanisms for managing the SMB Signing feature, depending on the
version. Refer to the Microsoft document "Overview of Server Message Block Signing" for details on a
specific version of Windows.
Most Windows clients use two registry keys to manage the configuration of the SMB Signing feature. The
registry keys are:
EnableSecuritySignature
RequireSecuritySignature
The location of these keys in the Windows registry is shown on the screen.
The settings for these keys determines the SMB configuration state:
Enabled
EnableSecuritySignature = 1
RequireSecuritySignature = 0
Required
Disabled
SMB Signing - Verifying Windows Clients
If running a version of DD OS that does not support SMB Signing, the client must not have SMB signing
configured as required. To determine if SMB signing is required by the Windows client, use the Regedit
utility to verify the configuration manually, or use the "reg" Windows command shown on the screen and
in the student guide.
regqueryHKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters
On screen is a sample of the output from the "reg query" Windows command:
ServiceDll REG_EXPAND_SZ%SystemRoot%\System32\wkssvc.dll
ServiceDllUnloadOnStop REG_DWORD0x1
EnablePlainTextPassword REG_DWORD0x0
EnableSecuritySignature REG_DWORD0x1
RequireSecuritySignature REG_DWORD0x1
OtherDomains REG_MULTI_SZ
SMB Signing - Configuring Windows Clients
The registry keys on the Windows client may be configured locally on the Windows client or through
centrally managed group policies. You can change the local registry's RequireSecuritySignature setting to
zero (0) by using the "Regedit" windows utility, or by using the following "reg" command:
regaddHKLM\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters/v
RequireSecuritySignature /tREG_DWORD/d0x0
You may need to execute the "Regedit" or "reg" command with the "Run as administrator" option.
Refer to the Microsoft document "Overview of Server Message Block Signing" for details on how to
configure SMB Signing through group policies.
Reboot the Windows client after changing this setting.
SMB Signing - Managing the Data Domain system
Verifying the SMB Signing State on the Data Domain system
To verify the state of SMB Signing on the Data Domain system, use the "cifs option show" CLI
command. The SMB signing feature is called "server signing" on the Data Domain system.
Configuring SMB Signing State on Data Domain system
Use the "cifs option set" CLI command to configure SMB Signing on the Data Domain system. To
disable the SMB Signing feature, use the "cifs option reset" CLI command.
Notes
EMC Data Domain encourages that SMB Signing feature be disabled unless it is required by your
enterprise.
SMB Signing may cause a decrease in server performance of around ten percent.
The default setting for the "server signing" feature is disabled.
Because there is a space in the option name, the "server signing" option is surrounded by quotes.
When configuring the "server signing" option, you will not receive an error if you specify an incorrect
option value.
SMB Signing - Errors and Messages
The Data Domain system clients.log file shows the state of SMB Signing (server signing) when the
Windows user establishes a connection.
[Signing server(disabled),client(disabled)]connecttoservicebackupinitiallyasuserdd120
train1\sysadmin(uid=100,gid=60001800)(pid24275)
If SMB Signing is required by client but disabled on server, the Windows client displays this error
message:
Systemerror1240hasoccurred.Theaccountisnotauthorizedtologinfromthisstation.
If SMB Signing is required by server but disabled on client, the Windows client displays this error
message:
Requiredbyserver,disabledonclient Systemerror5hasoccurred.Accessisdenied.
References
Reg Tool on Technet
181357 : SMB Signing on Data Domain
Overview of Server Message Block signing
CIFS
http://msdn.microsoft.com/en-us/library/Aa302188.aspx
File Access Troubleshooting
If the client request is failing with a STATUS_ACCESS_DENIED or STATUS_PRIVIELEGE_NOT_HELD error,
check the permissions of the file and the parent directory using the following methods:
If, from a Windows client, you are able to gain access to the file or directory as another user (such as
sysadmin), use Windows Explorer to examine the security properties of the file or parent directory.
If necessary, you can check the permissions on the file using the "se dd_xcacls" CLI command. This
command allows you to view and modify the object's owner and group SID. It also allows you to view
and modify the object's permissions and auditing information from the perspective of the operating
system.
Check if the file or directory is a replication target using the "replication show config" CLI command.
You cannot write to a replication target.
Verify that Write Once Read Many (WORM) retention-lock features are not configured on the file. The file
cannot be modified or deleted when retention lock has been applied.
References
180673 : Mapping a Network Drive to the Data Domain system
181517 : Unable to Write, Modify, or Delete a Directory or File on a Replication Target
LESSON SUMMARY - Share and File Access Control Troubleshooting
This lesson discussed the following topics:
Troubleshooting Share-Level Access Control Issues
Describing and Troubleshooting SMB Signing
Troubleshooting File-level Access Control Issues
Knowledge Check
LESSON - CIFS Performance Troubleshooting
Troubleshooting performance degradation due to slow network
Diagnosing session timeout issues
Listing CIFS performance troubleshooting recommendations
CIFS Performance - Slow Network
When troubleshooting CIFS performance, you need to verify or eliminate the network and NIC as an issue
by using the "net iperf" CLI command. Use the "window-size" option to specify a windows size of 1
megabit. If the test does not yield throughput that is close to the maximum capabilities of the NIC, then
check the network for the fault.
Another test is to create a file on the Windows client using the "fsutil" Windows program. You can use the
command shown on the screen to create a ten megabyte file on the Windows user's desktop.
Copy this file to a CIFS shared folder. If you reach the expected write speed, then the issue could lie with
the backup application.
You may also wish to analyze network traffic using the "net tcpdump" CLI command.
Finally, if the CIFS client is a Linux device, you can run the "lmdd" command.
References
fsutil
https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-
us/fsutil.mspx?mfr=true
fsutil
Wikipedia (iperf)
http://en.wikipedia.org/wiki/Iperf
LMDD Man Page
http://dev.justmanpage.com/web/man/lmdd.8
CIFS Performance - Session Timeout
If latency is high in a network, consider changing the CIFS session timeout value on the windows client
from the default value of 45 seconds to a value of 3600 second - which is one hour. You can view the
SESSTIMEOUT value by executing the "reg query", "regedit", or "regedit32" windows commands.
regqueryHKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters
If the field is not displayed, it means that the Windows client is using the default value. You can set (or
reset) and add the key by executing the "reg add" command as an administrator or by using the "regedit"
or "regedit32" program.
regaddHKLM\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters/vSESSTIMEOUT/t
REG_DWORD/d3600
Even though this step addresses the requirements for most systems, check the Knowledge base article
"Setting Network Parameters on the Client for Optimal Throughput with a Data Domain system" for
information on most versions of Windows.
References
180579 : Setting Network Parameters on the Client for Optimal Throughput with a Data Domain system
SMB 2.x and SMB 3.0 Timeouts in Windows
http://blogs.msdn.com/b/openspecification/archive/2013/03/27/smb-2-x-and-smb-3-0-timeouts-in-
windows.aspx
CIFS Performance Tuning
Consider increasing the TCP window size for better performance.
The knowledgebase article "Network Performance Troubleshooting" provides information on how to
configure different operating systems, including Windows, AIX, Red hat, and Solaris.
To make the new TCP window size take effect on the Windows client, restart the machine.
Using the "cifs option show" CLI command on the Data Domain system, verify the maxxmit option is at
the maximum value. The maximum value for this option is currently 65536.
The output from the "cifs option show" CLI command displays the options that have been changed from
their default values at the top. The default values are version dependent. In DD OS 5.1, the maxxmit
option has a default value of 16644. In DD OS 5.4, the maxxmit option has a default value of 65536.
To change the value of the maxxmit option, use the "cifs option set maxxmit" CLI command.
References
180512 : Network Performance Troubleshooting
LESSON SUMMARY - CIFS Performance Troubleshooting
Troubleshooting performance degradation due to slow network
Diagnosing session timeout issues
Listing CIFS performance troubleshooting recommendations
Knowledge Check
MODULE SUMMARY - CIFS Troubleshooting
You have completed this module. You should now be able to:
Describe CIFS
List possible CIFS problem areas
Troubleshoot CIFS authentication
Troubleshoot CIFS shares
Perform file access troubleshooting
MODULE - NFS Troubleshooting
Upon completion of this module, you will be able to:
Describe NFS
Configure NFS
Troubleshoot NFS
LESSON - NFS Overview
Describing NFS and NFSv3
Listing NFS configuration steps
Describing the nfs add CLI options
Mounting an NFS export on a client
Enabling authentication for NFS
NFS Description
The NFS network file system protocol was originally developed by Sun Microsystems in 1984. It allows a
user on a client computer to access files on an NFS server over a network. NFS clients include Unix, Linux,
and Windows operating systems. Verify the client is supported by the version of DD OS running on the
Data Domain system by reviewing the documentation and release notes.
There are four versions of NFS, versions 1, 2, 3, and 4. Data Domain supports version 3 (NFSv3).
No license is required to use the NFS feature on a Data Domain system.
References
RFC 1813 - NFS Version 3 Protocol Specification
http://tools.ietf.org/html/rfc1813
Wikipedia.org Network File System
http://en.wikipedia.org/wiki/Network_File_System
What is NFSv3?
NFSv3 supports 64-bit file sizes and offsets. This enables it to handle files larger than 4 gigabytes in size.
NFSv3 also supports asynchronous writes on the server. This improves write performance because writes
to disk are unacknowledged. The NFS client assumes the write worked unless the NFS server says
otherwise.
An NFSv3 server returns additional file attributes to the NFS client in response to certain requests. Using
this technique, the NFS server helps to avoid the need for the NFS client to issue subsequent requests to
obtain the attribute information.
NFSv3 servers also support read directory plus (READDIRPLUS) operations. This operation causes the NFS
server to get file handles and attributes, along with file names, when scanning a directory.
References
Wikipedia Asynchronous IO
http://en.wikipedia.org/wiki/Asynchronous_write
Review NFS Status
To review the NFS status, follow these steps.
First, verify NFS is enabled by using the nfs status CLI command. This command tells you if NFS is
enabled and the number of NFS requests that have been serviced by the system.
If necessary, you can enable NFS by using the nfs enable CLI command. The command lets you know if
it completed successfully or if NFS is already enabled.
Review NFS Exports
To review the directories that have been exported and shared with other devices on the network, use the
nfs show clients command.
This command displays:
A list of exported directories
The clients that are allowed to mount the exported directory
The export options associated with the directory
Add NFS Exports
Export (share) a directory
If you determine that you must export or share a directory on the Data Domain system, you can use the
nfs add CLI command. The nfs add CLI command returns the result of the operation and provides error
messages as appropriate.
The command arguments for the add nfs CLI command are path, client, and options.
Path
The path argument specifies the location of the target export directory on the Data Domain system. The
path starts with /backup, /ddvar, or /data/col1/.
Client
The client argument identifies the clients that have access to this mount point. Clients may be specified
through their IP address, IP address and subnet, host name, domain name, or by the use of the wildcard
character which is the asterisk (*).
The best practice is to use the host name when specifying clients.
Options
Options are available to provide more control over the access and management of the exported directory.
There is no option keyword associated with the nfs CLI command set. The options are identified by the
fact they are in parentheses at the end of the command. Individual items in a list of options must be
separated by commas. Spaces are not allowed in the option list.
Example (sec=sys,rw,no_root_squash,no_all_squash,secure,nolog)
nfs add CLI Command Options
When using the "nfs add" CLI command, you may exert more control over the access and management of
the exported directory through the use of the net add CLI command options.
ro, rw
The ro and rw options allow you to grant the assigned clients read-only or read-write access to the
exported directory.
Unless otherwise specified, the nfs add CLI command enables the rw option.
root_squash, no_root_squash
The root_squash option squashes root access through the exported directory by mapping the root UID to
the anonymous UID. This means that even though the user has root access on their local system, they
only have the access assigned to the anonymous user on the exported directory.
Squashing the root UID is done for security purposes. Just because somebody has logged in as root and
has root-level access to their local file system does not mean they should have root-level access to the
exported directory.
The no_root_squash option allows the NFS client's root UID to be used on the exported directory.
Unless otherwise specified, the nfs add CLI command enables the no_root_squash option and the
root_squash option is disabled.
all_squash, no_all_squash
The all_squash option causes the system to map a non-root UID from the local NFS client to the
anonymous UID on the NFS server when accessing an exported directory. This gives every non-root user
the same file access privileges.
The no_all_squash option allows the non-root UIDs to be used unchanged on the exported directory.
Unless otherwise specified, the nfs add CLI command enables the no_all_squash option and the
all_squash option is disabled.
secure, insecure
The secure and insecure options specify the ports from which the NFS requests must originate. When
enabled, the secure option only allows access to client requests originating from secure ports - those
below 1024. The insecure option allows client requests from any port.
Unless otherwise specified, the nfs add CLI command enables the secure option and the insecure option is
disabled.
NFS Add Options (Continued)
anonuid, anongid
The anonuid and anongid options enable you to map anonymous user to a specific UID and GID.
sec=sys, sec=krb5, sec=sys:krb5
The sec=sys option directs the system to use local system authentication. The sec=krb5 option directs
the system to use Kerberos Version 5 authentication. The sec=sys:krb5 specifies that both security
methods are to be used. When both security methods are to be used, remember to separate the sys and
krb5 options with a colon.
Unless otherwise specified, the nfs add CLI command enables the sec=sys option and the krb5 option is
disabled.
log, nolog
The log and nolog options enable or disable the NFS log.
Unless otherwise specified, the nfs add CLI command enables the nolog option and the log option is
disabled.
Using the nfs.log Option
The nfs.log option was introduced in DD OS version 5.5.1.
You can enable NFS logging on a per export basis. This means the activity on some exported directories
may be captured while the activity on other exported directories is not.
Use the nfs add CLI command with the log option to enable logging. On screen are example CLI
commands enabling and disabling the nfs.log option.
Log messages are written to the nfs.log file found in the debug directory. You can view the log file with
the log view CLI command.
Using the nfs.log option makes the system examine all NFS packets on the exported directory. This, of
course, can impact the performance of the Data Domain system.
nfs add CLI Command - Examples
On screen are examples of the nfs add CLI command.
Example 1 provides access to all IPv4 and IPv6 clients.
Example 2 provides access to all IPv4 and IPv6 clients and uses both local and Kerberos authentication.
NFS logging is enabled.
Example 3 allows myHost read-only access to the exported directory. myHost can be an IPv4 or IPv6
client.
Example 4 allows all workstations in the edu.emc.com domain access to the exported directory. The
insecure option means the NFS mount request can originate from any TCP port.
Examples
#nfsadd/data/col1/nfsGoodTest *
All IPv4 and IPv6 clients access
#nfsadd/data/col1/linuxDir *(sec=sys:krb5,log)
All IPv4 and IPv6 clients allowed.
Local and kerberos authentication.
NFS logging is enabled.
#nfsadd/data/col1/trainingmyHost (ro)
Allow myHost read-only access. myHost can be IPv4 or IPv6.
#nfsadd/data/col1/linuxDir *.edu.emc.com(insecure)
Allow access to all devices in edu.emc.com.
The NFS mount request can originate from any TCP port.
nfs add CLI Command Examples (Continued)
Example 5 allows the IPv4 device that meets the address and subnet mask requirements to access the
export.
Example 6 provides access to all IPv6 clients.
Example 7 provides access to all IPv6 clients from the specified IPv6 subnet.
Example 8 provides access to all IPv6 clients using their link-local address. The prefix, fe80, identifies
this as a link-local address. Link-local addresses are not routable, so only clients on the local subnet may
access the export.
Examples
# nfsadd/data/col1/test192.168.1.02/255.255.255.0
Allow access to an IPv4 device that meets the address and subnet mask requirements
#nfsadd/backup::/0
Allows access to IPv6 clients
#nfsadd/backup2620:0:170:1a01::/64
Allows access only from this IPv6 subnet using a 64-bit mask.
#nfsadd/backupfe80::/10
Allows all IPv6 clients on local IPv6 subnet access through their link-local address (prefix fe80).
Mounting an NFS Export on a Linux Client
When mounting an NFS export on a Linux client, follow these steps:
1. First, verify connectivity to the Data Domain system. You can do this using the ping command.
2. Next, decide where you wish the mount-point to be on the Linux client. It can be an existing
directory, or a new directory you create. In the example on the screen, a directory named /mtn-
ddsys is created on the Linux client to serve as the mount point.
3. Now, use the Linux mount command to link the exported directory on the Data Domain system with
the mount point on the Linux client. The examples on the screen show a mount commands targeting
a DNS name, an IPv4 address, and an IPv6 address.
4. The mount command has a number of options that follow the -o flag. All options should be included in
one string. Options are separated by commas. No spaces are allowed in the option string.
Note: On Sun Solaris systems, specify the lock option instead of the nolock option. Use the nolock option
on all other UNIX and Linux systems.
Refer to student guide for an explanation of the options shown.
Mount command options
This option specifies that the program using the NFS connection should stop and wait for the Data
Domain system to come back online, if the Data Domain system is unavailable. The user cannot
hard
terminate the process waiting for the NFS communication to resume unless the intr option is also
specified.
bg This option causes the device to execute a background mount.
intr This option allows NFS requests to be interrupted if the server goes down or cannot be reached.
This option specifies the maximum number of bytes the NFS client can receive when reading data from
rsize
This option specifies the maximum number of bytes the NFS client can write to the Data Domain
wsize
system.
nolock This option determines that the NFS client cannot lock files to prohibit access by other NFS clients.
proto This specifies if TCP or UDP is used for the NFS connection.
vers This specifies the version of the protocol that will be used.
Mounting an NFS Export on a Windows Client
NFS is not supported on Windows clients, but there are times when you may need to use a windows client
for expediency sake. To access NFS exported directories from a Windows 7 client, use the command
shown on the screen.
C:>mountomtype=hard,rsize=32768,wsize=32768,nolock\\ddsys\backuph:
In order to access the NFS server, the Windows client must have the Client for NFS service installed.
NFS Authentication Methods
Supported NFS Authentication Methods
There are three authentication methods supported for NFS - Local, Kerberos UNIX, and Kerberos
Windows. The Kerberos UNIX authentication method requires a UNIX (Linux) Key Distribution Center
(KDC). The Kerberos Windows authentication method requires a Domain Controller (DC).
The Data Domain CLI provides you with a number of tools to enable you to configure NFS authentication.
CLI command support
To disable Kerberos authentication, use the authentication kerberos reset CLI command.
To configure the Data Domain system for NFS Kerberos UNIX authentication, use the authentication
kerberos set realm CLI command with a kdc-type of unix.
To configure the Data Domain system for NFS Kerberos Windows authentication, use the authentication
kerberos set realm CLI command with a kdc-type of windows.
NFS Local Authentication
With NFS local authentication, the user and user's permissions are defined on the Data Domain
system. Local NFS Authentication is enabled by including the sec=sys option in the nfs add CLI
command.
Configuration Steps
To configure NFS Local Authentication, you must first add users to the Data Domain system with the user
add CLI command.
Next, when defining the export, you may include the sec=sys option to indicate the export uses NFS Local
authentication. This is the default setting for all exports.
NFS Kerberos Windows Authentication
When using the Windows Kerberos Authentication method, NFS authentication is performed by the Active
Directory (AD) service which usually runs on a Windows Domain Controller (DC). The AD also serves as a
Kerberos Distribution Center (KDC). Because of this, the terms AD, DC, and Windows KDS usually refer
to the same function.
When the Data Domain system is configured for Kerberos Windows authentication, NFS and CIFS clients
and servers use the AD for NFS authentication.
Implementation steps
The following tasks must be performed in order to successfully implement NFS Kerberos UNIX
Authentication.
1. First, create the keytab file on the Windows KDC for the Data Domain system.
2. After creating the Keytab file, copy it to the /ddvar directory on the Data Domain system. The Keytab
file must be name krb5.keytab.
3. Next, start the Data Domain system configuration process by resetting the authentication Kerberos
configuration to default values by using the authentication kerberos reset CLI command.
4. Now, use the authentication kerberos set realm CLI command to activate Kerberos Windows NFS
authentication. Up to three KDCs may be referenced when activating Kerberos Windows NFS
Authentication.
5. Next, import the krb5.keytab file by using the authentication kerberos keytab import CLI
command. This command moves the krb5.keytab file from the /ddvar directory to the /ddr/etc
directory.
6. Now, include the sec=krb5 option when creating an export to invoke Kerberos NFS authentication for
clients attempting to access that shared resource. CIFS clients will also use Kerberos authentication.
7. Next, verify the operation of Kerberos authentication on the Data Domain system.
8. Finally, verify the krb5.keytab file was deleted from the /ddvar directory.
NFS Kerberos Windows Authentication
When using the Windows Kerberos Authentication method, NFS authentication is performed by the Active
Directory (AD) service which usually runs on a Windows Domain Controller (DC). The AD also serves as a
Kerberos Distribution Center (KDC). Because of this, the terms AD, DC, and Windows KDC usually refer
to the same function.
When the Data Domain system is configured for Kerberos Windows authentication, NFS and CIFS clients
and servers use the AD for NFS authentication.
Implementation steps
The following tasks must be performed in order to successfully implement NFS Windows Kerberos
Authentication.
1. First, create the keytab file on the Windows Active Directory for the Data Domain system.
2. After creating the Keytab file, copy it to the /ddvar directory on the Data Domain system. The Keytab
file must be name krb5.keytab.
3. Next, start the Data Domain system configuration process by resetting the authentication Kerberos
configuration to default values by using the authentication kerberos reset CLI command.
4. Now, use the authentication kerberos set realm CLI command to activate Kerberos Windows
authentication. Up to three KDCs may be referenced.
5. Next, import the krb5.keytab file by using the authentication kerberos keytab import CLI
command. This command moves the krb5.keytab file from the /ddvar directory to the /ddr/etc
directory.
6. Now, include the sec=krb5 option when creating an export to invoke Kerberos Windows authentication
for clients attempting to access that shared resource. CIFS clients will also use Kerberos Windows
authentication.
7. Next, verify the operation of Kerberos authentication on the Data Domain system.
8. Finally, verify the krb5.keytab file was deleted from the /ddvar directory.
NFS Authentication (Continued)
A Kerberos keytab file is an encrypted file that is used to enable an NFS server to automatically
authenticate with the Key Distribution Center (KDC). Use the following steps create and import a keytab
file to the Data Domain system:
1. First, create the krb5.keytab file on the KDC.
2. Next, transfer the keytab file to the /ddvar directory on the Data Domain system. This is a sensitive
file, so use a secure transfer method when moving the file into this directory.
3. Now, use the authentication kerberos set realm CLI command to activate Kerberos authentication.
The import process If the kdc-type is windows, the krb5.keytab file should be imported automatically.
The import process moves the keytab file from the /ddvar directory to the /ddr/etc directory. The
/ddr/etc directory is not user accessible, providing the file a high degree of security.
4. If the kdc-type is unix, you must manually import the keytab file using the authentication kerberos
import keytab CLI command.
5. Test the system to make sure it works.
6. Verify the krb5.keytab file has been deleted from the /ddvar directory. This is a sensitive file and
should not be left in a location where it can be compromised.
LESSON SUMMARY - NFS Overview
Describing NFS and NFSv3
Listing NFS configuration steps
Describing the nfs add CLI options
Mounting an NFS export on a client
Enabling authentication for NFS
Knowledge Check
LESSON - NFS Troubleshooting
Using the showmount command for troubleshooting
Diagnosing NFS Connectivity Issues
Resolving NFS Performance Issues
Describing the effect of the hard and soft mount options
Troubleshooting with showmount
You can use the showmount command to verify the NFS configuration of the Data Domain system.
The showmount command provides a list of the remote directories that have been mounted (attached) to
the target system (local or remote) or the list of directories that have been exported (shared) by the
target. The showmount command also lists the clients allowed to mount the exported directory.
The showmount command is available on many clients including Linux and Windows.
On windows clients, it may require the activation of the "Services for NFS" feature. This can be activated
by navigating to the "Control Panel -> Programs and Features -> Turn Windows Features on or off" panel
and enabling this functionality.
Export added when directory doesn't exist
The showmount command can be useful if there is a discrepancy between what the Data Domain system
is reporting and what the client is seeing. For example , if an export is added, but the directory doesn't
exist, the "nfs add" CLI command informs the sysadmin of the problem. If the sysadmin does not see this
warning, they will be unaware of the issue.
CLI fails to show nfsBadTest points to non-existent path
A subsequent use of the "nfs show client" CLI command lists the export, but fails to show that there is a
problem with the path.
Note: The browser-based GUI does show that there is a problem with the path.
Client showmount command cannot see nfsBadTest
If you interrogate the Data Domain system from a client using the showmount command, you will see that
the client does not see the export with the bad path.
As you can see, you should test the validity of the exports from a client to ensure the client can see what
the Data Domain system is configured to advertise.
References
showmount man pages
http://www.unix.com/man-page/freebsd/8/showmount/
Utilities and SDK for Subsystem for UNIX-based Applications in Microsoft Windows 7 and Windows
Server 2008 R2
http://www.microsoft.com/en-us/download/details.aspx?id=2391
Troubleshooting NFS Connectivity
Verify connectivity with ping
When troubleshooting NFS connectivity, you can use the ping and rpcinfo commands.
The ping command can be used to verify general connectivity.
Gather NFS rpc program info from DDSYS
Next, use the rpcinfo command on the NFS client to gather information about the NFS program on the
Data Domain system.
The syntax for the rpcinfo command is shown on the screen.
The -t flag instructs the rpcinfo program to use TCP.
The -u flag instructs the rpcinfo command to use UDP as the transport protocol.
Use both -t and -u flags to verify both transport protocols are functional.
The target_host field is replaced by the IP address or DNS name of the target Data Domain system.
Finally, the rpcinfo command identifies the target service. In the first example on the screen, the NFS
service is targeted. In the second example, the mountd service is targeted.
Gather mountd rpc program info from DDSYS
To gather mountd rpc program information from the Data Domain system, use the rpcinfo command
targeting the mountd program.
Results from rpcinfo command
The rpcinfo command returns information and status messages about the target program. The display
shows what is returned when NFS and mountd are running correctly. The results of the rpcinfo command
also include the rpc program number which, for NFS, is 100003 and for the mountd program is 100005.
References
rpc.mountd man page - http://linux.die.net/man/8/rpc.mountd
rpcinfo man page - http://linux.die.net/man/8/rpcinfo
Troubleshooting rpcinfo Errors
Error - rpcinfo: RPC: Program not registered
If the rpcinfo command returns an error that states that the program is not registered, it may be that the
Data Domain filesystem is down.
Error: RPC: Port mapper failure - RPC: Timed out.
An rpcinfo error that identifies a port mapper failure could indicate the NFS service is not enabled on the
Data Domain system.
Running rpcinfo in BASH
If necessary, the rpcinfo command can be run on the Data Domain system in BASH.
DD OS 5.5.1 introduces support for rpcinfo -s
Starting with DD OS 5.5.1, the -s option is supported.
!# /usr/bin/rpcinfo -s
The -s option causes the rpcinfo program to list the RPC program number, version number, registered
network protocols (or netid), service name, and the owner of the RPC programs running on the Data
Domain system.
The IPv6 netids end with the number 6. The IPv4 netids do not.
NFS Performance Issues
NFS performance problems are most often due to the network or client-side mount options.
Network issues
To diagnose network issues, you can use the net show stats, nfs show stats, sys show stats view net,
and sys show stats view nfs CLI commands.
You can also use network analysis tools such as tcpdump, Wireshark, and iPerf.
mount sync / async option
Client-side mount options can have a significant effect on performance.
For example, the mount command supports a sync option. This option causes all I/O to the Data
Domain system to be be done synchronously. This means every I/O has to be acknowledge by the Data
Domain system before the NFS client will send another request.
If the mount command contains the sync option, consider using the async option
instead. The async option causes all I/O to the Data Domain system to be done asynchronously; that
is that I/O request do not require acknowledgment before the next request is made.
Reference
Mount command - http://ss64.com/bash/mount.html
Hard / Soft Mount Option Issues
Symptom
If performance symptoms include commands like df hanging on the client or the Data Domain system
does not respond to ping, then it may be due to the use of the mount hard option.
Hard Option Effects
The hard option causes the NFS client to retry requests indefinitely. This means that any command trying
to access the exported directory on the Data Domain system will seem to hang as it waits for its request
to complete.
Hard / Soft Mount Option Issue Resolution
Use soft option
The resolution to this problem could be to change the hard option used in the mount command to the soft
option.
Effects of the soft option
The soft option has the following effects:
First, it causes the mount command to fail after the configured number of retransmission requests have
been sent. It can also cause silent data corruption in certain cases.
Use when client responsiveness highest priority
Because of these possible issues, use the soft option only when client responsiveness is higher priority
than data integrity.
Mitigating soft option risks
You can mitigate the risks posed by using the soft option by specifying the mount use NFS over TCP. Use
the tcp mount option if it is supported by your NFS client.
If you choose to still use UDP as the transfer protocol, you can increase the number of times the NFS
client will retransmit a request using the mount command's retrans option. If the retrans option is
supported on your NFS client, try a value of 6 or greater. The default retrans value is 3.
Reference
nfs man page - http://linux.die.net/man/5/nfs
LESSON SUMMARY - NFS Troubleshooting
Using the showmount command for troubleshooting
Diagnosing NFS Connectivity Issues
Resolving NFS Performance Issues
Describing the effect of the hard and soft mount options
Knowledge Check
MODULE SUMMARY - NFS Troubleshooting
You have completed this module. You should now be able to:
Describe NFS
Configure NFS
Troubleshoot NFS
Course Summary
You have completed this course. You should now be able to:
Perform CIFS Troubleshooting
Perform NFS Troubleshooting
Knowledge Check
This concludes the Training. Thank you for your participation.

Data Domain CIFS and NFS Troubleshooting SG PDF

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Data Domain CIFS and NFS Troubleshooting SG PDF

Загружено:

Авторское право:

Доступные форматы

Welcome to the Data Domain CIFS and NFS Troubleshooting Course.

OS Date / Time Timezone

Copyright 2015 EMC Corporation. All rights reserved. EMCNFS

C:>wmic useraccount wherename='%username%'getsid

bg This option causes the device to execute a background mount.

Вам также может понравиться