Best Practices for

BlackBerry
Administrators

Phillip Lundie
Regional Technical Manager
Research In Motion
Agenda

• BlackBerry Administrator Tour

– IT Policies
– Role Based Administration
• BlackBerry® Enterprise Server Logs
• Troubleshooting: Enterprise Activation Issues
• Maintenance Recommendations
Administrator Tour
BlackBerry Manager initial view
Administrator Tour
IT Policies

• Over 360 Policies as of BlackBerry Enterprise

Server 4.1.4
• Policy/Regulatory Compliance
– Disable Cut/Copy/Paste
– Disable Forwarding Between Services
– Disable Alternate Or Un-Audited Communications
• PIN to PIN, SMS, BlackBerry Internet Service™
• BlackBerry® Smartphone Security
– Force Content Protection
– Disable Persisted Plaintext
– Enforce Password Rules
Administrator Tour
IT Policies

• Allow Outgoing Call When Locked

– Specifies whether users can place calls when the device is
securely locked
• Disable Forwarding Between Services
– Prevents the user from forwarding or replying to a message via a
different BlackBerry Enterprise Server than the one that delivered
• Confirm On Send
– Requires users to confirm before sending an email, PIN, SMS, or
MMS message
• Disable IP Modem
– Disables the IP modem feature on applicable device
• Disable JavaScript in Browser
– Disables execution of JavaScript scripts in the browser.
Administrator Tour
IT Policies suggestions

• Leave the default policy alone or have a blank ready

• Create new policy with your corporate policy
– Policies cannot be removed once loaded (Kill or Battery reset)
• Work with your security team to match corporate policy
– Our policies are designed to enable you to mirror your corporate
regulations
• Beware of the effect of certain policies
– i.e., enforcement of content protection stops the ability to change
password
• Begin with a test group before implementation

• Review all available policies

– www.blackberry.com/knowledgecenterpublic/livelink.exe?func=ll&o
bjId=1139827&objAction=browse&sort=name
Agenda

• BlackBerry Administrator Tour

– IT Policies
– Role Based Administration
• BlackBerry Enterprise Server Logs
• Troubleshooting: Enterprise Activation Issues
• Maintenance Recommendations
Administrator Tour
Role Based Administration

• Functions of administrative roles

– Reduce security risks
– Reduce operational risks
– Better distribute job responsibilities
– Improve accessibility of options needed to do the job
– Accommodate mobile support personnel
• When you start the BlackBerry Manager, it checks
your authentication credentials, determines your administrative role,
and then displays a list of the tasks that you can complete
Administrator Tour
Role Based Administration
Security administrator (rim_db_admin_security):
They are the only administrators who can manage role membership and change sensitive
security properties, such as licenses and encryption keys
Enterprise administrator (rim_db_admin_enterprise):
Can perform all tasks that relate to user accounts, services, BlackBerry Enterprise Servers,
and global application data. These administrators cannot view role membership, licenses, or
encryption keys.
Device administrator (rim_db_admin_handheld):
Can perform all tasks that relate to user accounts and BlackBerry smartphone management,
including supporting new user accounts, implementing BlackBerry smartphones, managing
software configurations, and managing the applications installed on BlackBerry smartphones
Senior help desk administrator (rim_db_admin_sr_helpdesk):
Can perform all user account management tasks, including adding, moving, and deleting user
accounts, changing IT policy assignments, and issuing IT administration commands
Junior help desk administrator (rim_db_admin_jr_helpdesk):
Can perform user account management tasks, including creating and sending wireless
enterprise activation passwords, and resending service books or IT policies. These
administrators cannot add, move, or delete user accounts or issue certain IT administration
commands.
(rim_db_admin_audit_<role>):
_ Use this view-only access to each role when training new administrators
Agenda

• BlackBerry Administrator Tour

– IT Policies
– Role Based Administration
• BlackBerry Enterprise Server Logs
• Troubleshooting: Enterprise Activation Issues
• Maintenance Recommendations
BlackBerry Enterprise Server Logs
Logs 101

• When to use logs?

– Use the logs for more than just finding out what went wrong
• What is logged?
– Everything! (or just about …)
– Sometimes you have to know what you are looking for (i.e., troubled
user, narrow down issue according to component and relevant log)
• Why use the log files?
– An easily accessible information repository
– Be proactive and recognize a potential issue
– Understand activity that happens on the server
• Where are the logs stored?
– Program Files/Research In Motion/BlackBerry Enterprise
Server/Logs
– Location editable via BlackBerry Server Configuration
BlackBerry Enterprise Server Logs
Logs 101 (Cont…)

• Logs for:
– Alerts
– Attachment Service, Dispatcher, Router, Controller
– Mailbox Agent, Manager, Policy Service, Synchronization Service,
Notes Connecter
– Backup Connecter and Management Connecter
• Logs are stored by date
• Logs require 60 Gig of free disk space
– Depends on Log Level, number of users and how long you retain
logs
• Plan on routine maintenance depending on your deployment
BlackBerry Enterprise Server Logs
Entry breakdown

Event ID Date/Time Thread ID Description

[30068] (04/13 00:03:06):{0x680} User auser@xyzcorp.com starting

up ...
• Event ID – indicates the severity of the debug log entry
[1####] = Error condition (Always Logged)
[2####] = Warning
[3####] = Informational
[4####] = Debug
[5####] = Other (Always Logged)
• Thread ID – identifies the application thread that is performing the particular
operation
• Description – outlines the detail of the work being performed
BlackBerry Enterprise Server Logs
Possible Error entries

• [1####] – Error condition

[10000] (04/20 18:17:19):{0x18C8} 400 - No Data to Update for 'ServerConfig‘
• [2####] – Warning
[20000] (04/18 16:09:03):{0x968} SRPClient::Connect: Error calling host
localhost" [127.0.0.1] (10061)
• [3####] – Informational
[30222] (04/19 14:56:22):{0x119C} {Administrator} MTH:
contentType=OTAKEYGEN, sizeOTA=216, sizeOTW=216,
TransactionId=1495007232, Tag=230
• [4####] – Debug
[46052] (04/19 14:57:40):{0x744} [SYNC-DSession] Contain 0 record(s) and
70 command(s) from Backup.Options/Browser Folders/Recipient
Cache/Browser Options/Profiles Options/Phone Hotlist/Profiles/Tasks
Options/Attachment Options/Key Store Options/Phone Call Logs/WTLS
Options/Default Service Selector/Browser Push Options/Quick Contacts...
[Administrator:10]
• [5####] – Other
[50096] (04/20 17:55:46):{0xEF8} [SRP] Dispatcher\SRP Connection
established
BlackBerry Enterprise Server Logs
Error Entries to look for…

• Important Log Entries to become familiar with:

– SRP Connectivity Issues
– Mail Server Connectivity Issues
– Memory Dumps/Exceptions
– User Configuration Issues
– Delayed Mail Delivery
• Common Server Routing Protocol (SRP) Connectivity Errors
– Connection time out (10060)
– Connection reset by peer (10054)
– Host not found (11001)
– Connection refused (10061) – 5 in 1 rule
BlackBerry Enterprise Server Logs
Managing/Tracking Message Flow

• Setting log levels and options

– Router
– Dispatcher
– Syncb
– IT Policy
– Message Agent
• Lets look at some examples
 BlackBerry Enterprise Server Logs
Track incoming contact
Contact is received by ROUTER:
[40000] (04/18 08:43:15):{0x1234} [SERVICE_RELAY_SESSION:T35723298:0079caf0]Relay GME packet
received. DESTINATION=T35723298, CONTENT=sync, TAG=-128926606,
SERVICESESSION_TAG=3868648, LENGTH=186

Contact packet is passed to the Dispatcher and relayed to the Sync Service:
[30222] (04/18 08:43:15):{0xFB0} {Cormier, Marc} MFH: contentType=sync, sizeOTA=146, sizeOTW=86,
TransactionId=954506655, Tag=3868648
[40324] (04/18 08:43:15):{0xB64} [BIPPe] (T35723298:C3,sync) Datagram SENT (3868648:120)

Sync Service receives the datagram and processes the information returning an acknowledgement back to the
Dispatcher:
[40000] (04/18 08:43:15):{0xBF8} [BIPP] Received datagram, Tag=3868648
[46008] (04/18 08:43:15):{0xAE8} [SYNC-DSession] Send 1 packet(s) with 8 bytes to the device. [Cormier,
Marc:1, SID=740677222, CLID=13, ECLID=2, TAG=15, ST=20]

Dispatcher relays the acknowledgement back to Router to send to handheld:

[30222] (04/18 08:43:15):{0x2CC} {Cormier, Marc} MTH: contentType=sync, sizeOTA=82, sizeOTW=8,
TransactionId=15, Tag=29

Router receives packet and transmits ACK back to the handheld and clears the transaction:
[40000] (04/18 08:43:15):{0x1234} [SERVICE_RELAY_SESSION:T35723298:0079caf0] Forwarding STATUS
packet to relay. DESTINATION=40077FD8, TAG=-128926606, SERVICESESSION_TAG=3868648
[40000] (04/18 08:43:16):{0x1234} [SERVICE_RELAY_SESSION:T35723298:0079caf0] (relay) Clear service
transaction. TAG=29
BlackBerry Enterprise Server Logs
Track New IT policy assignment

Policy Service:
[30000] (04/18 16:21:32):{0xA98} {bob@jwall.testnet.rim.net, PIN=40077FD8,
UserId=1}RequestHandler::SendQueuedITAdminCommandToDevice Sending data to device,
contentType=ITADMIN, size=310, RefId=0, TransactionId=-1002451416, Tag=4
….
[40000] (04/18 16:21:53):{0xAA4} {bob@jwall.testnet.rim.net, PIN=40077FD8,
UserId=1}RequestHandler::HandleITADMINDataCommand - ITPolicy Success Ack for the
command SET_IT_POLICY_COMMAND - Processing packet, Tag=3919184
Dispatcher:
[40329] (04/18 16:21:32):{0x974} [BIPPe] (T12855258:C3,ITADMIN,APPD,SERVICE_BOOK) Datagram
RECEIVED (4:400)
[30222] (04/18 16:21:32):{0x978} {Bob} MTH: contentType=ITADMIN, sizeOTA=314, sizeOTW=272,
TransactionId=-1002451416, Tag=23
….
[40700] (04/18 16:21:53):{0x998} {Bob} Receiving packet from device, size=93, TransactionId=-
1960003290, Tag=3919184, content type=ITADMIN, cmd=0x3
Router:
[40000] (04/18 16:21:32):{0x10A8} [SERVICE_RELAY_SESSION:T12855258:0079e4e0] Service V2
GME packet received. DESTINATION=40077FD8, CONTENT=ITADMIN, TAG=23,
RELAYROUTABLE=true, LENGTH=354
….
[40000] (04/18 16:21:53):{0x10A8} [SERVICE_RELAY_SESSION:T12855258:0079e4e0] Service
STATUS packet received. SERVICESESSION_TAG=3919184, RESULT=1
BlackBerry Enterprise Server Logs
Outgoing E-mail
[40000] (04/19 16:49:50):{0x1428} {Administrator/CSOA}
CN=R7MAIL1/O=CSOA!!mail\administ, fetching modified documents since 04/19/2006
04:49:32 PM

[40000] (04/19 16:49:50):{0x1428} {Administrator/CSOA} {Administrator/CSOA} Constructing

message (CMIME) (msgType=mail), size=175, RefId=-1514701180, TransactionId=0,
Tag=130, PHXCAP=0x00000005, PHXCFG=0x00000000, PHXMTR=0x00000000

[30305] (04/19 16:49:50):{0x1428} {Administrator/CSOA} Message sent to handheld (PIN

203B37B3, "mail\administ" on CN=R7MAIL1/O=CSOA): folder "($Inbox)", posted date
04/19/2006 04:51:01 PM, added date 04/19/2006 04:49:32 PM, TID=130, RID=-
1514701180, NID=B4A

[40000] (04/19 16:49:50):{0x1388} [BIPP] Send data, Tag=130

[40000] (04/19 16:49:52):{0x16CC} [BIPP] Received status DELIVERED, Tag=130

[30302] (04/19 16:49:52):{0x1428} {Administrator/CSOA} SRP: TID=130, RID=-1514701180,

NID=B4A, type MAIL returned DELIVERED
BlackBerry Enterprise Server Logs
Database Connectivity Issues

#1
[10249] (02/26 18:52:32) {0xAC3} UPDATE UserConfig SET DeviceType=3, PIN=‘10000000’ WHERE
Id = 1: COM Error 0x80004005 – Unspecified error – Source:”Microsoft OLEDB provider for SQL
Server” – Description “Transaction (Process ID 330) was deadlocked on lock resources with
another process and has been chose as the deadlock victim. Rerun the transaction” –
Command “”
[20061] (02/26 18:52:32) {0xAC3} Exception caught for PMDatabaseSQLImp::GetAgentIDMap

#2
[40000] (04/18 01:58:00):{0xFF4} [ODBCConnection::Connect] DB=SQL Server,
LoginConnectionTimeOut=30

#3
[14089] (04/18 09:05:06):{0x1D54} {Exception} {Database} System.Data.SqlClient.SqlException:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server
is not responding.
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior cmdBehavior,
RunBehavior runBehavior, Boolean returnStream)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at DBLibrary.SyncDatabase.ExecuteReader(String sSQL, SqlDataReader& oDR)
BlackBerry Enterprise Server Logs
Events to look out for…

BlackBerry Enterprise Server IBM Lotus Domino:

– 20089 (open failure)
– 20445 (connection to mail server %s is restored)
– 20446 (connection to mail server %s appears to be
dropped)
– 20143 (mail file is unavailable for OTAFM)
– 30392 (Scanning, but still in backoff mode)
BlackBerry Enterprise Server Logs
Words to look out for…

• Non case sensitive searches work best

• Log verbiage may change over service packs or
new Release (4.0.x vs. 4.1x)
• Words
– fail
– error
– interrupted
– Incomplete
– unsuccessful
Additional BlackBerry Troubleshooting Tips

• Collecting BlackBerry smartphone logs:

–From the smartphone:
•Alt LGLG, click and choose “copy day’s content”
•Next have the user compose an email and paste the contents of the
log in the body.
–From a PC:
•Using Javaloader Utility, javaloader –u eventlog > c:\filename.txt
•This file can then be collected and submitted for further investigation.
Additional BlackBerry Troubleshooting Tips

• Using BlackBerry autotext to collect information:

–Software version: myver (8800/4.2.1.74)
–BlackBerry smartphone PIN: mypin (pin:20021854)
–BlackBerry smartphone number: mynumber (555-987-
6543)
Agenda

• BlackBerry Administrator Tour

– IT Policies
– Role Based Administration
• BlackBerry Enterprise Server Logs
• Troubleshooting: Enterprise Activation Issues
• Maintenance Recommendations
 Enterprise Activation Issues

• The first step to identifying the issue is determining

at which point the Enterprise Activation is failing.

• By separating the Enterprise Activation into 4

stages, the problem can be identified much more
efficiently.
–Stage 1: Starting the activation
–Stage 2: Verifying encryption
–Stage 3: Receiving services
–Stage 4: Slow synchronization
Enterprise Activation Issues:
Stage 1 - Starting Activation

• Problem Scenario A
• The ‘Activating...’ status displayed on the BlackBerry
smartphone does not change.
• Every ten minutes a ‘Retrying...’ status persists
• After forty minutes, the process times out with one of the
errors below:
– ‘The server is not responding. Please contact your System
Administrator‘
– ‘An error has occurred. Please contact your System
Administrator’
Enterprise Activation Issues:
Stage 1 - Starting Activation

• Problem Scenario A
• Known reasons for the errors encountered
–One or more incorrect email addresses were typed on the
BlackBerry smartphone multiple times (review the
address)
–The BlackBerry smartphone user’s account mailbox is full
–The mail server is unable to send or receive messages
–The activation attachment is being stripped
Enterprise Activation Issues:
Stage 1 - Starting Activation

• Problem Scenario B
• The Enterprise Activation status remains on
‘Activating…’ and does not time out
–Is the BlackBerry smartphone responding to other events?
–Can the affected user register on the wireless network?
–Is the user in an area of adequate wireless coverage?
–Has their BlackBerry smartphone been provisioned
properly by the wireless carrier?
Enterprise Activation Issues:
Stage 1 - Starting Activation

• Problem Scenario B
• Does the BlackBerry® Enterprise Server Admin
account have the right permissions to retrieve the
ETP.DAT attachment?
–Check for user-defined filters, spam control or junk email
rules blocking the ETP message from arriving in the inbox
–Confirm that the BlackBerry smartphone has the correct
signal type and sufficient signal strength to send data
• Test BlackBerry smartphone PIN messaging to confirm this
Enterprise Activation Issues:
Stage 2 - Verifying Encryption

• Issue with Advanced Encryption Standard (AES):

–If the BlackBerry Enterprise Server is set to accept only
Advanced Encryption Standard (AES) encryption you may
get the following error:
–Activation error: Contact Service Administrator
•BlackBerry® Device Software v4.0 only supports Triple Data
Encryption Standard (DES) encryption for activation
•Set the BlackBerry Enterprise Server to accept Triple DES
encryption.
•Move the user to a BlackBerry Enterprise Server that accepts Triple
DES encryption.
Enterprise Activation Issues:
Stage 3 - Receiving Services

• The BlackBerry smartphone stops responding:

–This issue may occur if the BlackBerry Policy Service is
not running or encountered a problem
•The BlackBerry Policy Service cannot send the IT policy or service
books to the BlackBerry smartphone

–The enterprise activation process stops responding

because the BlackBerry Enterprise Server has network
connectivity issues or Microsoft® SQL Server errors.
•Check the application log on the BlackBerry Enterprise Server for
errors.
Enterprise Activation Issues:
Stage 3 - Receiving Services

• The BlackBerry smartphone rejected the IT policy

pushed from the BlackBerry Policy Service:
–The BlackBerry smartphone has a current IT policy from
another BlackBerry Enterprise Server.
•Use the Wipe Handheld function on the BlackBerry smartphone to
remove the policy key, so the BlackBerry smartphone will not reject a
policy from a different BlackBerry Enterprise Server.
•After the Security Wipe is complete reactivate the BlackBerry
smartphone
Enterprise Activation Issues:
Stage 4 – Slow Synchronization

• There are several potential problems that may occur

during the Slow Synchronization stage.
–The BlackBerry Synchronization Service is not running.
–The BlackBerry Synchronization Service does not start
because the Microsoft XML Parser (MSXML) is not
installed.
–The slow synchronization process stops responding
because the BlackBerry Enterprise Server has network
connectivity problems or Microsoft SQL Server errors.
Agenda

• BlackBerry Administrator Tour

– IT Policies
– Role Based Administration
• BlackBerry Enterprise Server Logs
• Troubleshooting: Enterprise Activation Issues
• Maintenance Recommendations
Monthly Maintenance
Distributed Components

BlackBerry Router Attachment Service Database

• Check logs for errors • Check logs for errors • Check Event Viewer
• Check for any SRP • Monitor and document for errors
disconnects deviations in Attachment • Run ODBC
• Do disconnects Service performance via connectivity checks
coincide with network Perfmon • Monitor and document
maintenance routines deviations in SQL
Server performance
via Perfmon
Weekly Maintenance
Cleanup

• Compact all BlackBerry-related databases regularly using the IBM®

Lotus® Domino® server Compact utility with the following settings:
– “-B” option
• recovers unused space
• helps maintain database integrity
• reduces on-disk file size
• allows access to the database while the Compact program runs
Note: If the IBM Lotus Domino server is transaction logged, perform a full
database backup soon after you run the Compact –B command
– BlackBerry Enterprise Server parameter
• compacts all databases in the Lotus/Domino/Data/BlackBerry Enterprise Server
directory
– Interval
• BES compacts state databases after every prune, so schedule weekly during
low traffic time
Weekly Maintenance
BlackBerry Infrastructure

BlackBerry Enterprise Server

– Ensure disabled users are removed from BlackBerry Enterprise Server
– BlackBerry Enterprise Server performance can be impacted by users who have been
disabled yet remain on the server
– Document Firewall/Proxy rules
– User inactivity should be monitored
• 30 days of inactivity - Wireless Reconciliation should be disabled
• 60 days of inactivity - User should be disabled
• 90 days of inactivity - User should be removed
Database Server
– Check configuration database growth patterns
– Document SQL maintenance scripts
– Backup database on a daily and weekly basis
Firewall
– Document rules
– Document all BlackBerry Enterprise Server IP addresses
– Document Proxy rules and authentication info
Routine Maintenance
Daily
• Check for errors
– Log files
– Event viewer
– BlackBerry Manager Console
• Backups
– SQL Database – configuration information, wireless backups
• MSDE – BlackBerryDBBackup.exe
• SQL – 3rd party backup software
– Users Mailboxes – hidden folder with BlackBerry info
• BlackBerryBackup.exe
• 3rd party backup tools
• BlackBerry Enterprise Server – server config in registry
– Soft and/or hard copy documentation
• Update Documentation
– Network Diagrams
– BlackBerry Enterprise Server Environment
– Messaging Environment
– Firewall/Proxy
• Server Maintenance
– Add users to appropriate BlackBerry Enterprise Server based on Messaging Server/BlackBerry
Enterprise Server location
– Check log file size
– Run Perfmon monitors to check CPU, Memory and Disk performance
Best Practices for BlackBerry Administrators
Resources

• System Administration Guide

– http://www.blackberry.com/btsc/search.do?cmd=display
KC&docType=kc&externalId=9174614414&sliceId=&dial
ogID=86348324&stateId=1 0 86346378
• BlackBerry Enterprise Resource Kit
– https://www.blackberry.com/BRK/entryPoint.do
• BB Resource Kit Library
– https://www.blackberry.com/BRK/entryPoint.do Æ View
Related Documentation
Questions?

Thank you for attending! Be sure to join the other BlackBerry sessions
and events happening at Lotusphere – pick up your schedule from
our booth.

BlackBerry VIP Customer Reception – Tuesday night, 6pm, Grand

Harbor South (Yacht & Beach Hotel)

Session Evaluations: complete our session evaluation to receive a

complimentary BlackBerry Application card that enables you to access
free downloads, discounted rates or trial periods for applications from
select BlackBerry Alliance Members.

For more resources, please visit:

www.blackberry.com/go/lotusphere

Download the BlackBerry presentation schedule application:

http://lotusphere.turtleweb.com/rim