Table of Contents

Domino Domain Monitoring Workshop..................................................................................... 1 Workshop Overview ........................................................................................................... 1 Lesson 1: Overview ........................................................................................................... 4 Lesson 2: Introduction to DDM.......................................................................................... 5 Lesson 3: Probes ................................................................................................................. 9 Lesson 4: Collection Hierarchy ........................................................................................ 16 Lesson 5: User Interfaces.................................................................................................. 18 Lesson 6: Event Filtering .................................................................................................. 22 Self-service resources on the Web:................................................................................... 23

Domino Domain Monitoring Workshop Workshop Overview

Introduction

This workshop introduces you to Domino Domain Monitoring, commonly abbreviated as DDM. DDM is a new feature in Lotus Domino 7 intended to provide a simple, flexible and powerful interface for Domino administrators to monitor the servers in their domain.

Objectives

After completing this workshop, participants will be able to Enable and modify the default probes included in the Monitoring Configuration database (events4.nsf). Create, modify and delete probes. Design and implement a Server Collection Hierarchy for your domain. Explain the purpose of the DDM database. Explore the views available to view the event documents. Assign events to different administrators in your domain. Allow authorized users and restrict other users from working with events in the DDM database. Demonstrate how to close events in the DDM database. Demonstrate how to create, modify and delete DDM Filters.

The Philosophy of a Workshop

The philosophy behind a workshop is that less structured learning (when used in the appropriate situations) leads to more actual learning. In a workshop, you figure out many things on your own, using your research and testing skills. The goal is for you to feel confident that you figured out many of the key items on your own and that you are capable of repeating those key steps again back at your desk, whether it is next week or six months from now.
Continued on next page

Last edited 12/20/2005

IBM 2005

Workshop Overview, Continued

Requirements

The training room setup requirements may vary depending on how many people will be participating in this workshop, but all participants will need the following: A Domino 7 server that is a member of a domain with at least 2 other servers A Domino 7 Administrator client and Notes client A user id for at least one user who has rights to administer the domain Access to the Domino Administrator Help database

If a training room is being used, it is recommended that all servers be in the same Domino domain. If separate domains are used, ensure that there are at least 3 servers per domain. More servers per domain are preferred, because it allows for better simulation of real world Domino domains. The required server and client can either be set up in the classroom ahead of time, or set up by the participants in the workshop. Each participant should have a unique user who is listed as a Manager in the Domino Directory with all Administration roles. Note: The Server documents in the Domino Directory need the field Server Build Number to be populated with a Domino 7 build number. To accomplish this, replicate the Administration Requests database to the Administration server for the Domino Directory and run the command tell adminp process all. Once the Server Build Number field has been populated on all Server documents, replicate the Domino Directory with the other servers in the domain. If the servers are being set up ahead of time, this can be accomplished by setting up Connection documents that replicate names.nsf and admin4.nsf regularly between the first server in the domain and servers set up as additional servers. The Server Build Number field will be populated automatically.
Continued on next page

Last edited 12/20/2005

IBM 2005

Workshop Overview, Continued

In this Course

This workshop is divided into the following lessons: Lessons Lesson 1: Overview Lesson 2: Introduction to DDM Lesson 3: Probes Lesson 4: Collection Hierarchy Lesson 5: User Interfaces Lesson 6: Event Filtering See Page 4 5 9 16 18 22

Timing

This workshop takes approximately 4.5 hours to complete.

Last edited 12/20/2005

IBM 2005

Lesson 1: Overview

Introduction

Domino Domain Monitoring was created in response to feedback to a Domino administrators survey taken in 2003. Administrators were pleased with new tools and features in Domino 6 that lowered the total cost of ownership but felt that monitoring and troubleshooting servers still takes too much time. Monitoring features in Domino 7 were designed with this feedback from customers in mind and the result was the new set of features known as Domino Domain Monitoring or DDM. Domino provides several tools which allow for monitoring of Domino servers. While they all provide a mechanism for monitoring Domino servers, they each have their own role to play.

DDM v. stats/events

Stats and Events, which was the mechanism introduced in 4.x, relies on Event Handlers, and the event task, and reports to a database called statrep.nsf. DDM also recognizes these events, but does not replace the event handlers that are already configured.

DDM v. ITALD

The IBM Tivoli Analyzer for Lotus Domino, or ITALD, was a separate product available in Notes/Domino 6.x, and includes two features: Activity Trends and Server Health Monitoring. Activity Trends is a tool for analyzing historical data to prevent future problems. DDM does not include the analytical features of ITALD, and it is focused on problems as they exist now.

DDM v. TAME

Tivoli Autonomic Monitoring Engine (TAME) is a part of IBMs overall Autonomic Computing initiative. An effort to integrate TAME was part of Notes/Domino 7 development. As part of that effort, TAME plugins were developed as operating system probes that monitor Disk, CPU, Memory, and Network utilization. These probes do double duty as TAME plugins and as the DDM Operating System probes, which will be covered in the section on probes below.

Last edited 12/20/2005

IBM 2005

Lesson 2: Introduction to DDM

Introduction

This workshop focuses only on DDM and its features, and is not intended to cover all of the possibilities for monitoring Domino servers.

DDM architecture

Domino Domain Monitoring can be broken into four main architectural components: Probes organized by server features Collection Hierarchy Event filtering (DDM and non-DDM) User Interfaces

Note: DDM runs on top of the Event task, so the Event task must be running for DDM to work.

Probes

Probes do the majority of the work of DDM. Domino has a default set of DDM Probe documents that are shipped with the product. These documents contain the following information about the DDM probes: A general description of the probe, including its purpose and intended use. Probe type and probe subtype, for example, messaging is a probe type and its associated probe subtype could be router process state. This combination of probe type and probe subtype creates a mail router status probe. Configurable specifics on what the probe monitors and how it should report, for example, thresholds for events it generates. Configurable probe targets, that is, which servers, database, etc., that the probe runs against. Where applicable, configurable scheduling information. You can use the default Probe documents with their default settings to get you "up and running" with DDM or you can configure probes to run in a manner that is specifically tailored to your organization by creating new Probe documents in which you define and customize your own probes.
Continued on next page

Last edited 12/20/2005

IBM 2005

Lesson 2: Introduction to DDM, Continued

Probe Types

Probes are organized by the functional area of Domino that they monitor. The nine probe types are: 1. 2. 3. 4. 5. 6. 7. 8. 9. Application code Database Directory Messaging Operating System Replication Security Server Web

Scheduling Probes

Probes run according to a default schedule, a schedule that you specify, or on an as-needed basis as determined by the probe and the events on the server. You can schedule probes to run daily, weekly, on a specific day during the month, or at almost any other time. You also specify the repeat interval for the probe. You can schedule probes to run on individual servers or on groups of servers.

Collection Hierarchy

The probes report their results to the Domino Domain Monitoring database (ddm.nsf). Instead of administrators having to open the ddm.nsf on each server to observe the probe results, they can define a collection hierarchy. Once one or several collection hierarchies are defined, the probe results are still reported to each servers own replica of ddm.nsf, but those results are then replicated to servers designated as collection servers. This process is referred to as data roll-up. One collection server can then roll up its results to another collection server higher in the hierarchy. Multiple collection hierarchies can be set up in a domain, and the collection servers can differ between multiple collection hierarchies.
Continued on next page

Last edited 12/20/2005

IBM 2005

Lesson 2: Introduction to DDM, Continued

Example:

Collection Hierarchy

In this three-tiered example, there are three collection servers represented by rectangles, and six servers running DDM which are not collection servers, represented by ovals, but from which DDM event information is being collected. Collection Server 2 (CS2) collects and reports information about itself and the non-collection servers S1 and S2. CS3 collects and reports information about itself and the non-collection servers S4, S5 and S6. CS1 collects and reports information about every server shown, since it collects and reports information from CS2 (and its servers S1 and S2), CS3 (and its servers S4, S5 and S6) and the non-collection server S3. In this example, CS1 represents the top tier. The middle tier contains CS2, S3 and CS3. The lower tier contains S1, S2, S4, S5 and S6. The information reported exists in the ddm.nsf on each DDM server, that is, a server running DDM probes or collecting DDM information or both, and contains information about the server it is on and the servers below it in the collection hierarchy. For example, ddm.nsf on CS2 would contain information about itself and servers S1 and S2. To view just the information for CS2 and the servers from which it is collecting, open ddm.nsf on the CS2 server. In this example, opening ddm.nsf on CS1 would reveal DDM event information about this entire domain. There are many ways to design a DDM server collection hierarchy. For example, you can set it up to represent an existing Domino server topology, for example, hub and spoke servers. Or, you can set it up to ensure that the administrator receives only information that is pertinent to the servers he or she maintains.
Continued on next page

Last edited 12/20/2005

IBM 2005

Lesson 2: Introduction to DDM, Continued

User Interfaces

The UI for DDM is made up of two databases, events4.nsf and ddm.nsf. The purpose for each database is summarized below: Database Monitoring Configuration: events4.nsf Domino Domain Monitoring: ddm.nsf Function This database is used for configuring probes and DDM Filters, and contains all events and formulas for how corrective actions are completed. The Domino Domain Monitoring database is where the results of probes are reported. They can be viewed in several different ways, such as by severity, date, etc. This database is also where events can be assigned to administrators.

Event Filtering

DDM event filters determine the type and volume of event information that is reported to the Domino Domain Monitoring database (ddm.nsf). Filtering is important in cases where an administrator might be seeing too much or too little information. For example, DDM Filters can be applied to either simple events or to both enhanced and simple events*. Filters can be configured to log some or all event types and severities. And filters can be configured to run on specific servers in the Domino domain (or all servers in the domain). * There are two kinds of events: Simple & Enhanced. Simple events are those events without specific target information. Generally, this includes events that are reported to the server console. Enhanced events are those events which are: generated by a DDM Probe Configuration document generated by a Domino Event Generator any other event that contains specific target information (for example, server, database, or agent that generates the event) Note: Filters will be applied to new Event documents generated after a Filter document is saved. Existing Event documents in the Domino Domain Monitoring database will be unaffected by new filters created after an event has been reported.

Last edited 12/20/2005

IBM 2005

Lesson 3: Probes

Introduction

In this lesson, you will learn about probes. Probes actually do the work behind DDM.

Objectives

Upon completion of this lesson you will be able to Enable the default probes in the Monitoring Configuration Database Modify the default probes to change their schedule or other configuration options Create, modify and delete new probes when the default probes do not satisfy your needs Identify which probe to use to address a specific need

Exercises

In the following series of exercises you will enable and configure probes. While completing these exercises, you may consult the following: Lotus Domino support page http://www.ibm.com/software/lotus/support/domino/support.html Domino Administrator Help Hint: Most of the documents relevant to this lesson can be found in Domino Administrator Help under the Contents view, in the Monitoring Section.

Exercise 3.1 enabling probes

To enable probes: Step 1 2 3 4 5 Action In the Domino Administrator, select the Files tab. Open the Monitoring Configuration database (events4.nsf). Select DDM Configuration., DDM Probes, By Type Click the action button Enable Probes. Choose Enable All Probes in View. (A message that some probes could not be enabled will occur; this is normal, as some probes require more information, such as a server name, to be filled in; click OK on this message.)
Continued on next page

Last edited 12/20/2005

IBM 2005

Lesson 3: Probes, Continued

Exercise 3.2 modifying probes

To modify the default probes:

Step 1 2 3 4 5 6

Action In the Domino Administrator, select the Files tab. Open the Monitoring Configuration database (events4.nsf). Select DDM Configuration, DDM Probes, By Type Find a probe with a clock icon to the left of the Probe Description column. Edit the Probe document. On the schedule tab, modify the field How Often should this probe run? While changing this field, answer the following question: If the server is down when a probe is supposed to run, which scheduling choices result in being able to specify how soon the probe should be run, and what is the result of each choice?

Exercise 3.3 create probes

If the default Probes do not meet your needs, you can create your own probes of any of the nine feature areas. In the following exercise, you will create a replication probe to monitor replication of the Domino Directory: Step 1 2 3 4 5 6 Action In the Domino Administrator, select the Files tab. Open the Monitoring Configuration database (events4.nsf). Select DDM Configuration. Choose any DDM Probe view. Click the Action button New DDM Probe. Choose Replication.
Continued on next page

Last edited 12/20/2005

10

IBM 2005

Lesson 3: Probes, Continued

Exercise 3.3 create probes (continued)

Step Action 7 On the Basics tab, set the fields to the following values: Field label Probe Subtype Probe Description Value Scheduled Replication Checks Descriptive text; this can be anything, but you must supply a description in order to save the document. Only the following Servers: <your servername> Note: The server chosen here must be a Domino 7 server. Only the following Servers: <your servername> names.nsf Leave blank Change to Failure 1 hours

Which Servers should run this Probe

Server(s) with which the specified database(s) must replicate Select one or more databases to probe Select one or more databases not to probe Severity Generate an event if the included databases have not replicated within the following interval: Check the following modes of replication:

Attempt to diagnose problems 8

Leave both Push and Pull checked (Often replication is set to only push or pull, in that case, checking only the appropriate choice will lead to the most accurate results of this probe) Leave Checked

On the Schedule tab, set the probe to run Multiple Times per day, every 5 minutes.
Continued on next page

Last edited 12/20/2005

11

IBM 2005

Lesson 3: Probes, Continued

Exercise 3.3 create probes (continued)

Step 9 10

Action Save and exit the new document. Locate the document in the All by Author view and ensure it is enabled.

Exercise 3.4 exploring probes

In this exercise, you will be presented with a specific area you might like to monitor using DDM, or a situation in which you might suggest a particular type of DDM probe as a uniquely appropriate solution. Fill in the table with the Probe Type and Probe Subtype. Use the Notes column to provide any detail of specific fields that need to be filled out to achieve the desired result. You may use any available resources to answer the questions; in particular, the Administrator Help document Creating and Configuring DDM probes, and the documents it links to, will be useful. You may also want to look at the default Probe documents and use the field help in those documents in the Monitoring Configuration database (events4.nsf).

What to Monitor/situation.
Ex.

Probe Type Database

Probe Subtype Compact

Notes Set the Which servers should run this probe to Special Target Servers Mail Server.

You would like to monitor for any errors encountered when compacting databases on mail servers in the domain. You have a Discussion Database located on several servers in the domain, but you suspect that only a few of the replicas are being actively used. Suggest a probe that would help monitor which replicas are least actively being used.

Continued on next page

Last edited 12/20/2005

12

IBM 2005

Lesson 3: Probes, Continued

Exercise 3.4 exploring probes (continued)

What to Monitor/ situation. Probe Type 2 Which Probe type would you configure to monitor which AdminP requests are being generated, and whether they are being processed? You have had several web server issues which resolved through several changes to the server documents. As part of the corrective action, you implemented a default configuration for all Web servers in your domain. You would like to find a way to ensure that if someone changes the Server document Web Server settings, you will know about it and can easily change it back. Which DDM probe would you use to help with this? You would like to set up a probe that will tell you which documents are not replicating in the Domino Directory.

Probe Subtype

Notes

Continued on next page

Last edited 12/20/2005

13

IBM 2005

Lesson 3: Probes, Continued

Exercise 3.4 exploring probes: (continued)

What to Monitor/situation. 5 You would like to monitor whether your IMAP servers are listening on the appropriate ports, which probe type will accomplish this? How often does the probe run by default? You want to monitor if name lookups are taking too long. Which probe type would you recommend to do this? You would like to monitor for Web agents that take longer than 2 minutes to complete. Which probe type would you recommend? Also in the Notes column, answer the questions: a) Will the default probes work for this, or do you need to modify the default or create your own probe? b) Could you monitor whether an agent run by Agent Manager took longer than 2 minutes to complete?

Probe Type

Probe Subtype

Notes

a)

b)

Why or why not?

Continued on next page

Last edited 12/20/2005

14

IBM 2005

Lesson 3: Probes, Continued

Exercise 3.4 exploring probes: (continued)

What to Monitor/ situation. Probe Type 8 Your IT department has spent months debating how their server documents should be set up. Finally they have come up with a default set of Server and Configuration document settings that they are going to apply to all servers. They are particularly concerned with fields related to security. What type of probe will help them ensure that the settings all stay the same over time? Which probe type and subtype would you configure to monitor CPU Utilization on your Domino Servers? You would like to monitor whether NRPC mail routing is occurring on the Mail Hub. In the Notes section, explain whether this probe will be enough for you to be confident that mail is routing from the hub to the users mail servers? What other probes might help in that regard?

Probe Subtype

Notes

10

Last edited 12/20/2005

15

IBM 2005

Lesson 4: Collection Hierarchy

Introduction

In this lesson, you will learn to design and implement a collection hierarchy.

Objectives

At the end of this lesson, participants should be able to Design and implement a Server Collection Hierarchy for a domain

Exercise 4.1 DDM collection hierarchies

In the Domino Administrator Help database, read the document entitled DDM server collection hierarchy and answer the following questions: Question 1 Answer In your own words, Explain the purpose of the DDM Server Collection Hierarchy. Will most administrators who are using DDM want to create a collection hierarchy?

In which Database is the DDM Server Collection Hierarchy configured?

What two circumstances cause ddm.nsf to replicate?

Can the DDM Server Collection Hierarchy be the same as your replication topology? If so, is this required?

Under which circumstances can you add a Domino 6 server to a collection hierarchy? Should you do this? Why or why not?

Continued on next page

Last edited 12/20/2005

16

IBM 2005

Lesson 4: Collection Hierarchy, Continued

Exercise 4.2 implementing collection hierarchy

In groups designated by your instructor, design and implement a collection hierarchy in the classroom.

Exercise points

Some points to be aware of when designing and creating your collection hierarchy: Use the procedure outlined in the Domino Administrator Help database document Creating a DDM server collection hierarchy when implementing your collection hierarchy. Each participant should designate their own server as a collection server. Since multiple collection hierarchies are possible, this will allow each participant to get data rolled up from several other servers in the domain. Only Domino 7 servers may be included in the hierarchy. If this domain was recently created, or if AdminP has not yet populated the Server build number field, you may need to replicate your names.nsf and admin4.nsf databases and issue the command tell adminp process all in order to get that field populated. If the value of the Server build number field is blank or contains information for a pre-Domino 7 server, you may not be able to add that server to the collection hierarchy.

Last edited 12/20/2005

17

IBM 2005

Lesson 5: User Interfaces

Introduction

In this lesson you will explore the user interface presented in the Domino Domain Monitoring database.

Objectives

By completing the exercises in this lesson you have completed the following objectives: Explain the purpose of the DDM database. Explore the views available to view the Event documents. Assign events to different administrators in your domain. Allow authorized users and restrict other users from working with events in the DDM database. Demonstrate how to close events in the DDM database.

User Interfaces

The UI for DDM is made up of two databases, events4.nsf and ddm.nsf. The purpose for each database is summarized below: Database Monitoring Configuration: events4.nsf Domino Domain Monitoring: ddm.nsf Function This database is used for configuring probes, and contains all events and formulas for how corrective actions are completed. The Domino Domain Monitoring database is where the results of probes are reported. They can be viewed in several different ways, such as by severity, date, etc. This database is also where events can be assigned to administrators to be worked on. In the exercises that follow, you will explore the DDM database and work with events by assigning them to others, and viewing the corrective actions you can take to close open events.
Continued on next page

Last edited 12/20/2005

18

IBM 2005

Lesson 5: User Interfaces, Continued

Exercise 5.1 DDM database

For this exercise open the DDM database on your server, and read the documents in Domino Administrator Help with the titles Domino Domain Monitoring database (ddm.nsf) and Viewing events in Domino Domain Monitoring and answer the following questions: Question 1 Answer List the views that are available for you view events: 1. 2. 3. 4. 5. 6. How often are event documents updated?

In the DDM database, navigate to the By Severity view. Do you have the action buttons pictured below?

If not, what do you need to do to get those buttons?

If you do have these buttons, how could you ensure that only administrators you want can see the action buttons?

Before proceeding, please ensure that both buttons DO show up for your administrator.
Continued on next page

Last edited 12/20/2005

19

IBM 2005

Lesson 5: User Interfaces, Continued

Exercise 5.1 DDM database (continued)

Question 4

Answer Explain the difference between an event state of Closed and Permanently Closed?

Exercise 5.2 DDM events

In this exercise you will complete some steps yourself, and in some cases will need to enlist a partner. At this point, you probably have quite a few events in your DDM database. If you do not, your instructor or mentor will assist in helping to generate some events to work with. For the tasks outlined below, use the procedures outlined in the Domino Administrator Help database. A good document to start with to navigate to the various procedures is Maintaining the Domino Domain Monitor. Complete the following tasks and where applicable, answer questions about the task. Task 1 Actions/Questions In the DDM Database, open the By Server view, and choose your server. Take a moment to look through the events that have been generated. 1. List at least 3 events that have corrective actions associated with them:

2. Can you find any that do not have corrective actions listed? 3. When looking at an event document, how can you tell which probe generated that event? 4. Find at least one event that has a button that can be clicked to take the suggested corrective action and list it here: Bonus Question: Can you find where the corrective actions and formulas for the buttons are taken from? (Hint: it is not in the DDM database)

Continued on next page

Last edited 12/20/2005

20

IBM 2005

Lesson 5: User Interfaces, Continued

Exercise 5.2 DDM events (continued)

Task 2

Actions/Questions Working with a partner who has your server included in his collection hierarchy, choose several events to assign to each other. (If necessary, you can simply assign events to yourself.) When assigning the events, include some comments. 1. Can you assign events that have a state of Closed to someone else? 2. Where do these comments appear after assigning the event?

Once you have several events assigned to you, change to the My Events view, and open the documents, take whatever corrective actions you would like to, and change the state to either closed or permanently closed. In your own words, describe what is meant by the term Correlating Events. List an example from your ddm.nsf of an event that has been correlated. Name the two databases that make up the User Interface of DDM and explain what each database is used for: a)

b)

Last edited 12/20/2005

21

IBM 2005

Lesson 6: Event Filtering

Introduction

In the previous lesson, we saw that having the default probes enabled can create a copious amount of Event documents in the DDM database. An administrator may decide that many of the events are not problems that need to be monitored on a regular basis. Event filters provide a way to control which type of events get reported to the DDM log file.

Objective

At the completion of this lesson, participants should be able to Demonstrate how to create, modify and delete DDM Filters.

Exercise 6.1 event filtering

To complete this exercise, follow the procedures in Domino Administrator Help linked in the document DDM Filters: Task 1 2 Actions/Questions Create a filter for ONLY YOUR SERVER to log only events of a severity of Fatal or Failure. Create another filter for ONLY YOUR SERVER to log only events of types Messaging, Database, and Server. You should receive an error at this point. What is that error?

Why did you get this error, and what can you do to resolve it?

3 4

Modify your existing filter to include more types of events. How would you delete an existing DDM filter?

Last edited 12/20/2005

22

IBM 2005

Self-service resources on the Web:

Lotus software

support

The Lotus software support Web site provides content to help you troubleshoot issues, plan deployments, and subscribe to product news. You can even submit and track problems with your IBM Customer Number. http://www.ibm.com/software/lotus/support

Lotus Domino support page

This product support page offers the latest troubleshooting resources, patches, product Flashes and other important content specific to Lotus Domino. http://www.ibm.com/software/lotus/support/domino/support.html

developerWorks: This page offers IBMs technical resources for Lotus Domino Developers, Lotus such as articles:

developerWorks: Lotus

http://www.ibm.com/developerworks/lotus

Notes and Domino http://www.ibm.com/developerworks/lotus/products/notesdomino

Notes/Domino 6 & 7 forum

The Notes/Domino 6 & 7 forum is an excellent source of information regarding Notes and Domino issues. The questions and answers posted by your peers can be quite helpful when you are researching an issue, sometimes preventing the need to submit a problem to software support! http://www.lotus.com/ldd/nd6forum.nsf

Product documentation

The product documentation page offers the latest Release Notes, Help files, White Papers, etc. for Lotus Domino. http://www.lotus.com/ldd/notesua.nsf/find/domino

Last edited 12/20/2005

23

IBM 2005