Data Mediator Install and Admin Guide R2 (13.1.10)

Data Mediator
Installation and Administration Guide

P/N D357009 R2
Important Notice
Important Notice
Allot Communications Ltd. ("Allot") is not a party to the purchase agreement under which NetEnforcer was purchased, and will not be liable for any damages of any kind whatsoever caused to the end users using this manual, regardless of the form of action, whether in contract, tort (including negligence), strict liability or otherwise. SPECIFICATIONS AND INFORMATION CONTAINED IN THIS MANUAL ARE FURNISHED FOR INFORMATIONAL USE ONLY, AND ARE SUBJECT TO CHANGE AT ANY TIME WITHOUT NOTICE, AND SHOULD NOT BE CONSTRUED AS A COMMITMENT BY ALLOT OR ANY OF ITS SUBSIDIARIES. ALLOT ASSUMES NO RESPONSIBILITY OR LIABILITY FOR ANY ERRORS OR INACCURACIES THAT MAY APPEAR IN THIS MANUAL, INCLUDING THE PRODUCTS AND SOFTWARE DESCRIBED IN IT. Please read the End User License Agreement and Warranty Certificate provided with this product before using the product. Please note that using the products indicates that you accept the terms of the End User License Agreement and Warranty Certificate. WITHOUT DEROGATING IN ANY WAY FROM THE AFORESAID, ALLOT WILL NOT BE LIABLE FOR ANY SPECIAL, EXEMPLARY, INDIRECT, INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, REGARDLESS OF THE FORM OF ACTION WHETHER IN CONTRACT, TORT (INCLUDING NEGLIGENCE), STRICT LIABILITY OR OTHERWISE, INCLUDING, BUT NOT LIMITED TO, LOSS OF REVENUE OR ANTICIPATED PROFITS, OR LOST BUSINESS, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Copyright
Copyright 1997-2013 Allot Communications. All rights reserved. No part of this document may be reproduced, photocopied, stored on a retrieval system, transmitted, or translated into any other language without a written permission and specific authorization from Allot Communications Ltd.
Trademarks
Products and corporate names appearing in this manual may or may not be registered trademarks or copyrights of their respective companies, and are used only for identification or explanation and to the owners' benefit, without intent to infringe. Allot and the Allot Communications logo are registered trademarks of Allot Communications Ltd. NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expense. Changes or modifications not expressly approved by Allot Communication Ltd. could void the user's authority to operate the equipment.
Data Mediator Installation & Administration Guide
iii
Important Notice
Version History
Doc Version
v2b11
Product
MED13.1
Date
24.07.13
Updates
List of required ports ammended Excluded Conversations CDRs clarified. GA Version, Record Export Rate clarification BDR Configuration Enabling commands, Obfuscation, Custom Attributes added Distributed Collection update, Conversations Priority CLI changes Distributed Collection and Data Export Rate information added General edits, XML information reorganized Field Descriptions New CDR Types
v2b10
MED13.1
09.07.13
v2b9
MED13.1
26.05.13
v2b8 v2b7
MED13.1 MED13.1
05.02.13 24.12.12
v2b6
MED13.1
16.12.12
v2b5
MED13.1
04.12.12
v2b4
MED13.1
02.12.12
v2b3 v2b2 v2b1
MED13.1 MED13.1 MED13.1
29.11.12 25.11.12 21.11.12
iv
Table of Contents
Important Notice .......................................................................................................................... iii Table of Contents........................................................................................................................... v
CHAPTER 1: OVERVIEW ................................................................................ 1-1

Data Types................................................................................................................................... 1-2 Record Export Rates ................................................................................................................. 1-3 DM Performance Rates ............................................................................................................. 1-3 Data Flow System Level .......................................................................................................... 1-4 Data Flow Data Mediator ....................................................................................................... 1-5 Glossary ....................................................................................................................................... 1-7
CHAPTER 2: HARDWARE .............................................................................. 2-1

Overview...................................................................................................................................... 2-1 AC Power Supply ....................................................................................................................... 2-2 AC Data Mediator (PA-MED-SRV and PA-MED-SRV-HAP) ............................................... 2-2 AC Storage Device ................................................................................................................... 2-3 DC Power Supply ....................................................................................................................... 2-4 DC Data Mediator (PA-MED-SRV-HAP-DC) ........................................................................ 2-4 DC Storage Device ................................................................................................................... 2-5
CHAPTER 3: INSTALLING DATA MEDIATOR ............................................... 3-1

Connecting to the Network ........................................................................................................ 3-1 Preparing the Network .............................................................................................................. 3-1 Connecting a Single Data Mediator .......................................................................................... 3-2 Connecting an HA Data Mediator Cluster ................................................................................ 3-2
CHAPTER 4: CONFIGURING DATA MEDIATOR ........................................... 4-1

Data Mediator Configuration .................................................................................................... 4-1 Configuring Network Parameters ............................................................................................. 4-1 Connecting to Data Mediator ................................................................................................ 4-1 Configuring Device Parameters ................................................................................................ 4-6 Usage ..................................................................................................................................... 4-6 Example ................................................................................................................................ 4-6 Configuring IMM Parameters ................................................................................................... 4-6 Rebooting the Data Mediator .................................................................................................... 4-9 Data Mediator High Availability .............................................................................................. 4-9 Pre-Installation Requirements................................................................................................... 4-9 Installing the HA Software ..................................................................................................... 4-10 Editing the /etc/hosts file ........................................................................................................ 4-11 Verify a Successful Installation .............................................................................................. 4-12 Enabling and Configuring Data Mediation............................................................................ 4-12 Enabling Subscriber Data Export License .............................................................................. 4-12 Enabling Distributed Collection ............................................................................................. 4-13
Data Mediator Installation & Administration Guide v
Adding a Data Mediator ......................................................................................................... 4-16 Enabling and Configuring CDR Types ................................................................................... 4-18
CHAPTER 5: HTTP DETAIL RECORDS (HDR) .............................................. 5-1

Enabling and Configuring HDRs .............................................................................................. 5-1 Enabling HDRs per In-line Platform ........................................................................................ 5-1 Configuring HDRs on the Data Mediator ................................................................................. 5-3 Filtering Rules ....................................................................................................................... 5-4 Naming Conventions .................................................................................................................. 5-5 HDR Interface Fields ................................................................................................................. 5-6 Output Example ......................................................................................................................... 5-8
CHAPTER 6: MINUTES OF USE (MOU) ......................................................... 6-1

Enabling and Configuring MOU .............................................................................................. 6-1 Enabling MOU per In-Line Platform........................................................................................ 6-1 Configuring MOU on the Data Mediator.................................................................................. 6-2 Naming Conventions .................................................................................................................. 6-4 MOU Interface Fields................................................................................................................. 6-4 Output Example ......................................................................................................................... 6-5
CHAPTER 7: VC RECORDS ........................................................................... 7-1

Configuring VCs on the Data Mediator ................................................................................... 7-1 Naming Conventions .................................................................................................................. 7-3 VC Interface Fields .................................................................................................................... 7-3 Output Example ......................................................................................................................... 7-6
CHAPTER 8: CONVERSATION RECORDS .................................................... 8-1

Enabling and Configuring Conversations ................................................................................ 8-1 Enabling Conversations Records per In-line Platform ............................................................. 8-1 Configuring Conversations Records on the Data Mediator ...................................................... 8-1 Prioritizing Conversations Statistics ..................................................................................... 8-3 Naming Conventions .................................................................................................................. 8-4 Conversations Interface Fields .................................................................................................. 8-5 Output Example ......................................................................................................................... 8-8
CHAPTER 9: SESSION DETAIL RECORDS (SDR) ........................................ 9-1

Enabling SDRs per SMP ............................................................................................................ 9-1 Configuring SDRs on the Data Mediator ................................................................................. 9-3 Naming Conventions .................................................................................................................. 9-4 SDR Interface Fields .................................................................................................................. 9-5 Custom Attributes ..................................................................................................................... 9-7 Service Plan ID ......................................................................................................................... 9-9 Open Reason / Close Reason .................................................................................................... 9-9 IMEISV Number..................................................................................................................... 9-10 Output Example ....................................................................................................................... 9-10
vi Data Mediator Installation & Administration Guide
CHAPTER 10: USAGE DETAIL RECORDS (UDR) ........................................10-1

Configuring UDRs on the Data Mediator .............................................................................. 10-1 Monitored Applications .......................................................................................................... 10-3 Naming Conventions ................................................................................................................ 10-4 UDR Interface Fields................................................................................................................ 10-5 Output Example ....................................................................................................................... 10-6
CHAPTER 11: BINARY DATA RECORDS (BDR) ..........................................11-1

Enabling and Configuring BDRs on the In-Line Platform................................................... 11-1 Legend .................................................................................................................................... 11-2 Configuring BDR Export on the NetXplorer ......................................................................... 11-3 Naming Conventions ................................................................................................................ 11-5
CHAPTER 12: SNMP INTERFACE .................................................................12-1

Data Mediator Traps................................................................................................................ 12-1 Data Mediator KPIs ................................................................................................................. 12-1 Incoming Records ................................................................................................................... 12-2 Processed Records .................................................................................................................. 12-2 Output Records ....................................................................................................................... 12-3
CHAPTER 13: APPENDICES .........................................................................13-1

Appendix A: Data Mediator Logs ........................................................................................... 13-1 Appendix B: Upgrade Procedure ............................................................................................ 13-1 Appendix C: Customizing Data Export ................................................................................. 13-3 DataExportConfig.xml Template ............................................................................................ 13-4 HDR XML Format.................................................................................................................. 13-5 UDR XML Format.................................................................................................................. 13-6 MOU XML Format ................................................................................................................. 13-6 VC XML Format .................................................................................................................... 13-6 Conversations XML Format ................................................................................................... 13-7 SDR XML Format .................................................................................................................. 13-7 Using Obfuscation .................................................................................................................. 13-8 Example .............................................................................................................................. 13-8
vii
Chapter 1: Overview
In addition to its powerful core functionality, the Allot solution elements can serve as a comprehensive data source. Working together with the Data Mediator, data can be compiled, collected, mediated and exported to external systems for the purpose of storage or analysis. Various types of statistics and detail records are generated by the Allot solution. The data must be collected periodically from the Data Source (Allot In-Line Platforms or Subscriber Management Platforms), transformed into a suitable format and then pushed to target devices that will process or store the data according to the needs of each individual customer. The Allot Data Mediator is an infrastructure component that provides:
Single and unified infrastructure for data mediation between producers and consumers, where consumers may be internal or external to the system Shorter implementation time to support collection, transformation and transfer of new data types Faster compliance with customer needs by supporting pluggable adapters without modifying generic infrastructure
1-1
Data Types
Allot in-line platforms, together the Allot Data Mediator, serve as a data source for the following data types:
Conversation Session (SDR/UDR) HDR VC MOU BDR

1.
Bandwidth per connection, service Customization of preferred and excluded application Bandwidth per subscribers Customization of filtering rules Web activity usage per subscriber Define Application to monitor Bandwidth per policy element
VOIP usage per service plan and service
Binary raw data
This data can be used to generate many different types of statistics, including: Usage based on Conversations records extracted from in-line traffic, Allot can provide a 360 degree view of network, subscriber, and app usage for monitoring and planning Mobile based on Mobile session records (SDR/UDR) generated by the Allot SMP (Subscriber Management Platform) from RADIUS messages it receives. Allot can provide statistics on Mobile handset usage that is useful for new handset introduction and monitoring as well as monitor Mobile network load e.g. by analyzing session bitrates. OTT based on MOU records, Allot can monitor the leakage of Voice calls to OTT VoIP. Domain-specific Allot can provide HTTP records extracted from HTTP traffic. The customer can use that for Web traffic analysis and subscriber profiling and trends and for generating new revenue streams (Proactive Analytics). Policy based on VC stats extracted from in line classified traffic providing usage based on the service plan and subscriber
2.
3. 4.
5.
1-2
level info, Allot can provide subscriber service plan quota usage, for service plan tuning and capacity planning NP> rephrase. VC records provide usage statistics per the policy elements managed by the system. The policy elements may represent subscribers and service plans. Allot data feeds are delivered with minimum impact on DPI performance. Records include a unique identifier which allows association between the different statistical record types for creating a single enriched record.
Record Export Rates

For SG-Sigma and SG-Sigma E Service Gateways, the following values are per Core Controller.
NOTE 1 Data Mediator supports up to 20 CC-300 blades.
PER INTERVAL (SECONDS) UDR HDR VC VC Conversation Conversation MOU 300 1 300 30 300 30 300
AC1400 AC3000 AC6000
SIGMA
SIGMA E
180,000 1,000 160,000 80,000 600,000 300,000 1,600
180,000 1,000 160,000 80,000 600,000 300,000 1,600
360,000 7,500 500,000 250,000
480,000 2,500 500,000 250,000
1,500,000 10,000 1,200,000 600,000 1,500,000 750,000 1,600
1,200,000 600,000 600,000 1,600 300,000
DM Performance Rates
5 MINS AGGREGATION UDR HDR 360M 720M 10-60 MINS AGGREGATION 180M 360M
1-3
5 MINS AGGREGATION VC* Conversation SDR MOU 288M 360M 22M 400K
10-60 MINS AGGREGATION 144M 180M 11M 200K
* The In-line platform limits the number of exported VCs to the # of supported VC in the distributed collector (4M per 5 Minutes).
The Aggregation value affects all exported records. For information concerning setting this value, see Configuring Conversations Records on the Data Mediator on page 8-1.
Data Flow System Level
Figure 1-1: System Level Data Flow

1-4
Step 1- The RADIUS server performs the AAA functionality for the new subscriber session. Step 2 The RADIUS Server passes the RADIUS accounting message to the SMP. Step 3 The SMP extracts relevant subscriber information from the RADIUS message and passes this on to the In-line Platform.
It also generates SDR buckets which aggregate this information on the SMP.
Step 4a The subscriber is granted access to the Internet and subscriber traffic flows through the In-line platform. The In-line platform records the relevant subscriber data (in a series of different data buckets) and Step 4b forwards the subscribers traffic to the internet according. Step 5 The Data Mediator polls the various data sources (in this case the In-line Platform and the SMP) for the configured data buckets, Step 6 The Data Mediator formats the data and sends it on to the Data Warehouse or third-party analytics system.
Data Flow Data Mediator

The Data Mediator performs three key functions:
Collects data from in-line platforms Transforms the data to an output format Pushes the data to the required targets
2
Collector Transformer (3)
Purge (6)
1
File List
Transformed Output (4)
Push (5)
Data Source
Data Mediator Figure 1-2: Data Mediator Data Flow
Data Target
1-5
The Data Mediator collects data from a list of configured Allot data sources. Periodically, the Data Mediator retrieves (1) the list of potential files to collect from each data source (in-line platform or SMP)
NOTE It is the responsibility of the Data Mediator to identify the files that have not been processed yet, and to avoid duplicate processing of files.
New files are retrieved by the Collect (2) function, and depending on the type of the collected data, a suitable Transform (3) function is executed. The Data Mediator supports different transform adapters per data type to facilitate changing adapters without making any change to the infrastructure. Transformations are either of the following types:
Simple assignments from input to output Data type conversions binary to textual (e.g. date, time, IP address)
A transformed file is stored in the Data Mediator storage (4). The system supports a push mechanism towards the Target consumers. Therefore, files are pushed by the Push (5) function.
At the completion of the life cycle, transformed files are purged from the Data Mediator storage by the Purge (6) function, according to the retention policy for the data type.
1-6
Glossary
TERM
BDR CDR Conversations Data Sources DM HDR In-line Platform MOU NX Pilot Packet SDR SMP UDR VC
DEFINITION
Binary Detail Records Call Data Record, a generic term used to indicate any of the seven data types translated by the Data Mediator Individual connections Can be in-line platforms (such as the SG-Sigma or NetEnforcer) and/or the SMP. Data Mediator (Also known as MED) HTTP Detail Records Allot NetEnforcers or Service Gateways Minutes of Use NetXplorer See BDR Session Detail Record Subscriber Management Platform Usage Detail Record Virtual Channel
1-7
Chapter 2: Hardware
Overview
The Allot Data Mediator is a 1U high appliance based on an IBM M3 x3550 server. The appliance is shipped from Allot with a Linux based CentOS operating system and Allot software already installed. The Allot Data Mediator is available in the following configurations:
PA-MED-SRV (AC standalone) PA-MED-SRV-HAP (AC HAP) PA-MED-SRV-HAP-DC (DC HAP)
A front and rear view of an Allot Data Mediator (PA-MED-SRV) is illustrated below:
Figure 2-1: PA-MED-SRV Front View
Figure 2-2: PA-MED-SRV Rear View A Data Mediator High Availability cluster is made up of two AC or DC Data Mediators along with a Storage device. The Storage device supports the same power supply (AC or DC) as the Data Mediators it is clustered with.
2-1
Figure 2-3: PA-MED-SRV-HAP cluster with Storage device Front View
AC Power Supply
AC Data Mediator (PA-MED-SRV and PA-MED-SRVHAP)
Two field replaceable and hot swappable 100 VAC to 240 VAC power supplies provide AC power to the PA-MED-SRV and to each Data Mediator Node in the PA-MEDSRV-HAP.
NOTE AC Power Supply units should be removed for maintenance purposes only.
The server is equipped with two AC power inlets which are located on the rear side of the server. The AC inlet connectors are also located on the rear of the server.
Figure 2-4: AC Power Inlets
2-2
Each AC power supply is capable of supplying 835w. One power supply is fully capable of supplying the power consumed by a fully occupied server. The second power supply is for use in cases where redundancy is needed.
NOTE Connect each power entry to a different power source for redundancy
Further power specifications for the Allot Data Mediator (PA-MED-SRV) are detailed in the table below:
Power Consumption (max) AC Power Supply 1+1 for redundancy 835 W 2,851 BTU/hour
100-127 VAC, 10A max 200-240 VAC, 5A max 50/60Hz
Form factor / height Network Interface Power Supply (std/max) Hot swap components
Rack/1U Integrated dual Gigabit Ethernet 835W 1/2 AC standard Power supply, fans and hard disk drives
AC Storage Device
The PA-MED-SRV-HAP comes with an IBM storage device
NOTE The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device. The device also might have more than one power cord. To remove all electrical current from the device, ensure that all power cords are disconnected from the power source. You must follow the power sequence in the order shown in the following procedure. To establish power redundancy for enclosures with two power supplies, use at least two power distribution units (PDUs) in the rack cabinet. Split the power connections from each enclosure into the separate PDUs. Then, connect the PDUs to external power receptacles that are on different circuits.
To connect the AC Storage Device to the power supply, follow the procedure below: To turn on the storage subsystem ac power, complete the following steps:
2-3
6.
Connect a power cord to each power supply in the storage subsystem.
Figure 2-5: AC Power Inlet, Storage (Right Side)

7. 8. 9.
Connect the other end of each power cord to the power receptacle. Turn on the power to the switches, if applicable. Turn on both power switches on all of the attached storage enclosures, and wait 60 seconds.
10. Turn on both power switches on the storage subsystem. NOTE When you turn off the power to the storage subsystem, complete the preceding steps in reverse order. Turn off the power to the storage subsystem first; then, turn off the power to the storage enclosures.
DC Power Supply
DC Data Mediator (PA-MED-SRV-HAP-DC)
Two field replaceable and hot swappable -48V DC power supplies provide DC power to the PA-MED-SRV-HAP-DC.
NOTE DC Power Supply units should be removed for maintenance purposes only.
The server is equipped with two -48V DC power inlets which are located on the rear side of the server. DC power cables are not provided by Allot.
Figure 2-6: DC Power Inlets
2-4
Each DC power supply is capable of supplying 675w. One power supply is fully capable of supplying the power consumed by a fully occupied server. The second power supply is for use in cases where redundancy is needed. Further power specifications for the Allot Data Mediator (PA-MED-SRV-HAP-DC) are detailed in the table below:
Power Consumption (max) DC Power Supply 1+1 for redundancy Form factor / height Network Interface Power Supply (std/max) Hot swap components 675W -48 V DC Rack/1U Integrated dual Gigabit Ethernet 675W 1/2 DC standard Power supply, fans and hard disk drives
DC Storage Device
The PA-MED-SRV-HAP-DC comes with an IBM storage device. To connect the DC Storage Device to the power supply, follow the procedure below:
1.
Make sure that the power on-off switches of both DC power supplies in the storage subsystem and all attached storage enclosures are in the off position.
Figure 2-7: DC Power Inlet, Storage (Right side)
2-5
2.
Connect a DC jumper cable to each DC power supply in the storage subsystem and to all attached storage enclosures: Using the supplied strap, tie the DC power supply cable to the rail to provide strain relief for the power cable. Connect the DC power cable to the power supply. See below for the DC power supply connector PIN positions.
Figure 2-8: DC Pin Positions

NUMBER FUNCTION DC POWER CABLE WIRE COLOR
1 2 3
3.
Pin 1: - 48 V dc Pin 2: POS RTN Pin 3: GND
Brown Blue Green/yellow
Connect the - 48V wire (brown) of the DC power cable to an approved disconnect device (circuit breaker) rated at 30 A. The disconnect device must be easily accessible from the back of the Storage unit.
The disconnect device (circuit breaker) must be rated at 30 A. Make sure that only 10 AWG copper conductor wires are used for all of the wiring between the DC power connectors and the DC power source. Complete the wiring from the disconnect device to the terminal marked -48V of the reliably grounded safety extra low voltage (SELV) DC power source. Connect the POS RTN wire (blue) and the ground wire (green/yellow) of the dc power cable to the terminals marked POS RTN and GND on the DC power source.
NOTE
2-6
Figure 2-9: DC Storage Device Wiring

NUMBER FUNCTION
1 2 3 4 5
4. 5. 6. NOTE
Pin 1: - 48 V DC Pin 2: POS RTN Pin 3: GND DC power source Disconnect device (circuit breaker) After connecting all of the DC jumper cables, turn on the power to the -48 V dc SELV power source. Turn on both power switches on all of the attached storage enclosures, and wait 60 seconds. Turn on both power switches on the storage subsystem.
When you turn off the power to the storage subsystem, complete the preceding steps in reverse order. Turn off the power to the storage subsystem first; then, turn off the power to the storage enclosures. Lastly, turn off the disconnect devices installed on the -48V SELV power source.
2-7
Chapter 3: Installing Data Mediator

Connecting to the Network
Preparing the Network
Certain ports need to be open to enable the required communication between the different components of the Data Mediator solution. The ports required are presented in the table below.
SOURCE DESTINATION APPLICATION PROTOCOL PORT COMMENTS Data transformation configuration DM DM DM DM NX DM Operator DM DM NX NX NX NX DM NX DM NE SMP RMI RMI RMI RMI SNMP SNMP SNMP HTTP HTTP FTP SFTP SCP File Copy tcp tcp tcp tcp udp udp udp tcp tcp 1098 3873 4446 1099 161 161,162 161,162 80* 80 21,22
(or any port required for communication with destination server) (DataExportConfig.xml)
Catalogs Subscriber data JNP server bind address set NX ip traps MIB retreival for KPIs Manifest and raw data files retrieval Manifest and raw data files retrieval Push transformed files to operator file server or DWH environment
DM
Operator
tcp
21,22
(or any port required for communication with destination server)
Operator DM
DM NX
SFTP NTP
tcp udp,tcp
Operator pulls transformed data If NX is the source of Time Synch
123
3-1
SOURCE
DESTINATION
APPLICATION
PROTOCOL
PORT
COMMENTS If an external Time server of Operator's network is the source of Time Synch
DM
External Time Server
NTP
udp,tcp
123
* If TCP port 80 is closed between the Data Mediator and the NE/SG, the NetXplorer will report the Data Mediator is unreachable. That is in addition to the ports which need to be open in every NetXplorer installation (see NetXplorer Installation and Admin Guide, Appendix E)
Connecting a Single Data Mediator

To connect a single Data Mediator to the network, follow the instructions below:
Connect a straight copper cable from port 1 illustrated below on the rear of the Data Mediator to a management switch. Connect a straight copper cable from the IMM port illustrated below on the rear of the Data Mediator to a laptop for direct connectivity and maintenance (in blue). Optionally, you may also connect an additional straight copper cable from the adjacent port (port 2) to a secondary management switch for backup purposes.
Figure 3-1: Connecting a Single Data Mediator (M3 Hardware)
Connecting an HA Data Mediator Cluster

Data Mediator Cluster Hardware Summary 3-2 Data Mediator Installation & Administration Guide
To connect the HA Data Mediator Cluster (2 x Data Mediators and a RAID storage device with dual controllers), refer to Figure 3-2 and follow these instructions:
Figure 3-2: HA Cluster Network Connections (M3 Hardware) In a High Availability Cluster configuration, the Data Mediator units are connected by two physical links (and each Data Mediator server to the RAID by dedicated SAS cables) as follows:
1.
Use a crossed copper cable to connect between Port 3 on one Data Mediator server and Port 3 on the second Data Mediator server. (illustrated in green above)
Figure 3-3: Close up Rear View of Data Mediator node in Data Mediator-HAP (M3)
2.
Use a null modem serial cable (RS 232) to connect between the Serial COM port on one Data Mediator server and the Serial COM port on the second Data Mediator server. (illustrated in red above) Use a Serial Attached SCSI (SAS) cable (see Figure 3-4) to connect between each Data Mediator server and the RAID storage server (as illustrated in Figure 3-5). (These connections and the exact ports required are illustrated in orange in Figure 3-2 above)
3-3
3.
Figure 3-4: SCSI (SAS) Cable
Figure 3-5: SCSI (SAS) Cable connections to RAID Storage Server

4.
Connect each Data Mediator server to the management network via Port 1 (shown in blue in Figure 3-2) with an additional link via Port 2. Additional management cables should be connected to the IMM module on each Data Mediator server for direct connectivity and maintenance (in blue). An additional optional management cable may be connected to the RAID storage server for storage management and storage traps.
5.
6.
The serial (SAS) cable, the crossed copper cable and the two SAS cables are provided with the Data Mediator High Availability kit.
3-4
Chapter 4: Configuring Data Mediator

Data Mediator Configuration
Irrespective of whether you are working with basic Data Mediator architecture or a high availability Data Mediator cluster, each Data Mediator server that you are working with must be prepared for use. Four initial steps are required
1. 2. 3. 4.
Configure network parameters Configure device parameters Configure IMM parameters Reboot the Mediator
Configuring Network Parameters

The following network parameters need to be assigned to each of the Data Mediators you are using:
NOTE
Data Mediator IP and mask Data Mediator Gateway IP for network access DNS Server IP Host name NTP Time Server IP
The default gateway must be configured. If not configured, in some circumstances, you may not be able to add the SMP to the NetXplorer.
Connecting to Data Mediator

Connect a keyboard, mouse and monitor to the front of the Data Mediator server as shown in Figure 4-1.
4-1
Figure 4-1: Connecting the Data Mediator (M3) Front View Log in as a root user: username: root password: bagabu
1. 2.
Configure the network settings via the CentOS UI. Open a network connection by selecting Administration/Network from the System menu as shown in Figure 4-2 below
Figure 4-2: Open Network Configuration

3.
From the Devices tab, select the eth0 interface as shown in Figure 4-3 below.
4-2
Figure 4-3: Select eth0 Double click the selected interface. The Ethernet Device dialog appears.
Figure 4-4: Ethernet Device dialog box

4. NOTE 5.
In the General tab set the IP address, Subnet mask and Default Gateway in the Statically set IP addresses section.
The Default Gateway MUST be set during the initial configuration.
Click OK to save and return to the Network Configuration dialog.
4-3
Figure 4-5: Network Configuration dialog box, Hosts tab

6.
Open the Hosts tab and click New to create a new Host. The Add/Edit Hosts dialog appears.
Figure 4-6: Network Configuration dialog box, Add/Edit Hosts dialog

7.
Enter the IP Address, Host name and Alias for the new host and click OK to return to the Network Configuration dialog. The new host will appear in the Hosts tab. To edit an existing Host, click the Edit button.
Figure 4-7: Network Configuration dialog box, DNS tab
4-4
8. NOTE 9.
Open the DNS tab and enter the Host name and DNS IP addresses.
The Hostname entered in the DNS tab must be the same as was added in the Hosts tab.
Select Save from the File menu to save all changes. and then Deactivate followed by Activate.
10. Restart the network process via the Devices tab by selecting eth0
4-5
Configuring Device Parameters

Once you have installed the MD package and configured the network parameters of the Data Mediator, you need to configure the appropriate device parameters in order to inform the Data Mediator server what functional elements should be enabled. The dev-setup command informs the Data Mediator what its identity is. You can see the list of available parameters by running the command dev_setup.sh without parameters.
NOTE Usage dev_setup.sh [ -v ] [ -m MODE ] [ -f ] [ -a ] If a Data Mediator has been shipped directly from Allot, this process has already been done and should be skipped.
-v -m <MODE> -f -a
View device setup properties and license details Device mode to set Force change of mode and key High availability mode and key
The proper MODE to use in the command to configure a Data Mediator is datamediation. Once that is set, reboot the Data Mediator. Once the device is set to datamediation you will need to open the NetXplorer GUI to assign the Data Mediator to that NetXplorer.
Example dev_setup.sh -m datamediation
Setting device mode to datamediation...
Request completed successfully. Device mode successfully changed to datamediation. Please reboot your device.
Configuring IMM Parameters

The default details of the IMM are as follows:
4-6
Default IP:
192.168.70.125 USERID
Default User Name:
Default Password:
PASSW0RD (where the 0 is not o but zero)
To configure the network settings of the Integrated Management Module, follow the steps below: 1. Connect directly from a laptop to the IMM interface on the rear of the Data Mediator. The interface is labeled SYSTEM MGMT as shown in Figure 4-8 below
Figure 4-8: IMM "System Management" Port 2. Open a web browser. In the address field type the IP address or host name of the IMM to which you want to connect.
NOTE If you are logging in to the IMM for the first time after installation, the IMM defaults to DHCP. If a DHCP host is unavailable, it uses the default static IP address 192.168.70.125. You can obtain the DHCP-assigned IP address or the static IP address from the server BIOS or from your network administrator.
3. Enter User ID and Password 4. You will be prompted to specify an inactive session timeout value. Choose a value from the dropdown list and click on Continue. 5. You will see the IMM User Interface, with the default System Status in view, as seen in below:
4-7
Figure 4-9: IMM System Status Screen 6. Select Network Interfaces from the system tree on the left side of the screen. 7. In the Ethernet section, make sure that interface is enabled, and IPv6 DHCP is disabled. In addition, DDNS status should be set to Disabled and Domain Name Used should be set to manual. 8. In the IPv4 section, make sure that the DHCP field is set to: Disabled Use Static IP configuration. Assign an IP, mask and default gateway as seen above and click Save. You can now access the IMM remotely using these network settings.
4-8
Figure 4-10: IMM Network Interfaces
Rebooting the Data Mediator

To reboot the Data Mediator server, log in as a root user and enter: reboot
Request completed successfully Broadcast message from root (Sun Mar 1 16:59:18 2009):
Data Mediator High Availability

The High Availability (HA) solution for Allots Data Mediator Platform is based on using two installations of Data Mediator for high availability applications and a shared storage unit for Sybase database files.
Pre-Installation Requirements
Before starting a high availability installation, you will need to ensure that both the Data Mediator servers which you will be using have the same Data Mediator software version.
NOTE: Make sure that both DMs are configured with the dev-setup -a flag for high availability i.e: dev-setup.sh m datamediator a
4-9
Confirm that the same software version is installed on both Data Mediator units by checking the install log on each Data Mediator server unit, located at /opt/allot/conf/install_log. You may also run the command dctype. Confirm that Network Parameters (IPs, Gateway, DNS, Hostname, etc.) are defined on both machines and that both nodes have the same subnet and DG. Decide which Data Mediator unit will be designated as DM-0 (initially active unit) and as DM-1 (initially passive-standby unit). To do this, use a logical role designation and host name.
Installing the HA Software

NOTE 1. Only a root user can install the HA (High availability) package.
Login to the active Data Mediator node su -
2. Change to root with the following command: 3. Shut down any external storage units and disconnect it. 4. Import the HA_smp.sh and HA_smp.tgz files to the package
folder.
5. Run the following script
/opt/allot/bin/dev_setup.sh -a
6. Make HA_smp.sh executable by running the following command:
chmod +x HA_smp.sh
7. Run the following command:
./HA_smp.sh -m
8. Repeat steps 2-7 on the passive node. 9. Reconnect the storage 10. Wait a few seconds and turn on the storage. 11. Reboot the Data Mediators.
The next steps should be done directly on the server via the console port, not via SSH/Telnet
12. Use the cat/proc/partitions command to have the server to see
the storage
13. Login as root and run the following command on both nodes:
./HA_smp.sh -e
4-10
If there is an error message about sdb -reboot, then run the command again
14. Answer the question 15. Reboot and wait 16. On the active node only run the following command:
./HA_smp.sh -p
17. Run the following command on both nodes:
./HA_smp.sh -I
18. On the active node only run the following command:
./HA_smp.sh -c
19. Enter root password when asked. 20. Wait a few seconds and enter the crm_mon command and verify
that all parameters are online. Example

[root@dm-0 ~]# crm_mon Updating... Signing on... Querying... Refresh in 15s... ============ Last updated: Wed Jun 13 13:11:12 2012 Current DC: dm-0.allot.local (f66cafd8-d3b7-4e27-821c-594a2e70dfd5) 2 Nodes configured. 1 Resources configured. ============ Node: dm-1.allot.local (41b08a9f-67e7-4197-8b0a-d45361a614a9): online Node: dm-0.allot.local (f66cafd8-d3b7-4e27-821c-594a2e70dfd5): online Resource Group: dm_ha vip (ocf::heartbeat:IPaddr2): db (ocf::heartbeat:Filesystem): Started dm-0.allot.local Started dm-0.allot.local
smp (lsb:md):
Started dm-0.allot.local
Editing the /etc/hosts file

On each Data Mediator node, you should edit the /etc/hosts file with the IP addresses of the nodes as shown below
127.0.0.1 192.168.168.1 localhost MED-0 4-11
192.168.168.2 x.x.x.x
MED-1 host.example.com host
In this example, x.x.x.x represents the management interface IP address - it should be different in each Data Mediator node.
Verify a Successful Installation

After running the hainstall script, there are several checks that you can run to verify a successful installation. Verifying the creation of a cluster using the display file system command df-h Verify network parameters for the cluster using ifconfig a Check the High Availability status by entering the crm_mon command. Confirm storage connectivity by entering the cat/proc/partitions command.
Enabling and Configuring Data Mediation

The process of configuring Subscriber Data Export consists of 4 distinct stages:
1. 2. 3. 4.
Enabling Subscriber Data Export license via activation key Adding IP addresses to each Core Controller Adding a Data Mediator Configuring and enabling each type of CDR.
Each of these stages is examined in detail below.
Enabling Subscriber Data Export License

In order for the Subscriber Data Export feature to work, the operator must first have purchased the correct license for each Core Controller blade on one or more of the Service Gateways in the network. Only those Service Gateways for which Subscriber Data Export have been enabled will be able to collect usage data to be later exported. There are two different licenses for subscriber data export functionality:
4-12

1.
Statistics Export License for Conv, VC, MOU, SDR, UDR export HTTP CDR License for HTTP CDR and BDR export
To Verify that the Proper Subscriber Data Export License is Activated: In the Navigation pane, select and right-click the NetEnforcer or Service Gateway in the Navigation tree and select Configuration from the popup menu. OR Select the NetEnforcer or Service Gateway in the Navigation tree and then select Configuration from the View menu. OR Select the NetEnforcer or Service Gateway in the Navigation tree and then click the Configuration icon on the toolbar.
The Configuration window for the selected NetEnforcer or Service Gateway is displayed.
2. 3.
Select the Identification & Key tab. In the Activation Key Details pane, scroll down to check if Statistics Export has been activated for each Core Controller in the Service Gateway as shown in Figure 4-11 below. If the correct number of Core Controllers do not appear in the Statistics Export line, contact Allot Customer Support at support@allot.com.
Figure 4-11: Subscriber Data Export License Enabled

NOTE This may also be verified from the CLI using the following command: gcy key
Enabling Distributed Collection

In multi-blade In-line Platforms Allot has two modes of data collection:
Data Mediator Installation & Administration Guide 4-13
Centralized Collection: where data is collected on each Core Controller blade and transferred to the Host Blade (SGSV-100 on SG-Sigma or SFB-300 in slot 7 on SG-Sigma E) where it is stored. The host blade maintains a list of each data file that is stored. Distributed Collection: where data is collected on each Core Controller blade and stored there. The host blade maintains a central list of each data file, and the core controller informs the host blade which files should be added or removed from the list and on which blade each one is stored.
Distributed collection must be used with the Data Mediator. In order to enable distributed collection you will need to assign IP addresses from the management network to each Core Controller and SFB in the system and then enable direct access to them. The procedures for doing this are detailed below. To set the IPs for CC blades in slots 1-5 and 10-14 in two steps: Enter the following commands:
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 1:5 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 10:14
In the commands above x.x.x.x:255.255.0.0 represents the IP:Subnet Mask while the y.y.y.y represents the Default Gateway. The slots value 1:5 means that the slot number will increment up from 1 to 5, while 10:14 means the slot number will increment up from 10 to 14. Using these commands will cause the IP address (x.x.x.x in the example) to increment up as follows:
CC n will get IP address x.x.x.x CC n+1 will get IP address x.x.x.x+1 etc.
To set the IP for each CC blade individually:

1.
Enter the following commands for the relevant slots:

go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 1:1 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 2:2 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 3:3 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 4:4 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 5:5
4-14
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 10:10 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 11:11 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 12:12 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 13:13 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 14:14 2. To view the configuration, type the following command: sysadmin@EXC-SBH[1/6]:~$ go config view blade_mngt_ips
To set the IP for each SFB blade individually:

1.
Enter the following commands for the relevant slots:

go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 6:6 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 7:7 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 8:8 go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 9:9
Output Example
==== Blade Management IPs |Slot |IP ==== |Gateway |Mask
-------------------------------------------------------------|4 |10.150.6.34 |255.255.0.0 |10.150.0.1
-------------------------------------------------------------Request completed successfully.
To enable Distributed Collection in the NetXplorer:

1.
In the Navigation pane, select and right-click the desired In-line Platform in the Navigation tree and select Configuration from the popup menu. OR Select the desired In-line Platform in the Navigation tree and then select Configuration from the View menu. OR
4-15
Select the desired In-line Platform in the Navigation tree and then click the Configuration icon on the toolbar.
The Configuration window for the selected NetEnforcer or Service Gateway is displayed.
2. 3.
Select the IP Properties tab. In the Direct Access pane, check the Management IP per CC checkbox.
Figure 4-12: IP Properties tab
Adding a Data Mediator

1. Open NetXplorer. 2. In the Navigation pane, right-click Servers in the Network pane in the Navigation tree and select New Data Mediation from the popup menu. The Data Mediation Properties - New dialog is displayed.
4-16
Figure 4-13: Data Mediation Properties
4-17
3. Enter the name and IP address of the Data Mediator. 4. In the Source Units area, use the arrow keys to move In-line Platforms and SMPs from the Available to the Selected lists. Those In-line Platforms and SMPs selected will provide data to the Data Mediator. 5. In the Record Types to Collect area, select those record types that this Data Mediator will collect. Multiple data types may be selected, depending upon the capabilities of the platforms selected as Source Devices as well as the current license key. The possible data types are VC, Conversations, HTTP Detailed Records (HDR), Minutes of Use (MOU), Binary Detail Records (BDR), Usage Detail Records (UDR) and Session Detail Records (SDR).
NOTE SDR collection is only possible if you have included an SMP in the Selected Sources.
6. Click Save to add the Data Mediator to the network.
7. Click Save. The Monitoring Collector is added to the Navigation tree. The New Collector operation can take up to a couple of minutes to complete.
Enabling and Configuring CDR Types

The way that the Data Mediator works with each of the seven types of CDRs (VCs, Conversations, HTTP Detailed Records (HDR), Minutes of Use (MOU), Binary Detail Records (BDR), Usage Detail Records (UDR) and Session Detail Records (SDR)) must be configured from the Network Configuration dialog within the NetXplorer GUI. In addition, HDRs, MOU and Conversation records must be enabled on each relevant In-line Platform individually. SDRs must be enabled on each relevant SMP, which will also enable UDRs. For details, see the subsequent chapters outlining the requirements for each CDR type.
NOTE In some circumstances you may wish to customize the format of the data exported. Allot provides direct access to the configuration file to enable you to do this if required. For more information, see Appendix C on page 13-3.
4-18
Chapter 5: HTTP Detail Records (HDR)

HTTP CDRs are exported in CSV file format. Every HTTP complete transaction is represented by a record in the CSV file. The CSV files are pushed uncompressed or compressed in GZIP format from the Data Mediator via SCP, SFTP, Copy or FTP (Default is None) to the external system frequently (every 1-2mins). These settings can be changed from the Data Mediation tab in the NetXplorer GUI. The Allot system can be configured to purge the files after a pre-defined period of time (default is 24 hours).
Enabling and Configuring HDRs

Enabling HDRs per In-line Platform
1.
In the Navigation pane, select and right-click the relevant In-line Platform in the Navigation tree and select Configuration from the popup menu. OR Select the In-line Platform in the Navigation tree and then select Configuration from the View menu. OR Select the In-line Platform in the Navigation tree and then click the Configuration icon on the toolbar.
The Configuration window for the selected In-line Platform is displayed.

2. 3.
Select the Service Activation tab. To enable HDRs, select Enable from the HDR Generation Enablement drop down field.
5-1
Figure 5-1: HDR CDR area in the Service Activation tab

4.
In the HDR CDR, the following other fields may be configured.
HDR Generation Rate Limit This defines the maximum number of entries generated per second. The maximum possible value depends upon which In-line Platform or Core Controller blade is being used, as follows:
CC-300 Blade 10000 CC-200 Blade 2500 AC-6000 7500 AC-1400/AC-3000 - 1000
HDR File Compression This sets the file compression method to be used when storing the file on the In-Line Platform. The possible options are GZIP and no compression. GZIP is selected by default. Save the configuration changes to apply the new settings.
The following environments do not support HDRs - Asymmetric Deployment - Captive Portal set to Use Request A Websafe blacklist external server will not create HDRs In Active Redundancy, the record will be generated on the NetEnforcer receiving the HTTP request HTTP requests created by a steering service will not be included in HDR
5. NOTES
5-2
HTTPS traffic is not included in HDRs
Configuring HDRs on the Data Mediator

1.
In the Navigation pane, select and right-click the Network in the Navigation tree and select Configuration from the popup menu. OR Select the Network in the Navigation tree and then select Configuration from the View menu. OR Select the Network in the Navigation tree and then click the Configuration icon on the toolbar.
The Configuration window for the Network is displayed.

2. 3.
Select the Data Mediation tab. To configure HDRs, open the Data Definition tab and select the HDR sub-tab.
Figure 5-2: HDR Data Definition Properties

4.
In the HDR sub-tab, the following fields may be configured.
File Size Limit (in bytes) This defines the maximum possible size of each exported file before compression. The default is 100,000 bytes (100 MB) and the maximum possible value is 1,000,000,000 bytes (1 GB).
5-3
File Closing Interval (in seconds) This defines how long after being created the export file is closed. The default is 300 seconds (5 minutes) and the maximum is 3600 seconds (1 hour). File Compression method This sets the file compression method to be used when exporting the file. The possible options are GZIP and no compression. GZIP is selected by default. Delete files older than (in minutes) This defines the amount of time the system retains files. The default is 1440 minutes (24 hours) and the maximum is 4320 minutes (72 hours). Push Properties Push properties, once enabled, define how the export files are sent to their proper locations for analysis and storage. The four push method options are SFTP, FTP, SCP and Copy. If SFTP is selected, the Username, Password, Server and Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the path must be defined. The default is Enabled. Filtering Rules See Below.
Filtering Rules
Using the Filtering Rules area, you may elect to Include All Data in your HDRs or to Exclude All Data from your HDRs by selecting the appropriate radio button. You may then add exceptions to the list below by clicking on the Add button.
Figure 5-3: Add Rule dialog When the Add Rule dialog appears, you may create an exclusion rule by entering a name and clicking Add to add attributes to the rule.
5-4
Figure 5-4: Add Attribute dialog Clicking the Add button on the Add Rule dialog allows you to select to add a ContentType, Path, Method or Domain and then enter the appropriate value. Click OK to save your selections.
KEY ContentType
DESCRIPTION The MIME type of this content (Multipurpose Internet Mail Extensions) The URI
EXAMPLES text/html image/gif image/jpeg /index.path /news
Path
Method
The desired action to be performed on the resource. The domain name of a server.
GET, CONNECT, POST
Domain
www.cnn.com www.ynetnews.com
Naming Conventions
HTTP CDRs files are generated according to the following naming conventions: <Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Data type>_<Version>.[<FE>]
FIELD NAME < Source unit name > <Source file IP>
DESCRIPTION The name of the source unit that generated the file The IP Address of the Core Controller.
NOTES Each name must be unique
5-5
FIELD NAME <Mediation IP> <Seq> <Date> <Data type>
DESCRIPTION The IP address of the Mediation unit that created the file A sequencer that is never reset. Creation date of the file in the form yyyymmddhhmmss Represent the file content: HDR for files containing HTTP CDR records
NOTES
<Version>
Version of the entry format. Format: V<format number> The version is set from the DataExportConfig.xml file where the required output structure is defined.
<FE>
File extension. The file extension is csv.gz for HTTP CDR files. While the file is being copied a .tmp suffix will be added. This suffix will be removed after the copy has been completed successfully.
HDR Interface Fields

ATTRIBUTE NAME
Enforcer Name StartTime
DESCRIPTION
In-line Platform name which generated the CDR Date and Time that the HTTP request detected in the device. By default represented as yyyy-MMddThh:mm:ss+hh:mm This field can also be set to be represented as a simple numeric string containing UTC time in seconds since 1.1.1970
FORMAT
String (in quotes) Date String
5-6
ATTRIBUTE NAME
SubscriberID
DESCRIPTION
Unique subscriber identifier. Zero if deployment is without SMP. Value is external identifier (in mobile networks) or internal identifier (in Fixed networks).
FORMAT
String (Numeric String in mobile environments)
SessionKey
Unique identifier of the subscriber session. The same SessionKey is used for the Subscriber session in HDRs, SDRs and UDRs. Present in subscriber management environment only. If no SMP is in the Network, this value appears as 0.
Numeric String
ClientIP ClientPort ServerIP ServerPort Service name HTTPMethod
IP (IPv4 or IPv6) of the HTTP request originator TCP port of the request originator IP (IPv4 or IPv6) of the designated HTTP server. TCP port of designated HTTP server The Service name mapped from the DPI application identifier HTTP method type of the request (GET, HEAD, POST, etc.) Domain of the HTTP request, taken from the Host header URI field of the request
Dotted Numeric String Numeric String Dotted Numeric String Numeric String String (in quotes) String (in quotes)
Host
String (in quotes)
URI
String (in quotes)
UserAgent Referer DownloadContentL ength
UserAgent header from the HTTP request Referer header from the HTTP Request The content length field from the server response as set by the server application. Limited to 4GB.
String (in quotes) String (in quotes) Numeric String
5-7
ATTRIBUTE NAME
UploadContentLen gth ResponseCode
DESCRIPTION
The content length field from the request as set by the client application. Limited to 4GB. The response code from the server response Time interval in milliseconds from identifying the request until receiving the server response. Transaction duration in milliseconds (until last packet of the response is received) Actual byte count of outgoing traffic as counted by the DPI engine. Actual byte count of incoming traffic as counted by the DPI engine. DNT or x-do-not-track header from the HTTP request
FORMAT
Numeric String
Numeric String
ServerInitialRespo nseTime
Numeric String
Duration
Numeric String
RequestActualByte Count ResponseActualByt eCount DNT
Numeric String Numeric String String (in quotes)
Output Example
Figure 5-5: HDR .csv file example
5-8
Chapter 6: Minutes of Use (MOU)

MOU CDRs present data for the minutes of use for different over the top VoIP applications running through the network. MOU CDRs are exported in CSV file format. Every complete transaction is represented by a record in the CSV file. The CSV files are pushed uncompressed or compressed in GZIP format from the Data Mediator via SCP, SFTP, Copy or FTP (Default is None) to the external system frequently (every 1-2mins). These settings can be changed from the Data Mediation tab in the NetXplorer GUI. The Allot system can be configured to purge the files after a pre-defined period of time (default is 24 hours).
Enabling and Configuring MOU

Enabling MOU per In-Line Platform
1.
In the Navigation pane, select and right-click the relevant In-line Platform in the Navigation tree and select Configuration from the popup menu. OR Select the In-line Platform in the Navigation tree and then select Configuration from the View menu. OR Select the In-line Platform in the Navigation tree and then click the Configuration icon on the toolbar.
The Configuration window for the selected In-line Platform is displayed.

2. 3.
Select the Service Activation tab. To enable MOU, select Enable from the VOIP Data Collection drop down field in the VOIP Reports area.
6-1
Figure 6-1: VOIP Reports in the Service Activation tab

4.
Save the configuration changes to apply the new settings.
Configuring MOU on the Data Mediator

1.

2. 3.
Select the Data Mediation tab. To configure MOUs, open the Data Definition tab and select the MOU sub-tab.
6-2
Figure 6-2: MOU Data Definition Properties

4.
In the MOU sub-tab, the following fields may be configured.
File Size Limit (in bytes) This defines the maximum possible size of each exported file. The default is 100,000 bytes (100 MB) and the maximum possible value is 1,000,000,000 bytes (1 GB). File Closing Interval (in seconds) This defines how long after being created the export file is closed. The default is 300 seconds (5 minutes) and the maximum is 3600 seconds (1 hour). File Compression method This sets the file compression method to be used when exporting the file. The possible options are GZIP and no compression. GZIP is selected by default. Delete files older than (in minutes) This defines the amount of time the system retains files. The default is 1440 minutes (24 hours) and the maximum is 4320 minutes (72 hours). Push Properties Push properties, once enabled, define how the export files are sent to their proper locations for analysis and storage. The four push method options are SFTP, FTP, SCP and Copy. If SFTP is selected, the Username, Password, Server and Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the path must be defined.
6-3
Naming Conventions
MOU files are generated according to the following naming conventions: <Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Data type>_<Version>.[<FE>]
FIELD NAME < Source unit name > <Source file IP> <Mediation IP> <Seq> <Date> <Data type>
DESCRIPTION The name of the source unit that generated the file The IP Address of the file origin. The IP address of the Mediation unit that created the file A sequencer that is never reset. Creation date of the file in the form yyyymmddhhmmss Represent the file content: MOU for files containing Minutes of Use records
<Version>
Version of the entry format. Format: V<format number> The version is set from the configuration file where the required output structure is defined.
<FE>
File extension. The file extension is csv.gz for MOU files. While the file is being copied a .tmp suffix will be added. This suffix will be removed after the copy has been completed successfully.
MOU Interface Fields

ATTRIBUTE NAME
6-4
DESCRIPTION
FORMAT
ATTRIBUTE NAME
Start Time
DESCRIPTION
Start of the time interval measured by this record. By default this is represented as yyyy-MMddThh:mm:ss+hh:mm Can also be set to be represented as a simple numeric string containing UTC time in seconds since 1.1.1970
FORMAT
Date String
End Time
End of the time interval measured by this record By default this is represented as yyyy-MMddThh:mm:ss+hh:mm Can also be set to be represented as a simple numeric string containing UTC time in seconds since 1.1.1970
Date String
Service Plan Name Service Name Total seconds
Service plan name The name of the application mapped from the DPI VoIP usage in seconds
String (in quotes) String (in quotes) Numeric (D)
Output Example
Figure 6-3: MOU .csv file example

Chapter 7: VC Records
NOTE VC Records are automatically enabled for all In-line Platforms.
This information concerns traffic that is classified into different rules in the Allot system. The data is based on the service plan and the subscriber. Depending on the implementation, VC statistics may provide policy enforced application or application group volumes based on the service plan definitions for traffic that needs to be shaped, blocked, steered or TOS-marked. They are extracted from the in-line traffic and are compiled on the in-line platform (e.g: SG-Sigma) VC CDRs are exported in CSV file format. Every complete transaction is represented by a record in the CSV file. The CSV files are pushed uncompressed or compressed in GZIP format from the Data Mediator via SCP, SFTP, Copy or FTP (Default is None) to the external system frequently (every 1-2mins). These settings can be changed from the Data Mediation tab in the NetXplorer GUI. The Allot system can be configured to purge the files after a pre-defined period of time (default is 24 hours).
Configuring VCs on the Data Mediator

1.

2. 3.
Select the Data Mediation tab. To configure VCs, open the Data Definition tab and select the VC sub-tab.
7-1
Figure 7-1: VC Data Definition Properties

4.
In the VC sub-tab, the following fields may be configured.
File Size Limit (in bytes) This defines the maximum possible size of each exported file. The default is 100,000 bytes (100 MB) and the maximum possible value is 1,000,000,000 bytes (1 GB). File Closing Interval (in seconds) This defines how long after being created the export file is closed. The default is 300 seconds (5 minutes) and the maximum is 3600 seconds (1 hour). File Compression method This sets the file compression method to be used when exporting the file. The possible options are GZIP and no compression. GZIP is selected by default. Delete files older than (in minutes) This defines the amount of time the system retains files. The default is 1440 minutes (24 hours) and the maximum is 4320 minutes (72 hours). Push Properties Push properties, once enabled, define how the export files are sent to their proper locations for analysis and storage. The four push method options are SFTP, FTP, SCP and Copy. If SFTP is selected, the Username, Password, Server and Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the path must be defined.
7-2
Naming Conventions
VC CDRs files are generated according to the following naming conventions: <Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Data type>_<Version>.[<FE>]
DESCRIPTION The name of the source unit that generated the file The IP Address of the file origin. The IP address of the Mediation unit that created the file A sequencer that is never reset. Creation date of the file in the form yyyymmddhhmmss Represent the file content: VC for files containing VC CDR records
<Version>
<FE>
File extension. The file extension is csv.gz for VC files. While the file is being copied a .tmp suffix will be added. This suffix will be removed after the copy has been completed successfully.
VC Interface Fields
ATTRIBUTE NAME DESCRIPTION FORMAT
7-3
ATTRIBUTE NAME Enforcer Name Start Time
DESCRIPTION Device name which generated the CDR Start of the time interval measured by this record. By default this is represented as yyyyMM-ddThh:mm:ss+hh:mm Can also be set to be represented as a simple numeric string containing UTC time in seconds since 1.1.1970
FORMAT String (in quotes) Date String
End Time
End of the time interval measured by this record. By default this is represented as yyyyMM-ddThh:mm:ss+hh:mm Can also be set to be represented as a simple numeric string containing UTC time in seconds since 1.1.1970
Date String
Line Pipe VC
Name representing the Line Name representing the Pipe Name representing the Virtual Channel See Note 1 See Note 1 See Note 1 Cell ID Number of live Connection in the time interval Number of dropped Connection in the time interval
String (in quotes) String (in quotes) String (in quotes)
Instance Type Instance Key 1 Instance Key 2 Net Unit ID Live connections
Numeric String Numeric String Numeric String Numeric String Numeric String
Dropped connections
Numeric (D)
7-4
ATTRIBUTE NAME New connections
DESCRIPTION Number of new Connection in the time interval Byte count of incoming traffic Byte count of outgoing traffic
FORMAT Numeric (D)
Octets in Octets out Note 1: INSTANCE TYPE 0 1 2 3 4 5 6
Numeric (D) Numeric (D)
INSTANCE KEY 1 Empty, no instance IP address (IPv4 or IPv6) as Pipe instance IP Address (IPv4 or IPv6) as VC instance Host group name as Pipe instance Host group name as VC instance Subscriber Identifier as Pipe instance Subscriber Identifier as VC instance
INSTANCE KEY 2 Empty Prefix for IPv6 Prefix for IPv6 Empty Empty Session Key Session Key
7-5
Output Example
Figure 7-2: VC .csv file example
7-6
Chapter 8: Conversation Records

This information includes data regarding individual connections on the network. Depending on the implementation, Conversation statistics may provide policy enforced application or application group volumes based on the service plan definitions for traffic that needs to be shaped, blocked, steered or TOS-marked. These statistics are further broken down into a DPI-identified service name. They are extracted from the in-line traffic and are compiled on the in-line platform (e.g: SG-Sigma). Conversations CDRs are exported in CSV file format. Every HTTP complete transaction is represented by a record in the CSV file. The CSV files are pushed uncompressed or compressed in GZIP format from the Data Mediator via SCP, SFTP, Copy or FTP (Default is None) to the external system frequently (every 1-2mins). These settings can be changed from the Data Mediation tab in the NetXplorer GUI. The Allot system can be configured to purge the files after a pre-defined period of time (default is 24 hours).
Enabling and Configuring Conversations

Enabling Conversations Records per In-line Platform
Conversations records must be enabled on each relevant In-line Platform by logging into the platform and entering the following CLI command: go config data_collect conv_export enable The may be disabled by using the following CLI command: go config data_collect conv_export disable
Configuring Conversations Records on the Data Mediator

1.

2. 3.
Select the Data Mediation tab. To configure Conversations CDRs, open the Data Definition tab and select the Conversations sub-tab.
Figure 8-1: Conversations Data Definition Properties

4.
In the Conversations sub-tab, the following fields may be configured.
File Size Limit (in bytes) This defines the maximum possible size of each exported file. The default is 5 minutes and you may opt for bytes (100 MB) and the maximum possible value is 1,000,000,000 bytes (1 GB). Aggregation Period (min) This defines how long after being created the export file is closed. The default is 5 minutes and you may opt for 10, 15, 30 or 60 minutes from the drop down list. This value will affect ALL record types, not just Conversations. File Compression method This sets the file compression method to be used when exporting the file. The possible options are GZIP and no compression. GZIP is selected by default. Delete files older than (in minutes) This defines the amount of time the system retains files. The default is 1440 minutes (24 hours) and the maximum is 4320 minutes (72 hours). Push Properties Push properties, once enabled, define how the export files are sent to their proper locations for analysis and

8-2
storage. The four push method options are SFTP, FTP, SCP and Copy. If SFTP is selected, the Username, Password, Server and Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the path must be defined.
Prioritizing Conversations Statistics

It is possible for the operator to prioritize the Conversation records by application as follows:
Golden: Conversation CDRs are exported in full detail due to high interest. Excluded: Conversation CDRs are exported in an abridged format due to lack of interest (only number of connections and throughput is saved, with no rule ID and client IP). Normal: Conversation CDRs are exported in summary detail. This priority level is set to applications by default.
All Conversation records are collected in detail and are summarized to reduce the storage space required. This feature allows the operator determine which applications should be kept and exported in full Conversation detail, which applications can be summarized and which are exported in a highly abridged format. Sufficient storage space is then set based on these priorities. The operator may further define whether the Conversation data should be exported in 5minute or 1-hour aggregations. All Conversation records are collected in detail and are summarized to reduce the storage space required. This feature allows the operator determine which applications should be kept and exported in full Conversation detail, which applications can be summarized and which need not be exported at all. Sufficient storage space is then set based on these priorities. Assigning Priority Priority levels need to be configured via the CLI using the following commands: To set the priority level: go add app_collection [app_name] priority golden/excluded app_name service catalogue name entry (excluding service groups) To show the priority list by application: go list app_collection Output example
sysadmin@AC3K-5:~$ go list app_collection Golden services: Viber Skype Tango Line MGCP Yahoo Chat Jabber Google Play iTunes Facebook Chat Baidu Hi QQ Chat Twitter Facebook Yandex VKontakte Gmail Call Of Duty GoogleTalk WhatsApp MailRu Yahoo Mail DiabloIII Lineage2 Kakao Excluded services: BGP DNS EGP ICMP IGMP OSPF RIP ARP PPPoE-Discovery PPPoE-Control DHCP-Client DHCP-Server NTP IS-IS LACP Request completed successfully. sysadmin@AC3K-5:~$
To change/remove the configuration for a specific application: go update app_collection [app name] priority golden/excluded go delete app_collection [app name] priority golden/excluded
Naming Conventions
Conversations records are generated according to the following naming conventions:
8-4
<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Data type>_<Version>.[<FE>]
DESCRIPTION The name of the source unit that generated the file The IP Address of the file origin. The IP address of the Mediation unit that created the file A sequencer that is never reset. Creation date of the file in the form yyyymmddhhmmss Represent the file content: CONV for files containing Conversations CDR records
<Version>
<FE>
Conversations Interface Fields

ATTRIBUTE NAME Enforcer Name DESCRIPTION In-line Platform name which generated the CDR FORMAT String (in quotes)
8-5
ATTRIBUTE NAME
DESCRIPTION Start of the time interval measured by this record.
FORMAT
Start Time
By default this is represented as yyyyMM-ddThh:mm:ss+hh:mm Can also be set to be represented as a simple numeric string containing UTC time in seconds since 1.1.1970 End of the time interval measured by this record.
Date String
End Time
By default this is represented as yyyyMM-ddThh:mm:ss+hh:mm Can also be set to be represented as a simple numeric string containing UTC time in seconds since 1.1.1970
Date String
Internal IP External IP Service Name Line Pipe VC Instance Type Instance Key 1 Instance Key 2 Net Unit ID
Internal IP address; both IPv4 and IPv6 are supported External IP address; both IPv4 and IPv6 are supported The name of the application mapped from the DPI Name representing the Line Name representing the Pipe Name representing the Virtual Channel See Note 1 See Note 1 See Note 1 Cell ID
Dotted Numeric String Dotted Numeric String String (in quotes) Numeric String Numeric String Numeric String Numeric String Numeric String Numeric String Numeric String
8-6
ATTRIBUTE NAME Live connections
DESCRIPTION Number of live Connection in the time interval Number of dropped Connection in the time interval Number of new Connection in the time interval Byte count of incoming traffic Byte count of outgoing traffic
FORMAT
Numeric String
Dropped connections
Numeric (D)
New connections Octets in Octets out Note 1: INSTANCE TYPE 0 1 2 3 4 5 6
Numeric (D) Numeric (D) Numeric (D)
INSTANCE KEY 1 Empty, no instance IP address (IPv4 or IPv6) as Pipe instance IP Address (IPv4 or IPv6) as VC instance Host group name as Pipe instance Host group name as VC instance Subscriber Identifier as Pipe instance Subscriber Identifier as VC instance
INSTANCE KEY 2 Empty Prefix for IPv6 Prefix for IPv6 Empty Empty Session Key Session Key
8-7
Output Example
Figure 8-2: CONV .csv file example
8-8
Chapter 9: Session Detail Records (SDR)

SDRs are extracted from the AAA function (e.g: RADIUS) and are compiled on the SMP Server. An SDR is created for every segment of a session (called a sub-session) on a periodic basis (20m by default. To change the interval, contact support@allot.com). A session is broken into a sub-session whenever there is a change in the session data (e.g: switch of service plan) or by time interval. SDRs are exported in CSV file format. Every complete transaction is represented by a record in the CSV file. The CSV files are pushed uncompressed or compressed in GZIP format from the Data Mediator via SCP, SFTP, Copy or FTP (Default is None) to the external system frequently (every 1-2mins). These settings can be changed from the Data Mediation tab in the NetXplorer GUI. The Allot system can be configured to purge the files after a pre-defined period of time (default is 24 hours).
NOTE SDRs are only available from an SMP, not from an In-line Platform.
Enabling SDRs per SMP

Session Detail Records (SDRs) must be generated on the SMP server to extract the relevant information from the RADIUS message. This is enabled via the NetXplorer GUI.
1.
In the Navigation pane of the NetXplorer User interface, rightclick the Network and select Configuration from the popup menu OR Select Network in the navigation pane, and then choose Configuration from the Actions menu. The network tabs will be displayed in the Applications pane.
2. 3.
Select the SMP tab. In the SMP preferences section, 4 additional tabs will appear. Select Session Parameters. From the Session Parameters tab, determine the extra RADIUS parameters which you wish to be included in the SDRs. Check whichever fields you wish to include, as shown in Figure 9-1 below.
9-1
Figure 9-1: Adding Additional RADIUS Parameters to the SDR

NOTES You can also set a default value for some fields. This default value will be included in each SDR for the particular field specified, should no actual value be received. The AVPs which you select in this dialog to be included in the SDR must have first been configured to be extracted from the RADIUS message in the radius_params.conf file. The procedure for extracting these AVPs is detailed in Error! Reference source not ound. 4. NOTE
Select the Policy and Charging tab. Set the SDR Collection Status field to On
Unless SDR Collection Status is defined as On in the Policy and Charging Tab, no SDRs will be generated even if SDR Export is enabled in other areas of the GUI.
9-2
Figure 9-2: Policy and Charging Tab

5.
You will be prompted to reboot the SMP server, whereupon SDR collection will commence.
Configuring SDRs on the Data Mediator

1.

2. 3.
Select the Data Mediation tab. To configure Conversations CDRs, open the Data Definition tab and select the Conversations sub-tab.
9-3
Figure 9-3: SDR Data Definition Properties

4.
In the SDR sub-tab, the following fields may be configured.
File Size Limit (in bytes) This defines the maximum possible size of each exported file. The default is 100,000 bytes (100 MB) and the maximum possible value is 1,000,000,000 bytes (1 GB). File Closing Interval (in seconds) This defines how long after being created the export file is closed. The default is 300 seconds (5 minutes) and the maximum is 3600 seconds (1 hour). File Compression method This sets the file compression method to be used when exporting the file. The possible options are GZIP and no compression. GZIP is selected by default. Delete files older than (in minutes) This defines the amount of time the system retains files. The default is 1440 minutes (24 hours) and the maximum is 4320 minutes (72 hours). Push Properties Push properties, once enabled, define how the export files are sent to their proper locations for analysis and storage. The three push method options are SFTP, FTP and Copy. If SFTP is selected, the Username, Password, Server and Path must be defined. For FTP and Copy, only the path must be defined.
Naming Conventions
SDRs files are generated according to the following naming conventions:
9-4 Data Mediator Installation & Administration Guide
<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Data type>_<Version>.[<FE>]
FIELD NAME
< Source unit name > <Source file IP> <Mediation IP> <Seq> <Date> <Data type>
DESCRIPTION
The name of the source unit that generated the file The IP Address of the file origin. The IP address of the Mediation unit that created the file A sequencer that is never reset. Creation date of the file in the form yyyymmddhhmmss Represent the file content: SDR for files containing HTTP CDR records
NOTES
Each name must be unique
<Version>
<FE>
SDR Interface Fields

ATTRIBUTE NAME
Enforcer Name Record creation timestamp
DESCRIPTION
Device name which generated the CDR SDR generation timestamp
FORMAT
string (in quotes) Date String
9-5
ATTRIBUTE NAME
Subscriber ID
DESCRIPTION
Unique subscriber identifier Zero if deployment is without SMP. Value is external identifier (in mobile networks) or internal identifier (in Fixed networks).
FORMAT
Numeric String
Session Key
Numeric String
Sub-Session Key
Unique sub-session identifier per subscriber session. Sub session is created if there is a change in the service plan of the subscriber during the session. Present in subscriber management environment only
Numeric String
Sub Session Start Time Sub Session End Time Service Plan Open reason
Start time of the sub session End time of the sub session Service plan name Reason for opening the session. One of the following values: START SERVICE_PLAN_CHANGE INFO_CHANGE SCHEDULE
Date String Date String String (in quotes) String (in quotes)
Close reason
Reason for closing the session. One of the following values: SERVICE_PLAN_CHANGE INFO_CHANGE SCHEDULE CLOSE
String (in quotes)
IMEI
This field contains the International Mobile Equipment Identity (IMEI) of the equipment served, if available (3GPP-IMEISV)
Numeric String
9-6
ATTRIBUTE NAME
SGSN IMEI-TAC RAT Source IPv4 Source IPv6 APN Location MCC MNC
DESCRIPTION
Initial SGSN addresses used received on session start (3GPP-SGSN-Address) The TAC part of the IMEI for device type identification (3GPP-IMEISV) Radio Access Type used to start the session (3GPP-RAT-Type) Source address of the session in IPv4 format Source address of the session in IPv6 format Access Point through which the session started (Called-Station-Id) Initial location where the session started (3GPPUser-Location-Info) Mobile Country Code and Mobile Network Code (MCC-MNC) of the subscriber's home operator (3GPP-IMSI-MCC-MNC) Attributes that can extracted on-demand by the end-user Attributes that can extracted on-demand by the end-user Attributes that can extracted on-demand by the end-user
FORMAT
Numeric String Numeric String Numeric String Dotted Numeric String Dotted Numeric String String (in quotes) String (in quotes) String (in quotes)
CustomAttr1 CustomAttr2 CustomAttr3
String (in quotes) String (in quotes) String (in quotes)
Custom Attributes
To define the three custom attributes (CustomAttr1, CustomAttr2 and CustomAttr3), the fields must be added to the radius-dictionary file on each SMP and then added to the DataExportConfig.xml file. These three custom fields will then be extracted from the radius message and saved in the SDR. For example:
9-7
RadiusAttr=x the numeric identifier of a RADIUS AVP RadiusVendorID=y the numeric value of a vendor identifier when RadiusAttr is set to 26 (vendor specific) RadiusVendorAttr=z the numeric value of a vendor specific attribute
To define a custom attribute:

1. 2. 3.
Edit RadiusConfig.xml on the SMP and add the desired fields. For more details see the SMP Installation and Administration Guide. Edit DataExportConfig.xml to include the fields you added to RadiusConfig.xml. Upload the edited DataExportConfig.xml file to the NetXplorer GUI.
For details about working with DataExportConfig.xml, see Appendix C: Customizing Data Export on page 13-3. In the xml example below, the custom attributes appear in red:
- <DataType name="SDR"> <FilteringRules defaultAction="include" /> - <OutputStructure version="1"> <Field ID="41" Type="EnforcerDBID" /> <Field ID="42" Type="RecordCreationTimestamp" /> <Field ID="3" Type="SubscriberID" Obfuscate="Hash" /> <Field ID="4" Type="SessionKey" /> <Field ID="39" Type="SubSessionKey" /> <Field ID="43" Type="SubSessionStartTime" /> <Field ID="44" Type="SubSessionEndTime" /> <Field ID="45" Type="ServicePlanID" Obfuscate="Encrypt" /> <Field ID="46" Type="OpenReason" /> <Field ID="47" Type="CloseReason" /> <Field ID="48" Type="SessionIMEI" /> <Field ID="49" Type="SessionSGSN" /> <Field ID="50" Type="SessionTAC" /> <Field ID="51" Type="SessionRAT" /> <Field ID="52" Type="SessionIPV4" /> <Field ID="53" Type="SessionIPV6" /> <Field ID="54" Type="SessionAPN" /> <Field ID="55" Type="SessionLocation" /> <Field ID="56" Type="SessionMCC" /> <Field ID="67" Type="CustomAttr1" RadiusAttr="26" RadiusVendorID="2603" RadiusVendorAttr="8" /> <Field ID="68" Type="CustomAttr2" RadiusAttr="26" RadiusVendorID="2603" RadiusVendorAttr="9" /> <Field ID="69" Type="CustomAttr3" RadiusAttr="26" RadiusVendorID="10415" RadiusVendorAttr="107" /> </OutputStructure> </DataType>
9-8
Service Plan ID
Every service plan configured in the system has an internal ID number. To determine the internal ID number of each service plan, run the pccAdmin.sh p command on the SMP. One of the logs created is the ServicePlanCatalog.log. This log will list the ID of each Service Plan created, mapping between ID and Service plan name as shown in the sample output below:
less ServicePlanCatalog.log ===== Service Plan Catalog ===== Service Plan Catalog Size= 26 Service Plan Id= 2 ,Service Plan Name= Block Service Plan ,Gx Session Exclude Group Id= 1025 Service Plan Id= 7 ,Service Plan Name= gold ,Gx Session Exclude Group Id= 1030 Service Plan Id= 1 ,Service Plan Name= Unlimited Service Plan ,Gx Session Exclude Group Id= 1024 Service Plan Id= 9 ,Service Plan Name= Unlimited Service Plan_1318349232984 ,Gx Session Exclude Group Id= 0 Service Plan Id= 19 ,Service Plan Name= Block Service Plan_1318349038792 ,Gx Session Exclude Group Id= 0 Service Plan Id= 6 ,Service Plan Name= vc_sp3 ,Gx Session Exclude Group Id= 1029
Open Reason / Close Reason

The open reason and close reason detail why a new sub-session (represented by a new line within the SDR) has been opened. A value between 1 and 6 is used to detail the reason, in accordance with the table below:
VALUE DEFINITION
1 START
Session radius start message received. The first sub-session record of a session will always have an open reason of Start.
2 START_REPORT You will only see this value if Allot Customer Support have set the Session Report on Start status configuration to "enable". In this case, a record with open reason Start is created. The close reason for such records is Start_Report. Start_Report will then also be used as the open reason of the following sub-session. 3 SP_CHANGE 4 INFO_CHANGE 5 SCHEDULE A new SP is assigned to the session. Any session information change received from Radius. Used as both close and open reason. You will only see this value if Allot Customer Support have
9-9
VALUE
DEFINITION
changed the value of the Sub-session lifetime field. By default the configuration is 24hr. When the duration of a sub-session reaches the configured time, the sub-session is closed by the SMP with a Schedule record type and a new sub-session is opened. This is used as both close and open reason. 6 CLOSE Session radius stop message received.
IMEISV Number
The IMEISV number is extracted from the RADIUS message. IMEISV contains 15 digits and has the following format: AA-BBBBBB-CCCCCC-D AA: These two digits are for the Reporting Body Identifier, indicating the GSMA approved group that allocated the TAC (Type Allocation Code). BBBBBB: The remainder of the TAC CCCCCC: Serial sequence of the Model D: Luhn check digit of the entire model or 0 (This is an algorithm that validates the ID number)
Output Example
Figure 9-4: SDR .csv file example
9-10
Chapter 10: Usage Detail Records (UDR)

NOTE UDRs are automatically enabled if SDRs are enabled. If SDRs are not being generated by one or more SMPs, then UDRs cannot be generated.
UDRs collect usage information per session as well as usage information per application within a session. UDR information is extracted from the in-line traffic and is compiled on the in-line platform (e.g: SG-Sigma). UDRs are created on a periodical basis (1hr by default. To change the interval, contact support@allot.com) and count the sub-session uplink and downlink volumes of traffic. When not interrupted by session changes, these records are aligned by the hour (e.g: 12.00, 13.00, 14.00 etc.) UDRs are exported in CSV file format. Every complete transaction is represented by a record in the CSV file. The CSV files are pushed compressed in GZIP format from the Data Mediator to the external system frequently (every 1-2mins). The Allot system can be configured to purge the files after a pre-defined period of time (default is 24 hours).
NOTE A UDR is generated for each monitored application configured, in addition to a UDR with total usage for that session. Allots in-line platforms use proprietary monitoring rules to count the number of bytes used per subscriber for each configured application and for the total bytes used by the subscribers session if configured to do so. The in-line platform will use a single monitoring rule per each application (and for the session counted bytes if configured to do so) defined for UDR collection. Allot supports a total of 13 monitoring rules per subscriber. Every UDR monitoring rule is coupled by an SDR monitoring rule. Therefore the total number of applications which can be monitored per subscriber is 6. Monitoring Rules are also consumed when implementing Gx based volume reporting, an online charging policy, an offline charging policy and service flow detection (event based triggers.
Configuring UDRs on the Data Mediator

1.
In the Navigation pane, select and right-click the Network in the Navigation tree and select Configuration from the popup menu. OR
10-1
Select the Network in the Navigation tree and then select Configuration from the View menu. OR Select the Network in the Navigation tree and then click the Configuration icon on the toolbar.

2. 3.
Select the Data Mediation tab. To configure UDRs, open the Data Definition tab and select the UDR sub-tab.
Figure 10-1: UDR Data Definition Properties

4.
In the UDR sub-tab, the following fields may be configured.
File Size Limit (in bytes) This defines the maximum possible size of each exported file. The default is 100,000 bytes (100 MB) and the maximum possible value is 1,000,000,000 bytes (1 GB). File Closing Interval (in seconds) This defines how long after being created the export file is closed. The default is 300 seconds (5 minutes) and the maximum is 3600 seconds (1 hour). File Compression method This sets the file compression method to be used when exporting the file. The possible options are GZIP and no compression. GZIP is selected by default.
10-2
Delete files older than (in minutes) This defines the amount of time the system retains files. The default is 1440 minutes (24 hours) and the maximum is 4320 minutes (72 hours). Push Properties Push properties, once enabled, define how the export files are sent to their proper locations for analysis and storage. The four push method options are SFTP, FTP, SCP and Copy. If SFTP is selected, the Username, Password, Server and Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the path must be defined. Monitored applications See below.
Monitored Applications
The Monitored Applications list is where applications-based UDRs are configured. The applications to be included in the UDR must be selected. In order for an application to be selected, it must have already been added to the Charging Applications catalog. You may add Monitored Applications to the list by clicking on the Edit button.
Figure 10-2: Change Monitored Applications dialog When the Change Monitored Applications dialog appears, use the Arrow buttons to move applications from the Available Applications list to the Selected Applications list. Only those applications that have previously been added to the Charging Applications catalog will appear in the Available Applications list. Click OK to save and return to the UDR tab. Those applications you have selected now appear in the Monitored Applications list. Modification of the Selected Applications list while the system is running will affect only new sessions. Statistics for sessions that started prior to the modification will continue reporting according to the previous Selected Applications list.
NOTE A UDR is generated for each monitored application configured, in addition to a UDR with total usage for that session. The Session application is created automatically and cannot be removed.
For more information on Charging Applications, see the NetXplorer Operations Guide.
Naming Conventions
UDR files are generated according to the following naming conventions: <Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Data type>_<Version>.[<FE>]
DESCRIPTION The name of the source unit that generated the file The IP Address of the file origin. The IP address of the Mediation unit that created the file A sequencer that is never reset. Creation date of the file in the form yyyymmddhhmmss Represent the file content: UDR for files containing UDR records
<Version>
<FE>
File extension. The file extension is csv.gz for UDR files. While the file is being copied a .tmp suffix will be added. This suffix will be removed after the copy has been completed successfully.
10-4
UDR Interface Fields

ATTRIBUTE NAME
Subscriber ID
DESCRIPTION
Unique subscriber identifier in subscriber management environment as received from SMP. Zero if deployment is without SMP. Value is external identifier (in mobile networks) or internal identifier (in Fixed networks).
FORMAT
Numeric String
Session Key
Numeric String
Sub-Session Key
Unique sub-session identifier per subscriber session. Sub session is created if there is a change in the service plan of the subscriber during the session. Present in subscriber management environment only
Numeric String
Monitored Application Name
The name of the monitored application as defined by the end-user in the application groups catalogs Start of the time interval for which the statistics measures are given (e.g. Octets in) End of the time interval for which the statistics measures are given (e.g. Octets in) Byte count of incoming traffic counted by the DPI engine.
String (in quotes)
Start Time
Date String
End Time
Date String
Octets in
Numeric String
10-5
ATTRIBUTE NAME
Octets out
DESCRIPTION
Byte count of outgoing traffic counted by the DPI engine.
FORMAT
Numeric (D)
Output Example
Figure 10-3: UDR .csv file example
10-6
Chapter 11: Binary Data Records (BDR)

Enabling and Configuring BDRs on the In-Line Platform
To enable BDR identification and data records generation, use the following procedure:
1. 2. 3. 4. 5.
Connect to the in-line platform, as sysadmin Create a directory somewhere called WAConfig Copy the configuration files into that directory. i.e. cp /opt/allot/conf/WebAnalytics/* ./WAConfig Edit your LOCAL ./WAConfig/server.cfg Change the relevant parameters as in the following example:
# # Pilot Packet configuration file # #################################################### # Connection Tuple # protocol list: 0-TCP / 1-UDP / 2-ANY # port: 0-65535 (0 for ANY) # IP: a valid ipv6 address (::ffff:0.0.0.0 for ANY) #################################################### intIp="::ffff:127.0.0.1" intPort=0 extIp="::ffff:255.255.255.255" extPort=8080 protocol=1 #################################################### # DPI Parameters #################################################### # establish the connection: 0-Parsed / 1-Established shouldEstablish=0 # DPI connection timeout optimization ( 0 - disabled / 1 - enabled ) special_timeout=1 # predefined servers mode ( 0 - disabled / 1 - enabled )
11-1
predefined_servers_enable=1 ################################################### # special tuple parameters ################################################### behaviour=0 ################################################### # WebAnalytics ################################################### hdr_enable=1 gcdr_enable=1
Legend
gcdr_enable Enable(1) or disable(0) BDRs.

The default configuration is for BDR to be disabled. For customers using the BDR feature, this flag HAS to be changed to 1. IMPORTANT
11-2
operation_mode 1 is enabled, 0 is disabled. This disables the entire list of servers. extIp / intIp One of these fields is mandatory. The other is optional. Therefore, there should be at least one IP address provided here. intPort , extPort Optional. protocol Indicates whether it's TCP (0) or UDP (1) or ANY (2). For BDRs you should use UDP, 1 , as in the example above. shouldEstablish Must be 0 for BDRs. appId Must be 407 , which is the AppID for "Other IM Applications" which was altered on the previous instructions. hdr_enable Not relevant, this feature is controlled via the NetXplorer GUI. special_timeout 1 = Enabled, 0 = disabled. This enables the use of HDR duration and a real byte count for timeout purposes. After editing this parameter, restart the DataSrv (using keeperMgr -SW R DataSrv) or reboot the host blade.
Use 0 if HDR duration or a real byte count fields are not required.
IMPORTANT 6.
After changing the configuration, please apply using the following command: /opt/allot/bin/wa-update.sh <WACONFIG DIRECTORY> Output Example
# /opt/allot/bin/wa-update.sh /home/sysadmin/WAConfig Creating a tar file... genericcdr.xml webanalytics.xml webanalytics.xsd Updating CCs... Done!
7.
This will copy the files in the directory into /opt/allot/conf/WebAnalytics and apply the configuration.
Configuring BDR Export on the NetXplorer

1.
To enable BDR configuration from the NetXplorer GUI, contact Allot Customer Support.
11-3
2.
Once BDRs have been enabled in the NetXplorer GUI, select and right-click the Network in the Navigation tree and select Configuration from the popup menu. OR Select the Network in the Navigation tree and then select Configuration from the View menu. OR Select the Network in the Navigation tree and then click the Configuration icon on the toolbar.

3. 4.
Select the Data Mediation tab. To configure BDRs, open the Data Definition tab and select the BDR sub-tab.
Figure 11-1: BDR Data Definition Properties

5.
In the BDR sub-tab, the following fields may be configured.
File Size Limit (in bytes) This defines the maximum possible size of each exported file before compression. The default is 100,000 bytes (100 MB) and the maximum possible value is 1,000,000,000 bytes (1 GB). File Closing Interval (in seconds) This defines how long after being created the export file is closed. The default is 300 seconds (5 minutes) and the maximum is 3600 seconds (1 hour).
11-4
File Compression method This sets the file compression method to be used when exporting the file. The possible options are GZIP and no compression. GZIP is selected by default. Delete files older than (in minutes) This defines the amount of time the system retains files. The default is 1440 minutes (24 hours) and the maximum is 4320 minutes (72 hours). Push Properties Push properties, once enabled, define how the export files are sent to their proper locations for analysis and storage. The four push method options are SFTP, FTP, SCP and Copy. If SFTP is selected, the Username, Password, Server and Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the path must be defined. The default is Enabled.
Naming Conventions
BDR files are generated according to the following naming conventions: <Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Data type>_<Version>.[<FE>]
DESCRIPTION The name of the source unit that generated the file The IP Address of the file origin. The IP address of the Mediation unit that created the file A sequencer that is never reset. Creation date of the file in the form yyyymmddhhmmss Represent the file content: B40 for files containing BDR records
<Version>
Version of the entry format. Format: V<format number> The current version is V1
11-5
FIELD NAME <FE>
DESCRIPTION File extension. The file extension is bin.gz for BDR files. While the file is being copied a .tmp suffix will be added. This suffix will be removed after the copy has been completed successfully.
NOTES
11-6
Chapter 12: SNMP Interface

Data Mediator Traps
The Data Mediator sends 5 different types of traps to the Service Providers NOC.
NAME DESCRIPTION OID
alDmCopyToFailure
alDmRecordRateUp
alDmRecordRateDown
alDmCopytoFailureDown
alDNGetFileFailureUp
This trap is sent whenever a file push failed. The alarm is raised provided that a configured interval has passed since the last time a copytofailure alarm was raised towards the same target This trap is sent if the rate of CDRs processed per second passes the configured threshold for each interval. This trap is sent if the rate of CDRs processed per second returns to below the configured threshold for each interval. This trap is sent if there was a failure to push a file to the target host. The trap will include the failed target host. This trap is sent if the get file operation from the in-line platform failed. The trap will include the relevant in-line platform and the relevant filename.
1.3.6.1.4.1.2603.12.4.1
1.3.6.1.4.1.2603.12.4.2
1.3.6.1.4.1.2603.12.4.3
1.3.6.1.4.1.2603.12.4.4
1.3.6.1.4.1.2603.12.4.5
Data Mediator KPIs

The Data Mediator can be polled by an external SNMP server for the purpose of health monitoring. Using the standard HOST-RESOURCES MIB, you can poll each appliance for health monitoring and key performance indicators. The standard LINUX Host-Resources MIB has the OID of 1.3.6.1.2.1.25. It can be polled for information about:
12-1
System Parameters (1.3.6.1.2.1.25.1) Storage Parameters (1.3.6.1.2.1.25.2). This contains information about the storage. Device Parameters (1.3.6.1.2.1.25.3). This contains information about the devices including processor table and disk storage table.
Aside from HOST-RESOURCES, the data mediator can also be polled for the standard cluster software mib and the standard shared storage system mib files. In addition, the following KPIs can be polled from the ALLOT-DATAMED-MIB module
Incoming Records
alDmKpiRecordsIncomingNu m
The number of incoming records since the last boot per data type (VC, Conv, SDR, UDS, HDR, BDR, MOU)
1.3.6.1.4.1.2603.12.3.1.1.1.9
Processed Records
A processed record is one that has been collected, transformed and sent by the DM.
alDmKpiRecordsProcessedNum
The number of records processed by the data mediator since the last boot The number of records processed by the data mediator since the last (configurable) interval The rate of records processed by the data mediator since the last boot The rate of records processed by the data mediator since the last (configurable) interval
1.3.6.1.4.1.2603.12.3.1.1.1.2
alDmKpiRecordsProcessedLastInt ervalNum
1.3.6.1.4.1.2603.12.3.1.1.1.3
alDmKpiRecordsProcessedRate
1.3.6.1.4.1.2603.12.3.1.1.1.4
alDmKpiRecordsProcessedLastInt ervalRate
1.3.6.1.4.1.2603.12.3.1.1.1.5
12-2
Output Records
alDmKpiOutputFilesCreatedNum
The number of files created by the data mediator since the last boot The number of files successfully pushed by the data mediator to the operators SFTP server since the last boot The attempts to push files by the data mediator to the operators SFTP server since the last boot which falied (note: this is not the number of files which have not been pushed) Files to be pushed the total number of files waiting to be pushed.
1.3.6.1.4.1.2603.12.3.1.1.1.6
alDmKpiOutputFilesPushSuccess Num
1.3.6.1.4.1.2603.12.3.1.1.1.7
alDmKpiOutputFilesPushFailNum
1.3.6.1.4.1.2603.12.3.1.1.1.8
alDmKpiOutputFilesPendingPush
1.3.6.1.4.1.2603.12.3.1.1.1.10
12-3
Chapter 13: Appendices

Appendix A: Data Mediator Logs

Datamediation.log a log providing all system messages at two configurable levels, info and debug (debug mode will print additional information in the log). App-datamediation.log a user oriented log, prints messages at 3 levels: failure (logging all failures to push to target or pull from source) push (logging those files successfully pushed) all (logging all files successfully pushed, pulled or created on the Data Mediator)
An entry in the App-datamediation.log contains the following information:
Name or address of the source unit Name or address of the target unit File name Transfer start date and time Transfer end date and time File size in Mbytes Number of records in file - for stored or pushed files Status success or failure Reason description of cause of failure
Appendix B: Upgrade Procedure

Use the following procedures to upgrade your Data Mediator devices:
13-1
To upgrade a Data Mediator:

1.
Contact Allot Customer Support for information about downloading the latest MD rpm package.
2. Place the MD rpm package in a temp folder on the Data Mediator. 3. 4. 5.
Change to root Run the following command: rpm -Uvh <FILE NAME> Ensure that the /etc/hosts file includes a fully qualified domain name (FQDN). Ensure that the file includes the following lines: 10.17.13.24 <IP ADDRESS> localhost.localdomain <DOMAIN> localhost <HOST NAME>
6. 7.
Reboot the device. The new version will be automatically installed Configure the Data Mediator as it had been previously. To do this you will need to change to the root user and configure the following:

8.
Network settings Device settings (dev_setup)
Repeat this process for each Data Mediator.
To upgrade a Data Mediator HA cluster:

1.
Contact Allot Customer Support for information about downloading the latest MD rpm package. Mediator.
2. Place the MD rpm package in a temp folder on each Data 3. Change to root using the following command:
su 4. On the passive node stop the heartbeat service by using the
following command: service heartbeat stop

5. On the active node stop the heartbeat service by using the
following command: service heartbeat stop
13-2
6. On both nodes make sure that there are no DB processes running
by using the following command: ps ax |grep -v grep |grep dbsrv -c
The output should be 0.

7. On both nodes make sure no storage is mounted by entering the
following command: mount |grep dm-1 c
The output should be 0.

8. On DM-0 run the following command:
mount /dev/dm-1 /opt/sybase/data

rpm -Uvh <filename>.rpm Where <filename> is the name of the new MD installation package stored in a temp folder on the Data Mediator/
rpm -Uvh <filename>.rpm Where <filename> is the name of the new MD installation package stored in a temp folder on the Data Mediator/
11. Reboot both nodes.
Appendix C: Customizing Data Export

In some circumstances you may wish to customize the format of the data exported. Allot provides direct access to the configuration file to enable you to do this via the following process.
1.

2.
Select the General tab.
13-3
3.
The current DataExportConfig.xml file is displayed. This file controls what fields are included in the exported CDR for each of the data types. If no custom file has been created, a default file is loaded and displayed here. To customize the contents of the export files, you may compose a new version of the DataExportConfig.xml file following the template shown below. Use the Interface fields as listed for each CDR type in the following chapters. Click Upload and browse to the location of the new DataExportConfig.xml to load it.
The DataExportConfig.xml cannot be edited from the NetXplorer GUI, it is only displayed for reference.
4.
NOTE
DataExportConfig.xml Template
<DataExportConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="./DataExportConfig.xsd"> <DataType name="HDR"> <FilteringRules defaultAction="include"> <Rule action="exclude"> <Condition type="ContentType">image</Condition> <Condition type="ContentType">text</Condition> <Condition type="Host">www.google.com</Condition> </Rule> <Rule action="exclude"> <Condition type="Host">www.ynet.co.il</Condition> <Condition type="ContentType">application/xjavascript</Condition> </Rule> </FilteringRules> <OutputStructure version="1">  <Field ID="2" Type="RequestHeader" Name="Host" MaxLength="70"/> <Field ID="3" Type="StartTime" Format="Numeric/DateTime/String/IPV4/IPV6"/> <Field ID="4" Type="SubscriberID" Obfuscate="None/Hash/Encrypt"/> <Field ID="50" Type="ClientIP"/> <Field ID="51" Type="ClientPort"/>
13-4
 </OutputStructure> </DataType> <DataType name="SDR"> <FilteringRules defaultAction="include"/> <OutputStructure version="1"> <Field ID="50" Type="ClientIP"/> <Field ID="51" Type="ClientPort"/> <Field ID="67" Type="CustomAttr1" RadiusAttr=26 RadiusVendorID=2603 RadiusVendorAttr=8/> </OutputStructure> </DataType> </DataExportConfig>
HDR XML Format

NOTE It is recommended that the OutputStructure version value of the section be increased by one before saving if any changes were made to this section.
- <DataType name="HDR"> <FilteringRules defaultAction="include" /> - <OutputStructure version="4"> <Field ID="2" Type="EnforcerID" /> <Field ID="1" Type="StartTime" /> <Field ID="3" Type="SubscriberID" /> <Field ID="4" Type="SessionKey" /> <Field ID="5" Type="ClientIP" /> <Field ID="6" Type="ClientPort" /> <Field ID="7" Type="ServerIP" /> <Field ID="8" Type="ServerPort" /> <Field ID="9" Type="ServiceID" /> <Field ID="10" Type="HTTPMethod" MaxLength="10" /> <Field ID="11" Type="RequestHeader" Name="Host" MaxLength="70" /> <Field ID="12" Type="URI" MaxLength="150" /> <Field ID="13" Type="RequestHeader" Name="User-Agent" MaxLength="150" /> <Field ID="14" Type="RequestHeader" Name="Referer" MaxLength="150" /> <Field ID="15" Type="ResponseHeader" Name="Content-Type" MaxLength="50" /> <Field ID="16" Type="DownloadContentLength" /> <Field ID="17" Type="UploadContentLength" /> <Field ID="18" Type="ResponseCode" MaxLength="20" /> <Field ID="19" Type="ServerInitialResponseTime" /> <Field ID="20" Type="Duration" /> <Field ID="21" Type="RequestActualByteCount" />
13-5
<Field ID="22" Type="ResponseActualByteCount" /> <Field ID="23" Type="RequestHeader" Name="DNT,x-do-not-track" MaxLength="1" /> </OutputStructure> </DataType>
UDR XML Format

- <DataType name="UDR"> <FilteringRules defaultAction="include" /> - <OutputStructure version="1"> <Field ID="3" Type="SubscriberID" /> <Field ID="4" Type="SessionKey" /> <Field ID="39" Type="SubSessionKey" /> <Field ID="40" Type="ChargingApplicationID" /> <Field ID="1" Type="StartTime" /> <Field ID="59" Type="EndTime" /> <Field ID="32" Type="OctetsIn" /> <Field ID="33" Type="OctetsOut" /> </OutputStructure> </DataType>
MOU XML Format

- <DataType name="MOU"> <FilteringRules defaultAction="include" /> - <OutputStructure version="1"> <Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="45" Type="ServicePlanID" /> <Field ID="9" Type="ServiceID" /> <Field ID="58" Type="SessionDuration" /> </OutputStructure>
VC XML Format
<DataType name="VC"> <FilteringRules defaultAction="include" /> - <OutputStructure version="1"> <Field ID="2" Type="EnforcerID" /> 13-6 Data Mediator Installation & Administration Guide
<Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="25" Type="LineID" /> <Field ID="26" Type="PipeID" /> <Field ID="27" Type="VCID" /> <Field ID="28" Type="InstanceType" /> <Field ID="29" Type="InstanceKey1" /> <Field ID="30" Type="InstanceKey2" /> <Field ID="31" Type="NetUnitID" /> <Field ID="32" Type="LiveConnections" /> <Field ID="33" Type="DroppedConnections" /> <Field ID="34" Type="NewConnections" /> <Field ID="37" Type="OctetsIn" /> <Field ID="38" Type="OctetsOut" /> </OutputStructure> </DataType>
Conversations XML Format

- <DataType name="CONV"> <FilteringRules defaultAction="include" /> - <OutputStructure version="1"> <Field ID="2" Type="EnforcerID" /> <Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="5" Type="ClientIP" /> <Field ID="7" Type="ServerIP" /> <Field ID="9" Type="ServiceID" /> <Field ID="25" Type="LineID" /> <Field ID="26" Type="PipeID" /> <Field ID="27" Type="VCID" /> <Field ID="28" Type="InstanceType" /> <Field ID="29" Type="InstanceKey1" /> <Field ID="30" Type="InstanceKey2" /> <Field ID="31" Type="NetUnitID" /> <Field ID="32" Type="LiveConnections" /> <Field ID="34" Type="NewConnections" /> <Field ID="37" Type="OctetsIn" /> <Field ID="38" Type="OctetsOut" /> </OutputStructure> </DataType>
SDR XML Format

- <DataType name="SDR">
13-7
<FilteringRules defaultAction="include" /> - <OutputStructure version="1"> <Field ID="41" Type="EnforcerDBID" /> <Field ID="42" Type="RecordCreationTimestamp" /> <Field ID="3" Type="SubscriberID" /> <Field ID="4" Type="SessionKey" /> <Field ID="39" Type="SubSessionKey" /> <Field ID="43" Type="SubSessionStartTime" /> <Field ID="44" Type="SubSessionEndTime" /> <Field ID="45" Type="ServicePlanID" /> <Field ID="46" Type="OpenReason" /> <Field ID="47" Type="CloseReason" /> <Field ID="48" Type="SessionIMEI" /> <Field ID="49" Type="SessionSGSN" /> <Field ID="50" Type="SessionTAC" /> <Field ID="51" Type="SessionRAT" /> <Field ID="52" Type="SessionIPV4" /> <Field ID="53" Type="SessionIPV6" /> <Field ID="54" Type="SessionAPN" /> <Field ID="55" Type="SessionLocation" /> <Field ID="56" Type="SessionMCC" /> </OutputStructure> </DataType>
Using Obfuscation
Obfuscation can be defined for any attribute in DataExportConfig.xml by editing and uploading the file. The function will be executed as the last function after any other transformation function that is defined for the attribute. As an example, Subscriber ID would first be transformed from binary to a string, and then would be obfuscated (if so requested). Two obfuscation methods are supported:
Hash irreversible, using SHA-1 algorithm Encrypt reversible, using AES 128 bit algorithm
Example
In this example, the HDR section of the DataExportConfig.xml file has been edited to define with obfuscation for the SubscriberID and URI attributes, appearing in red:
- <DataType name="HDR"> <FilteringRules defaultAction="include" /> - <OutputStructure version="4"> <Field ID="2" Type="EnforcerID" /> <Field ID="1" Type="StartTime" /> <Field ID="3" Type="SubscriberID" Obfuscate="Hash" /> <Field ID="4" Type="SessionKey" /> <Field ID="5" Type="ClientIP" /> <Field ID="6" Type="ClientPort" /> <Field ID="7" Type="ServerIP" /> <Field ID="8" Type="ServerPort" />
13-8
<Field <Field <Field <Field <Field />
ID="9" Type="ServiceID" /> ID="10" Type="HTTPMethod" MaxLength="10" /> ID="11" Type="RequestHeader" Name="Host" MaxLength="70" /> ID="12" Type="URI" MaxLength="150" Obfuscate="Encrypt" /> ID="13" Type="RequestHeader" Name="User-Agent" MaxLength="150"
<Field ID="14" Type="RequestHeader" Name="Referer" MaxLength="150" /> <Field ID="15" Type="ResponseHeader" Name="Content-Type" MaxLength="50" /> <Field ID="16" Type="DownloadContentLength" /> <Field ID="17" Type="UploadContentLength" /> <Field ID="18" Type="ResponseCode" MaxLength="20" /> <Field ID="19" Type="ServerInitialResponseTime" /> <Field ID="20" Type="Duration" /> <Field ID="21" Type="RequestActualByteCount" /> <Field ID="22" Type="ResponseActualByteCount" /> <Field ID="23" Type="RequestHeader" Name="DNT,x-do-not-track" MaxLength="1" /> </OutputStructure> </DataType>
13-9

Data Mediator Install and Admin Guide R2 (13.1.10)

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Data Mediator Install and Admin Guide R2 (13.1.10)

Загружено:

Авторское право:

Доступные форматы

Data Mediator

Installation and Administration Guide

Data Mediator Installation & Administration Guide

v2b3 v2b2 v2b1

MED13.1 MED13.1 MED13.1

29.11.12 25.11.12 21.11.12

Data Mediator Installation & Administration Guide

CHAPTER 1: OVERVIEW ................................................................................ 1-1

CHAPTER 2: HARDWARE .............................................................................. 2-1

CHAPTER 3: INSTALLING DATA MEDIATOR ............................................... 3-1

CHAPTER 4: CONFIGURING DATA MEDIATOR ........................................... 4-1

CHAPTER 5: HTTP DETAIL RECORDS (HDR) .............................................. 5-1

CHAPTER 6: MINUTES OF USE (MOU) ......................................................... 6-1

CHAPTER 7: VC RECORDS ........................................................................... 7-1

CHAPTER 8: CONVERSATION RECORDS .................................................... 8-1

CHAPTER 9: SESSION DETAIL RECORDS (SDR) ........................................ 9-1

CHAPTER 10: USAGE DETAIL RECORDS (UDR) ........................................10-1

CHAPTER 11: BINARY DATA RECORDS (BDR) ..........................................11-1

CHAPTER 12: SNMP INTERFACE .................................................................12-1

CHAPTER 13: APPENDICES .........................................................................13-1

Data Mediator Installation & Administration Guide

Data Mediator Installation & Administration Guide

Conversation Session (SDR/UDR) HDR VC MOU BDR

VOIP usage per service plan and service

Binary raw data

Record Export Rates

AC1400 AC3000 AC6000

180,000 1,000 160,000 80,000 600,000 300,000 1,600

180,000 1,000 160,000 80,000 600,000 300,000 1,600

360,000 7,500 500,000 250,000

480,000 2,500 500,000 250,000

1,500,000 10,000 1,200,000 600,000 1,500,000 750,000 1,600

1,200,000 600,000 600,000 1,600 300,000

Data Mediator Installation & Administration Guide

10-60 MINS AGGREGATION 144M 180M 11M 200K

Data Flow System Level

Figure 1-1: System Level Data Flow

Data Flow Data Mediator

Transformed Output (4)

Data Mediator Figure 1-2: Data Mediator Data Flow

Data Mediator Installation & Administration Guide

Data Mediator Installation & Administration Guide

Data Mediator Installation & Administration Guide

PA-MED-SRV (AC standalone) PA-MED-SRV-HAP (AC HAP) PA-MED-SRV-HAP-DC (DC HAP)

Figure 2-1: PA-MED-SRV Front View

Data Mediator Installation & Administration Guide

Figure 2-3: PA-MED-SRV-HAP cluster with Storage device Front View

Figure 2-4: AC Power Inlets

Data Mediator Installation & Administration Guide

100-127 VAC, 10A max 200-240 VAC, 5A max 50/60Hz

Data Mediator Installation & Administration Guide

Connect a power cord to each power supply in the storage subsystem.

Figure 2-5: AC Power Inlet, Storage (Right Side)

Figure 2-6: DC Power Inlets

Data Mediator Installation & Administration Guide

Figure 2-7: DC Power Inlet, Storage (Right side)

Data Mediator Installation & Administration Guide

Figure 2-8: DC Pin Positions

Pin 1: - 48 V dc Pin 2: POS RTN Pin 3: GND

Brown Blue Green/yellow

Data Mediator Installation & Administration Guide

Figure 2-9: DC Storage Device Wiring

Data Mediator Installation & Administration Guide

Chapter 3: Installing Data Mediator