Академический Документы
Профессиональный Документы
Культура Документы
RF Acceptance Document
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
Approvals
Huawei
GP
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
Contents
1. Introduction 2. KPI Requirement 2.1 PAT KPI Requirements 2.2 FAC KPI Requirement 3. Deliverables 3.1 Deliverables for Drive Test 3.2 Deliverables for Traffic Performance Statistics
.. 4 .. 4 .. 4 .. 5 .. 8 8 9
4. Acceptance Test Methodology and Procedure 10 4.1 General Conditions 10 4.2 Acceptance Test Procedure 110 4.3 DT Acceptance Test Conditions ...12 4.4 Traffic Performance Statistics Acceptance Test Conditions 15
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
1.
Introduction:
This document constitutes the Benchmark test and Network Acceptance report for the Grameen Phone Huawei GSM Network Swap Project. The purpose of this document is to present the acceptance procedures and the standardize and the methods of the testsing the for radio network. The tests for Huawei consist of two parts: Drive tests and traffic statistics. This document provides the detailed methodology for the drive test and traffic performance statistics measurements. The following areas are covered in this document: KPI Requirement Deliverables for Drive test and Traffic performance statistics Acceptance Test Methodology and Procedures
Order of precedence of this document, if any conflict arises, is as follows: a. Section-2 of this document b. RFQ (Annex 30)
2.
KPI Requirement:
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
- 1. Major Network Outage (Power, Transmission) - 2. Festival, Event - 3. Major Transmission Fluctuation - 4. Environmental impact
Operational issues (1 and 3) have to be communicated to BO/SOC for validation. GP will provide feedback within 2 days of submitting the request. Non operational issues (2 and 4) will be validated by KPI management team. For FAT weekly aggregated (3 BH where mentioned) KPI performance will be measured. If the weekly aggregated (3 BH where mentioned) targets are achieved for at least 3 consecutive weeks the FAT process can be carried out. Additionally during these three weeks period traffic should be at least 90% of the average traffic in the cell or cluster.
Retainability (MPD) No. of drops must not be more than pre-swap drive test. (Post drive test duration will be the same as pre drive test duration) Integrity (MOS) Rxqual Rx level (street level) Coverage Area of Probability E2E call establishment success rate Call setup time e2e (voice)
KPI Statistics Benchmarking: SDCCH Drop (excluding congestion) Handover Success Rate, Intra BSC
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
Handover Success Rate, Inter BSC/MSC Minutes Per Drop (MPD) Paging Success Rate Random Access Success Rate PDCH Allocation Success Rate TBF Establishment Success Rate Throughput TCH Congestion (3BH) SDCCH Congestion (including reattempt) (3BH)
2.2 FAC KPI Requirements For FAT all KPI targets mentioned in Annex 30 (and also mentioned below) in both cell and BSC wise cluster will have to be achieved.
For FAT weekly aggregated (3 BH where mentioned) KPI performance will be measured. If the weekly aggregated (3 BH where mentioned) targets are achieved for at least 3 consecutive weeks the FAT process can be carried out. Additionally during these three weeks period traffic should be at least 90% of the average traffic in the cell or cluster.
BSC along with all890% cells (in case of some exceptions ) should meet FAC target within 812 weeks (provided that consecutive 3 weeks FAC target will meet) out of 6 months FAC time . The rest cells must achieve FAC target in remaining FAC period.
W 1 W 2 W 3 M1 W 4 W 5 W 6 W 7 M2 W 8 W 9 W1 0 W1 1 M3 W1 2 W1 3 W1 4 W1 5 M4 W1 6 W1 7 W1 8 W1 9 M5 W2 0 W2 1 W2 2 W2 3 M6 W2 4
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
1. This one can be accepted with some modification such as instead of 2 months it can be 3 months and there can be a percentage of cells (70%) to achieve within the timeframe.
2. During FAC, all cell level KPI should be maintained above 80% of benchmark until FAC signing,
unless there is any major shortcoming (if the benchmark is higher than the FAC)
During FAC, all cell level KPI should be maintained above benchmark until FAC signing, unless there is any major shortcoming (if the benchmark is lower than the FAC). ?? 3. KPI relaxation can be given for FAC for the following cases only after thorough review by GP KPI Management Team - Major Network Outage (Power, Transmission) - Festival Event - Transmission Fluctuation
2.2.1
DT KPI Requirement
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
Sec.
KPI Retainability Integrity Rxqual Rx level (street level) Coverage area Probability E2E call establishment success rate Call Setup Time (E2E for Voice)
% %
99 97
Sec.
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
2.2.2 Traffic Performance Statistics KPI Requirement The detail statisti cs KPI standard in year 2008 is as below:
Dense Urban + Urban Cell Cluster Level Level 1% 92% 92% 75 N/A 99% 90% 99.8% 3 3 0.5% 98% 96% 170 94% 99.5% Sub Urban Cell Cluster Level Level 1.2% 92% 90% 70 N/A 99% 0.6% 97% 95% 160 95% 99.5% Rural Cell Level 1.4% 90% 85% 60 N/A 99%
Radio KPI SDCCH Drop (excluding congestion) Handover SR (Intra BSC) Handover SR (Inter BSC/MSC) MPD Paging SR Random Access SR PDCH Allocation SR TBF Establishment SR Throughput % of TCH Congestion (3BH) SDCCH Congestion including reattempt (3 BH)
1.5 1.5
3 3
1.5 1.5
3 3
1.5 1.5
Radio KPI SDCCH Drop (excluding congestion) Handover SR (Intra BSC) Handover SR (Inter BSC/MSC) MPD Paging SR Random Access SR PDCH Allocation SR TBF Establishment SR Throughput % of TCH Congestion (3BH) SDCCH Congestion including reattempt (3BH)
Sub Urban Cell Cluster Level Level 1.2% 92% 90% 70 N/A 99% 0.6% 97% 95% 170 94% 99.5%
1.5 1.5
3 3
1.5 1.5
3 3
1.5 1.5
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
Radio KPI SDCCH Drop (excluding congestion) Handover SR (Intra BSC) Handover SR (Inter BSC/MSC) MPD Paging SR Random Access SR TBF Establishment SR Throughput % of TCH Congestion (3BH) SDCCH Congestion including reattempt (3BH)
Dense Urban + Urban Cell Cluster Level Level 1% 92% 92% 75 N/A 99% 99.8% 3 3 1.5 1.5 0.5% 98% 96% 185 94% 99.5%
Sub Urban Cell Cluster Level Level 1.2% 92% 90% 70 N/A 99% 0.6% 97% 95% 175 95% 99.5%
3 3
1.5 1.5
3 3
1.5 1.5
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
10
SDCCH Drop (excluding congestion) will be measured instead of SDCH Drop due to quality. Huawei will also work in developing measurements for SDCCH Drop due to quality. SDCSDCCH Drop (excluding congestion) will be measured instead of SDCH Drop due to quality. Huawei will also work in developing measurements for SDCCH Drop due to quality Since PDP Context Activation (in cell level) cannot be measured in Ericsson system, Throughput will be measured. The target values for Data Throughput will has beenbe given by GP which is 110 kbps for all clutter type. . HO target between Huawei and Ericsson system will be agreed upon. For Handover Success Rate, Intra BSC For cell level, 95% of the cells will meet target KPI while remaining 5% will maintain at least 85% success rate. For Handover Success Rate, Inter BSC/MSC For cell level, 95% of the cells will meet the target KPI, remaining 5% will maintain 85% success rate. For 2010, Data Throughput should be 11520 kbps for all clutter type. If swap executes in 2010, FAC target will be based on 2010 Throughput targets and benchmarked values will be decided by both GP and Huawei.
1.
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
11
Detailed statistical report from TEMS report generator (details of events such as HO fail, block call, call drop, total no. of samples, % of samples, unsuccessful attempts, timer expires etc.) KPI table: Pre vs. Post; Target vs. Achieved Drive test Logs Recommendations
The plots that will be provided are Voice Call Plots Rxlevsub Rxqualsub Coverage area probability Plot Y Y Y
The following information will be provided by Huawei: All DT KPI mentioned in section 2.2.1 (both pre and post swap) All the analysis for drop and access failures will be included
Rx-Lev Band
Quality Band
-70 to 0 -75 to -70 -80 to -75 -85 to -80 -90 to -85 -105 to -90
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
12
3.2 Deliverables for Traffic Performance Statistics: After the Benchmark Test (Pre and Post) for PAT and the Final Acceptance Test the following information will be provided by Huawei: Cell level traffic statistics (mentioned in section 2.2.2) Resolution: hourly (for MPD), daily and weekly. Cluster level traffic statistics (mentioned in section 2.2.2) Resolution: hourly (for MPD), daily and weekly. KPI table: Pre vs. Post; Target vs. Achieved Recommendations
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
TBF Establishment SR
% of TCH Congestion
PDCH Allocation SR
Random Access SR
Paging SR
Cluster ID
MSC ID
BSC ID
MPD
13
Network tuning to achieve benchmark or better KPI for Cluster Conditional Acceptance Test is within two weeks (after commercial traffic).
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
TBF Establishment SR
% of TCH Congestion
PDCH Allocation SR
Random Access SR
Cluster ID
MSC ID
BSC ID
Cell ID
MPD
14
Network tuning to achieve target or better KPI in BSC wise cluster and cell level for Final Acceptance Test is in a maximum of six months (after Conditional Acceptance). FAT will start only after all cells and clusters have met the target KPI listed in Annex 30 and/or section 2.2 of this document. 4.2 Acceptance Test Procedure: For the GP-Huawei Swap project benchmark test will be taken by Huawei. After swap CACPAT will be completed within two weeks. When the Cell and Cluster level KPIs get the FAC requirement, the FAC will be processed by Huawei and GP as per guideline mentioned in 2.2 FAC KPI Requirements.. Test tools and test routes should completely be the same for all tests. After each test a report will be submitted by Huawei and this will contain the deliverables mentioned in Section 3 of this document.
4.2.1 Cluster Definition and Test Routes A pre and post swap drive test will be performed by Huawei to verify the radio network performance in a cluster. Minimum drive test route to cover for Benchmarking will be provided by GP. Huawei has to cover this route as a minimum. The planning of the drive test route will consider handover performance, neighbor relations, coverage, interference and quality. The test route will be carefully planned in advance by GP. All sites on the cluster will be mapped on the site map. This map will indicate the theoretical cell borders. Stationary Data Test will be incorporated in the Acceptance Procedure. A separate document of complete guideline will be finalized between Huawei and Grameephone.. Huawei will do stationary drivetest in random locations provided by GP for data service without any affect to acceptance procedure. 1. Stationary test: Not applicable for KPI measurement In general the test routes will be planned according to the following criteria: The routes will pass through the cells of the concern in the cluster The routes will cover all the major areas The route will be planned so that handovers are performed between as many cells as possible, i.e. all of the most important cells will be crossed
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
15
The routes will pass through important areas including busy roads, high traffic areas, business areas, VIP areas etc.
In order to know the performance of existing network, drive test will be done before swap by Huawei. Benchmark test will be based on Cluster and the same test route and time will be used before and after swap. Before the swap, GP will provide the Ericsson existing network statistics to Huawei as the statistics KPI baseline. The statistics period is 2 weeks before each cluster is swapped. The statistics will be from Sunday to Saturday (7 days) on an hourly basis. 4.2.3 CACPAT Test As soon as all the sites in one cluster have been swapped, optimization may start. Continuing from 1-2 weeks of optimization and observation of network performance, CACPAT will be executed within two weeks. The complete PAT process for each cluster will be completed within 145 days from the date of cutover/of cluster swap. 4.2.4 FAC Test After CACPAT, Huawei will expand the network for whole network, based on the traffic increase. During this time, continuous optimization will be on-going and network performance will be monitored on a weekly basis. If the cluster and cell KPI achieves the target KPI for 3 consecutive weeks, FAC process will then be executed. During FAC cell level tuning and optimization can be carried out. For this purpose Radio Network Optimization (RNO) Process should be followed as mentioned 1. Worst Performing cells handling: Analysis, Follow-up and Rectification 2. Detail Technical Analysis: Parameter modification, rectification process
3. 3. SLA: Weekly reporting of SLA for worst performing cell and customer complain SLA for Customer complains will be according to GP/HW agreed standard*.
* Will be incorporated after commercial negotiation is settled between GP and Huawei. (without SLA??)
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
16
4. Worst Performing Cell Handling Process and Reporting as per below table -
4.3 DT Acceptance Test Conditions 4.3.1 Drive Test Tools 1) Test laptop configuration will be equipped with RAM higher than 256 M, Hard disk capacity higher than 30 G. 2) Drive test tool is TEMS Investigation at least above version 7. 3) MOS test tool is QVOICE; GP will let Huawei use its QVOICE only if any unit of Huaweis is out of service. 4) Handset will support all of the test items 5) External antenna with 0 dB gain
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
17
6) Digital map will be available. 7) SIM cards will be supplied by GP. Each one will be Voice/EDGE capable 8) Test numbers will be provided by GP. 1) Test laptop configuration will be equipped with RAM higher than 256 M, Hard disk capacity higher than 30 G. 2) Drive test tool is TEMS Investigation at least above version 7. 3) MOS test tool is QVOICE; GP will let Huawei use its QVOICE only if any unit of Huaweis is out of service. 4) Handset will support all of the test items 5) External antenna with 0 dB gain 6) Digital map will be available. 7) SIM cards will be supplied by GP. Each one will be Voice/EDGE capable 8) Test numbers will be provided by GP. 4.3.2 Network Acceptance Test Time Benchmark test will start 2 weeks before the swap starts.
For PAT network optimization should be finished within 2 weeks after the swap. Within 6 months after PAT, when cell and cluster KPI has achieved FAC KPI requirement, Huawei will initiate and complete the FAC test. Measurement of performance will be for 2 weeks before the swap. Data resolution will be both hourly and daily for each day of the week. After swap both hourly data (for MPD) and daily data in cell and cluster level will be measured. After PAT statistical data measurement will be on a weekly basis until FAT is complete.
4.3.3 Cluster Drive Test Procedure (Before and after swap) When the basic functionality of the site have been checked, the next tasks are to verify that the handover relations are working properly, to detect the interference areas, to note unexpected coverage holes and to verify parameter and frequency allocations. A drive test is performed to verify the radio network performance in cluster by Huawei. GP must be informed upon completion of the pre-swap drive test results to validate the accuracy of the data and the selection of the drive test KPI data. Dedicated Mode (short call and long call) and idle mode drive testing will be done for each task. The procedure is defined as follows:
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
18
1. Open a log file and then start a call to another mobile (for short call) located in same radio environment (both A and B party in same car); A party being TEMS Investigation phone. 2. For short calls adjust TEMS setup and call duration as follows: Test call length 15 sec. Delay between the end of one call and the start of the next 5 sec. 3. For long call one will be TEMS mobile (calling mobile) and the other a test number in the same vehicle (i.e. same radio environment). 4. There is no maximum duration for a long call. Call will continue through the whole drive test period. Call will be reconnected if drop occurs in drive test period. 5. During each drive test Huawei will try to maintain the vehicle speed will not exceed 30 km/hour.as per below mentioned be in the following limits.. Clutter Type Rural/Suburban Dense Urban/Urban Dense Urban/Urban Highway Vehicle Speed Theshold (km/hr) 2530-40 15-40 >40 405-60 Percentage to be achieved (%) 80 60 0 80
6. 7. Drive test recording will be paused when the vehicle stands still due to road traffic or any other disturbance. 8. All Drive Tests will be performed within 1000 Hours to 2200 Hours. 9. No major overlapping will be allowed in the tested Drive Test routes. 10. External antenna (with 0 dB gain) will be mounted on top of the vehicle for all drive tests. 11. The number of minimum samples per drive test will be provided by GP after the number of sites per cluster has been finalized. 12. Number of samples in pre and post Drive Test for CAC will not be deviated by moreby more than 10%. 13. Pre and Post swap drive test will be in similar period of the day of the week. 14. MOS test will be done in both Pre and Post swap drive test. 15. If poor RxLev or RxQual (worse than the thresholds in 2.2.1) continues for 150m for Rural and Highway type clutters and 50m for other clutter types then it will be considered as a Black Spot. If a black spot found in the post-swap Drive Test was not in the pre-swap Drive Test then for passing CAC and FAC, it must be removed. 16. Lengths of logs need to be increased to avoid bulk number of log files. 17. Drive Test for FAC will be performed continuously and on consecutive dates. No post-tuning log will be added with the original report. If network tuning work is done after BSC drive-test is finished then a new boundary will be given by GP
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
19
where tuning activities was initiated and the log file will be replaced accordingly.After doing all the tuning works one complete Drive Test need to be done for the whole BSC. In unavoidable minimum cases, after optimization all old logs between a call attempt and the call end of that call may be replaced by the new one in the original report.
Dedicated Mode:
Both short call and long call test will be performed as outlined above. Purpose: To evaluate the network accessibility, retainability and integrity of the network. KPI Definition: Signal Quality RxQual Downlink (Sub value): percentage of measurement report below or equal to RxQual = 5 E2E Call Establishment (mobile to mobile): No. of Successful Calls/No. of Call Attempts Preconditions: Predefined routes Assuming no loss of coverage (due to site down) Assuming sample drive test before and after are similar
Actions: 1. Start the TEMS unit in the drive test vehicle and record the measurements to log file simultaneously with the idle mode. 2. Configure command sequence in TEMS for dedicated calls. 3. Set quick logging to 15,000 messages 4. Stop TEMS recording after completing the drive test swap area or cluster Result: The results for the case will be stored in the log files. This makes a statistical analysis of the possible results. Import all the log files using Report Generator of the TEMS equipment to get the number of samples gathered during drive test. Import all the log files into MapInfo to plot the RxQual as specified in the range in the Deliverables in section 3.1.
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
20
Acceptance: Benchmarking is based on one drive test before and one drive test after swap. The drive test after swap should be equal to or better than the pre-drive test.
Idle Mode:
Purpose: To evaluate the Coverage area of Probability and RxLev. KPI Definition: Signal Quality RxLev (dBm) Coverage Area of Probability: the percentage of the sample of receive level (above threshold)/Total no. of samples Preconditions: Predefined routes Measurements to be conducted on Idle mode on the predefined route Assuming no loss of coverage (due to site down) Assuming sample drive test data for before and after are similar Required street level coverage (will be using 0 dB external antenna)
Actions: 1. Start the TEMS unit in the drive test vehicle and record the measurements to log files simultaneously with the dedicated mode. 2. Set quick logging to 15,000 messages Result: The results for the case will be stored in the log files. This makes a statistical analysis of the possible results.
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
21
Import all the log files using Report Generator of the TEMS equipment to get the number of samples gathered during drive test. Import all the log files into MapInfo to plot the RxLev as specified in the range in the Deliverables in section 3.1. Acceptance: Benchmarking is based on one drive test before and one drive test after swap. The drive test after swap should be equal to or better than the pre-drive test. 4.4 Traffic Performance Statistics Acceptance Test Conditions Measurement of performance will be done on a daily basis starting from Sunday through Saturday (7 days) from 08:00 H to 23:00 H (CAC). For at least the first 24 hours fromFrom the swap hourly statistics (for MPD) in both cell and cluster level will be measured. After PAT is done performance will be measured weekly up to the completion of FAT (24 hours data).
GrameenPhone Ltd., Huawei Technologies Co. Ltd. & Huawei Technologies (Bangladesh) Ltd.
22