High Speed Networks

Avish Shah - 12mict21

Evalvid
Introduction EvalVid is a framework and tool-set for evaluation of the quality of video transmitted over a real or simulated communication network. It is targeted for researchers who want to evaluate their network designs or setups in terms of user perceived video quality. Besides measuring QoS parameters of the underlying network, like loss rates, delays, and jitter, a subjective video quality evaluation of the received video is provided. Currently H.263, MPEG-4 and H.264 are supported. Framework and Design In Figure ,the structure of the EvalVid framework is shown. The interactions between the implemented tools and data flows are also symbolized. In Section 3 it is explained what can be calculated and Section 4 shows how it is done and which results can be obtained.Also, in Figure , a complete transmission of a digital video is symbolized from the recording at the source over the encoding, packetization, transmission over the network, jitter reduction by the play-out buffer, decoding and display for the user. Furthermore the points, where data are tapped from the transmission flow are marked. This informa-tion is stored in various files. These files are used to gather the desired results, loss rates, jitter, and video quality. A lot of information is required to calculate these values. The required data are (from the sender side): raw uncompressed video encoded video time-stamp and type of every packet sentand from the receiver side: time-stamp and type of every packet received reassembled encoded video (possibly errorneous) raw uncompressed video to be displayed Source: raw (uncoded) video files are usually stored in YUV format. VS (Video Sender): generates two trace files necessary for the subsequent video quality evaluation - sender trace file - video trace file

Assignment 1 : Evalvid

Page 1

High Speed Networks

Avish Shah - 12mict21

ET (Evaluate Trace): generates reconstructed erroneous video using also the receiver trace. FV (Fix Video): is only needed if the used codec cannot provide lost frames (empty or the last decoded frames for lost frames) PSNR (Peak Signal to Noise Ratio) is an objective quality measure. It is a derivative of the well-known signal to noise ratio (SNR), which compares the signal energy to the error one.

Where MAXI is the maximum possible pixel value of the image MSE is the Mean Square Error: I is the original image and K is the compressed one
Assignment 1 : Evalvid Page 2

High Speed Networks

Avish Shah - 12mict21

MxN is the dimension of both images MOS (Mean Opinion Score) is a subjective quality measure. MOS ranges from 1 (worst) and 5 (best):

PSNR approximated to the MOS scale:

Evalvid and NS2

To experiment video transmission using the NS2 simulator Simulation produces a receiver trace file containing information necessary to reconstruct the possibly corrupted video at the receiver side. To decode the video files obtaining a YUV (raw) video: - ffmpeg -i video.264 video_raw.yuv To create a compressed raw video: MPEG-4 - ffmpeg -s cif -r 30 -b 64000 -bt 3200 -g 30 -i video_raw.yuv vcodec mpeg4 video_cod.m4v To create a MP4 files containing the video samples (frames) and a hint track which describes how to packetize the frames for the transport with RTP: -./mp4box -hint -mtu 1024 -fps 30 -add video_cod.m4v video_encaps.mp4 To obtain the YUV file created by decoding the coded video - ffmpeg -i video_encaps.mp4 video_ref_raw.yuv To compute PSNR that shows the codec impact on video quality - ./psnr 352 288 420 video_raw.yuv video_ref_raw.yuv > psnr_prima.txt
Assignment 1 : Evalvid Page 3

High Speed Networks

Avish Shah - 12mict21

Encoding the raw YUV video

$mpeg4encoder.exe example.par

Assignment 1 : Evalvid

Page 4

High Speed Networks

Avish Shah - 12mict21

After encoding, a compressed video file foreman_qcif.cmp can be found Get the Video Traffic Trace File $MP4.exe send 224.1.2.3 5555 1000 foreman_cif.cmp > st This MP4.exe command will read the compressed file. Fragment each frame into small packets with the maximum size of 1000 bytes. Then send this packets to IP:224.1.2.3 Port:5555. Here, the IP and Port is not important. Because we just need the frame information (stored in st file).

Run the simulation script

set ns [new Simulator] set nd [open out.tr w] $ns trace-all $nd set max_fragmented_size 1000 #add udp header(8 bytes) and IP header (20bytes) set packetSize set s1 [$ns node] set r1 [$ns node] set r2 [$ns node] set d1 [$ns node]
Assignment 1 : Evalvid Page 5

1028

High Speed Networks

Avish Shah - 12mict21

$ns duplex-link $s1 $r1 10Mb

1ms DropTail

$ns simplex-link $r1 $r2 0.18Mb 10ms DropTail $ns simplex-link $r2 $r1 0.18Mb 10ms DropTail $ns duplex-link $r2 $d1 10Mb 1ms DropTail

set qr1r2 [[$ns link $r1 $r2] queue] $qr1r2 set limit_ 10 set udp1 [new Agent/myUDP] $ns attach-agent $s1 $udp1 $udp1 set packetSize_ $packetSize $udp1 set_filename sd_be set null1 [new Agent/myUdpSink2] $ns attach-agent $d1 $null1 $ns connect $udp1 $null1 $null1 set_trace_filename rd_be set original_file_name st set trace_file_name video1.dat set original_file_id [open $original_file_name r] set trace_file_id [open $trace_file_name w] set frame_count 0 set last_time 0 while {[eof $original_file_id] == 0} { gets $original_file_id current_line scan $current_line "%d%s%d%s%s%s%d%s" no_ frametype_ length_ tmp1_ tmp2_ tmp3_ tmp4_ tmp5_ # 30 frames/sec. (1 sec -> 1000000us) set time [expr 1000 * 1000/30]

Assignment 1 : Evalvid

Page 6

High Speed Networks

Avish Shah - 12mict21

if { $frametype_ == "I" } set type_v 1 if { $frametype_ == "P" } set type_v 2 if { $frametype_ == "B" } set type_v 3 if { $frametype_ == "H" } set type_v 1 puts $trace_file_id "$time $length_ $type_v $max_fragmented_size" incr frame_count } close $original_file_id close $trace_file_id set end_sim_time [expr 1.0 * 1000/30 * ($frame_count + 1) / 1000] puts "$end_sim_time" set trace_file [new Tracefile] $trace_file filename $trace_file_name set video1 [new Application/Traffic/myTrace2] $video1 attach-agent $udp1 $video1 attach-tracefile $trace_file proc finish {} { global ns nd

$ns flush-trace close $nd exit 0 } $ns at 0.0 "$video1 start" $ns at $end_sim_time "$video1 stop" $ns at [expr $end_sim_time + 1.0] "$null1 closefile" $ns at [expr $end_sim_time + 1.0] "finish" $ns run

Assignment 1 : Evalvid

Page 7

High Speed Networks

Avish Shah - 12mict21

sd_be

rd_be

Assignment 1 : Evalvid

Page 8

High Speed Networks

Avish Shah - 12mict21

Video Quality Evaluation

execute et program $et.exe sd_be rd_be st foreman_cif.cmp err_be.cmp 0

Packet/Frame lost report The first line indicates that the total number of packets sent (p->nA) is 1022.

It includes 350 I frame packets (p->nI); 222 P frame packets (p->nP); and 449 B frame packets (p->nB). The second line indicates that the total number of packets lost (p->lA) is 161.

It includes 109 I frame packets (p->lI); 22 P frame packets (p->lP); and 30 B frame packets (p->lB). Decode the video $mpeg4decoder.exe err_be.cmp err_be 176 144 > df_be Error concealment $myfixyuv.exe df_be qcif 400 err_be.yuv myfix_be.yuv Calculate the average PSNR $avgpsnr.exe 176 144 420 foreman_qcif.yuv myfix_be.yuv avgerage psnr:24.363058

Assignment 1 : Evalvid

Page 9

High Speed Networks

Avish Shah - 12mict21

Run the qos.tcl and compare the results from be.tcl

Assignment 1 : Evalvid

Page 10