Correlation? What’s that?

If you think correlation has something to do with the fit of data points to a
function curve on a graph, and the word has no meaning to you in the
context of Mercury’s tool, then this document is for you. It explains what
correlation is, why you have to do it, how to do it, and what to do when it
goes wrong. If this is the first time that you have used VuGen or if you have
been using it a little but are not a guru, then read on.

Introduction
VuGen, when recording a script, simply listens to the client (browser) talking to the
server (Web server) and writes it all down. The complete transcript of everything that
was said, the dates and time, content, requests, and replies can be found in the Recording
Log (View
Output Window
Recording Log). The script is an easier to read version
of this. The main difference is that the script only contains the client’s communication.

If you imagine that VuGen is an impersonator pretending to be the client (browser), the
script tells VuGen what to say to the server to successfully fool it. You want the server to
believe that VuGen is a real client, and thus send it the requested information.

This script has the hard coded information of the original conversation (Browser session)
that occurred between the client and server. This hard coded information may not be
enough to fool the server during replay however; it may have to be correlated.
What is correlation?
Correlation is where the script is modified so that some of the hard coded values in the
script are no longer hard coded. Rather then sending the original value to the server, we
may need to send different values.

For example, the original recorded script may have included the server sending the client
a session identification number, something to identify the client during that particular
session. This Session ID was hard coded into the script during recording.

During replay, the server will send the client a new Session ID. You need to capture this
value, and incorporate it into the script so you can send it back to the server to correctly
identify yourself for this new session. If you leave the script unmodified, you will send
the old hard coded Session ID to the server. The server will look at it and think it invalid,
or unknown, and so will not send the pages that have been requested. Replaying script
with old will Session ID not have successfully fooled the server into believing it is a
client.

Correlation is the capturing of dynamic values passed from the server to the client and
back. You save this captured value into a parameter, and then use this parameter in the
script in place of the original value. During replay, the replay engine will now listen to
what the server sends to it, and when it makes requests of the server, send this new, valid
value back to the server; thus fooling the server into believing it is talking to a real client.

Why do I have to correlate?

If you try to replay a script without correlating first, then most likely the script will fail.
The requests it sends to the server will not be replied to. Either the Session ID is invalid
so the server will not allow you into the site, or it will not allow you to create new records
because they are the same as existing ones, or the server will not understand your request
because it is not what it is expecting.

Any value that changes every time you connect to the server is a candidate for
correlation. A correlated script will send the server the information it is looking for, and
so allow the script to replay. This will allow many Vusers to replay the script many
times, placing load on your server.
What errors mean I have to correlate?
There are no specific errors that are associated with correlation, but there are errors that
could be caused because a value has not been correlated. For example, if an invalid
Session ID is sent to a Web server, how that server responds depends on the
implementation of that server. It might send a page specifically stating the Session ID is
invalid and ask you to log in again. It might send an HTTP 404 Page not found error
because the requesting user did not have permissions for the specified page, and so the
server could not find the page.

In general, any error message returned from the server after the script makes a request
that complains about permissions can point to a hard coded value that needs to be
correlated.

The tools (functions) used to correlate

There are 3 functions that you can use for correlation. A list of them, along with
documentation and examples, can be found in the on-line documentation. From VuGen,
go to Help
Function reference
Contents
Web and Wireless Vuser Functions

Correlation Functions.

The first function, web_reg_save_param, is the latest function with some extra features
for a more powerful usage. The second and third function, web_create_html_param and
web_create_html_param _ex are for backward compatibility. They are essentially the
same, and will be talked about together. The last function is associated with the second
and third function, and is not directly a correlation function but rather a setting. It will be
talked about later in a different section.

Web_reg_save_param

web_reg_save_param ( “Parameter Name” , <List of Attributes>, LAST );

Each of these parameters is a pointer to a string. That means that if they are entered as
literal text, they need to be enclosed in quotes, with each parameter separated by a
comma. The supported attributes include LB (Left boundary), RB (Right boundary),
RelFrameID, ORD, Search, SaveOffset, SaveLen, and Convert. These attributes can
appear in any order because they contain within them what they are. Detail information
about each attribute can be found on the function reference online.
Web_create_html_param

web_create_html_param ( “Parameter Name”, “Left Boundary”, “Right Boundary” );

This function is for backward compatibility only. It was a standard for LoadRunner6.5
and below. This function takes three parameters.

Each of these parameters is a pointer to a string. That means that if they are entered as
literal text, they need to be enclosed in quotes, with each parameter separated by a
comma.

Parameter Name - This is the name of the parameter, or placeholder variable for saving
the captured value. After successfully capturing the value, the parameter name is used in
the script in place of the original value. The replay engine will identify the
parameter/placeholder, and substitute the captured value for the placeholder during
replay. This name should have no spaces, but apart from that limitation, it is entirely up
to you what name to give it.

Left Boundary - This is where you tell the replay engine how to find the dynamic value
you are looking for. In the Left Boundary, you specify the text that will appear to the left
of the changing value.

Right Boundary - This is where you tell the replay engine how to identify the end of the
dynamic value you are looking for. You use the text that will appear after the value.

Web_create_html_param_ex

web_create_html_param_ex ( “Parameter Name”, “Left Boundary”, “Right Boundary”,

“Instance”);

This function is the same as the web_create_html_param function, except it does not look
for the first instance of the boundaries, but rather the nth instance of those boundaries.
The first three parameters (Parameter Name, Left Boundary, and Right Boundary) are the
same, the last parameter is a pointer to a string, and so it must be enclosed in double
quotes. It is the number of the occurrence. If you place the number 1 here, the function
behaves exactly as the web_create_html_param function; it looks for the first occurrence.
If you put the number 3 here, it will look for the third occurrence of the left and right
boundaries, and place what appears in-between into the parameter.
Identifying values to correlate
So you have the tools, and you know why you need to use them, but how do you know
what to use them on? What values in the script need to be correlated? The simplest
answer is “Any value that changes between sessions required for the script to replay.”

A hypothetical example

You are logging onto a website. When you send the server your user name and password,
it replies with a Session ID that is good for that session. The Session ID needs to be
correlated for replay. You need to capture this value during replay to use in the script in
place of the hard coded value.

To identify values to correlate, record the script, and save it. Open a new script, and
record the same actions and business process again. As much as possible, during
recording, enter the same values in both scripts, for example, user ID, password, and
fields and edit selections. Save the second script, and run it with Extended log (Vuser

Run-Time Settings
Log
Extended log. Check all three options).

Go to Tools → Compare with Vuser, and choose the first recorded script. WinDiff will
open and display the two scripts side by side. Lines with differences in them will be
highlighted in yellow. Differences within the line will be in red (If not, go to Options

View
Show inline differences.)
If WinDiff gives an error here, dismiss the error. WinDiff will be minimized in the task
bar, right-click on it and choose “Restore.” Go to File → Select Files/Directories and
manually select the action sections for the two scripts.

Differences like "lr_think_time" can be ignored. They are pacing functions, and do not
represent data sent to the server.

Locate the first difference, take note of it, and search the script open in VuGen for that
difference. That is the original value hard coded into the script that was different in the
second script. Highlight it and copy it.

Go to the recording log and place your cursor at the top. Press Control-F (CTRL+F) to
do a search and paste in the original value. You are looking for the first occurrence of
this value in the recording log. If you do not find the value in the recording log, verify
you are looking in the correct script’s recording log. Remember you have two almost
identical scripts here.

If you find the value, scroll up in the log, and make sure the value was sent as part of a
response from the server. The first header you come across while looking up the script
should be prefaced by a receiving response. This indicates that the server sent the value
to the client. If the value first appears as part of a sending request, then the value
originated on the client side and does not need to be correlated, but rather parameterized.
That is a different topic all together. The response will have a comment before it that
looks like this:

*** [tid=b9 Action1 2] Receiving response from host astra.merc-int.com:80 ( 25/11/2002 12:04:00 )

So, you have a value that is different between subsequent recordings, it was sent from the
server to the client; this value most likely needs to be correlated. If the value is not
different between recordings and not originating on the server and sent to the client then
the value probably does not need to be correlated.
How to correlate
Step 1
After confirming that the first occurrence was part of a received response from the server,
you need to figure out where to place the web_reg_save_param( ) function. The
web_reg_save_param statement needs to go immediately before the request that fetched
the dynamic value from the server. In order to find this request or URL in the script, you
need to replay the script once with extended log and all the three options (In Vuser →
Runtime Settings → Log) turned on.

In the Recording log, pick up the text that is before the dynamic value. This text should
remain constant no matter how many times you replay the script and highlight it and copy
it. This is the text that will identify where to find the start of the value you are capturing.
Now, go to the Execution log and search for the text that you just copied from the
Recording log. You should see a corresponding Action1.c() at the beginning of that line
with a number in the brackets. That is the number of the line in the script where you need
to put web_reg_save_param( ) function. The function should go right above that line in
the script.

So, add a couple of blank lines to your script before the function at that line, and then
type in web_reg_save_param(“UserSession”, but give it a name that means more to you
than UserSession.
Step 2
Go back to the recording log, highlight the text to the left of the dynamic value and copy
it. This should be some of the same text you searched for in the Execution log.

The amount of text you highlight should be sufficient so that it is unique in this reply
from the server. It is suggested that you copy as much as possible without copying any
special characters. These show in the Execution log as black squares, and the actual
character they represent is uncertain. After selecting a boundary, go to the top of the
Servers reply, and press CTRL+F and do a search for that boundary. You want to make
certain what you have selected is the first occurrence in the Servers reply. If it is not
select, use more text to make it unique, or consider using the ORD parameter of the
web_reg_save_param function.

Once you have finalized the static text that represents the left boundary, copy it into the
web_reg_save_param (or web_create_html[_ex]) statement. If it contains any carriage
returns, place it all on one line. If there are any quotes (“) in the text, place the escape
character before it so that the replay does not incorrectly think it is the end of the
parameter, but rather a character to search for. For example, if the left boundary was
'input type=hidden name=userSession value=' (without the single quotes) and you are
using the web_reg_save_param statement, then the function you have so far would be

web_reg_save_param (“UserSession”, “LB=input type=hidden

name=userSession value=”,
Step 3
You are now going to tell the replay engine how to identify the end of the value
you are trying to capture, that is, the right boundary of what you are looking for.
Again, look in the Execution log and copy the static text that appears to the right
of the dynamic value you are looking for. For example, let’s say the Execution
log contained the following:

… userSession value=75893.0884568651DQADHfApHDHfcDtccpfAttcf>…

Then, the example so far to save the number into the parameter UserSession
would be

web_reg_save_param(“UserSession”, “LB=input type=hidden name=userSession

value=”, “RB=>”,LAST);

In choosing a right boundary, make sure you choose enough static text to specify
the end of the value. If the boundary you specify appears in the value that you
are trying to capture, then you will not capture the whole value.
Recap
That was a lot of looking through the Recording and Execution logs and checking
of values. Let’s just recap what you have done. You have identified a value that
you think needs to be correlated. You then identified in the script where to place
the statement that would ultimately capture and save the value into a parameter.
You then placed the statement, and gave the text strings that appear on either
side of the value that you are looking for so that it can be found.

The flow of logic for this is the correlation functions tell the replay engine, what to
look for in the next set of replies from the server. The replay engine makes a
request of the server. The server replies. The replay engine looks thorough the
replies for the left and right boundaries. If it finds them, what is in-between is
then saved to a parameter of the name specified.

Remember, the parameter cannot have a value until after the next statement is
executed. The correlation statement only tells the replay what to look for; it does
not assign a value to the parameter. Assignment of a value to the parameter
does not happen until after generating request to the server and looks in the
reply. If you have a case where a correlation statement is followed by a function
that attempts to use the parameter, the statement is in the wrong place, and the
script will fail.
This is always incorrect:

web_reg_save_param(…);
Web_submit_data (… {Parameter}…);

In-between the two, there needs to be a request of the server that causes it to
reply with the value you are trying to capture.
Replacing the hard coded value in the script with the parameter
Once you have created the parameter, the next step is to replace the hard coded
occurrences with the parameter. Look through the script for the original value.
Where you find it, delete the value and replace it with the parameter. Note, only
the value you want replaces is deleted. The characters around it remain.

EXAMPLE:
Change:
…
"Name=userSession",
"Value=75893.0884568651DQADHfApHDHfcDtccpfAttcf", , .....
…

To:
…
"Name=userSession", "Value={UserSession}", ENDITEM,
…

At this point, you are ready to run the script to test if it works, or, if it needs further
correlation, more work on this correlation.
Common errors when correlating
When the replay fails to find the boundaries for a web_reg_save_param
statement, it will print a warning message in the execution log like this:

Action1.c(x): Error: No match found for the requested parameter "Parameter

Name". If the data you want to save exceeds 256 bytes, use
web_set_max_html_param_len to increase the parameter size
Action1.c(x): web_url highest severity level was "ERROR", 47489 body bytes,
1518 header bytes

By default, web_reg_save_param function issues an error if the boundaries

cannot be found. If you need to use this function for other purposes, and would
like to avoid errors, use the "Notfound=warning" attribute so that the replay will
just issue a warning.

Now, the replay actually suggested 2 possible causes for the error,

1. The boundaries defined cannot be found

2. The data you want to save exceeds 256 bytes

The advice given are just recommendation that you must take into consideration.
Was the value you were trying to capture more than 256 characters long? In the
above example it was only 40 characters long. Have a look at the Recording log
and see how long the original value is. Have a look at the second recording
made earlier and see how long it was in that script. Turn on the extended log
(Run-Time Settings → Log → Extended log → All data returned from server) and
have a look at how long it is in the Execution log.

If all occurrences of the script were much shorter that the maximum parameter
length, then the problem is either with the web_reg_save_param in the wrong
place, or that the boundaries are incorrect. Go back and look at the boundaries
that you selected, look at the placement of the web_reg_save_param. Is it
immediately before the statement that causes the server to reply with the data
you are looking for?

If, at any time, any of these values was close to being around 200 characters, or
exceeded 200 characters, then add a web_set_max_html_param_len statement
to the start of the script to make the maximum longer than 256 characters. For
example,

web_set_max_html_param_len ( “length” );

This statement tells the replay engine to look for larger matches for the left and
right parameter. When it finds the left boundary, it will look ahead the max
parameter length for the right boundary. This setting is script wide and takes
effect from when it is executed; it only needs appear in the script once. Looking
for longer matches uses up more memory and CPU processing to search
through the returned text from the server. For this reason, do not set it too high,
or you will be making your script less scalable. That is you will reduce the
number of Vusers that can run it on a given machine. Try to have the maximum
parameter length no more than 100 characters greater that what you are
expecting.

Special cases for the boundaries:

There are some special characters and cases when specifying the boundaries.
Double quotes should be preceded by a “\” so that it can be recognizes as part of
the string to look for. If your text includes any carriage returns, that are part of
the http, and not just part of the wrap around in the recording log, these need to
be specified as a “\r\n” character set. If the “\” character is part of the text, it too
needs to be preceded by a “\” to indicate it is a literal.

Recording log Left boundary Right boundary

Value=”57685” “Value=\”” “\””
Value_”\item\”value’7875’ “Value_\”\\item\\\”value’ “’”

Value= “Value=\r\n\”” “\””

“7898756”

Debug help
Sometimes you want to print out the value that was assigned to a parameter. To
do this, use the lr_eval_string function and the lr_output_message function. For
example, to print the value of the parameter to the Execution log:

lr_output_message(“ Value Captured = %s", lr_eval_string("{ParameterName}"));

If you find that the value being substituted is too long, too short, or completely
wrong, printing out the value will help identify the changes you need to make to
the correlation function. If you have extra characters at the start of the value, you
need to add them to the end of the left boundary, if you have extra characters at
the end of the parameter value, you need to add them to the start of the right
boundary. If you are getting the wrong value all together, do searches in the
Recording log for the left boundary, make sure that you have a unique boundary
and that LoadRunner is not picking up an earlier occurrence. You can then use
the ORD=x attribute to specify which occurrence you want to capture if the
boundary is not unique.
Other Correlation help resources
The examples in the Function reference contain a lot of data and examples on
how to use these functions. I would recommend looking over them.

The Customer Support site has a video for download that goes over correlation.
You can get it from http://support.merc-int.com.

After logging in, go to Downloads → Browse. Select the Mercury Interactive

downloads radio button, choose “LoadRunner” from the product selection drop-
down box, and click on the “Retrieve” button. Under “Training,” select the
“LoadRunner Web Script Correlation Training” link.