Software Functional

Testing: A Guide for

Software Leaders
Thanks for reading the checklist. It helps software leaders
improve web and mobile app customer experiences by thinking
more strategically about functional, regression, exploratory, and
automated testing.

Skip to the end to print out a one-page version of the checklist!

Table of contents

What is functional testing? ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������3

Step one: Integrate your systems and tools �������������������������������������������������������������������������������������������������������������������������������������������������������������4

Step two: Before you begin functional testing ��������������������������������������������������������������������������������������������������������������������������������������������������������5

Step three: Write better test cases and regression suites ��������������������������������������������������������������������������������������������������������������������������������6

Step four: Build your tester pool �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������7

Step five: Run initial functional tests ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������8

Step six: Combine automated and manual testing to maximize efficiencies �����������������������������������������������������������������������������������������������9

Step seven: Filter results������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 10

Applied Example Using a Functional Testing Feedback Loop������������������������������������������������������������������������������������������������������������������������� 11

Your printable checklist �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 14

What is functional
testing?
Functional testing validates that
web and mobile applications are
executing actions as they should.
For example, can a user log into
an app? Can she enter credit
card information? When you add
a photo, does it get added to the
digital asset library?
Software functional testing checklist
Types of Functional Testing
• Acceptance testing
• Alpha testing
• API testing Functional testing is a type of software
• Beta testing
testing that verifies product features
• Black box testing
according to requirements, identifying all
• Component testing
• Integration testing
areas where performance doesn’t meet
• Production testing expected results.
• Sanity testing
• Smoke testing
• System testing
• Regression testing
• Unit testing
• UI testing
3
Step one:
Integrate your systems
and tools
For efficiency, it’s imperative that Integrate with test management
you integrate various tools for test tools and issue trackers like
TestRail, GitHub, Azure, Jenkins,
management, CI/CD, automated
BrowserStack, mabl, Jira, Asana,
testing, and issue management.
Trello, Shortcut, Pivotal Tracker, and
If partnering with a managed more.
software testing platform, make
sure to integrate these critical Make sure your testing partner can
review stories from Jira.
systems with their software testing
platform and APIs. This automated Do the test cases written for each TestlioConnect for Jira - View Here
approach dramatically improves story link to Jira so you can review
communication and the speed of and verify the scripts for each story?
overall testing.

Software functional testing checklist 4

Step two:
Before you begin functional testing
Encourage a collaborative
approach to test case preparation.
Whether you write your test cases
or outsource them to a testing
vendor, a bit of collective prep
work amongst key stakeholders
goes a long way to improve testing
efficiencies.
Software functional testing checklist
Define critical stakeholders from QA,
product, and engineering.
Talk with product leaders to learn
more about the new updates
or features being released, and
understand the depth and breadth of
the testing required.
Make sure whoever owns feature
stories (developer, product owner,
scrum master, or project manager)
adds Acceptance criteria and
updates them in Jira.
Ask your QA or product manager
to capture video and screenshots
of the product in development for
reference during testing.
Start listing all of the functions and
features that need to be included
in the testing round. Login and play
around with the app to recall any
additional functionality in those
product areas.
Map out “what if “ scenarios to
increase the effectiveness of your
overall testing.
Decide whether or not test cases could
be combined or if you need to break up
one concept into multiple test cases
(such as upload a photo, add a photo
tag, search a photo tag).
Dedicate ongoing stand-ups to check
in on new features, stories, and desired
outcomes.
5

Step three:
Write better test cases and
regression suites
Precise test cases and regression
suites:
Ensure a stable app after code
changes
Reduce app maintenance
Cut customer support costs due to
production bugs
We recommend using a test
management tool to author test cases.
This way, you input the information
needed on the test case, assign it to a
tester, and allow them to connect this
test case to a bug report if needed.
What makes a great functional test
case? See the tips from our experts.
Software functional testing checklist
Name and describe the function or
feature.
Be very clear on the objective.
Define the purpose of the test.
Define preconditions such as being
logged in as a user with a certain
subscription level.
Include “what if” scenarios to test,
such as “what if the network goes
down mid scroll?”
Write clear steps that outline what to
do and what results to expect.
Validate if the test passed or failed
with the option to upload an image
for UI testing.
A great functional test case has a good level of detail and
information (e.g., test data, execution preconditions, expected
results) to ensure that all the functionality under test is working
as expected. Good testing results not only uncover bugs but they
also provide an overview of the coverage - what was tested, what
is working, what is not working - in order to assess risks and make
release-related decisions.
— Kaisa Paavo, Engagement Manager, Testlio
You want to make sure that your test cases reflect the actual state
of the application. So, for example, you want to update test cases
after a feature has changed. This reduces noise because the tests
uncover issues, not a bad test case.
— Oksana Lang, Engagement Manager, Testlio
Good test cases are all about scenarios. We come up with basic
flows and then ask testers to explore and come up with more
scenarios to test. If the client agrees with the new flows, we
incorporate them into our weekly functional regression tests.
— Marielle Tamkorv, Engagement Manager, Testlio
We think of scenarios the client never would have come up with. For
example, a perfectly good test case might test a normal flow like
app login using a correct password. But, what about the opposite,
bad flow - app login using the wrong password? Translating “what
if” scenarios into clear test cases takes experience and creativity.
— Paulo Feitosa, Testing Manager, Testlio
6
Pros and cons of writing your
own test cases
Not everyone has the time or skilled
resources to write test cases. In
addition, lousy test cases create
noise. So let’s look at the pros and
cons of writing your own test cases
vs. outsourcing the work to experts.
Pros
Writing test cases challenges product
owners to think differently about user
stories and design prototypes. First,
they must consider all the possible user
scenarios. And then translate them into
easy-to-follow test cases for testers to
execute.
Cons
Writing test cases takes time, dedicated
resources, and a unique ability to break
down functionality from a user’s point of
view. In addition, a single product owner,
QA, or testing manager cannot come up
with all possible test scenarios. For these
reasons, many companies outsource test
case writing. It saves time and introduces
a diversity of perspectives, environments,
and devices to your test cases.
Software functional testing checklist
Step four:
Build your tester pool
Imagine developers in New York finishing
their workday and handing builds to
testers abroad just starting their day.
Today it’s easy to build a team of on-
demand, part-time, or full-time QA and
QE freelancers in time zones that align
with your downtime.
“We release weekly now, so speed is
very important. We send builds on Friday,
then Testlio works overnight and over the
weekend to test and file detailed reports. By
Monday morning we know what is wrong and
if there are any blockers or showstoppers
that need to be fixed. The rollout is
everything and I know that we’ll release on
time with Testlio.”
Remya Ravindran
Senior QA Engineer, Quizlet
Quizlet tests on weekends. Learn why.
In addition to global locations, consider
how to compensate testers. Pay-per-
bug models encourage high-volume
issue detection. In contrast, testers paid
by the hour detect more critical issues,
offer clear reproduction steps, and
report on what’s working as intended.
The difference in quantity vs. quality
impacts management and dev time.
Tips for building your tester pool:
Consider how on-demand, part-time,
or full-time quality engineers and
QA freelancers might augment your
team and variable testing demands.
Select testers from the countries,
locations, and time zones that meet
your needs.
Identify which device and OS
combinations require testing and
match those needs with the tester
pool.
Consider paying testers per hour
rather than per bug, as it incentivizes
deeper testing.
7
Step five:
Run initial functional tests
Your test cases are written and Align runs with builds from your CI/
testers ready. Set. Go. CD systems.

Leverage a single test management

Run functional tests during new
platform so your test cases,
feature development to find bugs
runs, and issue reporting operate
early in the cycle.
seamlessly in one location.
Combine a suite of functional tests
for full app regression testing to
confirm that apps work as intended
(or have issues after new features
are merged to the main branch).

Testlio Runs module

Software functional testing checklist 8

Step six:
Combine automated and manual
testing to maximize efficiencies
An overemphasis on either
automated or manual testing can
create bottlenecks, putting your
release timelines at risk. Testlio
uses a methodology called fused
testing, which combines automated
and manual runs so they work in
parallel. This helps development
teams to quickly validate failed
automated tests and enables a
manual continuation of flakey tests.
Software functional testing checklist
Considerations before embracing Critical paths: parallel testing
fused testing:
Automated Run
Have you already made test
Manual Run
automation investments with tools
like Applitools, BrowserStack,
Headspin, mabl, and Sauce Labs? Flakey tests: manual continuation
What’s your comfort level integrating Automated Run
CI/CD, automated testing, and task
Manual Run
management tools with a manual
testing platform?
Failed test: manual validation
Do you have the right mix of QE
resources to own script creation Automated Run
and maintenance, as well as Manual Run
automated testing configuration and
coordination?
Short on QE talent? Testlio can help.
9
Step seven:
Filter results
Perhaps the most valuable testing
outcome for your dev team is the
testing results. Results should offer
traceable actions with visibility
into the entire testing journey:
what’s tested, when it’s tested,
what devices are tested, costs,
outcomes, and recommended
actions.
Software functional testing checklist
Review test results and reproduction
steps before reporting them to the
dev team. There may be known
issues or lower priority bugs that
slip through the cracks, and filtering
them saves developer time and
frustration.
Make sure the results include clear
steps to reproduce with logs/
screenshots/videos. This saves time
by helping the developers quickly
debug the issues.
Curious about app instrumentation? Learn more here
Push the results to the issue tracking
system for developers to begin
fixing the bugs. Remember that your
testing recap should include what’s
working and what’s broken to help
decide if it’s ok to release.
Consider app instrumentation
to equip testers to automatically
present screenshots, network
variables, location information,
versioning, configurations, hardware,
OS, and more. The additional data
helps developers reproduce issues
more quickly.
10

Applied Example
Using a Functional
Testing Feedback
Loop
Whether you’re releasing new features daily or
aligned with sprints, the pressure is on to code,
build, test, and release quickly. We recommend
implementing a functional testing feedback loop
to build test cases that grow with your features.
Here’s what the functional testing feedback loop
looks like for our clients at Testlio.
Software functional testing checklist
Step one:
Test case authoring and first round of
functional testing for new features
1. We participate in sprint planning meetings, demo
sessions, and review client-provided wireframes and
development tickets. We also download the app to
experience it as an end-user.
2. We map out the new features and author functional
test cases covering both happy and negative paths.
3. We review the list of test cases with the client
and discuss any feedback and modifications, if
necessary.
4. We set up a run, and our testers conduct functional
testing to validate both happy and negative paths and
identify issues. Validating only the happy path is a
limitation as it causes escaped issues.
Step two:
Author regression suites and run regression
testing to match ad-hoc, weekly, bi-weekly,
and monthly client sprints
1. We communicate with the client bi-weekly, including
attending stand-ups to keep aware of upcoming
testing needs and receive new builds to test.
2. Client hands-off builds at any time (often on Fridays
for weekend testing).
3. Before running the regression test, we often run a
quick smoke test to ensure the build is working.
4. Based on the client’s need, we begin running
regression testing. We run the regression tests on
Saturday for Friday builds and present results on
Sunday, at the latest.
5. Depending on the sprint and testing cadence, the
client’s dev team begins to fix issues.
11

One QA or testing manager can’t come up with all possible
test scenarios. But, when you have five people exploring
the same thing you see the creative power of testing.
Testers think differently, work in different environments,
and use different devices. That’s why we often augment
regression tests with structured exploratory testing. When
you see how creative testers get, it’s beautiful.
Marielle Tamkorv
Engagement Manager, Testlio
Software functional testing checklist
Step three:
Optimize and enhance test scripts through
structured exploratory testing
Regression tests sometimes fail to catch all the issues,
so we supplement them with open-ended exploratory
testing to uncover more negative path scenarios.
1. We first create structured exploratory test scenarios.
2. Senior testers with limited direction explore to
uncover issues.
3. We analyze the issue reports and filter out known
issues to share with the client.
4. With client review and feedback, we update the
regression test suites to include the new functional
tests.
12
 Conclusion
Functional testing is perhaps the most critical type of software
See how Testlio
testing, as it validates if your product features work according
to their intended requirements (or don’t work as intended). In
supplements your
either case, quickly learning about a fully functional app or one
laden with bugs is mission-critical for more confident releases.
internal QA and
Following the steps within this checklist is your fast track automation processes
to more efficient software testing. In addition to the list of
actions, we hope you consider the concept of fused testing
and take advantage of the functional testing feedback loop Learn More
that leverages functional, regression, and exploratory testing.
It’s a tried and true approach that will lead to better end-user
experiences.

Software functional testing checklist 13


Software
Functional
Checklist
Step one: Integrate your systems and tools
Integrate with test management tools and
issue trackers like Jira, GitHub, Asana,
Trello, Azure, Shortcut, and Pivotal Tracker.
Make sure your testing partner can review
stories from Jira.
Do the test cases written for each story
link to Jira so you can review and verify the
scripts for each story?
Step two: Before you begin functional testing
Define critical stakeholders from QA,
product, and engineering.
Talk with product leaders to learn more
about the new updates or features being
released, and understand the depth and
breadth of the testing required.
Make sure whoever owns feature stories
(developer, product owner, scrum master, or
project manager) adds Acceptance criteria
and updates them in Jira.
Ask your QA or product manager to capture
video and screenshots of the product in
development for reference during testing.
Start listing all of the functions and features
that need to be included in the testing
round. Login and play around with the app
to recall any additional functionality in those
product areas.
Map out “what if “ scenarios to increase the
effectiveness of your overall testing.
Decide whether or not test cases could be
combined or if you need to break up one
concept into multiple test cases (such as
upload a photo, add a photo tag, search a
photo tag).
Step five: Run initial functional tests
Dedicate ongoing stand-ups to check
in on new features, stories, and desired
outcomes.
Step three: Write clean test cases for better
results
Name and describe the function or feature.
Be very clear on the objective. Define the
purpose of the test.
Define preconditions such as being logged
in as a user with a certain subscription level.
Include “what if” scenarios to test, such as
“what if the network goes down mid scroll?”
Write clear steps that outline what to do and
Step six: Filter results
what results to expect.
Validate if the test passed or failed with the
option to upload an image for UI testing.
Step four: Build your tester pool
Look for tester ratings, profiles, and
leverage a tester recommendation engine
to create, update, and extend tester groups.
Select testers from the countries and time This saves time by helping the developers
zones that meet your needs.
Identify which device and OS combinations
quickly debug the issues.
Push the results to the issue tracking
require testing and match those needs with
the tester pool.
system for developers to begin fixing the
bugs. Remember that your testing recap
Make sure your testers have a similar profile
to your customers and end-users
should include what’s working and what’s
broken to help decide if it’s ok to release.
Consider app instrumentation to
Consider paying testers per hour rather
equip testers to automatically present
than per bug, as it incentivizes deeper
screenshots, network variables, location
testing.
information, versioning, configurations,
hardware, OS, and more. The additional data
helps developers reproduce issues more
Run functional tests during new feature
quickly.
development to find bugs early in the cycle.
Combine a suite of functional tests for full
app regression testing to confirm that apps Functional Testing Feedback
work as intended (or have issues after new Loop
features are merged to the main branch).
Step one: Test case authoring and first
Align runs with builds from your CI/CD round of functional testing for new features.
systems.
Step two: Author regression suites and run
Leverage a single test management regression testing to match ad-hoc, weekly,
platform so your test cases, runs, and bi-weekly, and monthly client sprints.
issue reporting operate seamlessly in one
Step three: Optimize and enhance test
location.
scripts through structured exploratory
testing.
Review test results and reproduction steps
before reporting them to the dev team.
There may be known issues or lower priority
bugs that slip through the cracks, and
filtering them saves developer time and
frustration.
Make sure the results include clear steps to
reproduce with logs/screenshots/videos.