Blog

Get the App

Jobs
Documentation
Home
How to write powerful automated API tests with Postman, Newman and Jenkins
Posted on 3 Sep, 2015Author abhinav
Unless your development team is running on a six-month or an year-long cycle, yo
u would be practicing Continuous Integration. From the ThoughtWorks definition o
f Continuous Integration:
Continuous Integration (CI) is a development practice that requires developers t
o integrate code into a shared repository several times a day. Each check-in is
then verified by an automated build, allowing teams to detect problems early.
While source control systems have made it trivial to set up shared repositories
of code and view what people have been sharing, few teams are able to practice w
ell the second part verifying check-ins through an automated build process. Writ
ing tests and especially tests for APIs, can be a tedious process. Nobody would
disagree that having tests is good and we should be running them as often as pos
sible. In practice, writing tests, verifying whether they are working, setting u
p the testing environment and then eventually plugging them with the build syste
m is hard. Teams tend to skip this part accruing a ton of technical debt in the
process.

We believe that in the context of API development, this process can be made a lo
t easier. This is where Postman departs from just being a REST client in your ar
senal. Postman contains a full-featured testing sandbox that lets you write and
execute Javascript based tests for your API. We won't go into the specifics here
but do check out this tutorial for more detail.
Screen Shot 2016-06-27 at 15.12.11
The next logical step is hooking up Postman with your build system. This is wher
e Newman, Postman s command line companion comes in. Think of Newman as Postman s Co
llection Runner engine that sends API requests, receives the response and then r
uns your tests against the response.
In this tutorial, we are going to use Jenkins as an example. The same steps appl
y everywhere though. Jenkins is one of the most popular continuous integration s
ervers available right now. Jenkins primary goal is to help you build and test s
oftware projects continuously. You can also monitor executions of externally run
jobs too.
Newman and Jenkins are a perfect match. Lets start setting this up. We are using
Ubuntu as a target OS as in most cases your CI server would be running on a rem
ote Linux machine.
Install Jenkins: The process is straightforward. Just follow the instructions he
re: https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Ubuntu
Install NodeJS and npm. Newman is written in NodeJS and we distribute the offici
al copy through npm. Follow the instructions for Linux to install nodejs and npm
here: https://docs.npmjs.com/getting-started/installing-node
Install Newman through npm install -g newman. This would set up newman as a comm
and line tool in Ubuntu.
Run a sample Postman Collection. We are assuming that you already have a Postman
Collection with some tests. I am using a collection that sends two requests to
echo.getpostman.com. You can download this sample collection if you want to foll
ow the example. This is what the output looks in Postman s Collection Runner:
Screen Shot 2016-06-27 at 15.14.02
Some of my tests are failing intentionally in the screenshot. We will fix this l
ater in the tutorial.
To run this collection inside newman, use the command [command here]. If everyth
ing is set up nicely, you should see the output below.
Screen Shot 2016-06-27 at 16.31.48
Jenkins exposes an interface at http://localhost:8080. This is what it looks lik
e:
step-3
Create a new job by clicking on the "New Item" link on the left sidebar. Then se
lect a "Freestyle Project" from the options shown. I am calling my project Jenki
ns_Newman_Test.
step-4
Add a build step in the project. The build step executes a Shell command.
step-5
The command is:
newman -c jenkins_demo.postman_collection --exitCode 1.
Note here that we are using the newman command parameter exitCode with the value 1
. This denotes that newman is going to exit with this code that will tell Jenkin
s that everything did not go well. Click the save button to finish creating the
project.
step-6
Lets run this build test manually by clicking on the "Build Now" link in the sid
ebar.
step-7
Jenkins indicates that the build has failed with a red dot in the title. We can
check why with the console output from newman.
step-8
Click on the "Console Output" link in the sidebar to see what newman returned.
step-9
Lets fix these tests inside Postman and then try again. You can download the upd
ated collection here.
step-10
I have updated my collection now. Running these tests inside Jenkins tells me th
at everything worked!
step-11
Jenkins indicates that the build succeeded with a blue ball. If you are wonderin
g why it is not green, check out this article.
Lets change the Jenkins build trigger as every 30 mins. You can do this by click
ing on "Configure project" in the main project window and then scrolling down.=.
The syntax for setting the frequency is H/30 * * * *
step-12
And we are all done! Jenkins will now run newman every 30 mins and will tell you
whether the build failed or succeeded. In a bigger set up, newman will be part
of your build process and probably not the entire process. You can set up notifi
cations and customise Jenkins as per your needs.
You can use a wide variety of other configurations to make your collection more
dynamic. Check out the other tutorials on our blog to see how to do this. Postma
n, Newman and Jenkins will give you a lot of power along with flexibility. We ar
e continuously evolving the testing sandbox and let us know your suggestions on
our Github issue tracker. Of course, do share this article on Twitter if you fou
nd it useful!
Share this:
Click to share on Twitter (Opens in new window)26Click to share on Facebook (Ope
ns in new window)26Click to share on Reddit (Opens in new window)Click to share
on Google+ (Opens in new window)More
Related
The Importance of 3rd Party Integrations for Postman Pro
March 21, 2017
In "General"
Meet Newman: a command line companion for Postman
May 12, 2014
In "Updates"
BigCommerce uses Postman before a single line of code is written
March 21, 2017
In "Case Studies"
31 thoughts on How to write powerful automated API tests with Postman, Newman and
Jenkins
Andreas Poppernitsch says:
September 3, 2015 at 13:29
Hi! Great job on this, works like a charm. Is ist possible to export / dump a po
stman collection automatically (via some shell script for example) ?
Reply
Postman says:
September 5, 2015 at 02:27
Glad you liked the tutorial. It is not possible to export the Collection automat
ically but we will have hooks for this through our Sync service soon.
Reply
tmack8001 says:
October 4, 2015 at 09:58
Would be awesome to be able to reference postman's environment setups for my Jen
kins integration. For the time being I'm just periodically exporting the environ
ment variables along with a snapshot of the collections to a common place for Je
nkins to reference.
Reply
Amit Kumar says:
September 30, 2016 at 17:24
Hi! Great job on this, works like a charm. Is ist possible postman can integrate
with Travis CI for testing.? If is is possible please help me.
Reply
Amit Kumar says:
September 30, 2016 at 17:27
Hi! Great job on this, works like a charm. Is ist possible to integrate postman
with Travis CI for API testing? if its possible please help me on this.
Reply
Pingback: How to write powerful automated API tests with Postman, Newman and Jen
kins http://blog.getpostman.com/2015/09/03/how-to-write-powerful-automated-api-t
ests-with-postman-newman-and-jenkins/ via @postmanclient
Pingback: This week in API land #20 | Restlet - We Know About APIs
Srinivas Kalyan says:
December 17, 2015 at 07:51
I am trying to Integrate Postman scripts using Newman with Bamboo. Can any one s
uggest help me. I am unable to install newman facing some issues with proxy sinc
e it is behind a corporate proxy.
Any tips on configuring the proxy to run behind corporate?
Reply
Claire Thomson says:
December 24, 2015 at 17:31
Hi I'm running on Windows, but due to this have to use a batch file rather than
shell the demo above has the fails as expected but is marked as Finished: SUCCES
S and makes as passed in Jenkins. I'm guessing it's something to do with the exit
code 1? any ideas how I get this to work as expected?
Reply
Claire Thomson says:
December 29, 2015 at 15:53
It's OK I've sorted this out was missing the -x code in the batchfile calling ne
wman
Reply
Pablo Sanz says:
January 23, 2016 at 00:17
I'm having the same problem. Can you write the content of your batch file (or a
similar example)? Thanks
Reply
Claire Thomson says:
December 29, 2015 at 15:55
OK another question, in the jenkins console output I'm not getting ticks or cros
ses instead I'm getting or which font do I need to correct this issue? thanks
Reply
Ahmet Anil Grbz says:
January 8, 2016 at 18:15
Hi, i run collections on command line at jenkins as url. Like this "newman -u ht
tps://www.getpostman.com/collections/XXXXXXX -e TEST.postman_environment -g glob
als.postman_globals stopOnError" but i can not see to fail cases on jenkins's con
sole output. its look like all of them passed but one case must be fail.
Reply
Lana says:
January 12, 2016 at 13:12
I have an issue running Newman command from Jenkins(below is Console output). I
can successfully run newman command from cmd prompt(newman -c C:UserslholmesDesk
topPostmanCollectionsReciva.json.postman_collection)
Console:
Building in workspace C:Program Files (x86)JenkinsworkspaceJenkins_Newman_Test
[Jenkins_Newman_Test] $ sh -xe C:WindowsTEMPhudson6298287477471424761.sh
The system cannot find the file specified
FATAL: command execution failed
java.io.IOException: Cannot run program "sh" (in directory "C:Program Files (x86
)JenkinsworkspaceJenkins_Newman_Test"): CreateProcess error=2, The system cannot
find the file specified
at java.lang.ProcessBuilder.start(Unknown Source)
environment: Windows, running Jenkins on http://localhost:8080/.
Any advise?
Reply
Uri Gorodetsky says:
July 28, 2016 at 18:26
Try using 'Execute Windows batch command' action instead of 'Execute shell'
Reply
Assaf Lowenstein says:
February 4, 2016 at 11:48
Hi, This looks very nice!
Is there a way to display a results historical trend line like the one available
for Junit? maybe using an additional plugin?
Thanks!
Reply
Pingback: 10 Continuous Integration Tools to Spur API Development | Nordic APIs
|
Pingback: Tutoriel Postman | Nouvelles Chroniques d'Amethyste
Ravinder Singh Tanwer says:
March 4, 2016 at 09:44
Hi Team, while configuring postman collection with Jenkins, i am getting below e
rror in Jenkins console output. i installed NodeJS and npm and nowman.
Building in workspace C:Users#####.jenkinsjobsAPI_Testworkspace
[workspace] $ cmd /c call C:Users#####AppDataLocalTemphudson#######.bat
C:Users#####.jenkinsjobsAPI_Testworkspace>newman -c D:/apitest.json.postman_coll
ection
'node' is not recognized as an internal or external command,
operable program or batch file.
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE
Reply
Shanmuga Raja Ramaswamy says:
March 27, 2016 at 19:58
You may have to specify the full path for newman instead of newman -c the command
should be C:node_modules.binnewman -c
Reply
Madhu CS says:
April 15, 2016 at 22:42
Is there a way to write data to external CSV file ?
Reply
amar says:
May 3, 2016 at 17:31
Hey Guys,
How will we handling login and logout flow:
While login we get authorization token and same token is need to logout, how wil
l we using postman.setNextRequest("");
and then performing tests on logout api.
Any help is highly appreciable.
Thanks in advance.
p.s new to api testing.
Reply
spouki says:
May 4, 2016 at 19:45
Hi!
For me I use a collection where I have my login request and logout request (this
way I don't have to call postman.setNextRequest("") BUT if you absolutely need
the request caller you need to write the query name in the double quotes like th
is :
login request name : GET login
logout request name : POST logout
in the test section of GET login, when you have done your verification you write
postman.setNextRequest("POST logout"); and it will work fine)
And then I recommend you to take a look at environment and global variables ! It
's a great plus to handle your session token in all your requests !
I hope this helps you!
spouki ??
Reply
Fadly Mahendra says:
July 1, 2016 at 06:28
What kind of API you test for every 30mins?
I plan to run my all of API (around 100 APIs) by daily, is it the right approach
?
Reply
Tariq Ghalib says:
July 21, 2016 at 01:14
Is there a way to run server side database queries (eg: SQL queries) to build te
st data from the database and then read it as an input for postman API calls?
Reply
arunan annamalai says:
September 13, 2016 at 21:19
I am trying to run the json collection through jenkins windows command prompt. I
given the "newman run C:Sanity.postman_collection.json". I am getting response
as
Building in workspace C:Program Files (x86)JenkinsworkspaceOne touch
[One touch] $ cmd /c call C:windowsTEMPhudson6168983955557336367.bat
C:Program Files (x86)JenkinsworkspaceOne touch>newman run C:Sanity.postman_colle
ction.json
'newman' is not recognized as an internal or external command,
operable program or batch file.
But when i am trying came from Windows command prompt its working. Looks weird w
hy through jenkins its not working and through windows cmd its working
Reply
Georgi says:
October 3, 2016 at 21:36
Has anyone tried to generate code coverage for Sonar, while running the Postman
tests?
Reply
ashish says:
October 17, 2016 at 14:41
How to integrate postman/Newman with failsafe maven plugin ?
Reply
Pingback: Introduction to Postman with examples | Automation Rhapsody
Sree Bandla says:
November 3, 2016 at 04:52
This seems nice and i've tried it works like charm, It would be great if any one
tried to use any plugins to display results, i.e.serenity plugin for serenity t
ests but in this case i'm not sure of what to use.
Reply
Pingback: Postman: gestiona y construye tus APIs rpidamente - Paradigma
Leave a Reply
Your email address will not be published. Required fields are marked *
COMMENT

NAME *

EMAIL *

WEBSITE

Post Comment
Notify me of follow-up comments by email.
Notify me of new posts by email.
Post navigation
PREVIOUS
Previous post:Writing front-end API code with Postman
NEXT
Next post:API integrations using Postman Building a Slack channel bot