| 0

SciVal
Evidence Based Evaluation

Alexander van Servellen

Consultant, Research Management (SEA)
Elsevier Singapore

UNIMAP, Malaysia
February 24th, 2016
 | 1

Agenda

1. Introduction to SciVal and Evidence Based Evaluation

2. Analysis and Reporting

3. Non-Performance Variables

4. Setting up SciVal

5. Accessing SciVal

6. Q&A
 | 2

Introduction SciVal and

Evidence Based Evaluation
 | 3

Typical uses
Measure scientific
production and
benchmark research
performance at
multiple levels

Assess the
Monitor research
international impact
trends
of research

Map collaboration
Identify leading
networks and
organizations and
identify collaboration
competitors
opportunities

Assess the impact of

research funding on Identify who is doing
the scientific output of what and with whom
researchers and in a variety of fields
graduate students

Source: http://www.science-metrix.com/en/expertise/bibliometrics/know-how
 | 4

Main purposes of metrics

Evaluation of performance
Typically top-down performed by an authority
Essential to account for variables that can affect metrics values besides
differences in performance, so that the evaluation is fair

Demonstration of excellence
Typically bottom-up performed by entity requesting resources
Generally aiming to find a way to showcase strengths compared to peers;
non-performance variables may be used to advantage in this case

Scenario modeling
Non-performance variables may or may not be important depending on
scenario
e.g. if modelling recruitment in nanoscience, might not need to worry about
different citation rates between fields because only looking at one field!
 | 5

SciVal is an online tool for evaluating research.

SciVal offers quick, easy access to the research performance of 220 nations and
6000 research institutions worldwide. You can easily create teams, departments,
and specific research topics, and evaluate them.

5
Data updated weekly from Scopus (same data source as the University Rankings use
and used in many studies about research performance)
 | 6

Reporting at various levels

Top
Research Office
Management

Faculty admins

Research Unit
Admins

Researchers
 | 7

The data in SciVal comes from Scopus

Scopus is the largest abstract and citation database of peer-reviewed
research literature from around the world. Its the core data source of
Elsevier Research Intelligence solutions, and used by academics,
government researchers and corporate R&D professionals who need a
comprehensive and efficient place to search, discover and analyze
research.

Over 21,900 titles from more than 5,000 international publishers and 105
different countries

Over 54 million records, 23 million patents from 5 patent offices worldwide

All content is vigorously vetted by an independent, 15-person, international

board of experts called the Content Selection and Advisory Board (CSAB)

More than 3,000 customers worldwide in all geographic regions

 | 8

Scopus content
10%
8%

5%

5%

2%
1%
Other
60% 1%
1%
1%
1%
1%
1% 1% 1%
| 9
 | 10

Through SciVal and Scopus, Elsevier has experience

with metrics
 | 11

Elseviers metrics approach in a nutshell

Figure 1 of http://www.elsevier.com/online-tools/research-intelligence/resource-library/resources/response-to-hefces-call-for-evidence-independent-review-of-the-role-
of-metrics-in-research-assessment
 | 12

Levels of
aggregation
 | 13

Everything
 | 14

Region
 | 15

Country
 | 16

University
 | 17

Faculty, Department or Research Centre

 | 18

Individual
 | 19

Notes
 | 20

Analysis & Reporting

 | 21

Now that you have customized SciVal you can

explore and analyze using SciVal

Overview module Benchmarking module is Collaboration module

provides dashboard flexible, it allows you to looks at specifically at co-
style reports based on a select a time period publication trends. It
3 or 5 year period. You anywhere from 1996 to allows you to identify the
can analyze countries, one month ago. You can top collaboration partners
institutions, any choose from over 17 of an institution or
researcher, group of metrics, and compare country, as well as
researchers and across different types of identify potential
research areas you entities (i.e. you directly collaboration partners.
have created. It gives a can compare researchers Research areas can be
comprehensive with departments, applied here as a subject
overview of one entity at institutions, countries in filter.
a time. one analysis)
 | 22

20 sets of metrics at your disposal

Slice and dice your data from multiple angles to identify your core strengths and
weaknesses

Productivity metrics Collaboration metrics

Scholarly Output Authorship Count
h-indices (h, g, m) Number of Citing Countries
Collaboration (geographical)
Citation Impact metrics Academic-Corporate Collaboration
Citation Count
Citations per Publication
Cited Publications Usage metrics
h-indices (h, g, m) Views
Field-Weighted Citation Impact Views per publication
Publications in Top Percentiles Field-Weighed Views Impact
Publications in Top Journal Percentiles
Disciplinarity metrics
Collaboration Impact (geographical)
Journal count
Academic-Corporate Collaboration Impact
Journal category count

Snowball Metric; www.snowballmetrics.com/metrics

 | 23

Notes
 | 24

Non-Performance Variables
 | 25

When making comparisons, we must take into

consideration Non-performance variables

Size

Discipline

Publication-type

Database coverage

Manipulation

Time

In some cases the difference in a metric value between two entities

(authors for example) might not reflect a difference in performance.
Instead the difference can be caused by non-performance variables.
 | 26

Publications types receive differing levels of citations

Because some publication types are cited more often than others, we
should not compare different types without applying normalization.
 | 27

Citation practices differ between disciplines

Because some subject areas are cited more often than others, we should
not compare papers from different fields without applying normalization.
 | 28

Size and Time

Citation count is
directly related to
volume
Do you think the declining
trend is indicative of a
decrease in quality or is it
caused by a non-
performance variable?

CPP normalizes for

differences in
publication volume

FWCI normalizes for

subject area,
document type, and
time
 | 29

Notes
 | 30

Setting Up SciVal
 | 31

My SciVal Researchers and Groups

Creating Researchers
Creating Groups of researchers (departments etc.)
 | 32

Master List
 | 33

First create a Master List of researcher names

 | 34

We start with the authors for who you do not know the
profile, or have more than one profile in Scopus
 | 35

Go to My SciVal and Define a new Researcher

 | 36

Do this only for the authors who have more than one
Scopus Author ID!!

add the Scopus

author ID field
twice

If you do not
know the profile
number, then
search by name

Fill in the
Author IDs
of one
author
 | 37

SciVal will ask you to confirm merging of profiles, in

cases where one researcher has more than one profile.

You can save the

researcher, or
validate the
publications first
 | 38

Add a tag to the researcher indicating which

department he/she belongs to

Here I added a tag, so that

once there are hundreds of
researchers in SciVal, each
one is clearly tagged.
 | 39

Notes
 | 40

Bulk Import Researchers that

have one Scopus Author Profile
 | 41

The researchers with one Scopus Author ID can be

bulk imported into SciVal

You can paste the

author IDs belonging
to one department in
the box. Past just the
ID, not the author
name etc.
 | 42

SciVal will show you if any Author ID is not found

If an author ID is not
found, then you
should search for
this author manually
by name in SciVal.
 | 43

Create the Department and give it a name. Remember

to tag the researchers.

Give the
group a
name

Check this box

to create a
group for these
researchers

Tag the
researchers
 | 44

Now you have created a department in SciVal which

you can easily share with other SciVal users.

Use the
sharing
function to
give others
access to this
department
 | 45

Groups can be combined into larger organizational units

Drag and drop units to

combine
 | 46

Notes
 | 47

My SciVal Research Areas

Creating Researcher Areas using keywords
Creating Research Areas by grouping entities
 | 48

Create Research Area

 | 49

Use Keywords to define a very specific area of science

Useful
search
tips
 | 50

Refine & Validate

Validate that
publications are
representative of
topic
 | 51

Prepare your Research Areas in advance. Areas larger

than 5000 publications can take 48 hours
 | 52

Accessing SciVal
 | 53

Accessing SciVal: go to www.scival.com

Who does not have access to SciVal? Please raise your
hand.

If you do not have a

username and password
for SciVal, then please
register here.
 | 54

Thank you very much

Alexander van Servellen
a.vanservellen@elsevier.com

www.elsevier.com/research-intelligence