Welcome

Get your free subscription to the Library Connect Newsletter at

http://libraryconnect.elsevier.com/subscribe

Librarians & altmetrics: Tools, tips and use cases

February 20, 2014
Mike Taylor, Research Specialist, Elsevier Labs (@herrison)
Mike's current areas of work include altmetrics, contributorship,
research networks, the future of scholarly communications and other
identity issues. He has worked in various capacities within the ORCID
initiative.
Jenny Delasalle, Freelance Consultant/Librarian (@JennyDelasalle)
Jenny's interests include bibliometrics and altmetrics, the changing
landscape of scholarly communication, and how researchers can and
do share and promote their research while also protecting their
professional image. Jenny has worked in a number of academic library
roles at various UK higher education institutions, including most
recently at the University of Warwick, managing the library's support
of researchers.
Kristi Holmes, Bioinformaticist, Washington University in St. Louis
(@kristiholmes)
Kristi's professional interests include open science, support and
training in genomic medicine, and understanding the impact of
research efforts. She serves as the outreach lead for the research
discovery platform VIVO and is a member of the ORCID Outreach
Steering Group.

#LCwebinar

Webinar

Mike
Taylor
Research Specialist, Elsevier Labs
http://orcid.org/0000-0002-8534-5985
mi.taylor@elsevier.com

Altmetrics: a primer
Where does the data come from?
Can it be gamed?
Buy in or build your own?
Webinar

#LCwebinar

What is the data?

A set of altmetric data is about a common document
and represents usage, recommendation, shares, reusage
Identified by DOI, URL, shortened URL, other ID
(e.g., arXiv, PubMed)
It does not show common intent: a tweet is not the
same as a Mendeley share is not the same as a Data
Dryad data download is not the same as mass media
coverage or a blog
Webinar
Altmetrics: a primer

Mike Taylor

Various providers

Altmetric.com
Impactstory.org
Plum Analytics
PLOS / PLOS code
Altmetrics is not Altmetric.com

Each has strengths and weaknesses, no canonical source

Webinar

Altmetrics: a primer

Mike Taylor

Different data have different characteristics

Example from 13,500 papers:
Highly tweeted stories focus on policy, gender,
funding, contentious science issues, mostly
summaries on Nature News
Highly shared papers in Mendeley are hard core
original research
Different platforms have discipline bias
Scholarly blogs both lead interest and respond
Webinar
Data from Altmetric.com
Altmetrics: a primer

Mike Taylor

Bringing together sources

Altmetrics isnt one thing, so attempting to express it
as one thing will fail
We favour intelligent clusters of data: social activity,
mass media, scholarly activity, scholarly comment,
re-use
Elsevier believes that more research is needed, and
that best indicators are scholarly activity and
scholarly comment
Webinar

Altmetrics: a primer

Mike Taylor

Gaming / cheating
If people take this data seriously, will they cheat?
E.g., Brazilian citation scandal, strategies used by people to
increase IF of journals
Expertise in detecting fraudulent downloads (e.g., SSRN),
self-tweeting when is normal corrupt?
One thing to buy 1,000 tweets, another to buy 10 blogs, or
mass media coverage
Do those Twitter accounts have scholarly followers?
Pattern analysis, usage analysis, network analysis
Webinar
Public data = public analysis = public response

Altmetrics: a primer

Mike Taylor

Other criticisms
Biggest criticisms are when people try and conflate
all the data into a single thing
Easy point of attack tweets are all about sex drugs
and rock n roll papers*
Using clusters is more intelligible to academic
community e.g., re-use, scholarly activity, scholarly
comment (blogs, reviews, discussions)

* this isnt true anyway

Webinar

Altmetrics: a primer

Mike Taylor

Buy-in, or bake-your-own
Buy-in: Altmetric.com and PLUM from Ebsco
Free-to-use: Impactstory.org, platforms that use
PLOS article-level-metrics code
Bake-your-own: Impactstory.org, PLOS
Or a root-and-branch build

Webinar

Altmetrics: a primer

Mike Taylor

Topics covered

Data sources
Providers
Different types of data, differences and similarities
Criticisms, weaknesses and strategies
Your next steps

Webinar

Altmetrics: a primer

Mike Taylor

Jenny
Delasalle
Freelance Consultant/Librarian

Altmetrics: why it's relevant to a librarian

http://jennydelasalle.wordpress.com/

#LCwebinar

Webinar

Consider the potential of altmetrics

They are appearing in library subscription
products consequences
Should they appear in institutional
repositories? How?
Possibly use when considering crowd
funding/citizen science projects
Spaces to watch
Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Appearing in library subscription &

library recommended products
1. User education/
information
skills/literacy
2. Appropriate use of
data & tools by
institution
3. Marketing and
outreach potential?
Altmetrics : why it's relevant to a librarian

Webinar

Jenny Delasalle

What the public sees

Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Should they appear in

institutional repositories?
1. A showcase: Bring a paper to life : track
related discussions and materials
2. For authors & admin staff to see attention
from newspapers, blogs, social media
3. Use by repository manager: collection
development, depositor encouragement &
proof of the IRs contribution

Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Which article level measures in an IR?

Pageviews, downloads, citations
Saves on Mendeley, Delicious, CiteULike, etc.
Shares through Twitter, Facebook, blog posts,
Mendeley, etc.
Choose a source & tool, e.g.,
Altmetric.com/Impactstory/Plum/PLOS
Allow authors control?
Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Plum
Analytics:
University
of
Pittsburgh
Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Altmetrics: University of Warwick

Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Impactstory
& figshare

Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Crowdsourcing/citizen science
1. Benefits of crowdsourcing include
investment of cash, expertise or resource
2. It is both a means of engaging with the
public, and also relies upon being successful
at such engagement
3. By measuring interest in your research, you
can consider whether there would be
interest in participation/investment
Webinar
Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Spaces to watch
1. Kudos, growkudos.com helps authors and
institutions maximize the impact and visibility of
their publications (Also publishers & societies)
2. ORCID aggregates an individuals work: authors can
use with impactstory.org : are they using it?
3. COUNTER compliant institutional repositories
4. Services (not only tools) that allow universities to
use analytics to develop actionable insights on how
to improve their research, student experience and
efficiency, and effectiveness. Times HigherEd
Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

References & Further Reading

Priem, Costello, Tyler (2012) Prevalence and use of Twitter among scholars http://figshare.com/articles/Prevalence_and_use_of_Twitter_among_scholars/10
4629
The San Francisco Declaration on Research Assessment (DORA declaration),
initiated by the American Society for Cell Biology (ASCB) http://am.ascb.org/dora/
Konkiel, S (2013) Altmetrics in Institutional repositories, ASIS&T 2013, Montreal
https://scholarworks.iu.edu/dspace/bitstream/handle/2022/17122/2013_ASIST_
Altmetrics%20%26%20Libraries.pdf?sequence=1
Smith, D (2013) What is Kudos? An Interview with David Sommer, Co-Founder,
Scholarly Kitchen blog - http://scholarlykitchen.sspnet.org/2013/12/17/what-iskudos-an-interview-with-david-sommer-co-founder/
Brody et al (2009) Publisher and Institutional Repository Usage Statistics (PIRUS)
final report http://www.jisc.ac.uk/media/documents/programmes/pals3/pirus_finalreport.pd
f
Parr, C (2014) Bett show: four areas of technology that could transform
universities, THE - http://www.timeshighereducation.co.uk/news/bett-show-fourareas-of-technology-that-could-transform-universities/4/2011070.article Webinar

Altmetrics : why it's relevant to a librarian

Jenny Delasalle

Kristi
Holmes
Bioinformaticist
Washington University in St. Louis

Going beyond counts: understanding impact

Webinar

#LCwebinar

http://icts.wustl.edu/

Translational Research
The bench to the bedside and beyond

Webinar

Going beyond counts: understanding impact

Kristi Holmes

Why do we need to think about research

impact?
Quantify and document research
impact
Justify future requests for funding
Quantify return on research
investment
Discover how research findings are
being used
Identify similar research projects
Identify possible collaborators
Determine if research findings are
duplicated, confirmed, corrected,
improved or repudiated
https://becker.wustl.edu/impact-assessment/model

Going beyond counts: understanding impact

Determine if research findings were

extended (different human
populations, different animal
models/species, etc.)
Confirm that research findings were
properly attributed/credited
Demonstrate that research findings
are resulting in meaningful health
outcomes
Discover community benefit as a
result of research findings
Progress reports
Tenure
Promotion dossiers

Webinar

Kristi Holmes

How do we think
about productivity?
Dont knock publications
Citations
Rates and rankings
Networks
Dissemination patterns

Collaboration
Dissemination
Grant funding
Research trends
Webinar

http://www.philnel.com/2010/10/14/procrastigrading/

Going beyond counts: understanding impact

Kristi Holmes

but theres so
much more!

Webinar
http://www.philnel.com/2010/10/14/procrastigrading/

Going beyond counts: understanding impact

Kristi Holmes

Research Impact
It is no longer enough to measure what we
can we need to measure what matters.

So how do we measure what

matters?
Webinar
Wells R, Whitworth A. 2007. Assessing outcomes of health and medical research: do we measure what
counts or count what we can measure? Australia and New Zealand Health Policy, 4:14

Going beyond counts: understanding impact

Kristi Holmes

Going beyond the counts to find meaningful

impact
In everyones CV, well find
Reviews
New funding awarded
New types of funding
mechanisms
New research studies
Invited lectures, new focus areas
at conferences
Membership on committees
Awards

Consider specific pathways*

to uncover meaningful
impact
Advancement of Knowledge
Clinical Implementation
Legislation and Policy Enactment
Economic Benefit
Community Benefit
*can be adapted to reflect any number of disciplines

Webinar

Going beyond counts: understanding impact

Kristi Holmes

Going beyond the counts to find meaningful

impact
New diagnostic criteria

http://icts.wustl.edu/

http://icts.wustl.edu/

Going beyond counts: understanding impact

New standard of care

Curriculum guidelines
Measurement instruments
Continuing education materials
Clinical/practice guidelines
Quality measure guidelines
Private healthcare benefit plans
Cost-effective intervention
Consensus development conferences
American Medical Association
Current Procedural Terminology
(CPT) codes
Change in delivery of healthcare
services

Webinar

Kristi Holmes

Slide title

https://becker.wustl.edu/impact-assessment

Going beyond counts: understanding impact

Webinar

Kristi Holmes

The Becker Model

Provides a supplement to publication analysis to provide a more
robust and comprehensive perspective of biomedical research
impact.
reporting templates, glossary of resources and terms, examples of relevant indicators
of impact across the research process, and readings

Straightforward framework for tracking diffusion of research

outputs and activities to locate indicators that demonstrate evidence
of biomedical research impact
individual, core, and institutional-level; modify for different disciplines

Guidance for quantifying and documenting research impact as well

as resources for locating evidence of impact.
Strategies for enhancing the impact of research
Webinar

Going beyond counts: understanding impact

Kristi Holmes

Strategies for enhancing the impact

of research
Optimizing discoverability and
access of your research is the
surest way to enhance its
visibility and impact.
The strategies focus upon
Preparing for Publication,
Dissemination, and Keeping Track
of Your Research.
Repetition, consistency, and an
awareness of the intended
audience form the basis of most
the strategies.

Suggestions for researchers and

recommendations to reach out to their
library for assistance.
Webinar

https://becker.wustl.edu/impact-assessment/strategies
Going beyond counts: understanding impact

Kristi Holmes

Understanding the impact of a single paper can

be challenging.
How do we understand the impact of a person?
A research center?
A university?

Webinar

Going beyond counts: understanding impact

Kristi Holmes

Research networking and discovery systems

Enable more efficient means of

collecting & representing
meaningful outputs en masse
Showcase achievements and
expertise
Facilitate diffusion of research
products
Support team-based science and
collaboration
Allow better understanding of the
research enterprise
Peer comparisons
Strategic planning
Emerging trends

Webinar

Going beyond counts: understanding impact

Kristi Holmes

What is VIVO?
1. An open source semantic
web application
2. An information model
3. An open community*
* A big, welcoming OS community! Let us know if you have
questions or need information, connections, or materials:
http://vivoweb.org/contact

Going beyond counts: understanding impact

Webinar

Kristi Holmes

Tying it all together

Continue to represent
typical profile
information
absolutely!
Enhance profiles by
incorporating
meaningful outputs, as
described by the Becker
Model or other
frameworks, to enhance
their content and value

http://vivo.ufl.edu/display/n86607

Going beyond counts: understanding impact

Can facilitate the

dissemination of
scholarship and more
Webinar

Kristi Holmes

Tying it all together

http://blog.plumanalytics.com/post/57
707501083/plumx-now-supports-vivo

Going beyond counts: understanding impact

PlumX supports VIVO

1. Aggregate
information about our
researchers
2. Harvest research
outputs from the
profile
3. Plums harvesting
engine calculates
metrics and provides
both an analytics
dashboard as well as
visualizations.
4. Widgets can be
embedded back into
the VIVO profile!

Webinar

Kristi Holmes

Tying it all together

PlumX supports VIVO

1. Aggregate
information about our
researchers
2. Harvest research
outputs from the
profile
3. Plums harvesting
engine calculates
metrics and provides
both an analytics
dashboard as well as
visualizations.
4. Widgets can be
embedded back into
FigShare, lab notebooks,
the VIVO profile!
http://vivo.ufl.edu/display/n86607

project outputs,
SlideShare, etc.

Webinar

http://blog.plumanalytics.com/post/57707501083/plumxnow-supports-vivo
Going beyond counts: understanding impact

Kristi Holmes

Acknowledgements
Thanks:

Cathy Sarli, MLS, AHIP

Karen Gutzman NLM Fellow
Jae Allen
Washington University ICTS and
WU ICTS Tracking & Evaluation
Team
Andrea Michalek at Plum
@amichalek
VIVO Community @VIVOcollab
Becker Medical Library

Funding:
Washington University Institute
of Clinical and Translational
Sciences, NIH award UL1
RR024992
VIVO - DuraSpace

Webinar

Going beyond counts: understanding impact

Kristi Holmes

Questions & Thank You!

Mike Taylor, Research Specialist, Elsevier Labs
@herrison
mi.taylor@elsevier.com
http://orcid.org/0000-0002-8534-5985
Jenny Delasalle, Freelance Consultant/Librarian
@JennyDelasalle
jennydelasalle@gmail.com
http://jennydelasalle.wordpress.com/
Kristi Holmes, Bioinformaticist, Washington University in St. Louis
@kristiholmes
Webinar
holmeskr@wustl.edu

#LCwebinar