You’re not getting all the credit you should be for your research.

As an early career researcher, you’re likely publishing open access journal articles, sharing your research data and software code on GitHub, posting slides and figures on Slideshare and Figshare, and “opening up” your research in many other ways.

Yet these Open Science products and their impacts (on other scholars, the public, policymakers, and other stakeholders) are rarely mentioned when applying for jobs, tenure and promotion, and grants.

The traditional means of sharing your impact–citation counts–don’t meet the needs of today’s researchers. What you and the rest of Generation Open need is altmetrics.

In this post, I’ll describe what altmetrics are and the types of altmetrics you can expect to receive as someone who practices Open Science. We’ll also cover real life examples of scientists who used altmetrics to get grants and tenure–and how you can do the same.

Altmetrics 101

Altmetrics measure the attention your scholarly work receives online, from a variety of audiences.

As a scientist, you create research data, analyses, research narratives, and scholarly conversations on a daily basis. Altmetrics–measures of use sourced from the social web– can account for the uses of all of these varied output types.

Nearly everything that can be measured online has the potential to be an altmetric indicator. Here are just a few examples of the types of information that can be tracked for research articles alone:

When you add research software, data, slides, posters, and other scholarly outputs to the equation, the list of metrics you can use to understand the reception to your work grows exponentially.

And altmetrics can also help you understand the interest in your work from those both inside and outside of the Ivory Tower. For example, what are members of the public saying about your climate change research? How has it affected the decisions and debates among policy makers? Has it led to the adoption of new technologies in the private sector?

The days when your research only mattered to other academics are gone. And with them also goes the idea that there’s only one type of impact.

Flavors of impact

There are many flavors of impact that altmetrics can illuminate for you, beyond the traditional scholarly impact that’s measured by citations.

This 2012 study was the first to showcase the concept of flavors of impact via altmetrics. These flavors are found by examining the correlations between different altmetric indicators; how does a Mendeley bookmark correlate to a citation, or to a Facebook share? (And so on.) What can groups of correlations tell us about the uses of scholarship?

Among the flavors the researchers identified were a “popular hit” flavor (where scholarship is highly tweeted and shared on Facebook, but not seen much on scholarly sites like Mendeley or in citations) and an “expert pick” flavor (evidenced by F1000 Prime ratings and later citations, but few social shares or mentions). Lutz Bornmann’s 2014 study built upon that work, documenting that articles that are tagged on F1000 Prime as being “good for teaching” had more shares on Twitter–uncovering possible uses among educational audiences.

The correlation that’s on everyone’s mind? How do social media (and other indicators) correlate with citations? Mendeley bookmarks are found to have the most correlations with citations; this points to Mendeley’s use as a leading indicator (that is, if something is bookmarked on Mendeley today, it’s got better chance of being cited down the road than something that’s not bookmarked).

Correlations with citations aren’t the only correlations we should pay attention to, though. They only tell one part of an impact story–an important part, to be sure, but not the only part.

Altmetrics data includes qualitative data, too

Many don’t realize that altmetrics data isn’t only about the numbers. An important function of altmetrics aggregators like Altmetric.com and Impactstory (which we describe in more detail below) is to gather qualitative data from across the web into a single place, making it easy to read exactly what others are saying about your scholarship. Altmetric.com does this by including snippets of the blogs, tweets, and other mentions your work receives online. Impactstory links out to the data providers themselves, allowing you to more easily find and read the full-length mentions from across the web.

Altmetrics for Open Science

Now that you have an understanding of how altmetrics work in general, let’s talk about how they work for you as an Open Scientist. Below, we’ve listed some of the basic metrics you can expect to see on the scholarship that you make Open Access. We’ll discuss how to find these metrics in the next section.

Metrics for all products

Any scholarly object that’s got a URL or other permanent identifier like a DOI–which, if you’re practicing Open Science, would be all of them–can be shared and discussed online.

So, for any of your scholarly outputs that have been discussed online, you can expect to find Twitter mentions, blog posts and blog comments, Facebook and Google+ shares and comments, mainstream media mentions, and Wikipedia mentions.

Open Access Publications

Your open access publications will likely accrue citations same as your publications that appear in subscription journals, with two key differences: you can track citations to work that isn’t formally published (but has instead been shared on a preprint server like ArXiv or other such repository) and you can track citations to work that appear in non-peer reviewed literature. Citation indices like Scopus and Web of Science can help you track the former. Google Scholar is a good way to find citations in the non-peer reviewed literature.

Views and downloads can be found on some journal websites, and often on repositories–whether your university’s institutional repository, a subject repository like BioRXiv, or a general purpose repository like Figshare.

Bookmarks on reference management services like Mendeley and CiteULike can give you a sense of how widely your work is being read, and by what audiences. Mendeley, in particular, offers excellent demographic information for publications bookmarked in the service.

Software & code

Software & code, like other non-paper scholarly products, are often shared on specialized platforms. On these platforms, the type of metrics your work receives is often linked to the platform itself.

SourceForge blazed the trail for data metrics by allowing others to review and rate code–useful, crowd-sourced quality indicators.

On GitHub, you can expect for your work to receive forks (which signal adaptations of your code), stars (a bookmark or virtual fistbump that lets others tell you, “I like this”), pull requests (which can get at others’ engagement with your work, as well as the degree to which you tend to collaborate), and downloads (which may signal software installations or code use). One big advantage to using GitHub to share your code is that it allows you to mint DOIs–making it much easier to track mentions and shares of your code in the scholarly literature and across general purpose platforms, like those outlined above.

Data

Data is often cited in one of two ways: citations to data packages (the dataset itself, stored on a website or repository) and citations to data papers (publications that describe the dataset in detail, and that link out to the dataset). You can often track the former using an altmetrics aggregator (more on that in a moment) or the Data Citation Index, a database that’s similar to Web of Science which searches for mentions of your dataset in the scholarly literature. Citations to data papers can sometimes be found in traditional citation indices like Scopus and Web of Science.

Interest in datasets can also be measured by tracking views and downloads. Often, these metrics are shared on repositories where datasets are stored.

Where data is shared on GitHub, forks and stars (described above) can give an indication of that data’s reuse.

More info on metrics for data can be found on my post for the e-Science Portal Blog, “Tracking the Impacts of Data–Beyond Citations”.

Videos

Videos are created by many researchers to summarize a study for generalist audiences. Other times, videos are a type of data.

YouTube tracks the most varied metrics: views, likes, dislikes, and comments are all reported. On Vimeo and other video sharing sites, likes and views are the most often reported metrics.

Slide decks & posters

Slide decks and posters are among the scholarly outputs that get the least amount of love. Once you’ve returned from your conference, you tend to shelve and forget about the poster that you (or your grad students) have put hours worth of work into–and the same goes for the slide decks you use when presenting.

If you make these “forgotten” products available online, on the other hand, you can expect to see some of the following indicators of interest in your work: views, favorites (sometimes used as a bookmark, other times as a way of saying “job well done!”), downloads, comments, and embeds (which can show you how often–and by whom–your work is being shared and in some cases blogged about).

How to collect your metrics from across the Web

We just covered a heck of a lot of metrics, huh? Luckily, altmetrics aggregators are designed to collect these far-flung data points from across the web and deliver them to you in a single report.

There are three main independent altmetrics aggregators: Impactstory.org, PlumX, and Altmetric.com. Here’s the scoop:

• Impactstory.org: we’re a non-profit altmetrics service that collects metrics for all scholarly outputs. Impactstory profiles are designed to meet the needs of individual scientists. We regularly introduce new features based on user demand. You can sign up for a 30-day free trial on our website; after that, subscriptions are $10/month or $60/year.

• PlumX: a commercial service that is designed to meet the needs of administrators and funding agencies. Like Impactstory, PlumX also collects metrics for all scholarly outputs. PlumX boasts the largest data coverage of all altmetrics aggregators.

• Altmetric.com: a commercial service that collects metrics primarily for publishers and institutions. Altmetric can track any scholarly output with a DOI, PubMed ID, ArXiv ID, or Handle, but it does publications the best. Uniquely, they can find mentions to your scholarship in the mainstream media mentions and policy documents–two notoriously hard to mine locations.

Once you’ve collected your metrics from across the web, what do you do with them? We suggest experimenting with using them in your CV, year-end reporting, grant applications, and even tenure & promotion dossiers.

Skeptical? You needn’t be. An increasing number of scientists are using altmetrics for these purposes.

Researchers who have used altmetrics for tenure & grants

Each of the following researchers used altmetrics, alongside traditional metrics like citation counts and journal impact factors, to document the impact of their work.

Tenure: Dr. Steven Roberts, University of Washington

Steven is an Associate Professor in the School of Aquatic & Fishery Sciences at the University of Washington. He decided to use altmetrics data in his tenure dossier to two ends: to showcase his public engagement and to document interest in his work.

To showcase public engagement, Steven included this table in the Education and Outreach section of his dossier, illustrating the effects his various outreach channels (blog, Facebook, Flickr, etc) have had to date:

For evidence of the impact of specific products, he incorporated metrics into his CV like this:

Steven’s bid for tenure was successful.

Want to see more? You can download Steven’s full tenure dossier here.

Tenure: Dr. Ahmed Moustafa, American University in Cairo

Ahmed’s an Associate Professor in the Department of Biology at American University in Cairo, Egypt.

He used altmetrics data in his tenure dossier in two interesting ways. First, he included a screenshot of his most important scholarly products, as they appear on his Impactstory profile, to summarize the overall impacts of his work:

Note the badges that summarize in a glance the relative impacts of his work among both the public and other scholars. Ahmed also includes a link to his full profile, so his reviewers can drill down into the impact details of all his works, and also review them for themselves.

Ahmed also showcased the impact of a particular software package he created, JAligner, by including a link to a Google Scholar search that showcases all the scholarship that cites his software:

As of August 2013, JAligner has been cited in more than 150 publications, including journal articles, books, and patents, (http://tinyurl.com/jalignercitations) covering a wide range of topics in biomedical and computational research areas and downloaded almost 20,000 times (Figure 6). It is somehow noteworthy that JAligner has claimed its own Wikipedia entry (http://en.wikipedia.org/wiki/JAligner)!

Ahmed received tenure with AUC in 2013.

Grant Reporting: Dr. Holly Bik, University of Birmingham

Holly was awarded a major grant from the Alfred P. Sloan Foundation to develop a bioinformatics data visualization tool called Phinch.

When reporting back to Sloan on the success of her project, she included metrics like the Figshare views that related posters and talks received, Github statistics for the Phinch software, and other altmetrics related to the varied outputs that the project created over the last few years.

Holly’s hopeful that these metrics, in addition to the traditional metrics she’s reported to Sloan, will make a great case for renewal funding, so they can continue their work on Phinch.

Will altmetrics work for you?

The remarkable thing about each of these researchers is that their circumstances aren’t extraordinary. The organizations they work for and receive funding from are fairly traditional ones. It follows that you, too, may be able to use altmetrics to document the impacts of your Open Science, no matter where you work or are applying for funding. After all, more and more institutions are starting to incorporate recognition of non-traditional scholarship into their tenure & promotion guidelines. You’ll need non-traditional ways like altmetrics to showcase the impacts of that scholarship.

You may have noticed a change in our blog in recent months: we’ve added a number of editorial, how-to, and opinion posts, in addition to “behind the scenes” Impactstory updates.

Posts on our blogs and commentary on Twitter serve two purposes for us.  First, they promote our nonprofit goals of education and awareness.  Second, they serve as “content marketing,” a great way to get awareness of Impactstory to a broader audience.

We’ve been tracking the efficacy of this new strategy for a while now, and thought we’d begin to share the numbers with you in the spirit of making Impactstory more transparent. After all, if you’re an Impactstory fan, you’re likely interested in metrics of all stripes.

Here are our numbers for August 2014.

Organic site traffic stats

• Unique visitors to impactstory.org 3,429
• New users 378
• Conversion rate 11.3% (% of visitors who signed up for an Impactstory.org account)

Blog stats

• Unique visitors 4,381
• Pageviews 6,431
• Clickthrough rate (% of people who visited impactstory.org from the blog) 1.6%
• Conversion rate (% of impactstory.org visitors to blog who went on to sign up for an Impactstory.org account) 9.8%
• Percent of new user signups 1.8%

Overall: Our blog traffic has been steadily increasing from May onward: from 3896 pageviews to 6431 pageviews per month. And the number of unique visitors to our blog has increased, too: from 2,311 a month to 4,381 per month. We published four blog posts in August, two of which could be considered “content marketing”: an interview with Impactstory Advisor, Megan O’Donnell, and our monthly Open Science and Altmetrics Roundup.

What about clickthrough and conversion rates? On the one hand, it’d be helpful to compare these rates against industry norms; on the other hand, which “industry norms” would those be? Startup norms? Non-profit norms? Academic norms? In the end, I’ve decided it’s best to just use these numbers as a benchmark and forget about comparisons.

Twitter stats

• New followers 215
• Increase in followers over previous month 5.11%
• Mentions 346 (We’re tracking this to answer the question, “How engaged are our followers?”)
• Tweet reach 3,543,827 (We’re tracking this–the number of people who potentially saw a tweet mentioning Impactstory or our blog–to understand our brand awareness)
• Referrals to impactstory.org: 271 users
• Signups: 32

Overall: Our Twitter follower growth rate actually went down from May, from around ~8% new followers to ~5%. I did not (and have not yet) crossed the 5,000 follower threshold: a milestone that I intended to hit around August 20th. That said, engagement was up from the previous month by ~23%, a change that reflects conscious effort.

What does it all mean?

Our August numbers were no doubt affected by our subscription announcements and the new Impactstory features. I’m interested to see how these statistics change through September, which has seen an end to the “early adopter” 30 day free trial, and the debut of all the features we deployed during the 5 Meter sprint.

Our blog receives more unique visitors than our website, at this point, so increasing the number of blog-referred signups is a priority.

We could also stand to improve our conversion rates from organic website traffic, too. Our rates are lower than average when compared to other non-profits, publishing-related organizations, and IT.

Looking ahead

Given our findings from this month’s stats, here are our goals for September (already half-over, I know) and October:

• Website: Jason and Heather will be working in the coming months to improve conversion rates by introducing new features that drive signups and subscriptions.
• Blog: Increase unique visitors and the conversion rate for new signups–the former to continue to build brand awareness by publishing blogposts that resonate with scientists, and the latter the latter for obvious reasons. 🙂 One tactic could be to begin offering at least 1 content marketing post per week–a challenging task.
• Twitter: Increase our growth rate for Twitter followers, pass the 5,000 follower mark, and continue to engage with our audience in ways that provide value–whether by sharing Open Science and altmetrics news and research, answering a question they have about Impactstory, or connecting them with other scientists and resources.
• In general: Listen to (and act upon) feedback we get via social media. Continue to create useful blog content that meets the needs of practicing scientists, and to scour the web for the most interesting and relevant Open Science and Altmetrics news and research to share with our audience.

Questions?

Are there statistics you’re curious about, or do you have questions about our new approach to marketing? I’m happy to answer them in the comments below. Cheers!

Updated Dec. 31 2014 to reflect more accurate calculation for conversion rates from blog traffic.

September’s Impactstory Advisor of the Month is (drumroll please!)
Guillaume Lobet!

Guillaume is a post-doc researcher at the Université de Liège in Belgium, in the Plant Physiology lab of Pr. Claire Perilleux. He’s also a dedicated practitioner of open, web native science, creating awesome tools ranging from a Plant Image Analysis software finder to an image analysis toolbox that allows the quantitative analysis of root system architecture. He’s even created an open source webapp that uses Impactstory’s open profile data to automatically create CVs in LaTeX, HTML, and PDF formats. (More on that below.)

I had the pleasure of corresponding with Guillaume this week to talk about his research, what he enjoys about practicing web native science, and his approach to being an Impactstory Advisor.

Tell us a bit about your current research.

I am a plant physiologist. My current work focuses on how the growth and development of different plant organs (e.g. the root and the shoot) are coordinated, and how modifications in one organ affects the others. The project is fascinating, because so far the majority of the plant research is focused on one specific organ or process and few has been done to try to understand how the different parts communicate.

Why did you initially decide to join Impactstory?

A couple of years ago, I created a website referencing the existing plant image analysis software tools (www.plant-image-analysis.org). I wanted to help users understand how well the tools (or more specifically, the scientific papers describing the tools) have been received by the community. At that time, an article-level Impactstory widget was available, and I choose to use it. It was a great addition to the website!

At the same time, I created a Impactstory profile and I’ve used it since then. (A quick word about the new profiles: they look fantastic!)

Why did you decide to become an Advisor?

Mainly because the ideas promoted by the Impactstory team are in line with my own. Researchers’ contributions to the scientific community (or even to society in general) are not only done by publishing peer-reviewed paper (even though it is still a very important way to disseminate our findings). The Web 2.0 brought us a large array of means to contribute to the scientific debate and it would be restrictive not to consider those while evaluating one’s work.

How have you been spreading the word about Impactstory?

I started by talking about it with my direct colleagues. Then, I noticed that science valorisation in general was not well known, so made a presentation about it and shared it on Figshare. To my great surprise, it became my most viewed item (I guess people liked the Lord of the Rings / Impactstory mash up :)). In addition, I also created a small widget to convert any Impactstory online profile into a resume. And of course, I proudly wear my Impactstory t-shirt whenever I go to conferences, which alway bring questions such as “I heard of that, what is it exactly?”.

You’re a web-native scientist (as evidenced by your active presence on sites like Figshare, Github, and Mendeley). When did you start practicing web-native science? What do you like about it? Are there drawbacks?

It really started a couple of years ago, by the end of my PhD. At that time, I needed to apply for a new position, so I set up a webpage, Mendeley account, and so on. I quickly found it to be a great way to get in touch with other researchers.

What I like the most about web-native science is that boundaries are disappearing! You do not need to meet people in person to build a new project or start a new collaboration. It brings together all the researchers of the same fields who are scattered around the globe, into a small digital community where they can easily interact!

As of the drawbacks, I am still looking for them 🙂

Tell us about your “Impact CV” webapp, which converts anyone’s Impactstory profile data into PDF, Markdown, LaTeX, or HTML format. Why’d you create it and how’d you do it?

A few months ago, I needed to update my resume and my IS profile contained all my research outputs. So I thought it would be nice to be able to reuse this information, not only for me, but for everyone who has an Impactstory profile. So instead of copying & pasting my online profile to my resume,  I took advantage of the openness of Impactstory to automatically retrieve the data contained in my profile (everything is stored in a Json file that is readily available from any profile) and re-use it locally. I wrapped it up in a webpage (http://www.guillaumelobet.be/impact) and Voilà!

What’s the best part about your work as a post-doc researcher at the Université de Liège?

Academic freedom is definitely the best part about working in a University. It gives us the latitude to explore unexpected paths. And I work with great people!

Thanks, Guillaume!

As a token of our appreciation for Guillaume’s hard work, we’re sending him an Impactstory t-shirt of his choice from our Zazzle store.

Guillaume is just one part of a growing community of Impactstory Advisors. Want to join the ranks of some of the Web’s most cutting-edge researchers and librarians? Apply to be an Advisor today!

Not long ago, Jason received an email from an Impactstory user, asking him to respond to the anti-altmetrics claims raised by librarian Jeffrey Beall in a blogpost titled, “Article-Level Metrics: An Ill-Conceived and Meretricious Idea.”

Beall is well-known for his blog, which he uses to expose predatory journals and publishers that abuse Open Access publishing. This has been valuable to the OA community, and we commend Beall’s efforts. But we think his his post on altmetrics was not quite so well-grounded.

In the post, Beall claims that altmetrics don’t measure anything of quality. That they don’t measure the impact that matters. That altmetrics they can be easily gamed.

He’s not alone in making these criticisms; they’re common. But they’re also ill-informed. So, we thought that we’d make our responses public, because if one person is emailing to ask us about them, others must have questions, too.

Citations and the journal impact factor are a better measure of quality than altmetrics

Actually, citations and impact factors don’t measure quality.

Did I just blow your mind?

What citations actually measure

Although early theorists emphasized citation as a dispassionate connector of ideas, more recent research has repeatedly demonstrated that citation actually has more complex motivations, including often as a rhetorical tool or a way to satisfy social obligations (just ask a student who’s failed to cite their advisor). In fact, Simkin and Roychowdhury (2002) estimate that as few as 20% of citers even read the paper they’re citing. That’s before we even start talking about the dramatic disciplinary differences in citation behavior.

When it comes down to it, because we can’t identify citer motivations when looking at a citation count alone (and to date efforts to use sentiment analysis to understand citation motivations have failed to be widely adopted) the only bulletproof way to understand the intent behind citations is to read the paper that cites.

It’s true that some studies have shown that citations correlate with other measures of scientific quality like awards, grant funding, and peer evaluation. We’re not saying they’re not useful. But citations do not directly measure quality, which is something that some scientists seem to forget.

What journal impact factors actually measure

We were surprised that Beall holds up the journal impact factor as a superior way to understand the quality of individual papers. The journal impact factor has been repeatedly criticized throughout the years, and one issue above all others renders Beall’s argument moot: the impact factor is a journal-level measure of impact, and therefore irrelevant to the measure of article-level impact.

What altmetrics actually measure

The point of altmetrics isn’t to measure quality. It’s to better understand impact: both the quantity of impact and the diverse types of impact.

And when we supplement traditional measures of impact like citations with newer, altmetrics-based measures like post-publication peer review counts, scholarly bookmarks, etc we have a better picture of the full extent of impact. Not the only picture. But a better picture.

Altmetrics advocates aim to make everything a number. Only peer review will accurately get at quality.

This criticism is only half-wrong. We agree that informed, impartial expert consensus remains the gold standard for scientific quality. (Though traditional peer-review is certainly far from bullet-proof when it comes to finding this.)

But we take exception to the charge that we’re only interested in quantifying impact. In fact, we think that the compelling thing about altmetrics services is that they bring together important qualitative data (like post-publication peer reviews, mainstream media coverage, who’s bookmarking what on Mendeley, and so on) that can’t be summed up in a number.

The scholarly literature on altmetrics is growing fast, but it’s still early. And altmetrics reporting services can only improve over time, as we discover more and better data and ways to analyze it. Until then, using an altmetrics reporting service like our own (Impactstory), Altmetric.com or PlumX is the best way to discover the qualitative data at the heart of diverse impacts. (More on that below.)

There’s only one type of important impact: scholarly impact. And that’s already quantified in the impact factor and citations.

The idea that “the true impact of science is measured by its influence on subsequent scholarship” would likely be news to patients’ rights advocates, practitioners, educators, and everyone else that isn’t an academic but still uses research findings. And the assertion that laypeople aren’t able to understand scholarship is not only condescending, it’s wrong: cf. Kim Goodsell, Jack Andraka, and others.

Moreover, who are the people and groups that argue in favor of One Impact Above All Others, measured only through the impact factor and citations? Often, it’s the established class of scholars, most of whom have benefited from being good at attaining a very particular type of impact and who have no interest in changing the system to recognize and reward diverse impacts.

Even if we were to agree that scholarly impact were of paramount importance, let’s be real: the impact factor and citations alone aren’t sufficient to measure and understand scholarly impact in the 21st century.

Why? Because science is moving online. Mendeley and CiteULike bookmarks, Google Scholar citations, ResearchGate and Academia.edu pageviews and downloads, dataset citations, and other measures of scholarly attention have the potential to help us define and better understand new flavors of scholarly attention. Citations and impact factors by themselves just don’t cut the mustard.

I heard you can buy tweets. That proves that altmetrics can be gamed very easily.

There’s no denying that “gaming” happens, and it’s not limited to altmetrics. In fact, there have recently been journals that have been banned from Thomson-Reuters Journal Citation List due to impact factor manipulation, and papers retracted after a “citation ring” was busted. And researchers have proven just how easy it is to game Google Scholar citations.

Most players in the altmetrics world are pretty vigilant about staying one step ahead of the cheaters. (Though, to be clear, there’s not much evidence that scientists are gaming their altmetrics, since altmetrics aren’t yet central to the review and rewards systems in science.) Some good examples are SSRN’s means for finding and banning fraudulent downloaders, PLOS’s “Case Study in Anti-Gaming Mechanisms for Altmetrics,” and Altmetric.com’s thoughts on the complications of rooting out spammers and gamers. And we’re seeing new technology debut monthly that helps us uncover bots on Twitter and Wikipedia, fake reviews and social bookmarking spam.

Crucially, altmetrics reporting services make it easier than ever to sniff out gamed metrics by exposing the underlying data. Now, you can read all the tweets about a paper in one place, for example, or see who’s bookmarking a dataset on Delicious. And by bringing together that data, we help users decide for themselves whether that paper’s altmetrics have been gamed. (Not dissimilar from Beall’s other blog posts, which bring together information on predatory OA publishers in one place for others to easily access and use!)

Altmetrics advocates just want to bring down The Man

We’re not sure about what that means. But we sure are interested in bringing down barriers that keep science from being as efficient, productive, and open as it should be.  One of those barriers is the current incentive system for science, which is heavily dependent upon proprietary, opaque metrics such as the journal impact factor.

Our true endgame is to make all metrics–including those pushed by The Man–accurate, auditable, and meaningful. As Heather and Jason explain in their “Power of Altmetrics on a CV” article in the ASIS&T Bulletin:

Accurate data is up-to-date, well-described and has been filtered to remove attempts at deceitful gaming. Auditable data implies completely open and transparent calculation formulas for aggregation, navigable links to original sources and access by anyone without a subscription. Meaningful data needs context and reference. Categorizing online activity into an engagement framework helps readers understand the metrics without becoming overwhelmed. Reference is also crucial. How many tweets is a lot? What percentage of papers are cited in Wikipedia? Representing raw counts as statistically rigorous percentiles, ideally localized to domain or type of product, makes it easy to interpret the data responsibly.

That’s why we incorporated as a non-profit: to make sure that our goal of building an Open altmetrics infrastructure–which would help make altmetrics accurate, auditable, and meaningful–isn’t corrupted by commercial interests.

Do you have questions related to Beall’s–or others’–claims about altmetrics? Leave them in the comments below.