Email or username:

Password:

Forgot your password?
All posts Alex's posts Post Back to profile
Top-level
Cameron Neylon

@alexwild

Nice work! This takes me back to speculative musings on the time domain behaviour of these interventions (hdl.handle.net/20.500.11937/32, your Figure 2 made me think of my Figure 4)

You've really captured the immediacy of the viewing effect and I'm wondering whether a citation effect might be clearer if analysed in a more time dependent way rather than at a three year census point...

...but you've given us the necessary information to make that analysis possible, which is fabulous! (whether I have the time is another question)

The other question I've got is whether the citations might show greater diversity (reaching a wider range of scholars) because they are coming through a set of followers that might have wider geographic or disciplinary diversity. And we can test that as well! (same caveats apply...)

21 March at 9:25 | Wall-to-wall | Open on hcommons.social
5 comments
John Towse

@alexwild @cameronneylon
Yes, this makes sense…
There are direct citations (citing X because I’m replicating X / extending X by taking the next step / assimilating X into a theory) and there are more indirect citations (citing X because it’s interesting & cool & maybe it can link to these data Y). Social media might be expected to pull more of the latter, but evidently not noticeably so. A deeper dive into the non-sig citation gain might examine this diversity

21 March at 12:32 | Open on mstdn.social
Cameron Neylon

@johnntowse @alexwild The other point is that using a bigger citation data source might give a different result if there is a real effect but the effect size isn't huge and the statistical power not quite there. That's another thing that would be relatively easy to test with OpenCitations and the DOIs (I'll put it on the list...)

21 March at 12:45 | Open on hcommons.social
John Towse

@cameronneylon @alexwild
The issue of power is discussed in the paper of course, but I am sympathetic to the argument there the effect size is not that impactful (even if it exists at the population level, it’s not making much difference for the individuals who do or don’t tweet about their papers)

21 March at 13:13 | Open on mstdn.social
Cameron Neylon

@johnntowse @alexwild

Agreed, my counter would be that in many of these cases the distribution of effects amongst individual outputs is wild, so effect sizes may look small on average but the effect when it happens can be quite large. And I would always have expected any effect to be large, but for a subset of papers.

Obviously randomised control trials like this to smear some of those effects out by design.

I feel that a Hidden Markov Model or time domain analysis would ultimately help in understanding the underlying pathways. But I also get that those approaches tell us about probabilistic associations, not causality - which is where the approach here is strong

And all of that said your main point is well supported - that for any specific paper, being tweeted about doesn't (didn't?) lead to significantly more citations on average

@johnntowse @alexwild

Agreed, my counter would be that in many of these cases the distribution of effects amongst individual outputs is wild, so effect sizes may look small on average but the effect when it happens can be quite large. And I would always have expected any effect to be large, but for a subset of papers.

21 March at 13:24 | Open on hcommons.social
John Towse

@alexwild @cameronneylon
Absolutely, these are really interesting questions to think about in response to a clever paper. (And in the meantime those who stay away from social media / certain social media can modulate their FOMO!)

21 March at 13:31 | Open on mstdn.social
Go Up