Email or username:

Password:

Forgot your password?
All posts Alex's posts Post Back to profile
Top-level
Cameron Neylon

@johnntowse @alexwild The other point is that using a bigger citation data source might give a different result if there is a real effect but the effect size isn't huge and the statistical power not quite there. That's another thing that would be relatively easy to test with OpenCitations and the DOIs (I'll put it on the list...)

21 March at 12:45 | Wall-to-wall | Open on hcommons.social
3 comments
John Towse

@cameronneylon @alexwild
The issue of power is discussed in the paper of course, but I am sympathetic to the argument there the effect size is not that impactful (even if it exists at the population level, it’s not making much difference for the individuals who do or don’t tweet about their papers)

21 March at 13:13 | Open on mstdn.social
Cameron Neylon

@johnntowse @alexwild

Agreed, my counter would be that in many of these cases the distribution of effects amongst individual outputs is wild, so effect sizes may look small on average but the effect when it happens can be quite large. And I would always have expected any effect to be large, but for a subset of papers.

Obviously randomised control trials like this to smear some of those effects out by design.

I feel that a Hidden Markov Model or time domain analysis would ultimately help in understanding the underlying pathways. But I also get that those approaches tell us about probabilistic associations, not causality - which is where the approach here is strong

And all of that said your main point is well supported - that for any specific paper, being tweeted about doesn't (didn't?) lead to significantly more citations on average

@johnntowse @alexwild

Agreed, my counter would be that in many of these cases the distribution of effects amongst individual outputs is wild, so effect sizes may look small on average but the effect when it happens can be quite large. And I would always have expected any effect to be large, but for a subset of papers.

21 March at 13:24 | Open on hcommons.social
John Towse

@alexwild @cameronneylon
Absolutely, these are really interesting questions to think about in response to a clever paper. (And in the meantime those who stay away from social media / certain social media can modulate their FOMO!)

21 March at 13:31 | Open on mstdn.social
Go Up