Wednesday, April 23, 2014

Is publishing debatable conclusions now encouraged?

I am increasingly concerned about how many papers, particularly in luxury journals, publish claims and conclusions that appear (at least to me) to simply not follow from the data or calculations presented.
Is this problem getting worse?
Or am I just getting more sensitive about it?

Last year Nature published
Bounding the pseudogap with a line of phase transitions in YBa2Cu3O6+δ
The abstract states
Here we report that the pseudogap in YBa2Cu3O6+δ is a distinct phase, bounded by a line of phase transitions. The doping dependence of this line is such that it terminates at zero temperature inside the superconducting dome. From this we conclude that quantum criticality drives the strange metallic behaviour and therefore superconductivity in the copper oxide superconductors.
Let me examine separately the three claims I have highlighted in bold.

1. The claim that the line terminates at zero temperature is based on two data points! (the red dots in the figure below).

To be convinced of this claim I would like to see a lot more data points and particularly, extending down to a few Kelvin. Furthermore, even if you believe the authors are seeing a real continuous phase transition one wants to see that it does not terminate in a first-order line at some non-zero temperature.

1b. A line terminating at zero temperature [a quantum phase transition] is not the same as quantum criticality. Establishing that requires observing distinct features and scaling laws [e.g. a dephasing rate that is linear in temperature] at temperatures above the quantum critical point.

For the next two claims, it is important to distinguish causality and correlation. Just because one sees two things together does not mean that one causes the other. They could both be caused by some other underlying effect.

2. "quantum criticality drives the strange metal behaviour".
Here, it could be simply that in this material the phase transition and the strange metal occur at roughly the same doping. Furthermore, there are alternative explanations of the strange metal behaviour.

3. "and therefore superconductivity  in the cuprates".
I fail to see the logic. Varma's theory certainly connects quantum criticality, the strange metal, and superconductivity. But, there are alternatives that do not make this intimate connection. Hence, I can't see how it is legitimate to make this conclusion.

In contrast to the abstract, the last few sentences of the paper makes the more modest claim:
Our observed evolution of the pseudogap phase boundary from underdoped to overdosed establishes the presence of a quantum critical point inside the superconducting dome, suggesting a quantum-critical origin for both the strange metallic behaviour and the mechanism of superconducting pairing. 
Now, the story gets stranger. Look at version 1 of the paper on the arXiv. Presumably this is the version that was originally submitted to Nature. The abstract does not have the debatable sentences but instead the reasonable statements:
In slightly overdoped YBCO that transition is 20K below Tc, extending the pseudogap phase boundary inside the superconducting dome. This supports a description of the metallic state in cuprates where a pseudogap phase boundary evolves into a quantum critical point masked by the superconducting dome.
So, I would love to know whose idea it was to change the abstract? Is there any chance it was a Nature editor who wanted to "sex up" the paper?

But my real problem is not so much with this specific paper, but the many other cases I see, sometimes in non-luxury journals. Science is all about using rigorous thinking and experimentation to find out what is actually true, as best as we can tell. It is fine to speculate and to suggest possible correlations and causality. But that is totally different to claiming you have shown something to be true when you have not. We need to be precise in our language. I do think science is broken.

If we don't practice rigorous evidence-based thinking in our own community what right do we have to challenge politicians and business people who embrace climate change skepticism, opposition to vaccines, AIDS denialism, .....

3 comments:

  1. This case also illustrates the frustratingly prevalent magical thinking that "quantum criticality" constitutes a mechanism. I assume the authors don't think that ordinary finite temperature critical phenomena are the "mechanism" that causes (say) magnetic ordering at the Curie temperature. The existence of phase transitions comes from microscopic physics. The remarkable thing is that certain features near the transition can be described by a macroscopic phenomenological theory, but that shouldn't be confused with an explanation for why the transition happens in the first place.

    ReplyDelete
  2. Gee, you noticed! We're never happy with how things turn out, are we? The tradition of dry, passive-voice, cautious rationality has come under attack for rendering scholarly literature "inaccessible" (a serious problem) and Nature has helped combat this by encouraging "voiced" language and sensationalism. But now we have the premiere scientific journals reading like popular science journalism, which may serve "dissemination" well but science itself not so much. What to do? (Check out http://opeer.org :-)

    ReplyDelete
  3. it should be pointed out that the lead author of this paper along with collaborators has also pushed for the view that there is a QCP cutting the SC dome and they have a pet theory they are always advocating, so one might just get the impression that they are pushing an agenda...

    ReplyDelete