Maybe that is why I stick to EEG (and because it is a whee dram cheaper). Read all about it in Mindhacks. Fabulous blog !
For example, if you're measuring an adult's height you want to make sure that your tape measure gives you similar results each time you use it on the same person. Of course, you may have readings that vary by a millimetre or two each time, but if you get wildly different results on Monday and Tuesday, you probably want to bin your tape measure.
In fMRI there are two types of results. One is 'where in the brain' and the other is 'how strong' is the activity.
We can examine the first by looking at how well the active brain areas overlap in scans taken at two different times, and we can examine the second by looking at the similarity of the strength of the results using a statistical test like a correlation.
The better the overlap and the statistical relationship between the results from the same test on the same people at different times, the more we can rely on our measurement technique.
This new analysis reviewed all the previous studies that have looked at the test-retest reliability of fMRI and found that overall, active brain areas overlap about 30% of the time and the correlation for the strength of the activity was about 0.5. To get some perspective a result of 1 would indicate perfectly reliable and reproducible results while a result of 0 would indicate no reliability at all.
An overlap of 30% and a correlation result of 0.5 shows fMRI has moderate reliability, but is much poorer than most people assume.
However, this overall result is perhaps a little too broad, and the authors make the point that the reliability varies depending on the type of scanner being used, what test is being carried out by the participants, what brain areas are being investigated and how the results are analysed