When statistics attack

I swear stats is trying to kill me. I’ve redesigned my experiment so that it’s a nice elegant “two-factor repeated measures” flavour. I won’t trouble you with exactly what that means, or exactly what the nine separate hypotheses I’m testing are. What I will trouble you with, for it’s certainly been troubling me, is this:

To analyse the data I will collect I need to use a stats test, which broadly speaking is a factory that converts numbers into truth (or lies if you’re not careful).

Jim, Mr Stats, has a stats handbook that tells you how to do this. It has a nifty little flowchart at the beginning that you can trace through to work out which of the several dozen different kinds of stats tests you need to use. Easy enough, I think to myself as my fingers follow the little arrows across the page. And where do I end up? At a little box that states helpfully: “It may be possible to devise an ad hoc statistical test for the design under consideration.”

That’s right – with my new, improved, elegant design, the Oracle of Statistics reckons it may be possible, with not so much as a hint as to how one might actually go about it.

Not to be defeated, however, I turn to the Oracle of Everything – Google – with which I stumble upon something called Factorial Logistic Regression. I certainly won’t trouble you with what this means, not because I don’t want to but because I currently have no idea myself. Neither of my two supervisors – one of whom is Jim Himself – does either.

My only hope appears to lie in a library book entitled Regression Modeling Strategies. So the campaign continues…

How does this experiment work?

Statistics. It all seems to easy until you have to do it.

No worries Dave, I confidently assured myself as I fitted the last details of my delicate experimental design into place, all set to be unleashed on as many undergraduates as I had chocolate to bribe. Now all I have to do is plug in the stats terminology and I’ll… oh shit.

As a research student, you know you’re onto a “winning” idea when you have to write a Python script just to work out what factors your experiment is testing. Somewhat like realising, after you’ve found the ultimate answer to life, the universe and everything, that you didn’t know what the question was.