To be specific, I consult on research and analysis design, and also do coding work. Most clients are small businesses or independent. A few bigger organizations, and the odd student rich enough to drop a few hundo on getting a prof to critique their term paper. Rarely, academia.
-
-
Prikaži ovu nit
-
The first big thing you notice is that a lot of people tend to think of statistics as a tool that serves a function, rather than as a method of analysis.
Prikaži ovu nit -
What I mean by this is that it's common to think of statistical analysis almost like an enterprise product. If I buy this "statistics" doodad, it will help me make my point. If it doesn't do that, it serves no purpose.
Prikaži ovu nit -
It's extremely common that I will get requests to "fix" the work if it doesn't give them the result they want. I've had people demand refunds. Plenty of job ads that tell the worker the results they're supposed to get. I don't respond to those ones.
Prikaži ovu nit -
Tempting to think of this as fraudulent, but in the conversations I have with people I think it's more a deep misunderstanding of the purpose of statistics and empiricism.
Prikaži ovu nit -
It's extremely difficult to even get across the point of why this is a bad thing. And believe me, I've tried this conversation many times. They want good statistics and "best practices." They just literally think that this means doing whatever gives you the "right" result.
Prikaži ovu nit -
Related, but p-hacking is just an incredibly natural temptation. I'd say that a good 50% of the insignificant results I return get a response asking how they can be changed. The most common phrasing is asking how they can be "fixed." No concept of that being bad.
Prikaži ovu nit -
I say "natural" from academic experience. I've never seen a stat/metrics class that teaches p-hacking as good. Many explicitly warn against it. But when I see those students in the follow-up class, for many it's just the obvious thing to do. Of COURSE you "fix" insignificance.
Prikaži ovu nit -
(I have one client at the moment who I keep telling that his results long ago stopped meaning anything as we round the corner on maybe 2000 different specifications, but he keeps adding 'em and looking for significance! The desire to beat the stock market is a strong one.)
Prikaži ovu nit -
The second thing that I see is that statistics is seen as a real push-button operation. You pour in the numbers, the smart person pulls the right lever, and the result pops out toasty, warm, and ready to eat.
Prikaži ovu nit -
There's very little concept that context is important, or that there's some sort of underlying model. I've on occasion gotten shocked surprise when I've told clients that I need to know what the variables in the data set actually represent before I build an analysis plan.
Prikaži ovu nit -
Also common to just have a dataset dropped on my lap so I can go do the statistics to it. Very hard to coax a research question or even a preferred dependent variable out of a client who doesn't understand why you need such a thing. I have the data don't I? Just run regressions.
Prikaži ovu nit -
Similarly, the (usually) students want robustness checks, but in a very cargo cult way. They see them done elsewhere and they want them too, but they're not sure why or what purpose they serve. Just add all the robustness checks, please.
Prikaži ovu nit -
There's a real desire for statistical analysis to be a completely objective, theory-free, and context-free affair divorced from the reality of where the data came from. And I get where they get that, that's probably what I'd assume statistics was if I didn't know more about it.
Prikaži ovu nit -
I should point out that many of my clients fall into none of these holes and grok this stuff well. And the ones that do have these issues aren't bad or unintelligent. They're just not statisticians or econometricians (if they were they wouldn't need me!)
Prikaži ovu nit -
It's valuable to know what the folk understanding of statistical analysis is. Even if by "folk" I mean people who have enough stats knowledge to hire a consultant. And especially if this is a group of people who are connected, by volume, to a huge chunk of produced analysis.
Prikaži ovu nit -
Whether you're teaching, consulting, or reading the analysis of others, you want to have a good sense of how people think, and the assumptions and attitudes they're coming to the table with. /fin.
Prikaži ovu nit
Kraj razgovora
Novi razgovor -
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.