Why are the incentives the way they are? Why value quantity over quality?
Why is he getting criticised? He is absolutely brilliant, one of my favorites. And despite that he is a physicist, he seems to have a wonderful knowledge about most natural sciences and makes awesome videos in completely different topics.
I don't want to derail the thread, so I'll spoiler it:
Talking exclusively for my field (but it is likely similar for many others):
Because getting extremely qualitative research is incredibly hard. Also, because most of the research is done by PhD students, who typically need 3 top tier first author papers to get their PhD. And there are thousands of them.
Thanks. I haven't watched that video, cause usually I tend to watch the science popularizers in fields I am not working on.
Publish or perish. Why I got out of academia. I saw the way my friends and colleagues lived and it never ever ends until you retire or leave.
It sounds like yet another problem which capitalism exacerbates and is unfit to resolve.
This is why business majors shouldn't be put in charge. I'm pretty sure that all academics know that it shouldn't be this way, but at the end of the day money talks.
Part of the problem is that promotion/tenure in academia is usually tied to your publication record so you have little choice. Some Universities are now creating both research and teaching pathways for academics but it is still reasonably rare.
That sounds really stupid.
Yes, it was actually a mistake to climb down from the trees.
I think it is absurd that paper reviewing is not done in a double-blinded system. I am happy that in my field, the top venues* are all double-blinded (the reviewers and area chairs/associate editors don't know the authors and vice versa).
I find the WEIRD issue tends to come up more often. Basically how most research is conducted using white, educated, heterosexual, non-neurodiverse participants, and how those findings are then applied to different groups.
Exactly, it's a big issue - to the point where there is fraud happening just for people to be able to produce papers that will improve their job prospects. This stuff is particularly egrarious:
Full article: https://www.nature.com/articles/d41586-021-03035-y?WT.ec_id=NATURE-20211118
Full article:https://www.nature.com/articles/d41586-021-01759-5
There are so many absurd things with the entire system. I have worked as a researcher in 4 separate countries and the core issues remain the same everywhere. The absolute power that tenured professors have is gonna lead to a massive crisis at the miro and macro-level over the next few decades in scientific research. Thankfully, I start my new job in January, so i am done with this shite.
That's because test subjects are often university students or people who are otherwise more in touch with the researchers conducting the study. That's a reproductivity issue to some extent, as people working with a different demographic might have trouble reproducing the findings even if the original study was well conducted otherwise; but you will never be able to deal with that entirely. I mean, you can assemble the most diverse set of Norwegians possible for your study, but that might still not mean much for people in Chile. You can't get rid of that entirely, you will basically always need complementing studies to see how a certain effect plays out in different parts of the world.
For sure it's a flaw and I suppose any findings should be taken with a pinch of salt as exactitude is always going to be a problem with any study involving people/ human behaviour. In recent years people have often used that as a stick to beat social sciences but if you frame it differently, you're basically reaching a conclusion until a point where a better conclusion can be drawn, which might include using better, more exact methods. That's no issue in my eyes except if the researchers/ organisation has the sense to point out such flaws. Edit - I'm basically outlining why the replication crisis exists in a roundabout way
Actually, I think you're lining out the scientific process - Popper's principle of falsificiation. (If it's impossible to falsify research findings, then it's probably not proper or not properly conducted science.)For sure it's a flaw and I suppose any findings should be taken with a pinch of salt as exactitude is always going to be a problem with any study involving people/ human behaviour. In recent years people have often used that as a stick to beat social sciences but if you frame it differently, you're basically reaching a conclusion until a point where a better conclusion can be drawn, which might include using better, more exact methods. That's no issue in my eyes except if the researchers/ organisation has the sense to point out such flaws. Edit - I'm basically outlining why the replication crisis exists in a roundabout way
Yep agree that's the ideal. Obviously it's dependent on whether they think the study is worth replicating but that might be nit-picking...
Thanks - I'll check it out!Yep agree that's the ideal. Obviously it's dependent on whether they think the study is worth replicating but that might be nit-picking...
Probably is Popper related! The idea came from a podcast series I've been listening to recently and an episode last night was about post-enlightenment thinking, about how the scientific method needs to account for limits of human observation.
https://www.philosophizethis.org/podcast/consequences-of-reason
Not saying you should listen but I really like the show
The problem with replication in (social) psychology is that a lot of the foundational work, then built upon by top researchers in the field over the next several decades are not only non-replicable, but even possibly fraudulent. It’s cast doubt on a lot of current thinking within the field. Some of the most famous experiments (Standford prison experiment being one) performed by some of the most eminent psychologists (Zimbardo, Milgram) are partially, if not completely misleading. These experiments and their implications span multiple chapters in dozens of academic text books. Of course this doesn’t speak to the frequency of non replicable research but it has somewhat set a narrative for social psychology. Perhaps psychologists do revisit previous studies more often but if for example Crick and Watson’s or Sanger’s research on DNA was found to be most probably fraudulent/non-replicable I’d imagine there would be similar uproar within Biology.
here a good blog post about the replicability of the book:
Because everything is weighted towards getting publications, nothing matters if you cant get it published, and the editors of the major journals only care about headline discoveries.
Exactly. Scientists are largely judged, both by their peers and employers, by the prestige of their work (i.e., publications in high-ranking journals that get cited a lot in other articles). Articles that disprove theories or links, or report on not having achieved a result when looking for something, don't get accepted in those journals (except if something major is being disproven), and so for many scientists aren't worth the effort. In fact, if a scientist has finished a project and didn't get what they were hoping, they will usually start searching the data for anything unexpected that might be interesting, rather than reporting on the 'non-result'.
