Twitter | Search | |
This is the legacy version of twitter.com. We will be shutting it down on 15 December 2020. Please switch to a supported browser or device. You can see a list of supported browsers in our Help Center.
wikipedia brown demands #MoreThanDiversity Jun 6
If you’ve followed me >5 mins you can guess that I diverge ideologically from the framing presented by . But what social scientists have also pointed out, which I’d like to boost, is the disturbing level of what I’d call methodological malfeasance in the paper.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
I’m a qualitative researcher but I did take advanced statistical methods during my doctoral program at Harvard (shoutout Prof Willett!) and after convos w/friends, the report raises an obvious question—why so many cities with many of these policies, but awful police?
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity
I begin with this question because a lot of people in our country receive a poor mathematical education and PoC are often marginalized in these classrooms. many folks have a math "allergy." when they hear something is data-driven, they're like, well, okay.
Reply Retweet Like More
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
what this means is that ostensibly "data-driven" approaches in arenas from education to health care to policing benefit from the discomfort of the people most affected by these policies, many of whom feel like they have to take the "data-driven" claims on faith.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
but my people! we can still ask common sense questions! such as! why are we talking about banning chokeholds when NYPD banned them in 1993 but still used one to kill Eric Garner? why is Chicago's mayor bragging about how many of the policies we've already adopted?
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
even if your math education failed you, I bet your mama still taught you common sense. please feel empowered to use that when you encounter these kinds of reports. does it pass the sniff test? now let me explain what I mean by "methodological malfeasance."
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
based on my own questions, I talked to my husband, who has a PhD in economics and teaches at one of the nation's top policy schools. university rankings are extremely fake but the point is, that boy smart
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
much of what he had to say is explained in this thread. to him, this was the biggest issue. there are MANY of what we call confounding variables, variables that affect both the input and the output. such as how progressive the city is?
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
sorry I didn't write out the whole thread in advance guys. i'm almost done.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
so, how do you figure out the effect of a policy in situations like this, where you're comparing apples and oranges? one example is what they did in this paper.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
the authors wanted to know the impact of Alaska's direct cash transfer program. "You can’t just compare Alaska after the policy was implemented in 1982 to Alaska before that, as there are plenty of other differences between the Alaskan economy in, say, 1981 versus 1983."
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
"there’s no state that’s otherwise identical to Alaska that didn’t enact the policy to use as a baseline."
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
so! "they combine a number of other states whose patterns of employment, part-time work, and related statistics roughly match Alaska’s in the years before the policy was enacted."
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
"None of the states alone is a good comparison, but if you combine them carefully, you can come up with a 'synthetic Alaska' for comparison."
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
they use what's called a "difference-in-differences" approach. what change do we see in Alaska before/after the policy? what change do we see in the "synthetic Alaska" before/after? the gap between THOSE changes--the difference in differences--tells us about the policy's effect.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
now let's go back to the issue. it's not just that there is a big methodological problem w/ the paper. it's that usually before you publish a paper like this or certainly use it as the sole basis of a massive campaign, you get feedback to find problems like this.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
We all write papers that have mistakes in them. The goal of the journal peer review process is to find these issues. But even short of that, we present papers at conferences, share them in working groups and writing groups, and ask colleagues to review them.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
there are a lot of things to hate about academia, but something I love is how important feedback is to academic culture. almost every scholar has multiple formal and informal places where we go for folks to tell us we messed up and how to fix it, from seminars to friend groups.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
was this methodological approach run by other people? i do not know. y'all can ask about that process if you wish. what I can say is that based on my own understanding as well as quant-focused folks i've spoken to, the level of error here indicates that that is very unlikely.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
one other thing. something a lot of folks don't like about academics is that we are prone to hedging. A LOT. we be like "well evidence suggests a likelihood that perhaps under certain circumstances it's possible that the sky might be what's referred to as 'blue'"
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
but we are trained to have a very very high threshold for claiming causality. so even if the paper were ironclad, the language that has been used to present would be considered a significant overstatement of "if we x, then y." that's just a really hard claim to make.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
Is it helpful to have a database that summarizes various departmental policies? in my view, even though we KNOW they don't adhere to them, yes. are my personal politics in line with thinking this reform-based campaign is the way forward? in my view, no. but, as important:
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
Is it malfeasance to have a poor methodological approach in a paper? not at all. but if that paper is going to be the foundation for a massive campaign, the basis upon which you seek funding, press, and policy, and you don't make sure it's sound? that is wrong.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
I already saw one reply that was like "so what if the studies...blah blah." but we are talking about STUDY, singular. 8cantwait website -> "read the use of force study" -> report screenshotted below -> Analysis -> "read the full study" -> one paper, the one i've discussed here.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
I have spent a lot of time stewing over this and debating what to say. I was hesitant to say something because 1) i'm so tired 2) because people are polarized in their views on DeRay this can all spiral into a lot of interpersonal drama stuff that is beside the point.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
ultimately I decided I had to say something because we are in a moment to capture people's imaginations and those moments don't come along all the time and they don't last forever. it's incumbent upon us to be responsible stewards of that moment. and this ain't it.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
factual correction: my highest stats course is NOW called Intermediate and Advanced Statistical Methods for Applied Educational Research but at the time I took it was called Applied Data Analysis. I took it with John B. Willett who was an amazing teacher.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
Sam wrote a thread explaining some of the methods and someone asked my thoughts. this makes me angrier. the paper is from 2016. you know what you do before you launch a site? take the money & goodwill people have for Campaign Zero right now to hire some RAs to google the dates.
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
OR!!!! publish your database, say "look we made a cool database" and leave it at that!! just because what you have in front of you is insufficient to make a causal claim doesn't mean you go ahead and make it and say "sorry it was the best we have."
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
further, there is reference in the thread to the fact that the paper was written "in collaboration with statisticians." well, they are not named as co-authors, nor is there any acknowledgments section on the paper, nor is there one on the "Police Use of Force Policy Analysis."
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
the website said "PROVES" a few days ago. PROVES. so don't tell me "oh we knew it wasn't causal this was the best we could do." no. hire an undergrad to use google for you, or publish the data you have and don't go around telling Rolling Stone what it "proves." if you don't get
Reply Retweet Like
wikipedia brown demands #MoreThanDiversity Jun 6
Replying to @eveewing
Reply Retweet Like