Sunday, November 29, 2020

blowing holes through the bubble

I just spent a few minutes cruising up and down the Information Superhighway to see if anyone relevant had test-driven my theory on why election polls were not just wrong, but likely to remain wrong for the foreseeable future. The Atlantic came close, identifying "the poll clique - pollsters and analysts alike" in this column posted a month ago, but it's slightly different from what I had in mind; cliques are keenly aware of what outsiders think, using that knowledge to maintain exclusivity, but in this case we know the so-called "poll clique" had very little idea what outsiders thought - that's why the polls were off. The clique idea is a good comparison, but I think there is a better explanation.

Pollsters, I suspect, fit the definition of a group working in a certain kind of bubble, which is a word I'm using outside the commonly invoked ideological sense - I'm thinking about a bubble defined by education. Articles like this FiveThirtyEight piece from September wander toward this idea, noting that white voters without a four-year degree were underrepresented in 2016 polling, but it stops short of tearing down the fourth wall behind which I suspect there is a more universal explanation waiting to be found. There are hints to this reality if you are the type to read into things, such as by using job descriptions as a proxy measure for the type of person who ends up on the payroll; my review of these for roles such as "Politics Reporter" or "Survey Data Analyst" uncovered requirements such as "experience working with data" or "Proficient with Python and/or R and/or Excel and/or SQL", which from my experience as a hiring manager almost always signals (at minimum) a candidate in possession of a four-year degree (1). The descriptions and their requirements are consistent with my understanding that over the past few years polling has shifted toward a data-driven mentality, which suggests its job market favors candidates with skills common to graduates of four-year programs.

I wonder if the type of polling error we've seen over the past few years is a consequence of this shift toward data-driven thinking, its subsequent effect on the type of person employed by the industry, and the inability of those people to relate to anyone dissimilar to them; the problem would be exacerbated by the widely alleged polarization of America, where people are spending more of their time in self-selecting echo chambers and becoming less capable of relating to neighbors holding different views. It would explain an odd claim that the above linked FiveThirtyEight piece calls a "theory that won't die" - Trump supporters are too shy to admit they support him, then gleefully vote for him on Election Day, which throws off the polling numbers. The article explains the lack of evidence for the theory - I gather there was never much evidence - but it stops short of offering an explanation for the persistence of the idea.

One possibility is that the media - which I suspect tends to employ people with four-year degrees, and is often accused of a leftwing bias - has been unwittingly keeping this idea alive because it speaks to a phenomenon they've seen firsthand. My speculation is based in part on my own experience - in 2008, a close friend told me that he felt unwelcome accepting a ride to the polls from a student group on our liberal (arts) campus; he was voting for McCain. I remember my reaction to this day - I was shocked he wasn't voting for Obama! But as I think back, I realize I didn't have the full picture, since I can hardly recall anyone at school supporting McCain, and it just isn't realistic that all of my classmates supported the same candidate (and that's before I get the demographics of the student body); I suppose it wasn't worth the trouble to announce your McCain support in the dining hall. It's a similar situation at my current workplace, where my most vocal colleagues seem to speak on the behalf of a silent majority that supports Biden, and the next word of support for Trump will be the first; we almost all possess, at least, a four-year degree. The only problem of being immersed in these overly liberal environments is if you start to think that it represents "normal", in the sense that it's representative of every American's experience; this is even worse if your job depends on you remaining objective, since you'll just become bad at your job if you think the whole world is like a small liberal arts college.

A shared perspective within teams, it seems, is almost inevitable - it's hard to envision any kind of organization that would have an entirely neutral appeal, attracting job-seekers in equal numbers on both sides of various issues. But homogeneity shouldn't be accepted as an inherent characteristic of large groups, especially if idea generation is important to its success - it doesn't take much for a collection of like-minded people to establish consensus because they will share a certain perspective regarding the way of the world, but consensus has a way of shutting down debate. In the world of polling, this can have ramifications in the results - during those Obama years, for example, pollsters struggled for some time to account for cellphone-only voters; I can only speculate that the decision-makers in the industry were predominantly landline users, and agreed with each other about their methods because they didn't know very many cellphone-only voters (they probably reached this consensus over phone calls made on landlines). The current situation, in some ways, doesn't seem much different - two consecutive national elections have raised questions about the way certain voters are being represented in the polls, and many are left scratching their heads over the matter; I'm wondering if the answer isn't as simple as those voters not being represented in the organizations trying to reach them.

Footnotes

1) I don't mean to come off as cynical with this kind of remark (longtime readers should know by now that if I suspect selfish motives, I'm more than happy to point those out in my writing). What I mean is that posting a job sans the "four-year degree" minimum requirement only means a resume gets through the first round of screening. Inevitably, if the job asks for skills that are strongly correlated with a degree, any applicant without a degree will inevitably end up competing against candidates with a degree. So who's your money on now? It's really just a question of whether you think a hiring manager will take a candidate without a degree over a similar applicant with a degree - speaking from my own experience, I can say that I've never seen anyone make such a decision.