18 tweets · April 2019 · 53 likes · 2 retweets · read on Twitter
@leashless - Just listened to your interview with @dthorson, and... holy shit. It's going to take me awhile to digest the whole thing. Maybe I'll say more later! Right now, just a comment on the Effective Altruism thing, since you said you "look forward to being corrected"
I want to note that I resonate with the core point you were making, both in my personal experience and in other EAs (I've been to 5 yearly summits & have many EA friends). This core point (I gather) being: EA prioritization is biased due to inability to BE WITH suffering.
So my aim here is mostly to give you a bit more context about EA's relationship with AI. One thing to know is that it isn't really a single institution. There's a Center for Effective Altruism which coordinates stuff, but EA is more of a "collection of movements under one banner"
At least one of these movements (from Eliezer) was oriented towards AI before altruism. early-Eliezer thought "make AGI, it'll fix everything" then realized that by default it would be like capitalism×100: a system that systematically optimizes for things-other-than-what-matters.
At that time (mid-00s) there was no global conversation around superintelligence as a risk at all, so he wrote the LessWrong sequences to convince people to care about it. That clearly worked to some degree: there are now dozens of people working fulltime on it, and others aware.
In the interview, Vinay, you posited that nobody in EA was in a position to do anything about AI risk. I'm not sure what you meant by that, but a bunch of EAs got jobs at DeepMind for this purpose, for instance. And the other work happening seems it should have *someone* doing it
Anyway, EA formed from the merger of these people and a few other groups that had come into being around questions like "how can I use my money—whether billions or just <the rest of my paycheck beyond what I need to live>—to best care for the world?"
So AI was part of it from the very beginning, not a distraction that showed up later. Then there definitely have been lots of people within EA who started out oriented towards human or animal suffering, and shifted priorities towards AI risk.
Lots of reasons for this: my own favorite has been that if we were to solve poverty then an unaligned superintelligence were to come into being, it would all be for naught. Meanwhile, an aligned superintelligence could deal with not just poverty but biorisk and nukes as well.
On a totally different level, many people who are working on AI risk are doing so because it seems like exactly the sort of thing that they *could* do something about, inasmuch as there are core mathematical challenges to solve there, and they are math geeks.
As I said, I do think that there's something importantly true about your comment that EAs are inclined towards AI because of not wanting to emotionally deal with suffering. It's easier, it seems, to contemplate "the future exists or not" than feel weight of present collapse.
Ofc, I can't speak for everybody, but this definitely resonates with my own experience of trying to think about this stuff, & my experience of others. And it's not about not thinking about suffering, but about it not being spiritually real to EAs. It's abstract; decoupled: QALYs
Altho I acknowledge this bias, I still think AI risk is very important, which I doubt will shift unless I become certain that collapse will prevent all progress on AI. But I'm starting personal work on collapse & suffering, so other priorities may become more thinkable for me.
Btw, it seems many EAs can't deal with their own suffering either, so they end up counterproductively sacrificing each other's well-being, rather than doing what you describe in the podcast: simultaneously considering people you see (homeless or friends) AND your long-term plan.
There's a growing body of people connected w/ EA who are clear that a different, more spiritual orientation is needed. I find this encouraging! It's not yet clear to what extent any large chunk of the movement will go in that direction, or if it will fork, or form something new.
Here's a talk+transcript if you (or anyone who's reading this) is interested in more context on AI alignment work: intelligence.org/2017/04/12/ens… I deliberately didn't @-mention people here, b/c this thread exists in the context of the interview, which they won't have heard.
On which note: anyone reading this thread who hasn't heard the VG+DT interview, it's here, in 2 parts: 1: anchor.fm/emerge/episode… 2: anchor.fm/emerge/episode… Very intense. I don't recommend listening if you're feeling emotionally unstable or you're heading to work or something.
Addendum, from another thread: