Jaan Tallinn, one of CSER’s three cofounders, has recently given an interview for John Brockman’s Edge.org. In this interview, Jaan discusses the difficulty of thinking clearly about existential risks:
Elon Musk said at his interview at the TED conference a couple of years ago, that there are two kinds of thinking. All of humanity, most of the time, engages in what you call metaphorical thinking, or analog-based thinking. They bring in metaphors from different domains and then apply them to a domain that they want to analyze, which is like things that they do intuitively. It’s quick, cheap, but it’s imprecise. The other kind of thinking is that you reason from first principles. It’s slow, painful, and most people don’t do it, but reasoning from first principles is really the only way we can deal with unforeseen things in a sufficiently rigorous manner. For example, sending a man to the moon, or creating a rocket. If it hasn’t been done before, we can’t just use our knowledge. We can’t just think about “how would I behave if I were a rocket” and then go from there. You have to do the calculations. The thing with existential risks is it’s the same. It’s hard to reason about them, these things that have never happened. But they’re incredibly important, and you have to engage in this slow and laborious process of listening to the arguments and not pattern-matching them to things that you think might be relevant.
In relation to risks from artificial intelligence, which have long been an object of his attention, Jaan draws a conciliatory line with AI researchers, while advocating that more research on safety needs to be done:
More generally, everyone who is on a causal path of new technologies being developed, is in some way responsible for making sure that the new technologies that are brought into existence as a result of their efforts, they are responsible for ensuring that they are beneficial in the long term for humanity.
I would say that I don’t have any favorites, or any particular techniques within the domain of AI that I’m particularly worried about. First of all, I’m much more calm about these things. Perhaps by virtue of just having longer exposure to AI companies and people who develop AI. I know that they are well-meaning and people with good integrity.
Personally, I think the biggest research that we need to advance is how to analyze the consequences of bringing about very competent decision-making systems to always ensure that we have some degree of control over them, and we won’t just end up in a situation where this thing is loose and there’s nothing we can do now.
You can read the full interview, or watch the video here.
