Asleep At the Wheel

If the world was ending in five years, would you write a book?

    This is fundamental question gripping me, but it is not too dissimilar from the fundamental question gripping humanity as a whole. We all know that we are going to die, and we have no idea if there is life after death. As a result, every action we take could very well be meaningless. Yet, we trudge on anyway. We live our lives in denial of this death, and generally we remain ignorant of the pressing existential dread bubbling under the surface of our consciousness.

    I see the potential rise of artificial general intelligence (AGI) as a generic extension of this problem. The inevitability of AGI seems clear, and in my opinion it is likely to arrive within this century. There is quite a bit of controversy over timelines, but it seems most researchers are now coming to a similar conclusion. The arrival of AGI will no doubt fundamentally alter the future of humanity, and it has the potential to be extremely positive or extremely negative. It seems clear that this technological revolution will quickly lead to artificial super intelligence (ASI), a term that is largely undefinable. The amount of control we will have over any sort of advanced artificial intelligence is unknown, and it is very likely that the control problem will be completely insurmountable. AI alignment, the research field aiming to ensure that AI systems are designed in a way that aligns with human goals and values, is extremely immature. I would be hard pressed to find a more important field in the history of human history, especially as humanity enters this critical period of rapidly advancing AI systems.

    The field of AI research has been around since the 1950’s, and AI has been a concern in sci-fi media for just as long. The first “Terminator” film debuted in 1984. The first real AI alignment discussions began in the early 2000’s, but until 2015 the field was not at all taken seriously. I ran into one of the most prominent early AI “doomers” in San Francisco recently, and I realized that if your leader is a fedora wearing neckbeard with a savior complex, people will probably not pay much attention. The lack of professionalism within AI alignment needs to be overcome, and the slow migration of respected professors, Stuart Russell being the most prominent, into the field has done wonders in terms of credibility. Still, the main problem, the complexity of aligning future AGI with human values, is at this moment, completely unsolved. Worse yet, it is somewhat likely that the problem is completely unsolvable.

    I have as strong desire to help, as I agree with Toby Ord that this is likely the most important issue humanity is currently facing. I've thought through starting an AI alignment company and have been pondering how it would be structured. If I go the nonprofit route, I will be beholden to many of the downsides that plague non-profit companies. Talent retention will be difficult, salaries will likely be compressed, and we will be beholden to outside donors for capital. However, structuring as a for-profit may wrongly signal that we are in it “for the money.” My purpose for starting this company would be to push forward the agenda of AI-alignment and make it easier for companies across the globe to ensure high standards of safety. I have no current thoughts about the company structure or potential product lines.

Reasons to start an AI safety company:

1. AGI is likely coming in the next 50 years. Humans evolved intelligence over the course of millions of years via an inefficient, largely random process. Given the current technological trends, humans should, at some point soon, be able to create something smarter than them.

2. AGI is likely to fundamentally change the human experience. The productivity gains alone will cause massive changes, but the inevitable development of ASI will make current life unrecognizable.

3. Morals are learned and not innate. Making an AI better at problem solving, even in a generic “human” sense, will not correspond to the AI learning objective moral values and ethics.

4. Human life is valuable, and the prosperity of the human race should be protected.

Because of the first three reasons reasons, I believe that AI alignment will become the most important field of research in the world in the coming decades. Because of the fourth reason, I should try to make a positive difference in this field. Note: the field of moral philosophy is very messy. As a consequence, I admit that number four is an axiom that I will continue to assume, even though it is unproven.

Asleep at the Wheel:

    I started reading Superintelligence in 2018, and completely agreed with Bostrom at the time. I remember at a job interview that year, I was asked some derivative of "what is something that you would disagree with most people about?" I answered "I am very convinced that artificial intelligence is going to have a massive impact on the world in my lifetime, and it is likely that it will be a negative impact." The book was difficult to get through, so I ended up picking it back up and finishing it in 2020. Since then, I have read a few more books about AI (Human Compatible being the best), but have largely ignored the issue, despite my deep concerns. As with 99% of the human race, I have been asleep at the wheel. 

    Starting this blog is an attempt to change my priorities. If I am forced to think about this issue enough, I believe over time I will discover ways to make a positive impact. If I start an AI alignment company and prevent the paperclip company from doing something stupid, all the better.

1.