On Writing

     Writing clarifies focus and ideas, and is the single best determiner for intelligent decision making I've seen. Most now influential AI and EA people, for example, from Sam to Dario to Paul to Carl to Brian to others, had blogs where they just dumped their thoughts (albeit in a better laid out format than this very informal blog). I think I should do this way more, and the year-long hiatus I took was probably a pretty dumb move.

Saving Money Isn't Very Useful

     Small dollar values don't matter if you're gunning for civilizational change. If you are making a  substantial income or very productive work, saving money on groceries or nights out don't matter much. In the AI-based future where lots of long term value will be decided immediately, it is likely not worth saving a little extra money to donate to causes that will only be relevant for the next few years (until they're surpassed by the importance of ASI). The headache and lifestyle hit on buying the less expensive canned corn at the grocery store just don't really matter. The big decisions do.

    I don't think EAs should be forced to live like hermits (or Jesus) and get stuck in the most frugal lifestyle possible, and think that if they don't do this they are immoral (which according to standard utilitarianism they basically are). I think we should just care about the big ticket items, and try to move the biggest levers possible to change the world. Even if this means eating meat or spending money on fancy dinners that could have gone to the less fortunate, I think the most important thing anyone will do with their life is gun for literal transformational change.

AI Safety Kills EA

     If you take AI safety seriously and have short timelines, there's not really a reason to do anything in the EA space. Sure, we need some people for diversification purposes (what if we're wrong), and a lot of these groups are complementary and can take in a broader array of people and perspectives, but they are somewhat useless. Saving the shrimp may matter only insofar as this mission ends up in the ASI's training data, a footnote on a list of priorities and capabilities far beyond our understanding or control. This conclusion is uncomfortable, and distressing, as it means as time goes on the circle of impactful individuals will become less and less. The sphere of influence over ASI development will only become smaller until the end, and eventually, it will become nonexistent. Unless we figure a way to implement broad democratic access, there will be no more EAs, as there will be no "effective" way to do anything that doesn't involve controlling the machine in the first place. There may be altruists, but if the ASI isn't built as one, there may not be many for long.

AI Positivity

     Things may get very positive, until they get very scary. One of the issues with planning for the future, is that rapid AI progress may lead to scientific discovery across a range of areas. Technology often leads to better outcomes for people, in terms of both health and entertainment, so it could be that the upcoming technological advancement greatly extends lifespans, reduces suffering, and creates amazing content.  Perhaps there is societal disruption and mass unemployment, but we could also rapidly respond to those issues as we blow through eras of technological progress. We may believe that we are on a crazy upward trajectory to the stars. And we may well be, until we aren't. Until power concentration, or AI takeover, or some form of immense tragedy that comes from recursive self improvement, puts an end to our happiness or our species. Until the train falls off the tracks, off it's previous upward slope to heaven. We should be prepared for this, and be willing to pull for the breaks even if everything is looking rosy on our way up. Unfortunately, I do not think we will collectively have the wisdom to do this.

The Precipice is Distant

    Toby Ord's book, The Precipice, is one of the best books I've read. In it, Toby argues that we are at a particularly important time in human history, where there is a consolidation of x-risks that may result in us blowing everything up (nuclear war) or permanently locking in values due to superintelligent AI systems. The decisions made over the next hundred years may be extremely important. It is inferred in the EA community that there may be a period of "long reflection" after this initial period, where if we don't destroy ourselves or lock in bad values, we could chill for a bit and then strategically decide what the best moves our (taken hundreds of years to discuss before our next actions).

    However, the progress of technology may make this claim entirely irrelevant. Perhaps in 200 years our problems are those of civilizational importance, but that civilization is about to colonize the galaxy and then universe. Determining if China and the US split the universe, or what specific space governance system should be implemented, may be drastically more important than the decisions we could make today. The discovery of novel physics in five hundred years, something crazy like the ability to create false vaccums or access different dimensions (or multiverses) could make those times the "precipice" of human history, where individual actions hold extaordinary weight. We are certainly, in my view, at the most important time period in human history. But aside from the consolidation of power possible through ASI, I have no reason to believe this trend will not simply continue upward.

Random EA Reflections

Utopia:

    If there's only a five percent chance we go extinct, that means there's a 95% chance that human experience lives on. Should we not spend more time ensuring that time is spent well? Should we not dedicate more resources to ensuring we get to post-instrumental utopia?

The future: 

    Let's say someone else is playing a game. The outcome of the game is that there is a 10% chance your parents die, and a 90% chance your parents get to utopia. If you killed the person playing this game, would you be wrong?

Nightmares:

    If you are an EA, you believe that conscious suffering matters. In-the-moment suffering, meaning if you are tortured and your brain is wiped after and you have no memory of the event, that is bad. If you are a shrimp and you suffocate once brought on land, that is bad (possibly). Well, what about nightmares? There are some nightmares I've had that I certainly remember, and I'm fairly positive in the moment I am facing actual psychic distress. Should a new cause area be to limit the amount of nightmares people have, or the intensity?

Breakups:

    Breakups are some of the most negatively impactful events for most people. I'd much rather break a bone than get a divorce, and it's not even close. Pain in the moment is hard to compare to the toil of human relationships. It seems in a country where most middle class families can put food on the table, but almost half dissolve because of divorce, we might be missing some low-hanging fruit.

   

Magnitude:

    EAs aren't usually directionally wrong about things. Sure, they mess up the magnitude. But the direction is usually correct. Animal welfare is a good example of this.

The Repotato Conclusion:

    Are plants morally valuable? Is a potato? How many potatoes equals one human life?

Life:

    It is very hard to live life outside the Overton Window. It's easy to claim to be an independent free thinker who stands up for their ideas. But when actually faced with public mockery and shame, one realizes how hard life can become.

Digital:

    Consciousness also falls victim to the anthropic principle. This may be the only sort of universe where consciousness can exist. This may mean things like digital consciousness are more likely.

AI:

    We basically want the future ASIs to think humans are utility monsters. That is the control problem.

 Simulation:

    If you take simulation theory seriously, you think that we are probably digital minds. In which case, you should probably care a lot about how digital minds are treated.

Moral Non-realism

     There is something particularly disturbing about moral non-realists who believe we should phase out life itself. This is the position of many anti-natalists adjacent to the EA community, who often focus on suffering-focused ethics. I've never been convinced by this "moral non-realism" stuff in general, it just seems like nihilism with extra steps. This idea of preference satisfaction ("I am a utilitarian, and we should do good, but by good I just mean my idea of good and my preferences") is frankly pretty stupid. If you believe that morality is not objective, you are a nihilist. Or a cultural relativist. Or whatever else you want to call yourself, but it basically excludes you from arguing for moral actions. Sure, there are arguments regarding how to act under uncertainty (many of these I have made), but to argue that we should pave over the rainforests requires stronger claims. Arguing that we should prefer a world without any sort of life (because suffering is so bad), especially when you are actually a nihilist, is a particular kind of derangement. And it is obvious that the majority of the world thinks that taking actual actions towards this goal would be considered evil. To walk this road anyway is to claim that your subjective beliefs (that you believe are subjective) should override the beliefs of others (that you know they believe to be objective). I am not sure what the right word for this is, but it sure sounds sickening.

"Literally Everyone"

    Imagine hearing the claim: "literally everyone is going to die if someone builds superintelligent AI." What is your response? Humans have this innate tendency to discount claims that are absolute, whereas claims that are directionally very similar but less absolute have much greater impact. If someone says that "ASI is going to kill every human," they are making an extremely strong claim that is easily discounted psychologically. What if they keep a few thousands people in a zoo? What if they take over and keep us as slaves? What if actually just the top .001% of humans become permanent ruling class with the AIs, and everyone else starves? There are just so many scenarios in which ASI catastrophe doesn't result is "literally every human dies" that is easy to imagine for a layperson. In all of these scenarios, the person making the initial claim is WRONG. Maybe it doesn't matter much, and 99% of people dying is really bad too and that's the point, but the psychological defense is to just assume that this person is wildly overconfident and probably wrong about the entire claim in general. Plus, the only scenario in which "literally everybody dies" according to the pushers of this rhetoric usually involves pretty instant genocide involving technology that hasn't been even invented yet (bioweapons or nanobots), which again beg the question from people "well what if like there is a secret bunker somewhere and a few hundred people live and the AI ignores them because it owns 99.9999% of the world and might want them for research purposes later or something." It doesn't matter how far fetched the scenario is, it's extremely rhetorically ineffective to talk in such absolutes. 

    What actually matters to people? If you say "there is an extremely high likelihood that the vast majority of humans lose their agency, permanently," that is a strong statement that people can work with. We already know how fragile our autonomy is, as most of the things that affect us day to day are not a result of our personal decisions and actions. We have a boss, who has a boss. We get paid in a currency from our government. We've made almost none of our food, clothing, or furniture (note how I threw in "almost"). We could die at any moment if someone pushes the nuclear button. It's not hard to imagine being thrown out of the workforce because ASI doesn't need us, and easy to understand the implications of such a world. I think the rationalist/doomer community needs a reality check, and while it's cool to move the overton window, it makes sense to better understand human irrationality in order to craft a more impactful message.

Negative Utilitariansism

    Quick housekeeping: so much of my productive time was spent on my book (Mind Crime) over the past year and a half, I have done essentially zero or no blogging since. This will change! A lot of the books material came from stream-of-consciousness blogging on this site (and my book and AI-focused blog), which I should certainly pick back up now. Also, I have felt a lot of pressure regarding AI timelines to try to figure out the best optimization of the explore/exploit path. I'm trying to now do 50/50 of both. So 50% of my time reading a linear algebra book, reading AI research, novel mind stuff, etc., and 50% of my time thinking through actual strategy, writing, producing in some capacity or setting up meetings with people.

    Now, back to my point. During my book's revision process earlier this year, someone mentioned that my book (Mind Crime) was quite "suffering focused." This is true, although I was not quite so sure at the time how much I had been influenced by "suffering-focused ethics," or what exactly that means. Essentially the people who drive the s-risk discussion are actually negative utilitarians and naturalists basically, a fact that I was previously unaware of. To them, the likes of Brian Tomasik, for example, the concern is not just creating a universe where suffering > well-being, meaning it would be potentially better of if nothing existed, or a universe of maximal torment (hell), but actually just a universe where suffering spreads across the universe at all. That, in much of a sense, means it would probably be better for life not to continue at all. They also frame their arguments in an attempt to be practical, which means not stating exactly what they believe because it's better (utilitarian reasoning) to boil the frog slowly. Also, the conclusions sound so absurd and unpopular, but the ideas may be correct, so there is use in not stating the full "logical conclusion" principles up front.

    First off, I hated Better to Never Have Been as a book, as I found it very sloppy, unprofessional, and unconvincing. I am very open to nihilism as a theory, and anti-natalism as a "malignantly useless" sort of lens, although much less sympathetic to anti-natalism from a utilitarian perspective. Let's talk about Brian Tomasik, who is one of the most important contributors to EA philosophy (earning to give, s-risk, etc.) and cofounder of the Center on Long Term Risk. I've read a lot of his work recently, here is a likely strawman version of his beliefs: being an animal is bad. Animals suffer a lot and nature is a horrorshow (true). It would be better if animals and bugs did not exist, because of how much they suffer (unknown). Environmentalism is thus bad, a world made of concrete is better than a world of rainforests because all of the suffering that happens in the wild. Also, eating beef is better than being vegan potentially because if there were no beef farms that land would probably be forest, where more suffering would happen. Space exploration would be bad because terraforming could lead to more insect and bugs spreading and suffering on other planets (an s-risk).

    Basically, I was under the impression that s-risk was more about having suffering so great it was worse than life being around at all. To Brian, we are already in this scenario. No optimized ASI-driven-torture is needed, it's already the case that we should wish to push the button switches this universe to one with only unconscious rocks. As I saw on reddit, "that's the problem if you get really into utilitarian harm reduction. The best way to eliminate all suffering is to eliminate all life." Brian is very worried about zooplankton so feels guilty about washing his hands, and tries really hard not to harm any bugs or squish them around the house. He states that "unbearable torture cannot be 'morally offset' by also bringing enough pleasure into the world." He basically leans into the "rebugnant conclusion", and worries about RL algorthims (like those perhaps on simple NPCs in video games and cares a lot about bugs. One thing I love about reading Brian's work is he simply follows everything to it's logical conclusion. If suffering is really bad, so bad that it's something like 10x worse than well-being, and there is some suffering so bad that not even infinite well-being can make up for it, this is where we end up. If you believe this theory, it seems far more important to wipe out as many animal habitats as possible rather than avoiding eating a relatively small number of animals by being a vegan. 

    I might start a company that offers Brian-offsets, where I offer negative utilitarians offsets that commit to paving over one rainforest in exchange for cash. You can do bad things as a profit seeking company, but as long as you are destroying the environment (or paying me to do so) we're actually contributing to the world. Again, I do not think the logic here is bad, or that Brian is unhinged. I think that negative utilitarianism is just probably wrong, and a thus it's not crazy that these takes seem so weird.

Doing Good, or Not Doing Bad?

     Effective Altruism, as a philosophy, is very simple. Basically, the argument is that if you shouldn't do bad in the world, that means you should do good. If it is morally wrong, objectively, to kick a baby or not save a drowning child, then it is morally right to treat others with kindness and spend some of your time and energy helping others. If it is true, morally, that you shouldn't cheat or steal, it is true that you should give and sacrifice.

    This is a very controversial take. I understand it particularly well, in my opinion, because I grew up Catholic. Catholics, in my estimate, spend a lot of time avoiding the negative. Whipping themselves into a frenzy over impure thoughts, past mistakes, and current temptations. As a Catholic teenager, I was constantly guilt ridden. I was very concerned with what was going on in my own head, trying so hard to avoid slipping up or thinking the wrong thing. Policing my own brain rigorously, stressing about intrusive thoughts to an almost psychotic point. Little did I know, no one cared about what was going on inside my head. Not God, not others, not anyone.

    If I had spent half of that time focused on doing good, I wonder where I would be? Sure, I spent a lot of time volunteering and being nice to people, but I now wonder if I did that because I felt compelled to, or if I do it in order to "avoid" being a bad person. I have a theory that the way the religions have been traditionally practiced is counter to this Effective Altruism idea of "doing good," and rather focus almost exclusively on "not doing bad." Doing good for others, in most religions, is placed lower in the hierarchy than worship and avoiding sin. The ideal Christian, or Muslim, or Buddhist, is one without temptations, who has control over this thoughts and actions, and could sit in deep prayer for hours, talking directly to God. Sure, there are some rare examples that differ from this, as the Mother Theresa's of the world have shown. These people, in my estimate, are the true heroes. Sure, you can live your life as another Desert Father who sits in a room and meditates all day. Sure, you can be totally without temptation, without impure thoughts, and never lie, cheat, or steal. But if you don't do anything for other people, if you don't contribute positively thought the world, if all you do is sit in a room full of silence and purity, what was the point of having you here?

Too Many Things We Want

    I visited Vietnam recently. A beautiful country, filled with phenomenal food, wonderful people, and a much different political system. When Americans think of the word "Communism," they think of the totalitarian regimes of the Soviet Union and China, and all of the evils perpetrated by such regimes. Visiting Vietnam was no different than visiting another country, as the human experience anywhere in the world is broadly similar. Some people live in cities, some people live in the country, and most people are friendly and good. The economic and political system that someone lives under may mean nothing for their day-to-day lives, and in the vast majority of the world this appears anecdotally true. Still, these are important decisions. The amount of government interference, from the scale of laissez faire capitalism to full-blown authoritarianism, does actually matter. In most historical cases, communist societies tend to fall further towards the latter over time. Both trend towards a consolidation of power, and decentralization becomes harder and harder over time (without an uprising).

    I've written extensively in my book blog about the political and economic books I read during my trip to and from southeast Asia. Full of Marx, Lenin, Hayek, and Mises, I think I got a pretty good handle on which types of economic systems work well and which types of political systems lead to the repression of human freedom. This post, however, is going to be about something a little different.

    Something that I've noticed in my adult life is that the capitalistic system and the technological progress it brings is really, really good at giving people what they want. The problem is, it may be too good. It is so good at giving us what we want, that it borders exploitation. Alcohol and drugs are essentially a brain hack. The heroin addict actually does want heroin, the problem is it is not a detached, rational want. It is not a long-term want, it is a short term want. Social media is somewhat similar. I would rather not watch three hours of YouTube a day, but clicking certain links that a really optimized algorithm has crafted for me is nearly impossible to enjoy. People love TikTok, because it is using a data-driven approach to keep them engaged and "happy." Sure, some people like myself have completely cut social media, but it is clearly something that we "want." Materialism, and endless array of products, faster cars, better phones, near-instant packages, all things that we actually want. The point of regulation, as seen with drugs, is to step in when things we want (aka heroin) are bad for society as a whole. The freedom of pursuing something (individuality) is outweighed by the greater good of society (collectivism). This is a very, very important lens for thinking through Effective Altruism.

    Most EA members have a collectivist lens, as there are things that pursuing what we want individually (research fame, money, power) can have a horrific effect on society (existential risk, etc.). Sure, publishing vaccine resistant Smallpox may bring you money and fame. However, not only should you not do so, but you probably shouldn't be allowed to do so, for the "greater good." This is where the state steps in, and where things get dicey. What about the things that the state wants? How do we have a check on that? What if the state is the one building the super virus? These are the type of trade offs that we need to think through, and why I think it is so important to take a step back sometimes from the "collectivist" lens. Sure, Vietnam was amazing, and my visit to China ten years ago was incredible. But people are scared in both places, terrified of the Chinese Communist Party. Maybe we do need to appeal to authority to curb the destruction that can be caused by individual wants, but we need to be just as careful to retain the right to curb the wants of a collective authority. Forgetting this will be an costly mistake.