No that's not how the world works. Most of climate change happens not because some decrepit old man lives on his yacht but poor people in third world countries get their first air conditioners. As they should.
The only way is through. Most of are not going to be living the suffocating life you think we should. We will grow, innovate and accelerate out of this.
> Most of climate change happens not because some decrepit old man lives on his yacht but poor people in third world countries get their first air conditioners.
You're confusing acceleration with velocity. Poor people are far from the greatest per capita contributors.
If everyone had devices that could do some compute worth paying for, people would be doing the work all of the time. The problem is actually useful POW is a lot more expensive compute-wise, making it non viable for normal users
I think in the future we'll have robot pets and look back in horror at how we enslaved animals out of their natural habitats and normalized (even glorified) it.
What about the animals that have been domesticated to be pets? It's arguable that their native habitat is the home at this point. Although, I do hope we stop capturing the animals which aren't domesticated (most "exotic" pets).
I agree with the horror of pulling animals out of their natural habitat, but if you are using this as a blanket statement towards all living animals then you should read up on the history of the domestication of wolves (dogs). It was born of a beneficial relationship to both humans and wolves and i personally find my relationship with my dogs to be extremely fulfilling and i believe my dogs do as well.
I have no interest in a relationship with a robot.
I know the history. We bred them to be our pets and normalized this relationship. Your pet might even love his life. But we can do the same with robot AIs, and it will be a better relationship. Your robot can talk to you. It can call 911 if you’re in trouble instead of just barking.
There are many many people for whom that will never be close to the same, let alone better. An immortal sack of wires that is passed down through generations is interesting and will appeal to many, but it’s not automatically a “better” relationship than a living creature.
> Your robot can talk to you.
If you want a conversation, may I suggest another person instead of a pet?
> It can call 911 if you’re in trouble instead of just barking.
If you want a way to automatically call 911, how about a smartwatch that tracks your vitals and does just that?
Anyway, stories of pets calling 911 are a dime a dozen.
Where would the dogs go? They don't have their own ecological niche and usually can't survive in the wild. In cases when they actually can survive in the wild, stray dogs are known to wreak absolute havoc on the natural native ecosystem. Total dog extermination?
I'd argue that emotional co-regulation is born out of biological origins. Unless we are making synth, it would be a long time to match the relationship with a living creature.
> I have no interest in a relationship with a robot.
I think I could, if it meant I could keep the experience of having a pet.
I've had cats going back to my twenties. I dearly love my fifteen y/o cat. She's my last cat, though. As I approach fifty I've grown tired of hefting bags of litter and food, stooping to clean up after her, taking her to the vet, and administering her medication.
When she's finally dead (which is hopefully not for a good long time) I'm not going to get another. This makes me really sad. I'll miss having a cat around. As I age, though, I don't want the encumbrance of taking care of a pet, nor do I want to cause hardship to family if I suddenly become unable to care for a pet (dying, being hospitalized, living arrangement changes and owning a pet isn't allowed, etc).
If there was a robot alternative I'd jump on it. I think my brain would adjust fine to a convincingly-executed robotic cat. Our brains instinctively ascribe agency to objects moving in non-inertial frames, for example. Think about all the people who name their Roomba, install googly eyes on it, etc. People easily accept inanimate objects that behave like living things, to the point of developing feelings for them.
When my cat is gone I hope to get the opportunity to choose a "relationship" with a convincing robotic cat versus not having one at all.
I wouldn't expect everyone to feel this way. I certainly don't want to legislate away pet ownership.
I bet that, if good robotic pets become available, we'll see a significant number of people favoring them over living pets for convenience, long-term cost, freedom from anxiety about the pet's health and eventual death, and the opportunity for companionship in living arrangements where pets aren't allowed.
I've said, for years, that I think there's hundreds of billions of dollars for the company who can make really good robotic pets. If somebody can get the "soft robotics" down and build / train good animal behavior models I think they'll be able to print money.
Yes, and robot pets aren’t going to look like clankers. They’ll be warm and furry and imitate the sounds and movements of real pets. I think people still imagine robots as some cold metallic insert-Stephen-Hawking-voice thing, and it’s pretty clear even with today’s tech that’s just not what they’re going to be.
Interesting how all of the replies so far to your comment assume we’d just release the existing animals into the wild instead of just not breeding more.
As one might guess, the "just" there is doing a lot of work.
Common pets reproduce just fine without our intervention - in practice it requires costly maintenance programs just to keep their population under control (not to mention the massive and ongoing eugenics program required to produce only the desirable breeds/attributes).
If your neighbourhood isn't overrun with stray cats and dogs, that's because someone is funding a spay/neuter program, and someone is funding animal control to round up (and ultimately euthanise) the rest.
All animal husbandry systems have to be maintained via regular culling, and "eliminating" pets would require orders of magnitude more killing
I always assumed the "enslaved animal-free" future would be something that came on gradually, likely motivated by economics (i.e. lab-grown meat becoming cheaper and higher quality than farmed, robotic pets being cheaper and more "trouble-free", etc).
I never felt like anybody seriously thought we'd release domestic animals into the wild to fend for themselves-- that's ludicrous.
> look back in horror at how we enslaved animals out of their natural habitats
Somehow, vegan-inclined folks never seem to factor in the mass-genocide of domestic breeds that is necessary to achieve their particular utopia. We're talking 1.5 billion pets, 5 billion livestock, and 28 billion poultry - none of which are remotely able to survive on their own in the wild.
Wait... do you really think that if everyone goes vegan these animals will get killed, but if everyone eats them they won't?
You are aware that all of these livestock animals didn't exist a few months to years ago? And that we go through these animals every couple months-years?
Everyone going vegan would just mean we'd stop repleneshing them.
Keep in mind that we already have to perform widespread culling of wild animals in this category. Having eliminated most of the apex predators, we're on the hook to keep populations of wild horses/cattle/deer/etc in check.
If we cease to use domestic animals, they end up in the same boat - we're no longer paying to feed and house them, and if they are capable of breeding in the wild, they will breed unchecked. Leading us to cull them, ad infinitum
The "genocide" that you describe is happening constantly. None of the chicken we have today will be alive in a couple months. We kill them to eat them.
If we cease to "use" them, we just keep that "genocide" going for a year, but we just stop breeding new ones. The result is no more animals after a few months/years. And you didn't kill any more than you would have anyways.
If you have a sink that is half full because the inflowing water and the outflowing water are in quilibrium, you drain it by turning off the tap (breeding).
> we just keep that "genocide" going for a year, but we just stop breeding new ones
"Carry on for a year and then it's all good" is not one of the available choices.
Your choices are (a) wipe out the species entirely, or (b) allow them to continue breeding on their own, and commit to culling them for the rest of eternity.
I detest the vegan movement. I'm not an animal lover. Pets are unsanitary nuisances. Robots are better. We don't need to commit a genocide against domestic pets for the transition. Just breed less.
This isn't great but a negative side effect of an otherwise positive change. I know a lot of comments will try to make this into a gotcha but its really not. Decreased friction leads to misuse. Just like before we'll figure out ways (reputation, light filter pass) to deal with it.
Tech people kind of do, we just dont notice it until trying to describe things to others.
I had some YouTube video on aanf they were benchmarking something, my wife was like "what does any of that mean?"
I realised that, yeah that last sentence was basically just nothing but abbreviations and acronyms. I understood it perfectly but from the outside it is total nonsense.
This isn't just a tech thing, this goes for almost all specialtys.
I love the internet. Its way more interesting than the world outside. Its not cool to say this yet people vote with their attention everyday. What's missing is a tech problem (touch, smell etc). Full matrix style immersion will solve that.
Would be an interesting world where we all live in chambers connected to the cloud. Would take care of overcrowded city problems with everyone trying to move to one place and driving up the cost of living. You can live on a farm in Ohio and have the New York experience.
You could be born in a poor country who's values you do not align with and become a citizen of one (hosted on a discord server) that feels like home.
Tech is beautiful. The ugly parts are just a midwit problem you falsely ascribe to runaway capitalism, other bogeymen, ludditism, a false shiny view of the past and projection of whatever thing isn't quite working out for you (your job, your relationship etc).
The irony is the best place to complain about it is on the internet.
Ya'll are not ready for AI to just keep getting better and better. The slop-midwit-doomerism you are being fed as part of some last ditch copium that your SE identity matters and is going to be needed prevents you from seeing what is inevitable. You can't beat em. Try joining em.
The month to month changes and improvements across free tools is noticeable. A number of skills SE and other roles are built on (or people started with) like more simple scripting and automation work are reachable by a BA with proper requirements, some background with development/coding, and access to AI tools.
It's a continuation of both low code development tools and the ongoing compaction of development groups through DevOps. AI is enabling (vibe) code development by non devs. It's enabling a BA or even direct business rep to take requirements into a prototype or example.
LLMs are out of the box and will continue developing. How/If you use them is up to you.
This is actually a very hard question to answer. If you’re truly AGI-pilled and believe in this progress continuing, then you basically have to contend with a scenario where AI is better than you at doing anything by an order of magnitude. And doing things suboptimally just for the sake of some notion of human independence doesn’t feel right to me.
Perhaps the only way forward will be if we figure out how to merge with the AIs so we can keep up. Otherwise, a soma-filled world likely awaits. And unlike Brave New World, I think it might actually be a lot more pleasant, but still one with a different set of tradeoffs.
yup - its the j curve of productivity phenomenon in action. and we'll see smaller companies reaping benefits faster becasue they can start with AI native development.
I just don't understand why people are incapable of extrapolating. AI can't do this blah blah, my brother in spaghetti monster, have you seen the curve? ChatGPT launched in November, 2022. Opus in December last year. Do you see where this is going?
AI may be a tool and add 50 IQ points to a 100 IQ person today but what happens when its adding 10000 IQ points. The 100 base doesn't quite matter as much does it? What does that world look like?
No, it's not just next word prediction. No you can't just wave your hand that it's probabilistic, so insert bad analogy to compiler or calculator and go about your day. And no, you can't just look at its limitations today, make a conclusion, and go back under your rock.
> ChatGPT launched in November, 2022. Opus in December last year. Do you see where this is going?
Maybe this is tongue-in-cheek, but in case it's not.
No, nobody does. Consider that a person in their lifetime could have seen the Wright Brothers first flight in 1903, then the first jet engine in 1939, then commercial jet travel in the 1950s. That is an amazing 50 years for air travel. If you were living in 1950 and were to extrapolate where air travel would be by 2026, you would think we would be taking routine trips to Mars. Instead what we got was TSA, cramped seats, and better safety. But "progress" leveled off dramatically. Commercial air flight has looked pretty much the same for my entire life.
We have no idea how far transformers and LLMs will take us. It certainly isn't obvious where this is going.
I think it's credible to believe that that a system which is able to meaningfully improve it's own intelligence will lead to a far weirder world than the one imagined by the article.
The parent comment could be interpreted as frustration from a lack of imagination about how weird the future could be.
Perhaps we're not at the recursive self improvement yet, but it seems increasingly naive to believe that it isn't possible.
Of course you might be right. But if we look at our past ability to predict the future by extrapolating from the present and recent past, you might also say it is naive to think that _this time_ is somehow different, that this time the future is clear.
Also, I think you've set up a straw man. I haven't seen anyone arguing that this future isn't possible. What I see is resistance to the idea that we can have any certainty about the future by drawing a straight line from the past.
I agree with you that there is no certainty, and in fact the future will almost certainly not look like anything we imagine. I think that's what makes it exciting.
We think about starships and humans mining Mars, and it'll probably be nothing like that. It might just be humanoids or AI drones that actually end up colonizing the solar system, with the humans never really leaving earth except as mind uploads...
But at the end of the day, things are changing. Things are accelerating. And our mental models of what works or doesn't work will almost certainly break. I don't fear the future, nor do I project any confidence on what that's going to look like, but I know I want to be part of it.
> What I see is resistance to the idea that we can have any certainty about the future by drawing a straight line from the past.
I'm arguing that if recursive self improvement happens, that the trend line will be reasonably predictable. (and that we are close enough to RSI that we should take this possibility seriously).
> I just don't understand why people are incapable of extrapolating.
I guess it's a matter of education. People with a mathematics or computer science background know that unless the dynamics of a process are known to be linear, extrapolation is usually wrong. Since we don't know that the dynamics here are linear and have every reason to believe they aren't, extrpolation is unlikely to teach us anything valuable.
What reason? To me recursive self improvement seems credible, it's just a question of when. It seems obvious that given RSI, trend lines will be at least linear.
The disagreement could just be about if RSI is currently meaningfully happening, but that seems very difficult to tell given the lack of public data.
Because most systems in nature and society, including technological progress, aren't linear.
It's certainly plausible that improvements will continue, but the pace is completely unpredictable. It's also plausible that the training material is polluted and progress will not continue. I'm just saying that predicting the rate of technological progress is not easy, and historically, it's rarely been smooth in the long run.
There are many complicating technical factors, but also non-technical ones. Technological improvement in the short term will not necessarily yield commensurate economic gains, in which case, investment may not grow enough to sustain the progress.
As for the recursive part, currently it's either hypothetical or based on too few data points. Not saying it won't happen, but it's far from being the only plausible trajectory.
> I suspect we just have different beliefs about how close we are to RSI.
I don't have any belief on the matter, but my scepticism isn't necessarily about RSI itself, but about how much it would matter even if it does happen soon. Too many things are limited by lack of resources that no brain in a jar can obtain. And if such a brain in a jar itself is very expensive to operate, it may not be easy for it to justify its existence. My point is that the technical aspect is uncertain, but it is also only a part of a larger system that's has many sources of uncertainty.
This makes sense. Interacting with the world is hard and poses real bottlenecks.
I think that resource acquisition is a solvable problem for such a system, robots seem mainly limited by software.
I concede that I am too certain about the linearity of it. But like, cmon man, in the limit entropy ensures that everything ends. The world could still get very weird very fast. I think that's what original comment was trying to argue, and I think it's possible within the limits you've laid out.
That's all I'm trying to argue at least. And I admit that waving my hand at linearity sounds uneducated.
First, I don't think so. Second, some resources would take even robots decades or centuries to collect. Even something as fundamental as energy production takes a lot of time and money to build. The question of where we'll get the energy to run the robots is not a simple one, and it's one of the simplest questions involved.
> I think that's what original comment was trying to argue, and I think it's possible within the limits you've laid out.
Possible isn't the same as likely, and the reason we don't extrapolate is that different extrapolations lead to very different results.
But regardless of what happens and when, I think that we, people educated in computer science, should remember that many questions are simply not answerable in a short amount of time, and we know with absolute certainty that answering them is not a question of intelligence but of computational power and time. We no that no human or machine, however intelligent, can predict or control the nonlinear systems that are all around us because they are computationally intractable.
no because we have no way to understand the full consequence chain of any stated goal. We see this time and time again in all complex system goals; unintended consequences.
So RSI toward what prompted goal? Then, how can you be so sure that reaching that goal looks like the straight line you envision?
edit: more accurately, it's about incentives. We generally understand that encoding a goal in a complex system is about aligning incentives. So now it's the incentives that are proxies for intended outcomes. But as mentioned, we have a growing body of evidence for just how bad we are at aligning incentives with intended outcomes. And it makes sense because complex systems are not a straight line.
Is a coherent, human-interpretable, goal necessary for recursive improvement?
> how can you be so sure that reaching that goal looks like the straight line you envision?
Because the thing pursuing the goal would be able to improve it's ability to pursue the goal. Doesn't that follow from the premise of RSI?
> we have a growing body of evidence for just how bad we are at aligning incentives with intended outcomes.
We are terrible at it! I don't think that will stop us letting something rip in some unknowable direction (with unintended outcomes). I hope that aligned incentives are a prerequisite, but it seems increasingly likely that they are not.
> Because the thing pursuing the goal would be able to improve it's ability to pursue the goal
But not necessarily without needing a huge amount of resources. It is a mathematical certainty that no intelligence can solve computationally intractable problems (including forecasting the weather, or the economy) without access to resources we simply don't have.
We comprehend, but these "scaling laws" have been in effect for less than a decade (and they're more historical observations over a very short period than actual laws), and while some technologies progress exponentially for some amount of time, the complexity of some computational problems grows exponentially forever. For example, if computational resources double every year, it may still be five centuries before some computational problems can become practically computable. It is mathematically proven that no amount of intelligence can compensate for resources, and even if exponential growth could be sustained for a long time, and that's a big if, an exponential curve still grows slowly at the beginning.
For example, suppose AI helps us figure out a way to exploit much more of the sun's energy. Accomplishing that necessary preliminary can, on its own, take many decades, and if we split our efforts among multiple approaches, it may take longer. Intelligence can't break the actual laws of mathematics or of physics. And that's before we consider things like how resources will be allocated when AI tells people that man-made climate change and transgenderism are real.
This generation of AI hasn't even hit its first crisis. Assuming there will be none is like settlers assuming their town will never suffer a major earthquake because they haven't had one in five years.
Of course, improvements in problems that may not grow exponentially also matter a great deal, but there are too many unknowns (look at how many unknowns there are around quantum computing).
but you’re just on the other end doing the same thing.
one end is luddites nothing to see here, and the other end is omg humanity is cooked because programming is solved! and consciousness is coding so -> gg
yes there’s a wide surface of emergent behavior we haven’t fully explored, but why do you get the pass to draw a straight line to a conclusion? the naysayers are doing the same thing.
At least I have trends backing up my claim. What precedent is there for saying, "Yeah, AI has kept getting better, and keeps doing things that we thought it would never be able to do, but now I think it’s going to plateau because I just can’t see the model writing good poetry or being smart enough to make determinsitic tool calls" or whatever?
Most people are not against AI getting better per se. They’re about AI boosters not able to prove their outlandish claims.
We’ve seen a lot of claims over the years substituting one metric (amount of code written) for another (useful code produced) to justify that AI are getting “better”. We just want a proper definition of better.
No, you don't have trends backing your claims that we are going to add 10,000 IQ points to the average person. Anymore than the S&P 500 is going to 100x in a decade years.
The line is not just AI. It’s our whole story. You can extrapolate it to Moore’s law, automation, agriculture, and so on. And there's no decay. It's just perception.
We are already in the singularity. We’ve always been. It just gets faster at each step.
I've read Sapiens. And I liked it. It's very insightful. But that perception is a perception so I don't understand the distinction you're making.
Human advancement is a story we tell ourselves. Agree. That it in any way looks like a straight line is just not true. If nothing else, it looks coherent after the fact... because it's an extremely lossy story our limited brains can comprehend. But it's not reality. No entity in the known Universe "comprehends" the march of progress. They weren't there when it started. So I just don't know what we're talking about anymore.
We still have far to go. You can tell by the pace if improvement in so many dimensions. If the popcorn's popping had started slowing down, I might believe you, but instead it's speeding up.
I encourage you to do research on extrapolation as a general concept. Not a deep dive, but to explore its general utility and limitations as a tool. I also encourage you to explore what metrics you use when evaluating the utility and impacts of LLMs.
Extrapolation leaves you in a world with no agency. Okay, so the models become superintelligent, you're kind of fucked at that point and there's no value you as a human can add so that reality isn't really productive to think about.
The only thing we can do is look at the current state of things, and we still have the same problems we've always had with these models, they need proper direction and input.
> [...] we are living in a society [that is out] virtually to satisfy and gratify each and every human need, except for one need, the most basic and fundamental need operant in man, the need for meaning.
I reject the premise that we need to go back to some old ways of living to find meaning. My life is full of meaning, but it would probably look like meaningless abundance to someone who was toiling on a farm 18 hours a day or being chased by saber-toothed tigers.
The next paradigm will be similar. It might seem meaningless to us. For example, most of our identity is tied to our jobs in our current society. This will have to change. What does that look like? Do we create content and share our art? Do followers and attention become the new currency?
I don’t think meaning is as malleable as implied. Yes, meaning is entirely subjective, but it’s not boundlessly so. The reason history is a guide is because the needs were more directly connected to the whole maslov’s hierarchy thing. Survival is more justifiably meaningful than “being famous”. I don’t think seeking attention is at all justifiably meaningful in regards to this proposed future attention economy.
“Man’s search for meaning” is arguably the most visceral human framed definition of meaning. Let’s just say it’s not about attention and the Metaverse.
Extrapolate the flight time from NYC to London in 1976 ... have you seen the J-curve?
Or extrapolate the curve for self driving cars once they were only "two years away".
Catch is, tech progress is not one curve. It's overlapping S curves, stair steps of progress and stagnation, with the next iteration starting off worse than the current one, until crossing it as the current one tops out.
Is there demand for 10000 iq intellegence to fund its construction if we already have +100 iq AI? You are assuming there will be demand that scales as fast as technology can to fund its continued fast pace development.
Yes. If it looks like 10000 iq is possible, then money will be thrown at whoever is most likely to achieve it first, and whoever has the most intelligent models right now strongly predicts who will be first.
This is driven by the belief that whoever gets to 10000 iq first will likely dominate the majority of all economic activity, since it will be more efficient to trade with them than with some less efficient (dumber).
This does not depend on traditional economic demand, at some point this becomes a self-fulfilling prophecy.
People with money to invest will throw money at it to build it, but who is the customer base after that? So one guy eventually controls the stock market, why would I as a human want anything to do with that market? Why would I invest my earning into sustaining a machine that ideally is extracting every cent possible without leeway and funneling it up the pyramid? Unless I start at the top, ill never really go meaningfully higher.
Instead I can spend far less to use my 200 iq machine that already exceeds everything I could ever want. Im not ever going to personally fund a war of intellegence against other intellegences so anything more is of marginal value. The same reason I don't own a 2,000 horsepower truck or tractor, there are almost no uses for me to want it.
That's the fun part, there is none (in the traditional way of people buying things at least).
> Why would I invest my earning into sustaining a machine that ideally is extracting every cent possible without leeway and funneling it up the pyramid.
You shouldn't! Unfortunately everyone acting in their own self interest still results in this getting funded, since if such a machine were to exist, would it not be better to have a share of it?
I agree with your perspective! I just don't think that, as a species, we have a good track record of saying no to the existence of 2000hp trucks with ALMOST no use.
The demand for frontier intelligence is basically infinite. The frontier doesn't just complete a task more effectively, it creates new tasks and new problems and new solutions. It grows the pie exponentially. It's like looking at the internet the first time and trying to imagine the economy that will be built using it: social media, YouTube, Online job boards, Blogs etc.
woa your nephew lives inside a height-optimization machine that gets redesigned every few months (weeks), learns from every growth chart on Earth, and keeps proving it can grow faster?
It literally is just next word prediction. If you don't understand that, everything else you're thinking and saying is based on an incorrect foundation.
Take a step back and really take a look at what's going on. You are not thinking clearly.
The only way is through. Most of are not going to be living the suffocating life you think we should. We will grow, innovate and accelerate out of this.
reply