As with the rest of the domains, AI/LLMs will do syntax and search better than any human.
In code, any developer whose differentiaton was clean code and knowledge of different technologies is now average.
In math, any mathematitian whose differentiation was to manipulate formal systems and know tricks of different domains will be average.
Fortunately, humans do more than syntax and search.
The bad news for developers is that if you know what the output of your program should be (which happens most of the time), almost all of the job is syntax and search to build the code that reproduces the output.
The good news for mathematitians is that for the majority of problems you never know the output, or you just know the output is either "True" or "False". There are some cases where you need something else, for example "a solution that blows up in finite time". For those cases AI will outperform you easily (see new Navier-Stokes solution)
So if as a mathematitian you were doing more than syntax and search, then keep doing that and use AI just for what its best.
I am not surprised that Silicon Valley researchers say they cannot solve the alignment problem... Here the leading voice in the region publishes an article about how to be more powerful without any other mention of alignment but "users".
For example, I am sure facebook users (the ones buying ads) are super happy, but is the company aligned?
How do we prove alignment? Often times, large regulatory bodies. Who supports that? Does the regulatory body actually have enough power to enforce those standards? (Hint: look at modern America). That's an example of power questions sneaking in even when you didn't want to.
Even "mere Darwinism" is less extreme because it pretends like there is some objective criterion called natural selection that will filter out bad candidates, when it's often the case that the bad candidates are the ones that weren't proactive to define the selection process themselves. Sitting back and optimizing for an externally imposed standard is worse than being able to impose said standard yourself.
(If this point was complicated - my point is that yes, Darwinism makes sense, but when applying it, you can get a mis-skewed definition of what "natural selection" really entails. I am calling out that caveat).
Of course, my hope is that that isn't the end all, be all solution. Philosophy doesn't end with Nietzche, hopefully. But the world does sure looks bleak right now...
And yet we have the saying "all's fair in love and war." Adultery, experienced in xx% of relationships, is a blatant power struggle. It's normal to edit your dating app pictures (or was back when people actually used dating apps) to look more attractive, or to get cosmetic surgery. There's custody battles and all that.
> The closest human parallel is self-deception, which is common and well studied by psychologists. Motivated reasoning, motivated cognition16 and the rationalizations that relieve cognitive dissonance (the discomfort of holding a belief that clashes with our actions) are all cases where thinking bends toward whatever justification suits one's interests, including one's moral self-image.
Come on, it’s way more common than that. We’ve invented 3000+ gods and almost as many religions, most of them are incompatible with each other. So, most of these must be incorrect, so a huge amount of self-deception. But as Harari argued in his book sapiens, humans can be inspired to great things by stories, even if false. Self deception has served humanity in a big way.
No one religion covers "most people." You could argue that Christianity and Islam (which add up to ~55%) are the same God because of their Abrahamic roots, but both religions have very important disagreements on the true nature of God that are fundamental to their beliefs and fundamentally incompatible with each other.
Their definitions of God do agree that there is exactly one God... which is fundamentally incompatible with the next two biggest religions (Hinduism and Buddhism) that both hold "there are many gods/divine heavenly beings" as core beliefs.
Christianity, Islam and Judaism are most people between then, as you agree.
They explicitly all worship the same God so not "incompatible gods". I did not claim there were no disagreements about the nature of God, but that it only adds one to the GP's claim of 3,000 incompatible Gods. Even if Christians are completely right theologically, Jews and Muslims are still mostly right - one God, a loving creator, omnipotent and omniscient etc.
AFAIK most Hindus are pantheists, so believe on one God, albeit of a very different nature.
Buddhists do not necessarily believe in any god at all.
Buddhism and Hinduism are definitely compatible with each other. I know lots of Buddhists who make offerings in Hindu temples, for example.
Gods of many polytheistic religions are compatible, you just add more gods or identify similar gods with each other (e.g. Sulis Minerva who was also Venus). They can also be compatible with pantheism - you just add more aspects of God.
At the most almost all human religions fit into a handful of broadly similar systems.
> Christianity, Islam and Judaism […] They explicitly all worship the same God so not "incompatible gods"
One of them (for the most part, at least) explicitly worships one God that exists as three distinct, co-equal divine persons: the Father, the Son, and the Holy Spirit. That view is explicitly rejected by the other two.
I’ll agree with your last 3 words: they’re broadly similar systems - systems of deceit and self-deception. They may share the same god, but for 99% they’re different enough that they think they can kill one another and their god will approve, or even give them gifts. For 99% these religions are incompatible.
Submissions are anonymous to the exterior viewers, but you do them from one account. You can extract impact metrics from that account for job applications
I like to replace thes AI text with "virus manipulation"
"Given the acceleratung rate of virus manipulation in labs, its my worry that in 6-12 months a virus could scape a take over the world and collapse health systems."
If a CEO of a health company was saying this, the reactions would not be that chill.
The worst failure of our society is to call this technology AI, intead of something line "artificial general automation". The formes allows the creators to be somehow less responsible of the consequences. The later clearly moves the responsibility to the creator/user.
Your point was that its an unfair comparison because AI its a race. My point is that bioweaponds are also a race, but a less public one. You dont have the equivalent of Dario publishing an essay every month.
This sounds to me like a cry for help from someone thats held hostage.
His previous self is writting this from the possition of his current self who is too deep in the economic consequeces to be able to do anything meaningful other than write a consequenceless text. Any other action would now carry too much personal risk for him.
You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely unsafe, because it'd provide unrestricted access to all the most capable/dangerous models to everyone, and hence doesn't do it. Why do you think it'd make the world more secure, if he did?
AI technology is now only audited by people who think like them. The ones that dont either dont join the company, or leave after a short stint as we have seen. That creates an information or feedback bubble which is not healthy nor productive.
Okay, but suppose that Hypothetical Opensource Anthropic trains a model that turns out to be very dangerous, and releases it. Suppose that the public investigates and, not being limited by an information bubble, correctly notices that it's very dangerous. What then? The model's already released, there's effectively no way to prevent it from being used. Whatever the risks of its release were, they will now materialize, regardless of what the public wants. How is this better than the current world, either by Dario's values or by yours?
By the same logic, suppose that Dario/Altman/Jensen accumulate all capital because they are the only one that have access to AGI and end up controlling democracy, turning the world into technofeudalism or whatever you want to call it. How is that better than an opensource Anthropic. At least an open source anthropic would allow to join economic forces to try to combat that situation, for example. But there are other more clear, less distopian benefits.
> By the same logic, suppose that Dario/Altman/Jensen accumulate all capital because they are the only one that have access to AGI and end up controlling democracy, turning the world into technofeudalism or whatever you want to call it. How is that better than an opensource Anthropic.
It'd be pretty bad but it's also a world where humanity lives on, which is better than a lot of other outcomes. A world where everyone has unrestricted access to AGI is like a world in which everyone has a tactical nuke in their pocket. If somehow AGI doesn't lead to x-risks, we will "merely" have to survive in a world where rogue agents can do whatever they want and defenders can only ever react. Technofeudalism would be bad, but this (technoanarchism?) is quite bad too.
However, neither does Dario seem to propose to become world dictator. Like, I'm sure he wouldn't say if he wanted it, but notice that he isn't particularly trying to aim for that outcome, either. The plan in this essay involves third-party oversight and federal control and global cooperation - IMO it's about as non-dystopian an outcome as we can hope for, if we build AGI at all.
I agree. Dario is a very smart guy and he is probably aiming for the best solution for humanity that at the same time keeps VCs/whoever gives him money happy, and keeps him in control because he does not trust the rest (probably with good reasons).
My point is more that he is aiming for a local optimum for society, but he could find a lower optimum if he was not trapped in the path that he chose early on.
I dont buy the tactical nukes/end of the world narrative. Why is he not speaking about the loss of cognitive skills of the population for example? That is a more real danger that is starting to happen. There are articles already speaking about the use of LLMs as cognitive viruses. Why are his aligment teams not reasearching and publishing about this?
Why don't you create your own auditing org to fix this? Or at the very least, publish a detailed critique of what you believe existing auditing orgs are missing.
Will they give my auditing org access to their proprietary confidential secrets? Or will that happen only if they know i agree enough with them so that i am not a risk?
We should create a platform where people can upload AI slop proofs anonymously without taking credit for it. That way the incentives of planting a thorem flag would go down. And if someone wants to clean an AI slop proof to advance the field, they can do it without cleaning the house for free of the person that planted the flag.
Would OpenAI have uploaded their proof to such a platform? If you know the answer then you know what the problem with what happened is.
Dont confuse mathematics with the formal system. If you beleive mathematics = formal system then AI is obviously better at it, and we dont need humans.
But then who decides why a statement is mor important than another? In the eyes of a formal systems all statements are born equal.
In code, any developer whose differentiaton was clean code and knowledge of different technologies is now average.
In math, any mathematitian whose differentiation was to manipulate formal systems and know tricks of different domains will be average.
Fortunately, humans do more than syntax and search.
The bad news for developers is that if you know what the output of your program should be (which happens most of the time), almost all of the job is syntax and search to build the code that reproduces the output.
The good news for mathematitians is that for the majority of problems you never know the output, or you just know the output is either "True" or "False". There are some cases where you need something else, for example "a solution that blows up in finite time". For those cases AI will outperform you easily (see new Navier-Stokes solution)
So if as a mathematitian you were doing more than syntax and search, then keep doing that and use AI just for what its best.
reply