Rendered at 00:47:40 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
andai 17 hours ago [-]
If they succeed here, won't they have to gate access to this feature, too? For the same reasons as, "if I so much as mention mitochondria, it downgrades me to Opus."
Step 1. Make it so Claude can do anything — the whole point of AGI
Step 2. Wait, if the user can do Anything, that would be Very Bad!
Step 3. Err on the safe side with blanket bans of entire fields
The latter actually seems to me a sensible reaction to e.g. the compartmentalization used in the large scale cyber attack using Claude last year. Where they were able to do Bad Thing by dividing it into many, many Small, Seemingly Harmless Things.
Gated access sounds bad (and I agree it sounds bad!) but it might actually be the only sensible response to such a set of conditions. I'm not sure though.
--
I saw some studies recently which showed LLMs provide much more detailed information to expert users. So we can distinguish between competence and incompetence based on use of language, and that is a reasonable metric for harm reduction.
But I don't think we can reliably detect "user has harmful intentions", at least not at a sufficient level of sophistication of the attacker.
MiracleRabbit 17 hours ago [-]
I just wait until a Opensource model inhaled all the chemistry books and papers. Lol.
They are following closely and the best offer 80-90% of the performance and come with a very small fraction of the costs.
inigyou 10 hours ago [-]
There are no open source LLMs, only LLMs you can download.
echelon_musk 16 hours ago [-]
Why wouldn't the US gov. outlaw the open source models?
skeledrew 12 hours ago [-]
Applicable to only those of the US (who choose to comply), which would mean giving non-US an advantage, which is a distinct no-no. They tried to restrict Fable to only-US, but that backfired since the world isn't at all clean cut, resulting in full unavailability of the model
MiracleRabbit 15 hours ago [-]
Streisand effect
Deepseek.v4.Pro.RePacked.LLMBoyz.part1.zstd
Daviey 13 hours ago [-]
Will each part be a 1.44mb rar/ace file?
skeledrew 12 hours ago [-]
Damn this reminds me of my days going around with a stack of - mostly recovered - diskettes.
MiracleRabbit 13 hours ago [-]
And screaming loud Keygen music
squidbeak 13 hours ago [-]
Without international treaty and regulation restricting frontier capabilities globally, any attempt to outlaw open source models will only be as effective as King Canute ordering the tide to turn. Unless the USA fancies bombing those who refuse.
skeledrew 12 hours ago [-]
China is definitely refusing. And definitely will bomb back.
10 hours ago [-]
throw1234567891 10 hours ago [-]
And build the wall, all around.
lelanthran 11 hours ago [-]
How would that affect the world?
15155 10 hours ago [-]
Prior restraint?
CamperBob2 3 hours ago [-]
"National security" is a cheat code in the courts that gets you past any conceivable Constitutional blocks.
OutOfHere 9 hours ago [-]
It cannot. The only available law is an export control law. A restriction on an open model would be strongly unconstitutional due to a freedom of speech guarantee in the Constitution.
UltraSane 15 hours ago [-]
Much harder to ban if they can run locally
inigyou 10 hours ago [-]
It's a good marketing strategy though, since nobody can disprove that Claude is the best chemist :)
siva7 10 hours ago [-]
"Making Claude a chemist" is a punch in the face of all us here after we got a taste of Claude fable blocking almost any STEM-related topic. So i don't know what Story Anthropic wants to tell us with this blog. Probably more like "Hey, look at what awesome tricks Claude can do which of course you will never experience because you're not american, not friends with the CEO, not rich or simply because fuck you"
HarHarVeryFunny 6 hours ago [-]
I think the story they want to tell is "we're not just taking your jobs, we can do positive stuff like Chemistry too, just like DeepMind!"
Except of course they mostly can't because Chemistry is about structure and molecular dynamics, not people's descriptions of experiments.
DeepMind if so far the only AI/AGI company - the rest are LLM companies.
NewsaHackO 16 hours ago [-]
I think one time I asked opus about copyfail when it just came out and it did treat me like some sort of criminal, but are there really people that run into this on a regular basis other than cybersecurity experts (which cannot be a big enough group to generate all of this criticism)?
poizan42 10 hours ago [-]
I have been trying to work on debugging tools using Opus 4.8. As it turns out working with low-level techniques that inspects and alters behavior of other processes is bordering close enough to cybersecurity research that it often hits the guardrails.
dmd 13 hours ago [-]
I asked it to clip a video onto segments based on location of spoken words (using separately done audio transcription) and it flagged that as cyber.
brookst 10 hours ago [-]
Maybe? It seems like their strategy is to accept that safety is imperfect, err on the side of over-triggering, and iterate.
I don’t think it’s a black and white “if fable 5 over triggers on bio safety in 2026, that’s the final pattern we should expect to see from post-Mythos 20 in 2036”
UltraSane 15 hours ago [-]
Fable wouldn't even explain what an amino acid is.
chaostheory 11 hours ago [-]
It would make more sense to have specialized versions of Claude instead of having a universal one ie BioClaude, ProgrammerClaude, NuclearClaude, CyberSecurityClaude, et al
This would be both safer and less annoying to use.
OutOfHere 9 hours ago [-]
That seems bad because it kills interdisciplinary science.
chaostheory 5 hours ago [-]
sure, but how else would you better guarantee safety? How else can we avoid annoying guardrails?
Besides, it's not like you can't have different agents with skills collaborating
OutOfHere 4 hours ago [-]
Safety is a lie. It's always possible to divide a task into subtasks, to use abstractions, and the LLM will allow each of these individually.
For example, If you want it to write malware, break that down into a hundred high level steps, then implement each one individually, then stitch them together.
A dumber but uncensored agentic LLM will even do this for you to control the censored smarter LLM.
matheusmoreira 5 hours ago [-]
> how else would you better guarantee safety?
You wouldn't. This safety nonsense is overrated.
gopalv 18 hours ago [-]
The non-professional side of Organic Chemistry is one place where I think AI would really shine.
Feels complex like solving a Rubik's cube to write down synthesis steps but it is all a sequence of memorized tricks. Do Cannizaro if you want this, Bergmann to do that.
But the synthesis plan is only 10% of the actual work.
The gap between writing down the synthesis step and actually doing it is also extremely large.
Even if you get the right molecule, it might be the wrong way around or just clump up into a useless mess.
The Ritonavir episode of Veritasium is a great example of how all chemistry on paper is a mere shadow of what actually happens in real life.
ElFitz 17 hours ago [-]
> Feels complex like solving a Rubik's cube to write down synthesis steps but it is all a sequence of memorized tricks. Do Cannizaro if you want this, Bergmann to do that.
I remember two years ago, when I actually got into using graph data structures, wondering if maybe the "space" of available reactions for any given starter and target molecules could be mapped as a graph, with intermediates as nodes and reactions as weighted directed edges, so synthesis becomes pathfinding through chemical space.
Turns out, it’s a thing! [^0]
Edit: Makes you wonder how much interesting stuff is sitting in plain sight, waiting for someone with the right cross-domain awareness / knowledge / whatever to notice it.
There is a lot of graph theory in Chemistry - modelling chemicals as (vertex/edge coloured) graphs, reaction networks, etc.
Of course some molecules (eg aromatic systems, like ferrocene) are not naturally representable as graphs. I wonder if it is the same with synthesis - are there reactions hard to model as a graph (or petri net or whatever). One simple example I know is that you have to be careful with including a node for 'water' as it gets connected to everything else! Or at least in biochemistry it does.
Obscurity4340 10 hours ago [-]
Why is ferrocene ungraphable or in this context unable to be modelled in that way?
A metal atom sandwiched between two Cp rings. You _can_ model this as 5 single bonds between each atom of a ring (so 10 total C-M bonds), or you have to have some kind of 'edge' (bond) between the ring as a whole and the metal.
The more general issue is that a graph model of a chemical assumes a 'bond' is between exactly two atoms. Three-center hydrogen bonds are another example where this model fails to capture the chemistry very well.
Of course, it's a tradeoff - you can model _most_ compounds with just graphs (plus atom type, charge, chirality) and the relatively few that do not quite fit are special cases.
tylergetsay 12 hours ago [-]
Hamilton Morris and his stuff on clandestine chemistry is super interesting in this domain. Sometimes the chemistry is straight forward but access to certain chemicals is hard, so the procedure must change based on what's available not necessarily what's ideal
jgilias 18 hours ago [-]
> Even if you get the right molecule, it might be the wrong way around or just clump up into a useless mess.
Sounds a lot like vibe coding lol
fakedang 18 hours ago [-]
Modern biochemistry (so far) IS vibe coding lol. You mostly have vibes on how the chemistry should work, based on (very strong) natural evidence coupled with theoretical development and lab studies. Then you mix and match, goading bacteria and praying that they produce what you want in good measure. Then you take their secretions and run chromatography studies on them to check if that's what you actually want, or whether it's just some random bullshit. If it's the latter, you have to toss that out and start all over again.
skeledrew 12 hours ago [-]
Or in some instances that "random bullshit" turns out to be something actually novel and/or useful.
moffkalast 17 hours ago [-]
At least vibe coding can only explode in your face metaphorically.
b112 16 hours ago [-]
Until vibe code is used for weapons system, or explosive manufacturers, or.. or...
The world today is coding.
inigyou 10 hours ago [-]
Already is. Missile that hit the Iranian girl's school was vibe targeted.
reinitctxoffset 18 hours ago [-]
Organic chemistry seems like a discipline better done by chemists than forward deployed staff with their payoff function sharply truncated at an IPO which at this point may or may not happen on schedule.
xvilka 17 hours ago [-]
Combine it with automated lab like this[1][2][3][4][5] (and many others) and it will iterate much quicker. Some already do but at a smaller scale, AFAIK.
What good is it if you can't use it? Or worse, if you can but it silently sabotages you?
yorwba 19 hours ago [-]
> We measured three Claude models (Opus 4.7, Opus 4.6, Sonnet 4.6)
You can use those and they probably won't intentionally sabotage you.
matheusmoreira 18 hours ago [-]
Yeah. Probably.
siva7 10 hours ago [-]
The thing about trust is.. it takes time to build and seconds to destroy.
sandermvanvliet 15 hours ago [-]
Waiting for Claude to end up on Derek Lowe’s list of things he won’t work with
smj-edison 17 hours ago [-]
I feel like chemistry is one thing that current models will struggle with for the next while, because it's inherently 3D. In the micro world, shape = function. Maybe enough textual patterns will let it under chemistry, but like how do you describe a hydrogen shift without showing how it moves positions and rebalances bonds?
FailMore 15 hours ago [-]
I think this is a very interesting concept/question. I feel like programming is more about shapes than anything else… but they seem to have mastered that fairly easily… but I totally get your point!
johncole 15 hours ago [-]
Agree here, chem may be more complex than language, and not as definable as language. I think this is the realm that physical ai will overlap with.
moonsu 12 hours ago [-]
I wonder if chemistry skills will be an emergent behaviour of world model research.
snovv_crash 13 hours ago [-]
I work a lot with the geometry side of computer vision (camera calibration, 3d reconstruction etc.) and LLMs have been really bad in this space. They throw stuff at the problem until a minor improvement is made by over fitting on a testset, then gaslight that this is the best thing possible. Then I go do something like working with raster-based data, or some JavaScript based visualization, and it goes super smoothly.
spaceprison 6 hours ago [-]
Fable pre-ban wouldn’t continue an old conversation about brain chemistry, sooo I guess they’re dusting off old tricks now that their new robot is offline.
anentropic 16 hours ago [-]
I guess the future is one where every idiot has access to a genius servant, and all that implies
sudhirkhanger 14 hours ago [-]
Are there organizations or individuals using AI to solve world problems if they are so powerful as these companies are saying?
throw1234567891 10 hours ago [-]
everything from amd in robotics space
thefounder 19 hours ago [-]
Let’s ban this before it gets too powerful !
simulator5g 19 hours ago [-]
I don't think it should be outright banned, automated intelligence is like a gun, hammer, knife, nuclear warhead, etc.
thefounder 12 hours ago [-]
Ok I may agree with this. Instead to ban it we should nerf it to the point it’s useless.
naveen99 11 hours ago [-]
Maybe nerf should start selling ai models.
Obscurity4340 10 hours ago [-]
How are they not already??
pkal 18 hours ago [-]
I see that intelligence itself is a tool, but that doesn't mean I want an automated gun, automated hammer, automated nuclear warhead, etc.
fakedang 18 hours ago [-]
You may not want them, but the NRA certainly wants that you want them.
ben_w 18 hours ago [-]
In the future perhaps it will mean "Neural Rifle Automaton"
tearwear 18 hours ago [-]
not really, though. you don't see automated intelligence in the hands of junkies et al. ... and you don't see it coming, either ...
simulator5g 18 hours ago [-]
Junkies et al can log into chatgpt right now and use it to create a phishing email to steal drug money from your Grandma. Many people saw this coming.
orphea 14 hours ago [-]
And they don't need castrated Fable for that.
xaxfixho 18 hours ago [-]
[dead]
ed_mercer 16 hours ago [-]
I wonder if every future announcement will be met with this criticism. Like, what's the point if they'll ban it in a few days.
UltraSane 15 hours ago [-]
I don't really want the average LLM being able to tell anyone exactly how to syntheize lethal nerve agents.
matheusmoreira 5 hours ago [-]
Don't worry, it is extremely likely that anyone who actually tries to synthesize such things will die trying before any serious damage is done.
bouncycastle 14 hours ago [-]
you can already get that information by reading books in the library.
The biggest barrier is not information, it's the ability to secure enough of the materials and equipment.
For example, information for how to make a nuclear weapon is already there in the library. However, mining enough yellow cake and then purifying it is an industrial scale operation, out of reach unless you are a nation state, and have good mountain tunnels, etc. To a lesser extent, this is also true for producing chemical weapons. The theory is there, but actual production extremely out of reach. No LLM can help you there. (You can verify by reading up on Aum Shinrikyo to get an idea of the staggering scale required)
dmd 13 hours ago [-]
Sure, information on all sorts of things is in “the library”. But (a) most people in the world don’t have access to a decent library (if you’re on HN you’re statistically much more likely to), (b) most people have no idea how to use one (same), and most importantly (c) AI will distill the knowledge from ten books you might need to read into step by step simple instructions if you ask.
defrost 14 hours ago [-]
> mining enough yellow cake and then purifying it is an industrial scale operation, out of reach unless you are a nation state
or a transnational (or even national scale) energy and or minerals company.
Might be hard to slip past the shareholders, but dark projects have flown under the annual reports of several large players.
bouncycastle 11 hours ago [-]
Highly unlikely. Just the energy cost of running centrifuges, let alone manufacturing them is impossible to hide. It's staggering when you consider it.
To put some more perspective, an LLM could show you how to make a Boeing 747 in detail, but the actual chance of someone making one with no tools and logistics would be 0.
LLMs haven't changed the economic realities.
defrost 33 minutes ago [-]
> Highly unlikely. Just the energy cost of running centrifuges, let alone manufacturing them is impossible to hide.
And yet that has happened twice already. Eg:
NUCLEAR ANXIETY: THE BLUNDERS; U.S. Blundered On Intelligence, Officials Admit
There are numerous large companies with the resources and technical know how to match either Pakistan or India's efforts here, companies already experienced in yellowcake extraction and complex production processing equipment manufacture.
> To put some more perspective, an LLM could show you how to make a Boeing 747 in detail, but the actual chance of someone making one with no tools and logistics would be 0.
Pakistan, India, and many companies have resources, tools, logistics experience .. I have no idea how this sentence is meant to meaningfully refute those facts.
> LLMs haven't changed the economic realities.
I don't recall mentioning LLM's and I remind all that the bulk of nuclear weapons development happened before LLM's were a thing.
FWiW there are better actual arguments why it is unlikely (but not impossible) for a non state actor to put together a nuclear weapon .. the comment above is not one.
14 hours ago [-]
CamperBob2 3 hours ago [-]
Aum Shin Rikyo didn't need an LLM to synthesize Sarin, and neither do you.
But when the police raid the hideout of the next Aum Shin Rikyo successor group and find evidence that they got the recipe from an LLM rather than traditional chemistry books, the resulting kneejerk legislative overreaction will cost us all some freedom.
That's not OK.
jgilias 18 hours ago [-]
inb4 someone calls Bessent to explain how this can be used in fentanyl production.
vzcx 5 hours ago [-]
I know your comment is about the absurdity of such a thing, and maybe the stupidity of the administration, but for the readers at home:
Fentanyl is actually a very simple molecule. The way it's (and other drugs) production is controlled is by controlling and monitoring the precursor chemicals. So if someone goes out and buys a lot of proprionyl chloride, they will pop up on the DEA's list.
This is a much more sane thing to do than hemming and hawing about how dangerous the AI chatbot is and placing yourself into a position superiority over your users, pretending you know better than them, assuming that you must protect them from certain knowledge for their own good, etc, etc.
Step 1. Make it so Claude can do anything — the whole point of AGI
Step 2. Wait, if the user can do Anything, that would be Very Bad!
Step 3. Err on the safe side with blanket bans of entire fields
The latter actually seems to me a sensible reaction to e.g. the compartmentalization used in the large scale cyber attack using Claude last year. Where they were able to do Bad Thing by dividing it into many, many Small, Seemingly Harmless Things.
Gated access sounds bad (and I agree it sounds bad!) but it might actually be the only sensible response to such a set of conditions. I'm not sure though.
--
I saw some studies recently which showed LLMs provide much more detailed information to expert users. So we can distinguish between competence and incompetence based on use of language, and that is a reasonable metric for harm reduction.
But I don't think we can reliably detect "user has harmful intentions", at least not at a sufficient level of sophistication of the attacker.
They are following closely and the best offer 80-90% of the performance and come with a very small fraction of the costs.
Deepseek.v4.Pro.RePacked.LLMBoyz.part1.zstd
Except of course they mostly can't because Chemistry is about structure and molecular dynamics, not people's descriptions of experiments.
DeepMind if so far the only AI/AGI company - the rest are LLM companies.
I don’t think it’s a black and white “if fable 5 over triggers on bio safety in 2026, that’s the final pattern we should expect to see from post-Mythos 20 in 2036”
This would be both safer and less annoying to use.
Besides, it's not like you can't have different agents with skills collaborating
For example, If you want it to write malware, break that down into a hundred high level steps, then implement each one individually, then stitch them together.
A dumber but uncensored agentic LLM will even do this for you to control the censored smarter LLM.
You wouldn't. This safety nonsense is overrated.
Feels complex like solving a Rubik's cube to write down synthesis steps but it is all a sequence of memorized tricks. Do Cannizaro if you want this, Bergmann to do that.
But the synthesis plan is only 10% of the actual work.
The gap between writing down the synthesis step and actually doing it is also extremely large.
Even if you get the right molecule, it might be the wrong way around or just clump up into a useless mess.
The Ritonavir episode of Veritasium is a great example of how all chemistry on paper is a mere shadow of what actually happens in real life.
I remember two years ago, when I actually got into using graph data structures, wondering if maybe the "space" of available reactions for any given starter and target molecules could be mapped as a graph, with intermediates as nodes and reactions as weighted directed edges, so synthesis becomes pathfinding through chemical space.
Turns out, it’s a thing! [^0]
Edit: Makes you wonder how much interesting stuff is sitting in plain sight, waiting for someone with the right cross-domain awareness / knowledge / whatever to notice it.
[0]: https://pmc.ncbi.nlm.nih.gov/articles/PMC9574932/
Of course some molecules (eg aromatic systems, like ferrocene) are not naturally representable as graphs. I wonder if it is the same with synthesis - are there reactions hard to model as a graph (or petri net or whatever). One simple example I know is that you have to be careful with including a node for 'water' as it gets connected to everything else! Or at least in biochemistry it does.
https://en.wikipedia.org/wiki/Metallocene
A metal atom sandwiched between two Cp rings. You _can_ model this as 5 single bonds between each atom of a ring (so 10 total C-M bonds), or you have to have some kind of 'edge' (bond) between the ring as a whole and the metal.
The more general issue is that a graph model of a chemical assumes a 'bond' is between exactly two atoms. Three-center hydrogen bonds are another example where this model fails to capture the chemistry very well.
Of course, it's a tradeoff - you can model _most_ compounds with just graphs (plus atom type, charge, chirality) and the relatively few that do not quite fit are special cases.
Sounds a lot like vibe coding lol
The world today is coding.
[1] https://www.ginkgo.bio/autonomous-lab
[2] https://www.emeraldcloudlab.com/
[3] https://www.kebotix.com/
[4] https://www.chemify.io/
[5] https://arxiv.org/abs/2507.01485
You can use those and they probably won't intentionally sabotage you.
The biggest barrier is not information, it's the ability to secure enough of the materials and equipment.
For example, information for how to make a nuclear weapon is already there in the library. However, mining enough yellow cake and then purifying it is an industrial scale operation, out of reach unless you are a nation state, and have good mountain tunnels, etc. To a lesser extent, this is also true for producing chemical weapons. The theory is there, but actual production extremely out of reach. No LLM can help you there. (You can verify by reading up on Aum Shinrikyo to get an idea of the staggering scale required)
or a transnational (or even national scale) energy and or minerals company.
Might be hard to slip past the shareholders, but dark projects have flown under the annual reports of several large players.
To put some more perspective, an LLM could show you how to make a Boeing 747 in detail, but the actual chance of someone making one with no tools and logistics would be 0.
LLMs haven't changed the economic realities.
And yet that has happened twice already. Eg:
NUCLEAR ANXIETY: THE BLUNDERS; U.S. Blundered On Intelligence, Officials Admit
https://www.nytimes.com/1998/05/13/world/nuclear-anxiety-the...
There are numerous large companies with the resources and technical know how to match either Pakistan or India's efforts here, companies already experienced in yellowcake extraction and complex production processing equipment manufacture.
> To put some more perspective, an LLM could show you how to make a Boeing 747 in detail, but the actual chance of someone making one with no tools and logistics would be 0.
Pakistan, India, and many companies have resources, tools, logistics experience .. I have no idea how this sentence is meant to meaningfully refute those facts.
> LLMs haven't changed the economic realities.
I don't recall mentioning LLM's and I remind all that the bulk of nuclear weapons development happened before LLM's were a thing.
FWiW there are better actual arguments why it is unlikely (but not impossible) for a non state actor to put together a nuclear weapon .. the comment above is not one.
But when the police raid the hideout of the next Aum Shin Rikyo successor group and find evidence that they got the recipe from an LLM rather than traditional chemistry books, the resulting kneejerk legislative overreaction will cost us all some freedom.
That's not OK.
Fentanyl is actually a very simple molecule. The way it's (and other drugs) production is controlled is by controlling and monitoring the precursor chemicals. So if someone goes out and buys a lot of proprionyl chloride, they will pop up on the DEA's list.
This is a much more sane thing to do than hemming and hawing about how dangerous the AI chatbot is and placing yourself into a position superiority over your users, pretending you know better than them, assuming that you must protect them from certain knowledge for their own good, etc, etc.