- cross-posted to:
- hackernews@lemmy.smeargle.fans
- cross-posted to:
- hackernews@lemmy.smeargle.fans
OpenAI blog post: https://openai.com/research/building-an-early-warning-system-for-llm-aided-biological-threat-creation
Orange discuss: https://news.ycombinator.com/item?id=39207291
I don’t have any particular section to call out. May post thoughts tomorrow today it’s after midnight oh gosh, but wanted to post since I knew ya’ll’d be interested in this.
Terrorists could use autocorrect according to OpenAI! Discuss!
I think what’s going miss here is that “CBRN groups” is very obviously and primarily shit made up by the military-industrial complex to justify itself after the Cold War
I don’t want to be dismissive of genuine attempts at being ready just in case, but the scale and scope of this is defined by politics, not by technical possibility
i mean, i don’t blame them, ritual dogfighting for congressional attention and money has became an art form. but the primary realistic concern of bioweapons preparedness would be, from what i understand, use of biological weapons by state actor, and i don’t really see a scenario where this happens before nukes start flying
100% agreed, what terrifies me is that our friend here seems to see the word “science” in here and immediately assume impeccable faith and perfect knowledge
I think the claim that cbrn is made up to self justify needs a lot more justification than you’re giving it. It’s just a profoundly confusing claim. They didn’t issue mopp4 in Syria for nothing…
And whether or not you think nuclear weapon proliferation is a problem, it’s hard to claim CBRN anti proliferation efforts are just a made up excuse to exist, it’s a very real reason to exist as a program concept. Maybe you wouldn’t have one if you could decide to, but that’s a far cry from whatever you seem to be claiming.
I expressly put “CBRN groups” in scare quotes to tag along with my line at the bottom “I don’t want to be dismissive of genuine attempts…but the scale and scope of this is defined by politics, not by technical possibility”
You, however, have me saying “cbrn is made up to self justify” - of course if I had said any such thing, then one counter-example would have sufficed. Although actually it wouldn’t have sufficed, because in this context we’re talking about terroristic or otherwise chaotic release of a novel weapon. We’re not talking at all about bad powerful people deliberately employing chemical weapons they already have, for which of course CBRN is a worthy use and “genuine attempt at being ready”.
“CBRN groups”, here, operates at the level of rhetoric, and that’s what I tried to draw attention to. The context in which “CBRN groups” the rhetorical and political device emerged was that in which Bill Clinton could become so enthused by a sci-fi novel about bioterrorism that he had its author up in front of the senate testifying as an expert on the subject. So on reflection, I should have deferred to Eisenhower’s original formulation: the military-industrial-congressional complex.
Edit: you could always try Alex Wellerstein for the aggressively obvious historical counter-point to this whole fantasy. In his Restricted Data he provides a useful companion to Barriers to Bioweapons in a chapter discussing the notorious “backyard atomic bomb built from declassified material” cases. But because it’s a work of history we learn the most salient fact of all: the only way anyone believed that the backyard bomb designs were viable was because somebody wanted them to believe it, or because they had some reason to want to believe it themselves.
Without that ingredient it was plain that the actual know-how was just not there, however that fact was fundamentally obscured by the desire to believe, and so people saw viability where there was none: plugging holes in their imaginary with meaningless verbiage about risk and but-what-if?
Incredible gymnastics to bend over backwards to interpret my response as one which doesn’t address what you say, when I specifically ask you to expand on what you mean and justify it. Ambiguous language doesn’t make you clever. Truly a discourse for the times.
So your problem is with politicians using fearmongering. Sure. That’s always frustrating, using fear mongering top drum up support has been a political passtime since politics.
I was not however, referring to fear mongering politics, but the practical and technical application of CBRN as a program and the actual, real, issues with bioterrorism and state bio weapons programs. Glad you got that soapbox out of your system though.
Bio and chemical terrorism are hardly akin to nuclear weapons. Refining uranium at any rate that could produce a bomb in someone’s lifetime takes industry that must be hidden at a state level.
This is simply not true of chemical and bio warfare.
If I may refer you back to the book cited, the (made up) fears of that time in fact incorporated the difficulty of obtaining fissile material during that period, when amongst the worries was that obtaining fissile material would not actually be that difficult. To simply state that biological and chemical warfare bear no resemblance is to depart from the lesson being related here to making excuses for that object of which you happen to be afraid. In each case the fear being constructed will make its own allowances for the real or supposed facts on the ground, and in this case there was no need to assume that a bombmaker would have to make his own plutonium - you’re drawing attention to an irrelevant distraction.
Another point which you’re glibly avoiding, with tellingly unnecessary recourse to insulting language, is that “CBRN” the construct cannot be so easily distinguished from the “practical and technical application” that the real enterprise has. Indeed the existence of the real enterprise is often driven in part by the made-up fears (which does not licence the fears) - this happened, for example, with security protocols around the management of fissile material. I refer you back to the same book and to the rather famous data point about Bill Clinton’s interest in manufactured diseases.
For more on stuff like this, although again not on the subject of bioterrorism because I don’t have that material in front of me, I recommend the confluence of two chapters in The Merger of Knowledge with Power by Rabitz (as well as the whole book), namely “Recombinant DNA Research: Whose Risks?” and “Hardware and Fantasy in Military Technology”. This isn’t paranoid soapboxing from a teenage Chomsky fan, it’s just part of the fabric of industrial science and technology as a social phenomenon.
Again, I’m not.
Right so fundamentally getting the ingredients for chem and bio warfare is objectively easier than fissile material. To dismiss them as the same implies you don’t realize you almost certainly have the ingredients to make a substantial amount of chlorine gas sitting in your home right now.
Yes bio is a bit harder than that, but not as much as you might think. Anthrax is a common soil bacteria. Ricin from grain. Isolating specific bacteria takes time and is sloppy, sure, but doable in a garage. Not easy, not something we should simply brush off, either.
Ultimately, you’re not going to be convinced. You want to paint something as the same ol false fear instead of a developing threat from genuine technological improvements that you are potentially not aware of. Oh well.
You can ramble about the politics of politicians and CBRN all day if you want, it won’t be responding to the focused discussion I was having about the practicality of bio warfare though.
“focused discussion”…right
Look, if you want to understand where I’m coming from, I can give you - at a glance - the two sources I’ve already given: Alex Wellerstein Unrestricted Data and J.R. Ravetz “The Merger of Knowledge with Power”, specifically the two chapters cited (the Wellerstein chapter is “Unrestricted Data: New Challenges to the Cold War Secrecy Regime”. I would also urge you to check out Lisa Stampnitzky Disciplining Terror. The introduction to the Ravetz book is also a must-read, not just for this, but also for a general understanding of how scientific research at the industrial level serves political and sectoral interests of all kinds - this is not radical pamphleteering about “the politics of politicians” but real empirical work about the real conditions under which science is done.
Stampnitzky is extremely useful here for understanding how the word “terrorist” (or similar) functions in the sorts of papers you cite at the very top. “Terrorist” and “state actor” are political words, and the risks (supposedly) measured which are attached to the threats you describe are weighted by those words, not by the scientific words pertaining to technical capability. To say that “terrorists” might get hold of this or that technology is to say that a particular type of person (who may or may not exist) will get hold of that technical capacity and make use of it.
The point is, in fact, that technical capacity has almost nothing to do with the measurement of risk from terrorist acquisition of that technical capacity. The measurement of risk is locused pretty much exclusively around the type of person who poses a threat. That type of person is a construction of politics, not a scientifically neutral object term in which people with medical or physical science qualifications have any expertise whatsoever.
To put it extremely briefly, this means that when you come across papers by CBRN professionals assessing speculative risks, much of the work being done is being done at the behest of political projects which have their home in the defense industry, not in assessment of the mere technical capacities available to people at large. As we learn from Ravetz, speculative risk created such an enormous bubble during the Cold War that it is almost impossible to take those measured risks remotely seriously - and as we learn from Stampnitzky, the idea of a “terrorist” has been constructed in such a way as to fuel that bubble. This means that CBRN professionals, however unimpeachable their contributions to the amelioration of those occasional disasters which do actually happen, are thoroughly questionable as unbiased witnesses to the scale that risks at large present.
Because, as your own inconsistencies show, you are not having a focused discussion (for example: you angrily claim in your second reply that in your first you asked me to expand on an earlier point, even though this never actually happened) it is extremely difficult to get this point across without appearing to just be dismissive of technical capacity as a factor. But in fact technical capacity has been factored in to my discussion this entire time. The fact that you’re unaware of the political environment in which your (non-)fear finds its sources is not anybody’s fault, but it is your fault if you don’t even acknowledge that other people might have a clearer idea about how this stuff works.
Look, ultimately why I’m focusing on technical capacity is because I don’t disagree on the political side of this. I think it’s highly unlikely that anyone would use bio weapons or develop them soon, and it’s definitely true that it’s inextricably political. My point is ultimately that bio weapons are getting easier and easier to produce and it’s not a non issue to consider. The top comments here were more than happy to dismiss bioweapons wholesale as even a thing that could happen.
I’ll check out your book recs. I’ll point out that the main book on this topic folks have cited is ten years old now, and we’re experiencing a legitimate bio tech revolution right now. How much further it’ll go is anybody’s guess.
I understand that you’re olive branching me here, but I don’t accept “How much further it’ll go is anybody’s guess”. Trends are analysable, and the sources of projections are equally analysable. A book that’s ten years old is far better than (a) 30-year old (and more) newspaper-level stuff, without citations, about backyard anthrax, (b) nothing, and © two links to tangentially related reports, and you’ve brought those three.
I am seriously concerned about the confluence of two things: (1) how closely your comments here mirror, right down to the level of language, press releases and opinion columns paraphrasing press releases, some of them (the anthrax stuff) extremely old hat; (2) the level of outrage and confidence you bring to the table when challenged on this and similar. Phrases like “How much further it’ll go is anybody’s guess” are press release language - they have absolutely no place in serious discussion, but they have a powerful rhetorical effect which allows them to displace serious analysis, and that displacement furthers specific, analysable, interpretable sectoral and political interests.
The same goes for “bio tech revolution” - you are never clear, in any of this, what that actually entails. What you do is cite possibility and unknowability, in a manner innovated precisely by sectoral and political interests from the 1950s onward. You have no detail of any value, and you write off actual detail with speculation and glib remarks about the age of the detail you’re given - that is a political innovation to which you have allowed yourself to be susceptible. You may also try on Naomi Oreskes for size as an author who grapples with this in both directions.
deleted by creator
deleted by creator
They aren’t their goalposts! They’re the goalposts already laid out in advance by the discourse and shaped in press releases since god knows when, that’s why it’s so easy to shift! There’s a whole avenue to be burrowed in Rationalism Studies, incidentally, about how Yud and his ilk inherited the same techniques from tobacco companies and the defense industry of the 1980s.
maybe SneerClub is the wrong outlet for whichever ax you’re currently grinding
I had fine discussions with others here and in the past. This particular poster wants to soapbox and dismiss rather than engage.