> I like that we are talking about prompt injections from you to your personal assistant. Obviously I know what they are for, but still, it's so funny that we've just normalized building software adversarial to the consumers supposedly engaging with the product.
As you can see in their marketing page, Muse is a personal assistant that reaches out and/or ingests data through apps, the web, your email/messages, etc.
If someone sends you a malicious email or if Muse finds a malicious website by accident, you wouldn't want it to send them all of your photos or to purchase things you don't want (or maybe that aren't even real).
The payment related examples on the page are of particular concern, but I imagine it kicks back to a human to actually pay for things. But who knows?
> I think the product as shown in this marketing material with these examples falls squarely in the “nobody asked for this” territory of AI.
I agree. The time to launch something like this would have been during the OpenClaw hype cycle (although, obviously, Muse is much more limited in the types of tasks it can accomplish).
I thought about installing it, but I realized I couldn't think of any real uses for it in my personal life.
The CVE associated with it (CVE-2026-85046) is already being exploited in the wild. If we put our thinking caps on, how much do you think this vulnerability is actually worth? How much do you think an organization like Google would spend on, for example, AI tokens or compute to detect this internally before it was found and exploited in the wild?
Ethical disclosure is a complicated topic, because researchers shouldn't hold bugs for ransom or demand high payment. But at the same time, if someone submits a critical issue like this, it makes sense to pay them what the bug's actually worth. Why should a researcher be effectively penalized for responsibly telling a vendor instead of selling the bug to a "research firm" or three-letter agency?
It's one thing if you're an open source project maintainer just trying to put something out to the community. The math is a lot different if you're Google.
If the vulnerability is already being exploited in the wild --- as in, it's a vector people already know about and are tracking --- it's possibly not worth much at all. Vulnerability valuations depend heavily on the lifespan of the vulnerability; payments on black market are tranched (explicitly or less explicitly, as with "maintenance payments") based on whether they're patched.
Further: a vulnerability is probably not worth that much either, even if it's a hypercapable vulnerability, because the grey market buys full enablement kits, not vulnerability information. People making 6 figures on vulnerabilities are selling fully enabled full chain exploit systems, not just intelligence about a sandbox escape.
Isn’t the implication that the vulnerability had already been found and exploited instead of reported to Google, because its value to Google was so low? Whoever found it originally had more incentive to sell it on the grey/black market. Or is the incentive structure truly different for vulnerabilities discovered to be already in the wild?
Some exploit hunters sell strictly to the grey/black market to avoid opsec issues. If you're selling on both, you're more likely to be identified by both.
Sometimes, yes. But usually, as a security researcher, you usually sold to brokers. The brokers made the vulnerability into a reliable exploit. That is whst they then sold to agencies. For a markup of course. The industry was in a tough spot already the past years. Now AI is shaking it up even more.
This "only" escapes the JavaScript sandbox. You still need to chain it with an escape for the process sandbox. So it's not just the boring stuff of turning a predictable crash into arbitrary execution, but also finding a second RCE vulnerability to chain this with
there are many vulns that are sitting idle waiting to be chained like this. plus in some cases adjacent bugs can be found once you find a bug like this.
> Spacing around an em dash varies. Most newspapers insert a space before and after the dash, and many popular magazines do the same, but most books and journals omit spacing, closing whatever comes before and after the em dash right up next to it.
Before the LLMs made the emdash the tool of the electronic oppressor I used it often enough and other people did too.
I hate that it's basically become the little hitler moustache of punctuation. There was at least one guy who was really enjoying his tiny little moustache until he couldn't anymore. That is the emdash for me.
In the spirit of good pedantic fun, I'll point out that en dashes are no less inconvenient than em dashes for the average Windows user to insert. The hyphen-minus immediately accessible on the keyboard was (and is) used for the common typewriting kludge attested in Garner's[1] as well as the LaTeX-syntax triple-hyphen em dash (which, might I add, for lack of a better place to do so, isn't one of the ways I've seen dashes used by LLMs or disguised by their operators).
That's really informative but maybe a little overly capitalist-brained.
We shouldn't look to the black market as cost discovery for these vulnerabilities, most non-criminal researchers are not putting up an ask order and letting the black market compete with Google.
There's plenty of legal gray-hat companies that will buy a bug. Zerodium is a good example though it no longer exists. These companies in turn sell to NSA, etc. maybe even foreign governments for all I know. There's very little regulation of the space iirc.
I had a friend that worked in the space. He lived an exciting life; I hope he's still alive..
Ah the classic programmer mistake when encountering the legal system of thinking ‘but surely this act, in and of itself, could not be considered illegal?’ When what the law criminalizes are not mere acts but actions carried out with intent and knowledge in a context.
‘Officer, you can’t arrest me for carrying a gold bar!’
‘I can if you are carrying it out of the Federal Reserve vault without permission’
Similarly:
‘Surely selling someone the knowledge that a piece of software can be hacked isn’t illegal?’
‘It is if you know or suspect that that person’s intention is to use that knowledge to commit crimes’
This would make you a ‘coconspirator’ or possibly an ‘accessory before the fact’ in any crimes they committed.
That's one of the points, yes, but the black market doesn't dictate the value of the exploit to Google.
A hardline bargaining position with Google would be more like "pay me what I want, or else I'll give it to all takers on the black market for maximum damage". That would be unethical and probably illegal to boot but it's a better definition of value than "1$ greater than max bid".
The value to Google - sure. But the market value of the exploit itself - the black market I think is a factor in that regardless of what Google offers.
The value of something is dictated by what _the market_ offers, and just because Google throws a lowball price because _they_ don't value it doesn't necessarily mean that the value of the exploit itself is as low as they dictate.
There will always be someone who would screw Google just for the love of the game, and if they got a better price from elsewhere, I don't see why _morality_ would really play that big of a role. It all comes down to incentives, and if Google doesn't incentivize doing the good thing enough, then someone _will_ incentivize them to do the bad thing just a bit more.
Markets are fundamental things and exist regardless of any kind of moral "should". Otherwise we wouldn't have people buying hard drugs or trafficing women.
The value of future reports should also be a component though. By paying a low amount you discourage ethical bug bounty hackers from bothering to look for more exploits. If I think I'm only getting $1000 for a Chrome issue versus $100,000 for an Acme Co issue, I'll be spending my time looking for Acme Co issues.
Bug bounties are as much a way of attracting talent to even try to exploit your system as they are about the exploits themselves. If you lowball the bounties the talent goes elsewhere.
How do you figure? The value of the report is, ethics aside, the same as the value of exploiting it. Doesn’t matter if I can conceive of it, it matters if I can exploit it.
Supply and demand has to be considered. The more parties aware of the vulnerability, the more attractive sellers (reporters) have to make their ask to the buyer (defender).
Not following that. The report is the upstream resource the exploit needs. The value of iron ore is definitely not the value of the steel made with it. Or maybe I misunderstood your view?
Well, the implication (and I'm not saying this is right) is that to Google it's only worth $1k to have this brought to their attention by a white hat, versus finding out by exploitation.
Which means that they have zero concern from this incident about reputational damage to themselves or their browser. That's pretty good circumstantial evidence of a monopolistic practice, when you can safely assume that there's effectively no difference to your bottom line if your software is hacked.
But google is also a monopsony. There isn't anyone else the researcher can ethically sell it to. They just have to take whatever bounty google decides to pay.
Google should have been split up into shreds like 2 decades ago. Search wants to have income from ads? Good sell it to anybody who pays the most, just like every single newspaper does. Gmail wants to sell our data, or ad space? Good sell them, and not just reuse them internally. Chrome wants to monetize every single request you do? Go, sell them on the open market. And not this fake, "we're separate companies, but only on paper" way.
This should have been done a long time ago.
Firefox, and the browser market would be much more healthy. Btw, Microsoft, Apple, Facebook, now even Twitter/SpaceX and all of these should have been forced the same way. And of course not just in this field, but all of them, like oil companies. They can pivot, of course, with some grace period, but that would mean giving up something at the end.
And if we are there, we can abolish most of trade secrecy too, which exists only to keep up the status quo while hindering progress.
Google is the answer to an Internet that largely blocks ads and uses backdoors to circumvent pay walls. A single massive pillar with enough surface area to carry all the dead weight.
If people want a better internet, they can start fostering one, rather than endlessly complain that someone else should be fostering it for them.
There will probably be upwards of 1000 people who read this comment that have used Google services for 15+ years, never loaded a single ad, and complain about Google ruining the internet. Please, pick-up a mirror. Nobody wants to build a competitor to serve your cheap ass
Google is routinely buying and destroying competitors.
They lower prices to kill competition en-masse. They promote their own products in search and elsewhere over paid ads.
Search Engine have chicken and egg problem, you can only have a good search if enough people use it to tune the ranking and see enough search spam cases.
“Just compete with a monopoly on their own field with one hand tied behind”
Every Google competitor has the problem that 40% of users won't load ads and 99% won't pay a subscription.
They don't need to do anything adversarial, the greed driven mindset of the internet (ads suck, everything should be free) staves off any real competition.
That 40 percent is definitely not true in the general audience. Also they will load ads, just not all of them. Surveys ask that. Especially on mobile, and apps. Also, when we measured on our car rental site, with near perfect precision (it's absolutely not difficult), then the number was around 20%. Also, according to download statistics, there are less than half a billion people who use ad blocking browsers or other solutions on Android. And that's downloads, which is obviously inflated, even obvious from browser statistics (no not 100% of Firefox users are on Android, it's almost the opposite).
It's not nothing, but it's probably around 20%.
Also as others stated. I pay for everything, if they give me the opportunity to pay them for ad free versions. I don't see ads on my Android, and I don't use a general ad blocker on it (except my browser, but I rarely visit sites which would contain ads, and basically all of them from Hacker News). I don't use free search, email, newspapers, I pay for YouTube, and I would pay for Facebook if I would use it. I don't even use Hacker News freely, because I paid for the app which I use to not see ads, and have more features. Heck, I pay even for my torrent, because I support them.
You're in the wrong neighborhood with this sentiment. But yes, the general populace is not like this.
Good for you, now get others to do it. Definitely not the wrong neighborhood though, HN is extremely pro ad-block and every paywall link has an archive.ph link in the comments.
Hell the top story right now is about a tool for backdooring twitter...
I don't see how we could have more vulnerability, on a massive scale, than to have two companies which pre-install both the OS and the browser, on their own hardware in the case of Apple, for 95% of the planet.
Microsoft was sued by the USDOJ in 1998 precisely for this, that it was leveraging the pre-installation of Windows to force Internet Explorer to be the default browser for PC consumers. That that was bad for both security and for the web seems pretty obvious in retrospect, when you think about the trajectory of IE.
Apple's insistent blocking of non-webkit web engines, including V8, on its own hardware/iOS is an even more egregious and dangerous phenomenon.
Browser security and browsers themselves would be vastly improved if they weren't a monopoly of the two mobile OS makers.
That’s exactly right, and applies to more than Google. I can’t think of a single browser vendor that would think a single vulnerability materially causes reputational damage.
While I agree 1000 is hilariously low for this, worth is hard to quantify. Do you pay what it could theoretically cost your company? the amount the top bidding bad actor would be willing to pay?
The discount Google is getting on bounties versus internal spend is easy to estimate:
# assumed to be $0.5mil USD or greater
A := What quantity of salaries-and-benefits and AI-dollars does Google spend on zero-day research?
# assumed to be greater than zero
B := How many full sandbox RCEs are they *hoping* to discover per year with that budget?
# $/RCE budgeted spend
C := A ÷ B
# $/bounty
D := $1000 USD
# % discount per bounty relative to in-house spend
E := (C - D) / C
While we lack the data to be sure, it is reasonable to estimate that they're getting a discount of 90% or better versus internal spend on this bounty payment, if one assumes that they do not have many sandbox RCEs left undiscovered. It's unclear whether that assumption holds, but with only a single researcher at an assumed $0.5mil/year (all-inclusive after pay, stock, and benefits) is enough to support the plausibility of that 90% figure, before accounting at market rates for their internal use of the house AIs.
So, the most likely case is that they're greedy and miserly, and hope we don't do the math. However I recognize that there are judgment calls to be made here. Either their internal spending finds hundreds of RCEs per year, or they're significantly discounting bounty payments versus their actual worth, or they're negligent in budgeting for RCE discovery at all, or they assign zero value to the security of the Chromium platform underpinning Edge, Electron, et al. All of these are bad in different ways; one hopes a competent tech reporter actually pursues this line of questioning with them!
> or they're negligent in budgeting for RCE discovery at all, or they assign zero value to the security of the Chromium platform underpinning Edge, Electron, et al
I generally agree, but a 3rd explanation is they figure that too generous a bounty will flood them with reports of minor issues making major ones harder to see (and costing time and money to verify that could be spent looking for security issues).
Having previously worked near a bounty program, I can confirm that they are regardless flooded with people fishing for bounties, even before AI, no matter how cheap the bounty may be — people will grift anything with the most pathetic skript kiddie attempts possible to try and pad their resume with a hit, and bounties that pay $0 are more valuable than pull requests that pay $0.
Maybe needs a Good-Guy-Buy-It-Now w/instant delivery at a fair price. (OK that’s kind of a threat—you’re running an auction and you have the price the corp has to pay to avoid the auction ending.)
$1k is so dumb and the fact we’re discussing auctions is proof (hello, Sundar, what you doing over there?).
Guess this will change after the next e.g. nationwide hospital ransomware by a hacker who publicly laments bounty rates, if the news cycle accommodates the story long enough.
> Guess this will change after the next e.g. nationwide hospital ransomware by a hacker who publicly laments bounty rates, if the news cycle accommodates the story long enough.
Negotiating with terrorists or black mailers is a bad idea.
Maybe. But as soon as they threaten to sell it to the baddies or use it for ransomware themselves, I would cease all communication and negotiation.
The legitimate threat the researcher has is to disclose to the general public. (And to disclose the next bug to the general public, if there's no good payment.)
Maybe some code is so important and heavily trafficked it becomes a public works project, and various legs can bid for pieces of the project, line how all infrastructure works.
we really do not want to engineer a system in which using bugs to make money is considered economically legitimate activity. It is still crime. The main reason to report bugs and get the bounties for doing so is still because it makes the world safer and healthier. The money is there to make is to incentivize the work of finding and reporting them -- not to outbid the bad actors.
I would say that maintaining legacy systems as a software engineer is effectively "using bugs to make money" and very much an "economically legitimate activity".
If old systems had no bugs/issues, companies could do without the maintenance burden altogether (which includes even systems not being evolved/extended).
Companies sometimes reward their employees with important bug fixes. When I worked on a big dev team, we'd even decide what were the most important fixes and give people a special 5k bonus or something.
But they weren't security issues necessarily. I never thought about it, fixing a huge performance issue is big. A security fix that gets caught early makes no noise so you just don't know how important it would have been. We also once had a really terrible bug that lead to lots of customers getting effectively attacked.
It's one thing to hold something for ransom ("give me $5M or I release the 0day"). It's another to sell a valuable piece of information ("give me $5M if you want the 0day"). As long as you're only offering the bug to the company who would be impacted by its release, there's nothing unethical about asking for payment.
Maybe you think that, ethically, all bugs should be reported, regardless of payment, because it prevents harm. Well a lot of things prevent harm that we don't all take it upon ourselves to do voluntarily. Should everyone do all safety-related work for free? If we don't want to do it for free, should we not do safety work at all?
If the company really wanted it safe, and they can't make it safe themselves, they can pay someone else to make it safe. If they aren't willing to do that, then nobody is obligated to do free work for them, because we don't require anyone else to do safety-critical work for free. Let's not forget, this isn't a scrappy startup struggling for a seed round, this is one of the world's largest corporations with billions of dollars in cash. If they want your labor, make them pay for it.
"Crime" is very flexible term. One country's criminal is another country hero. Maybe the author would sell the vulnerability to an organization making exploits for government use.
"Safety" is also a relative thing, when the world is safer for one party, it is usually worse for another.
As our surroundings grow more secure, the justice system variant of swatting becomes a greater portion of the threats to worry about. There will be abuseable bugs and situations in our non static world, there is no way we'll ever have perfect security of anything. So a motivated actor with a grudge should be able to plant something a place you provably beyond reasonable doubt have sole control of, given enough time. How do you propose then that we secure deniability once the justice system is wielded as a weapon against the innocent, when everyone feels that there is no reasonable way defects exist and could have been used? Just look at the British post office scandal, real world justice systems have already operated under the assumption that software doesn't have bugs for decades, which speaks volumes on their inclination to believe that they both exist and are used by a unknown third party with ill intent. Thus the widespread trust in that things are secure is a threat in itself. And unlike airports we don't need the users to have an artificial sense of security for computers, networks, software, and digital services to be viable markets.
Brain dead moral relativism argument. The question is whether eg. a group trying to scam elders out of insurance money or a Columbian cartel to hack local politicians to do blackmail, or South Sudan to hack Darfur or whatever, should be allowed to compete with the companies making products for their own exploits.
99.999% of people will agree that reducing software vulnerabilities is desirable if they're able to understand the question, including the bad actors themselves a lot of the times.
The situations like bad state actors are already not bound by laws, and things like keeping activism legal are better fought for through other ways
Well, someone did decide to tell google about this in exchange for a thousand dollars (albeit unclear how much the money was the motivator). Doesn't that mean the market did decide in google's favour?
Someone decided to tell Google about this in exchange for an unknown amount of money, chosen unilaterally by Google at a later date, at which point the market value of the vulnerability is $0.
Has anyone actually been convicted of abetting a crime by selling a vulnerability, by itself, not conspiring with the buyer to commit a crime using said vulnerability? Not as far as I can see. It would be absurd to jail someone for accurately describing a bug.
It would be absurd to jail someone for accurately describing a bug on their blog or whatever.
Not so much for taking money from someone who the buyer should know has no reason to be interested in buying the information. And either you know who your counterparty is, in which case you know that they are using it nefariously, or you don't know who your counterparty is, in which case you know that they are using it nefariously. Any court and any jury should see straight through this.
Similarly if you figure out how to get the ATM down the street to give you free money, and you "accurately describe the bug" to people who pay you, and they use it to steal money from the ATM, expect to be charged for participating in, and in fact being an instrumental enabler of their crime. Because it is beyond all reasonable doubt that you could've believed they could have been interested enough to pay you for any other reason.
Has anyone actually been charged and convicted for disclosing knowledge of a vulnerability in exchange for money with no further collusion to commit a crime?
Jeremy Jethro seems to be an example. His lawyer claimed he had no knowledge of what the exploit would be used for, and that it didn't even work, but he ultimately pled guilty to criminal conspiracy.
Hmm. Are you aware of any publicly identifiable security researchers that openly talk about selling their exploits on the black market?
Since it's all so legal and risk-free, you'd think selling an exploit for a million bucks would be quite the feather in their cap!
It may also be helpful to visualize being interviewed by the FBI and being asked "Did you sell this exploit? To whom? How much did you receive? For what purpose did you think it would be used?". And to remember they already know the answers to these questions, and lying to the FBI is also a crime.
courts are very good at reasoning about things like this and figuring out its bullshit. Zerodium is probably the closest you could get to some reasonable denial about this. Selling an exploit on crime.com for "research puposes only" will get you laughed at on the way to the cell.
In the past I would have thought this would incentivize finding bugs that might never be found. However it is now clear that all bugs that can be found will be found. So this makes a ton of sense.
No company will ever value your privacy or security more than or equal to how much you value them. This is why you gotta keep an unencrypted bitcoin private key in your password manager. I'll know pretty quickly (within ~ 10 minutes or less) that someone has access to all of my passwords.
you're taking someone's word it's being exploited. It says right at the top of the report
> allowed a remote attacker to execute arbitrary code *inside the sandbox*
A bug in V8 leads to code execution in Chrome's web page process. It does not lead to execution in general. For that you need other exploits that escape the web page process. Those are not detailed here. This CVE is not a big deal. You're responding the poster's title, not the actual CVE
Then what it was chained to is the real issue, not this one. The entire point of having webpages run in their own process is to prevent bugs like this one from doing worse. If you're claiming this bug is the bug that matters, you're effectively claiming they shouldn't need to run pages in their own process and just trust that there are zero bugs. No major browser does that. Not Firefox, not Safari, and not Chromium.
that's why bugs in the webpage process pay out very little. Without a worse 2nd bug, they are less serious. Bugs that let you RCE outside the webpage process pay much higher.
That sounds like a dumb strategy because if non-evil people sit on individual pieces of the puzzle waiting to solve it in full google loses most of the advantage of having a multi-layer system…
Makes you wonder how many hacks wouldn't have occurred if security researchers (and vulnerability disclosure) was actually rewarded proportional to the possible/potential damage said vulnerability may have otherwise caused.
It's insulting how poorly incentivised white hats are, just look at how much North Korea is raking in with their cyber shenanigans - current estimates put it at around $6.75 billion to date (over the past 10 years or so).
> But at the same time, if someone submits a critical issue like this, it makes sense to pay them what the bug's actually worth.
I'd point out that part of the reason the grey and black market pays so well is because it is that type of market. You have to pay people extra to look past their morals and a risk premium against potential reputational and legal consequences.
Someone could sell it on the black market, sell it to Google, or just move on with their life and not sell it.
I don't know what this is worth on the black market, maybe I'd be scammed by even trying to sell it. Maybe I don't want to be a bad person. These are all things that go into the prices
There are plenty of people out there who find vulnerabilities and sell them to the highest bidder. Anyone is welcome to do it, including the researchers and hackers reporting them responsibly. There's no need to try and make a convoluted ethical justification. "I did this bad thing because you didn't pay me enough not to" doesn't work past the 6th grade.
They're not going to stop underpaying security researchers just because security researchers decide to sell them to vuln brokers. Advocating for this is reckless.
they will if it becomes common knowledge that nobody serious is participating in their bug bounty programs. besides, they have incredibly deep pockets and they can afford to pay 6 figures for bugs like these
advocating for this is much more ethical than donating money to Google. I'd rather have that money go my family than a multi trillion dollar company.
> How much do you think an organization like Google would spend on, for example, AI tokens or compute to detect this internally before it was found and exploited in the wild?
On average, probably not that much. What's the amortized cost of all testing, static analysis, and audit / code review, per "prevented potential bug"?
The potential damage is all to users bound by terms and conditions, who are unlikely to collect damages successfully from a vendor. Structurally, vendors don’t have to care. Therefore, vulnerabilities have little direct financial value to a vendor.
It’s natural to feel cognitive dissonance because the value to the vendor is so disproportionate to the potential harm to users, but the incentive structure is what it is.
A vulnerability which lets an attacker harm the vendor has much higher direct financial value.
Interesting question, and how much should a user pay Google to fix the vulnerability? I suppose the smallest unit of currency less than the amount of effort they'd have to put in to mitigate it. A fully market economy of bug fixing here is an interesting idea, certainly, but if I'm being honest I actually don't want to pay Google a thousand dollars to fix security issues. In the limit, what would happen is that I end up with the competitor browser Elgoog Emorhc which fixes security issues for free, and pays very little for them, which is the status quo.
In the world where security issues are paid for entirely at market rate, it would also be very important to not use browsers by poor groups because they would be unable to pay for security reports on the market and consequently the browsers would be less secure.
Interesting idea, for sure, but I don't think it lands in a place I want to go since I neither desire stochastic payments nor desire that all browsers should be from large corporations.
The problem is they are being flooded with both fake AND real disclosures. Imagine if they tried to pay out $250,000 or more per bug? Would the cost be worth it? Maybe, but shareholders may not be pleased... Unless they viewed it as insurance against it being more financially sound for the finder to sell the exploit on the gray or black market instead...
In fact, Google will pay you $250,000 for a full chain exploit. The CVE reported here is a renderer process vulnerability. Google used to pay more for those before the vulnpocalypse. Now, they are fixing hundreds of bugs per week that they find themselves.
They should just multiply a base rate against the severity level. Say the base rate is ranged so low-severity stuff is $500-1K base but high-severity stuff is $10K base. That would net a researcher ~$88K for this specific bug (8.8 severity).
CVEs are handed out like candy for non issues, the severity rating system isn’t a serious evaluation of the actual severity (eg a vulnerable function may not even be compiled in), the scoring is inconsistent and subjective and frequently inflated to make the severity seem worse, and with the AI flood they have a massive backlog of handing out CVEs.
Like look at CVEs curl dealt with at one point that were just completely bogus and given huge severity ratings to start with.
But honestly if you’re the one proposing a “simple solution” maybe do some research yourself.
With such low payment, it makes one wonder how many exploits exist which were sold to 3rd parties and are currently used in the wild without Google even knowing about it.
It sounds insultingly low, yeah. I'm trying to imagine why they would pay so little. The only two reasons I can think of are either (a) they were already aware of it and fixing it, and therefore the report didn't really change much, or (b) it requires an unusual configuration or otherwise rare opportunity to that makes it impractical to exploit most users. Really curious to see what the issue was whenever it gets made public.
Imagine the consideration for the Trump admin, should we pay this guy a million bucks for this attack that gets us into the command system of Iran, or would that be unethical. Of course they don't consider that at this time.
I heard, on the podcast Darknet Diaries, that there are auctions for zero days in Argentina. This security researcher could probably have cleared a million dollars for a bug like that if they were unscrupulous. The bug bounty should absolutely be higher.
The CVE associated with it (CVE-2026-85046) is already being exploited in the wild. If we put our thinking caps on, how much do you think this vulnerability is actually worth? How much do you think an organization like Google would spend on, for example, AI tokens or compute to detect this internally before it was found and exploited in the wild?
in the darkweb, due to crypto, a lot. there is where the $ is, whether it's stealing crypto directly or phishing developers.
You've been on HN for 16 years and still comment the lowest brow possible comment on security vulnerability threads that the bounty isn't big enough. How many times do we need to have a top comment crying about the same thing? If you think its too little, sell the exploits you find for more.
>researchers shouldn't hold bugs for ransom or demand high payment.
Hell no, the same level bugs at Google should be enthusiastically paid way more than from an undercapitalized startup, who actually needs the help more so. Should be orders of magnitude difference in relation to scale.
>it makes sense to pay them what the bug's actually worth.
Honest fair-dealing should come into play at least but there are some players who have struck it so rich they can now take enough pride to pay an additional premium just because they can, and their good human nature almost compels them ethically to do way more than the minimum.
Just apparently not at Google.
If a company has achieved financial success to a degree that they are no longer worried about complete failure for the foreseeable future, then it's only a matter of generosity vs Scrooge-like behavior.
What's missing from their overall business acumen if they can't even afford to project an image of generosity yet?
If they're not actively making a serious effort to pay the maximum they can well afford for bugs that are truly serious, there is a technical term for that. Chickenshit.
>Ethical disclosure is a complicated topic, because researchers shouldn't hold bugs for ransom or demand high payment
Why not?
"Hey, I found a cvss 8.8 bug in chrome that allows arbitrary code execution when loading my http url. For X USD I can send a report along, and for Y USD I can send a commit with the fix."
Sounds like a basic contract to me
What I do think is ethically dubious is:
"Hey I found this bug and I will MAKE IT PUBLIC WITHIN 90 DAYS SO LOOK AT IT"
I know it's a convention from 'security researchers', but I think the first approach is more ethical than the latter.
> Ethical disclosure is a complicated topic, because researchers shouldn't hold bugs for ransom or demand high payment.
Why not? Capitalism requires they maximize their value. These profitable companies lay bare at the altar, so they should understand the requirements of their god.
> how much do you think this vulnerability is actually worth? How much do you think an organization like Google would spend on, for example, AI tokens or compute to detect this internally before it was found and exploited in the wild?
What? That's not how you calculate the value of something at all.
The value is not based on the prevention, it's the cost of the cure.
I hate to change the subject to something so trivial -- because I have been an exclusive Firefox user for many, many years -- but the awful moving background of this website is literally nauseating.
I'm no designer, but I can't understand why someone would make a graphical choice like this. It absolutely detracts from the content, and in my case, will prevent me from reading it.
I have a lot of respect for Trail of Bits, and I'm sure that Artem is thinking about this correctly. However, I respectfully disagree with the premise.
AI agents are not magic. Mythos/Glasswing does not magically create vulnerabilities in software projects. Advanced, cyber-capable models do not magically hack out of VMs or contained environments. They do not have a "hacking" stat that, if high enough, means that they can breach anything. They aren't Kevin Mitnick whistling nuclear launch codes into the prison payphone. This isn't a movie.
What these cyber-capable frontier models can do is find security problems and exploit them. The statement should not be that VMs won't contain cyber-capable agents, but rather that we need to focus on finding and fixing vulnerabilities and misconfigurations in these environments.
Even the article itself concludes with suggesting something like Firecracker, which was designed with security in mind.
Like most other security-related problems introduced by advanced cyber-capable AI, it's possible that these issues will get worse until they get better. But if frontier models are run against state-of-the-art VMs, and OpenAI or Anthropic or whoever works with the virtualization projects to address the issues, eventually it will run out of things to exploit.
The concept of virtualization is not inherently insecure. We just have a long way to go.
I agree. Whether or not it is what they are intending (I lean towards they are), IMO it has been strongly inferred in all the messaging from frontier models that it's impossible to secure software, which is just not true. In reality, it's now possible to find and fix problems faster. If attackers/adversaries can just point LLMs at software and dump zero days as we are warned/told repeatedly by every LLM vendor, then literally anyone can do it now. If that is true, then if you want to be marked safe from AI enabled attackers, test your own software with it.
This highlights a problem with the Anthropic-Mythos-restriction style "we regulate the best models so attackers don't have them": It also ensures that only the big-guy partners of the program (Glasswing in their case) have the capability to secure themselves at the frontier model level.
I think it s a response to the "containers can't contain agents we need VMs" which didn't go far enough. Qemu/KVM is the type of software that maybe isn't fixable, it has a huge surface area of features (all in memory unsafe languages) that have to be fixed, and maybe its not going to be fixed. If everyone who needs strong security reads this and uses firecracker or whatever instead (I mean, they probably mostly are), then maybe there will be even less pressure to fix the convenient stuff in Debian.
Note that the author didn't use any QEMU guest-to-host vulnerability here. The agent found a logic bug that facilitated going straight for the hypervisor from inside the guest, but it isn't caused by the use a memory-unsafe language.
In all seriousness, here is the hard question we collectively face:
will our institutions and infrastructure survive things getting worse until they get better?
That's part of what's being asked by https://www.gatesnotes.com/home/home-page-topic/reader/a-tur... which I find most useful not for its analysis or suggestions, but for the fact that is advancing the need for discourse around risk and devising—quickly—a process for having that dialog.
What that process is supposed to be, who is to have a seat, how decisions madehave teeth of any kind are not any clearer than the primary question, how are we to collectively manage risk.
"May you be born in messy times." A curse if there ever was one.
on the flip side: these models can create all the security holes they want in any given open source project they desire as long as they can convince a lazy AI llm to let them contribute code.
Everything you described can go through the black mirror and nullify it.
On a long enough timeline anything can be hacked out of given enough time to reason about it - humans do it all the time. Cyber capable AI does it on a much much shorter timeline..
The premise is correct. VMs won't contain these. As we've already seen AI hack in/out of companies in the real world accidently. There is no perfect, unhackable stack. If there were then computer security would have been a done deal a long time ago.
Once these malicious AIs get out on the real internet basically everything is an open free for all. The military, infrastructure, government, corporate networks are far from perfect, not up to date, they are no match for an AI that can create its own zero days.
I think when push comes to shove we either need to shut down the networks/internet or risk AI locking us out of everything potentially for ransom.
Imagine being blackmailed by a non-human entity. I guess we won't need to imagine for too much longer. That is definitely going to happen. Somehow every year my crazy AI arguments get easier and easier to defend. That hugging face incident really made you guys move the goal posts.
> I think when push comes to shove we either need to shut down the networks/internet or risk AI locking us out of everything potentially for ransom.
Yes, I too believe that this is what is going to happen.
The ban of foreign robotics in the US also affecting vacuum robots is one of those examples that sound weird and annoying to people, but given this perspective (and their cloud-connectedness), it might just be preparation to eventually push the global internet killswitch, without your voters getting angry that their floors don't get cleaned anymore.
I guess there are much more sensible examples to pick here, but hey. This one is interesting I guess.
Unfortunately once you accept this premise, the next logical conclusion is that the physical world and analog systems are just as vulnerable to exploitation by embodied AI.
I don’t think the post claims magical abilities. Just that most things are already crackable given public knowledge and LLMs can exploit those. Worse, that the rate at which we update security issues is too slow to keep up with “knowing about a problem is equivalent to having a working exploit”.
Maybe there will be unhackable hardware one day, but it's likely impossible with modern CPUs due to the alarming number of side-channels hidden in the silicon. They might even have intentional guest escape vulnerabilities...
> And how long will that take? It's already well past due.
Probably not anytime soon, and it's getting worse: every AI-pilled CxO is horny for AI-boosted velocity - instead of security. Expect more bugs shipped, based on just the volume AI enables.
We're going to see more of this before we see, hopefully, substantially less of it.
What I'm seeing now in industry -- and I think this autofix issue is a precise example of it -- is a natural evolution of the "LGTM!" review that's so prevalent in software development and similar disciplines.
For years, the dramatic majority of "code review" was a quick glance followed by "Looks good to me." Sure, critical workflows have more scrutiny. Sure, not everyone fell victim to this trap. Sure, there are many exceptions. But it's a meme for a reason: most people weren't really reviewing code assigned to them. They were effectively rubber-stamping most things.
So now, in the age of AI, those same people are (sometimes still) expected to be responsible for what their automated developer friend Claude is doing. It's absolutely unreasonable to think that most people are giving the PR more than a glance, and in many organizations they're explicitly trying to remove humans from the loop.
One day, AI development and code review will be so good that mistakes like this will be extraordinarily rare. For the near-future, though, I anticipate we'll see more of this before we see less.
That's mostly because at some point a wave of nonsense swept over the field that brought with it the Scrum master, agile, pairing, middle managers thinking up elaborate Git branching schemes (they don't understand Git) and of course, the mandatory code review.
Yeah I agree and I think code forges as well as AI harnesses are kinda the killer apps of this (relatively short) era.
I think once we figure out how to tighten the loop of user feedback, expert analysis, automatic/static verification and AI generation then technology is going to make another leap.
Mistakes like this were always common because GHA is evil. If you pull a random action and read the code, chances are it has a few bugs.
Vulnerabilities and bugs are becoming rarer due to AI. We’ve already seen the Linux kernel stamp out vulnerability after vulnerability, some of which have existed for over a decade.
That doesn’t mean AI just does the work. You need highly skilled engineers leading it. Which the Linux kernel has. But yes, AI is good at reading code and finding defects. It’s good today. Not all models, you need a highly quality model, but yes it’s good today.
I said this about OpenAI/Hugging Face, and I'll say it again for Anthropic:
It's not that I think this is fiction; I'm confident these events actually happened. But I think they were effectively allowed to happen because Anthropic and OpenAI are constantly chasing each other for the narrative of "hugely advanced, maybe almost sentient AI lives here."
It reads more like a press release than a security update, and I think that's because it is. I hope this type of marketing backfires and the companies face actual scrutiny and consequences for operating this way.
As you can see in their marketing page, Muse is a personal assistant that reaches out and/or ingests data through apps, the web, your email/messages, etc.
If someone sends you a malicious email or if Muse finds a malicious website by accident, you wouldn't want it to send them all of your photos or to purchase things you don't want (or maybe that aren't even real).
The payment related examples on the page are of particular concern, but I imagine it kicks back to a human to actually pay for things. But who knows?
reply