Good observation. I’m certainly not trying to over complicate things. That just tells us, though, that the bio-mechanical computation part is there before language, which I took as a foregone conclusion, it being meatware and all.
Good lord, why are you so angry that people are treating an obvious advertisement with due skepticism? Your post is chock full of emotional language - no one is “screeching” anything.
Many who rush to the defense of AI companies' marketing departments seem to take criticism personally, as if not buying into it all hook, line, and sinker is an affront to them. A foreseeable consequence of becoming cognitively dependent on LLMs.
And for what it's worth, I'm not an AI skeptic. I fully believe that the frontier models are capable of exploiting (chains of) vulnerabilities, having seen GLM and now Kimi do it myself. What I find no reason to accept is the sci-fi existential risk subtext peddled by the salesmanship around it. We will reach a new equilibrium with more secure software, and LLMs (by finding vulnerabilities, generating proofs, etc) will help us along.
This was reported 1 week before by huggingface. It was in no way PR-ish or marketing friendly to the closed labs. They said, in no uncertain terms, that they couldn't use the paid APIs to properly assess the intrusion, as they were blocked when trying to send logs and IoCs to these paid models. They made a point of saying that they had to use open models running on-prem.
Whatever oAI might have said about the incident, and their PR spin bs, the facts here are not in contention. This is not a marketing stunt in any way. Stop "parroting" this every time something happens. It gets stale.
Yes, I think this idea, that it should be "magical", is what makes it feel scummy. (Apparently I am not alone https://news.ycombinator.com/item?id=48988475). It makes AI providers sound like snake oil salesmen, and rightfully so.
Meanwhile, technological and engineering (STEM) progress have always been made by emphasizing externalization of the deductions (as opposed to reference to an opaque expert judgement) and reproducibility of experimental results.
I would even call the frontier AI labs anti-scientific. We need to understand how inference is done to avoid mistakes, not rely on intuition, even if the intuition is enclosed in a reproducible machine. The idea that AI should be this closed is a return to pre-scientific days.
The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. We can acknowledge it as a useful tool while understanding it has limits and is not “alive” or “thinking” in the same way a human is.
That’s not a solution. Humans are wired to anthropomorphise; we do it with sticks and rocks. How much more so with LLMs.
We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.
The marketing and ecosystem surrounding these tools feeds into the anthropomorphization to a huge extent, though. You’re doing the opposite - throwing up your hands and going “it’s our evolutionary instinct, what can you do” while willfully ignoring how manipulated this space is. OpenAI alone has a higher advertising budget than Coca-Cola.
> The marketing and ecosystem surrounding these tools feeds into the anthropomorphization to a huge extent
And especially their design. The con artist politeness and confidence is not an accident of output, but a design choice by the developers.
I tell non-technical people: It's a software program; there is always a person behind it, and they have consciously chosen this output, to serve their own interests.
Both claims seem to obviously contradict reality, imho:
> Humans are wired to anthropomorphise; we do it with sticks and rocks.
I do not see people anthropomorphize their TVs, books, paintings, cars, houses, rocks, sticks, ... I'm not sure what you're talking about here. I haven't seen someone call a rock or stick by a pronoun, for example.
I've seen people fancifully do it, but it's exaggeration for a cute effect, and works because everyone knows they're exaggerating.
> We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.
People learn to look at things in different ways. Humans learn, society changes. It's changed quite a bit in recent years; it's much different than a century ago, a century before that, etc.
In a way, it's more worship of technology to say humans are hopelessly, powerlessly dumb, and technology inevitably rules them. The inevitability of technology is of course a self-serving trope of the industry: you are powerless, we are inevitable. How convenient, and short-sighted.
I did not say that humans anthropomorphise every rock or stick. The point is that we are hard-wired for this possibility. Perhaps something more relatable? A cat? We ascribe all sorts of emotions to instinctive feline behaviours. It doesn’t matter with cats.
> In a way, it's more worship of technology to say humans are hopelessly, powerlessly dumb, and technology inevitably rules them.
You as an individual have some autonomy against the machines. Some. (Scroll facebook an hour a day and show me your resilience to engagement engineering). But the point of this is what happens at population scale.
To ignore our biology is as stupid as suggesting we must be ruled by it.
What I'd like is the TNG ship computer: an extremely smart system that nonetheless limits its "creativity" to the clearly marked holodeck, where simple prompting is treated like the casual throwaway diversion it is, and actual authors are still putting in substantial effort and personal preferences and viewpoints into their work.
> The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do.
It's not just fanboys doing that, but "official" educational channels. Both an internal training (more like an all-hands knowledge share) and some dumb training session by a guy from Anthropic explicitly, repeatedly recommended users anthropomorphize their chatbots on no uncertain terms. "Treat it as your super-smart coworkers", "I named mine Becky and I go to her with all my work challenges", etc.
reply