I just find this an interesting comment. Having an electrical engineers education, there is this interesting repetition of formula which is unmistakable.
First you learn the physical equations, mostly applied to sound interestingly. Then at some point you learn about transmission line theory and maximum power transfer theory. Lastly, you get introduced to Maxwells equations, and how they produce the phenomena you’ve been learning about for years.
Unfortunately that is where my learning stopped, and I never got an answer to how quantum theory resolves with Maxwells equations, and how it interacts with standard model and special/general relativity.
Honestly I’d love an explanation of the standard model that wasn’t covered with mystery and math. It makes the idea of understanding particle physics impossible.
Unfortunately, in my experience the more advanced/nuanced/complete of an explanation of physical phenomenon you want, the more math heavy and abstract the "real answer" becomes. Quantum Electrodynamics is some beefy stuff, the math is gnarly, and it is built on an increasing pyramid of mathematical reasoning and intuition as well as the steps of physical reasoning and intuition that you mention.
Remembering my graduate physics days im astounded that i was able to understand it, and am unable to really follow it now, as that higher math has gotten preeeety rusty.
Feynman diagrams are the attempt to loop back around and represent this field without all the math, but my experience was you have to have done the math first otherwise they are just so much arcane heiroglyphs. the wikipedia illustrates this well, where you can see a relatively clear diagram and corresponding equation, but i recall whole pages of a notebook to get one _line_ of an equation down. https://en.wikipedia.org/wiki/Quantum_electrodynamics#Feynma...
I say all of this not to suggest a clear explanation is impossible, but it sure wouldn't be easy. Love to see it though.
Fellow excited. Sol has been revolutionary for me. It’s been the first time I’ve let a model “go ham” on a production code base and it actually worked, and also didn’t bankrupt my company in the process.
I’m far more excited for the trajectory that OpenAI has chosen. I’ve been listening to mates whose companies adopted Claude wholesale, only for their AI use to become a double digit percentage of their salary.
I get that AI is a force multiplier, but that level of expense isn’t a path to mass adoption.
Honestly, this feels like a real revolution now in a way that 90s kid never really experienced. We grew up with technological progression, we never experienced the obsolescence of skill.
The internet revolution made for more skills and innovation, it didn’t obsolete entire careers. My kids are almost certainly going to grow up knowing less but being capable of more.
Imagine being a 1950s “human calculator” on the dawn of a computer revolution. That’s what it feels like right now.
> "Imagine being a 1950s “human calculator” on the dawn of a computer revolution. That’s what it feels like right now."
My personal / family history is a real-world example of that evolution. My grandfather was a "computer", my father was a traditional "programmer" (lots of Perl), and I'm a SWE / frontend architect / budding "AI Engineer".
Tahoe has destroyed macOS performance. I’m running a maxed out last gen intel and was getting stuttering and slide showing straight out of the update. It was completely unusable until I turned on reduced transparency.
Colleagues on much better spec’d M1/2/3 all noted similar, but not quite so drastic, slow downs.
The hugest issue is search appears to be universal broken. Spotlight doesn’t work, finder doesn’t work, email doesn’t work. The other day I had to login to Gmail through the web app for the first time in about 5 years to find an email I knew existed that mail decided didn’t exist.
Insult to injury is the search indexing service appears to be somewhat responsible for slowdowns. It’s right up there with the famous “windows updating in the background” slowdowns.
Look to Australia if you want to see how these energy policies work out in the long run.
Energy is big money. Anything that disrupts the status quo is immediately vilified. It got to the point that our major party at the time lead a “gas fired” green energy policy.
They got voted out, the new guys implemented a subsidised home battery policy. In the space of a year it has destroyed the economics of a carbon fuelled grid. All the economics pointing to renewables being the way forward were proven correct.
Everyone gets their back scratched. If you’re a consumer and you invested, you have a massively discounted personal power station that generates you money. If you operate the grid, your transmission costs and peak transmission costs went down. General consumer prices went down because of a reduction in peak electricity use driving up retail costs. Everyone in won.
Except big gas, who cannot turn a profit on a renewable success story. Don’t let rhetoric become your reality.
It's absurd how the propaganda against green energy on certain political spectrums essentially turned into, "It's all money/cash grab for green energy companies!"
As if the established industry has no incentive in the opposite direction? Everyone is going to act in their best interests in this system, that's how it's designed, the whole point of government is to mediate the system in the best interests of the whole.
RE "...how the propaganda against green energy...."
One problem is statement are made , that depend on certain definitions , where the definition is not what a reasonable person would expect.
Such as statements about costs of renewables. How exactly is the cost of renewables measured.?
I'd have to say there are lots people who do not believe the government is acting in their best interests. Its one of the reasons why Pauline Hanson's "One Nation" party has gained a lot of traction.
> Except big gas, who cannot turn a profit on a renewable success story.
If only.
Chevron’s Gorgon project will be ongoing for decades and making money hand over fist flogging LNG out of Australia to overseas customers.
It'll be a massive source of previously sequestered CO2 and claimed to be "the largest CO2 sequestration project on the planet* .. with the minor catch of (re-) stashing away a very very tiny portion of the CO2 released. Something, something, not let it out in the first place?
Still, could be worse, at least they're not planning on fracking the Kimberley.
> with the minor catch of (re-) stashing away a very very tiny portion of the CO2 released
First, I want to clarify that when raw natural gas is extracted from underground wells, some portion of the gas is carbon dioxide. In this project, it is about 15%. The project's sequestration goal was to capture this CO2. So far, they are only able to capture about 1/3 of this CO2 for various technical reasons. I would not characterize 1/3 as a "very very tiny portion". Do I misunderstand your comment?
Well the tax payer, would be paying the 30% subsidy for Home Battery deployments, which I understand is severeral $Billion 8.5 AU at present.
One group of loosers will be people who can not afford to participate in this program.
It would seem another group would be users who cannot shift their electrity demands from peak times when electricty will become quite expensive.
The comment is factually correct, It's a inconvenient truth.
Battery deployment is not free.Someone (the taxpayer) pays.
I suspect after they are quite widely deployed. Government could change conditions in future that they must be available as virtual power plant (VPP)
There has been a rush on these since people want access to cheaper power and they believe to avoid the power increases to come. That there will be big increases in power , especially during peak time is the forced introduction of smart meters which allow easy tariff change etc ....
> Power prices to fall for most customers, with bigger drops for businesses
> What is driving [lower prices] is a reduction in the cost of producing electricity. We've seen a lot more batteries and solar systems come into the electricity market in the last 12 months. They've been making the market much less volatile," Ms Savage said.
> "We've not needed as much gas and hydro generation in the evening peaks, and that's what's really cut that cost of wholesale generation."
Yep, exactly. Due to the deranged Net Zero policies of successive Australian federal and state governments, Australia have experienced skyrocketing electricity prices, so any small decrease in residential and industrial prices now is effectively a shit sandwich.
> Pre-merge, this took 5.9 billion uncached input tokens, 690 million output tokens, and 72 billion cached input token reads — around $165,000 at API pricing. By hand, I think this would've taken 3 engineers with full context on the codebase about a year, during which time we wouldn't be able to improve Node.js compatibility, fix bugs, fix security issues or implement new features. We never would've done that. The realistic alternative was to do nothing and keep fixing the bugs at the top of this post forever.
> THE SPINNER MESSAGE CAUSES 100% GPU USAGE ON AN MBP M5!!
One conspiratorial idea I had was that this isn't a bug, and that Codex was actually doing computation on users' hardware under the guise of "thinking". Like Folding@home, or bitcoin mining malware, involuntarily on paying customers. Your usage is being subsidized by your personal compute hardware that you can't take advantage of unless it was being applied at massive scale.
This would make even more sense when you consider that thinking and response time metrics aren't publicly being tracked. There is an assumption that LLM interaction is being processed as fast as possible, but this doesn't align with the reality of fixed hardware and oversubscription. Of course throttling is occurring. So, if you can take advantage of local compute, delay the responses and you have even more access compute!
I find it difficult to believe that given the scale, number of users, and money involved, that someone hasn't fixed this "bug".
People conflate the ideas of happiness, and comfort. Money buys access to increasing levels of comfort, but comfort becomes normalized very quickly. Once you've become accustomed to a certain level of comfort, the luxury of it wears off and it becomes a new norm. You also have an expectation to, at a minimum, maintain wealth so that you don't lose access to your current level of comfort.
When people with 1X see people with 10X or 100X and go hey! Why aren't you doing more? That gives me hope. When these people succeed, they are exactly the type of people who will give back and derive happiness from it. The right person who acquires wealth can do a lot of good in the world.
This was a genuinely thought provoking article. I had to challenge some personal assumptions.
Coming from an electrical engineering background, I disagree with how the author presented "Two types of quantizers". Mathematically rigorous, but not grounded in practical systems.
In ADCs, there is always an inherent +-1/2 LSB of quantisation uncertainty. The transfer characteristic is always mid-tread sampling, or at least I haven't come across any counter examples. This is true for bipolar or unipolar ADCs.
The lowest code is negative voltage reference, and the highest code the positive reference. The transfer characteristic plot will show what the author has demonstrated, that the highest and lowest bins will effectively be 1/2LSB in width.
In a unipolar system, this has the consequence of not being able to represent the midpoint voltage precisely, or in other words, the gray problem. In a bipolar system, 0V will be mid-tread N/2 value, but that doesn't mean it has "256 ranges".
So, I'll be sticking with (VREF+ - VREV-) * k / (2^N - 1). Or in other words I agree with the normalisation by 255. It's the fence post error all over again, you have N values, but N-1 ranges. If you have less ranges than you do values, you need to distribute 1 of those ranges between two values, hence the 1/2LSB range endpoints.
All ADCs I have looked at document that they can't represent the positive full scale. For instance, for an 8 bit ±1 V ADC, -128 represents -1 V, +127 represents 127/128=0.99219 V. The transition from 126 to 127 happens at 1.5 LSB from the positive full range. 1 LSB difference represents 1/128 = 0.00781 V difference, and not 2 / 255 = 0.00784 V.
But if you actually care about what the voltage (and uncertainty) is, most of this is difference is mostly pointless, you're reference will have a bias, there are linearity errors and so on. 1 LSB will not match either the 1/128 or 2/255, you will need parameters to compensate for it.
You've made me do even more digging and now I'm even more confused.
ATMega328P, data sheet specifically calls out using that it cannot represent full range, ADC = VIN * 1024 / VREF. The STM32F4 datasheet shows an idealised transfer function which is mid-tread, but the "actual transfer function" is sort of a shifted mid-tread (lowest code is larger than a single LSB, highest code is VREF and 1/4 LSB). Other STM32 references I found show that it should follow an ideal mid-tread, unless you are using VADC as your voltage reference. High resolution differential input ADCs are, as best as I can tell, always mid-tread with both end codes representing the positive and negative voltage reference.
The best data sheet I've found was for a PIC32, very detailed transfer characteristic diagram. This shows that each LSB is VREF/1024, but also why! The transfer function shows that is is still a mid-tread transfer, but the highest code is assigned to (1023/1024) * VREF, hence the divide by 1024, but it doesn't solve the 1/2LSB end bin problem.
This is different to the ideas presented the article, there is no reason a single ended ADC couldn't use mid-tread quantisation with VREF as the end code. What it does show is that by using divide by 256, you are truncating the range and arbitrarily deciding where the end points should be. This doesn't fix any inherent 1/2LSB quantisation uncertainty, which was the main argument against using divide by 255.
As you say, in ADCs it's a moot point with all of the other sources of error.
Largely those datasheets are poorly written... but in practice it doesn't matter much because the manufacturing variations of offset and gain error are larger than the quantization level.
The datasheet specifications are largely standardized across ADC manufacturers and do not represent the exact mathematical norm but rather the range of ADC performances. So it's not that "the transfer characteristic is always mid-tread sampling" but rather that "the ideal model of the transfer characteristic is always X".
I do have some historical examples where mid-riser was used, but in practice it doesn't matter when you consider ADC variations and the variation of gain and offset are multiple counts. (8-bit or more; this exact definition would have been important for ADCs less than 8 bits.)
That’s the mental model that works for people, specifically those that come from VM workflow.
Ironically that’s how Docker works on every platform where it’s running a non-native OS. On macOS that’s how all images are run. Linux on Linux is the only Docker combination that is particularly problematic from a security perspective.
Virtualisation has advanced greatly since docker was introduced, if your running in local hardware that’s supports virtualisation, Docker should be running images fully virtualised. There is no good reason to use the OS kernel for most use cases as the performance impact is negligible. If you need kernel access there are better options, like systemd containers.
I agree that virtualization has seen great advances. Kata containers on k8s are almost (not quite 100%) drop in replacement. Regardless those last 10% remain a problem.
I run a personal server for few open source applications for personal use. I was thinking with all the supply chain attacks, and how carelessly I run `docker pull`s to update things I should probably consider hardening things a bit. I thought before jumping to full virtualization with Kata I can easily try gvisor/runsc first. Only to realize that DNS resolution is completely different with runsc vs runc and had to switch back.
Another sticking issue with virtualization is resource allocation. With namespace docker you can easily oversubscribe each container CPU/memory and rely on the single kernel letting individual containers burst as needed. With full virtualization this is still a big problem. Even with balloon devices and dynamic memory and CPU etc, the resource allocation is still not optimal. On a basic 8 core/16GB machine you can run 1 or 2 dozen services and things generally workout fine. Trying to run each of those in a virtualized VM you suddly can maybe run 6 or 7 maybe. There is no way to tell VM 3 kernel to drop its file system cache because VM 6 needs to load a large file in memory. Even if you script it out, now VM 3 is slow because it dropped all its cache while VM 6 finished processing 3 hours ago. These are not unsolvable problems, but despite how far virtualization has come, are still friction points.
Not to mention issues like sharing hardware devices (GPUs, disks, USB devices etc) between multiple VMs
First you learn the physical equations, mostly applied to sound interestingly. Then at some point you learn about transmission line theory and maximum power transfer theory. Lastly, you get introduced to Maxwells equations, and how they produce the phenomena you’ve been learning about for years.
Unfortunately that is where my learning stopped, and I never got an answer to how quantum theory resolves with Maxwells equations, and how it interacts with standard model and special/general relativity.
Honestly I’d love an explanation of the standard model that wasn’t covered with mystery and math. It makes the idea of understanding particle physics impossible.
reply