I did try some finance analysis earlier this year. I was using a DGX Spark, so I could run some relatively large models, but the results were pretty mixed at the time. I honestly can't remember which models I used anymore.
I'm having a really hard time doing on twin DGX spark what I could do on my quad 3090 rig (which is a scaled down version of what I was using before, the power requirements and the noise were really an issue but I loved the speed and the amount of VRAM). The results tend to be inconsistent, there is lots of looping, far more tokens generated for the same job and lower quality output. I suspect there is some kind of regression in the B12X kernels or something to that effect because none of that should happen, the exact same model on both machines gives wildly different results. Probably this will sort itself out over time. If I may ask, what model / software combo were you using?
I was only using a single DGX Spark, and this was earlier in the year, so I was running some pretty aggressively quantized models — probably in the 1–3 bit range.
My main issue at the time was that my financial data had lots of messy notes, comments, and irregular annotations. The quantized models often failed to process all of that context consistently and would miss things. So I ended up generating a fake dataset with the same structure, asking Claude Code to work out the analysis on that, and then bringing the result back to the local model for the final pass.
I was mainly using llama.cpp at the time, before B12X support was integrated into vLLM, so I think I wasn't using it then.
Thank you, interesting info! I think the Sparks are an interesting platform, the power consumption / memory bandwidth / memory amount trade-off is completely different from the regular cards and it will take a while for the software to really take advantage of them.
What do you think of AI generated infographics? Someone at my job is creating them on the daily and sending them around to leadership. I find it lazy and too busy.
Part of the problem is there’s so much garbage on the internet that an LLM can consume in search of the answer. As humans, we look into source quality: we can tell it by reputation of the website, how carefully crafted that site is, what qualifications the author has, and much more. There are many subtle tells on the internet that indicate source quality to humans, that AI cannot pick up on.
I experienced this first hand this week while searching for a specification on how much oil my car holds. AI gave me different answers multiple times. I manually searched, found an official manufacturer page, and got the proper information. Had I listened to the AI the first time, I would’ve severely underfilled my car.
No. As a human I like reading human written text over computer written text. I want something a human composed with thought put into it. Not something a human tried to save time with by having the machine write it.
Same, have a very old MBP. Not sure what to do because I don’t want to wait a year and a half. That coupled with today’s price increases make it a tougher decision.
reply