yup, 87.8% of $42,412 is just two GPUs and a PSU. Even RAM is less than 5%.
coffeebeqn 2 days ago [-]
Really just the GPUs. PSUs aren’t that expensive maybe 100-200 for a good 750W-1000W one
embedding-shape 2 days ago [-]
Kind of feels like if you spend $40K on your GPU setup, you don't want to cheap out on the PSU.
Still, you're right, I have a 1200W PSU for a single RTX Pro 6000, cost ~500 something, so still the PSU shouldn't be that large part of the total budget.
slowmovintarget 2 days ago [-]
Funny thing is, you don't really want to just plug that into a wall socket. You want an uninterruptible power supply that can keep the machine running for the 5 to 8 minutes you need to get to it and shut it down safely during a power cut.
But you'll find most household power sockets aren't actually rated for the kind of power these things draw at peak. If you go for a dual GPU you're going to want to sustain a higher current. Think washer / drier / fridge kind of hookups for the UPS and that rig.
You spent $42K on the rig, you're also going to need to get an electrician in and have the electrical hookup seen to. The single 4090 I'm running put me right up to that edge where I could use a standard electrical hookup with a 1000w UPS.
So another $1K for the UPS, and $500 worth of electrical work (or more if they have to up the switch to 200 amps, for example).
Not for the faint of wallet.
coffeebeqn 20 hours ago [-]
True enough, depends how good your house electricity panel and wiring is. My basic plugs have 230V x 10A fuses so it would definitely take it to the limit. I also have some 16A which should be fine but I’d have to unplug one of the washing machine or heat pump type things. A 45k machine I would probably run on a UPS like you said. A lot of resistive heating household items use easily 2000W+ so it’s not super exotic but you do need to pay some attention
embedding-shape 2 days ago [-]
Kind of feels like if you're noticing $1K here and $500 there, maybe $42K is too much to start with :)
slowmovintarget 22 hours ago [-]
When you start getting into things like this, there's the initial price, then all the pile on of tax and the infrastructure needed to run thing. It's like getting a boat. For a mere $55K you can get a small one... but that is merely the cost of entry.
So you're right. If the $42K price tag didn't make you blink, the additional $2700 in tax, and the $2K to $3K for the work in your house are going to financial hiccups. But it means the cost to start is $50K, not just the price tag.
Keyframe 2 days ago [-]
$42k and a toy CPU at that. Threadripper or EPIC if it must be AMD camp, although I'd rather it weren't (_always_ some random issues on linux). TBH, if I were buying workstation at such prices, I'd probably first take a look at what HP has (since their cooling and immunity to dust is unprecedented), and then probably BOXX and Puget. For DIY there's always supermicro at such levels.
anaisbetts 2 days ago [-]
That's what I don't understand, they're shipping RTX 6000s that are PCI lane-starved. Buying a Maserati and driving it around in 3rd gear.
Keyframe 2 days ago [-]
it's probably a fantasy setup, not built nor tested. Or worse, someone there things x8/x8 vs x16 GPUs are 'good enough' for this setup at that price.
sourweasel 2 days ago [-]
I've noticed that some companies have flat out dropped the high memory configurations from their lineup that were offered just months ago at inflated prices. OnePlayerX had a 128GB Strix Halo tablet, RedMagic had a 24GB Android phone - both have been removed from their sites. It's a similar situation for the new Thinkpad lineup.
alightsoul 2 days ago [-]
The honor 400 used to be sold for 300 dollars with 12 gb of ram. Now the replacement the honor 600 comes with 8 GB at twice the price.
2 days ago [-]
cute_boi 2 days ago [-]
SSD too. The priced have tripled...
It would be great if China had more lithography machine....
t-3 2 days ago [-]
All memory. SSDs are expensive, microSD is expensive, even spinning rust has inflated.
jaggederest 2 days ago [-]
hoist by our own strategic export controls, rip
vsgherzi 2 days ago [-]
Taiwan would disagree
mcbuilder 2 days ago [-]
I had to do a double take myself, scrolled down and arrived at a similar number.
Wish I built this a few years ago!
slowmovintarget 2 days ago [-]
Yeah, I bought a Thelio Major three years ago. I'm kicking myself for not going dual-4090s for a mere $10K back then.
mandeepj 2 days ago [-]
Hopefully it will come down to that price point or cheaper again in the next 3-4 years?
matheusmoreira 2 days ago [-]
Hopefully... If it doesn't, mere mortals like us consumers are going to be priced out of computers altogether. It'll be the end of the personal computing era and a return to big iron mainframes.
AngryData 10 hours ago [-]
On the highest end maybe. But people have had some sucess DIYing some semi-modern lithography semiconductor production.
slowmovintarget 2 days ago [-]
"All of this has happened before, and will happen again."
felixfurtak 2 days ago [-]
Me three
f6v 2 days ago [-]
And people still don't believe Apple is giving a good deal on Macs.
pulse7 2 days ago [-]
Prompt processing on Macs is VERY slow...
larodi 2 days ago [-]
which engine? cause there are dozens already, and some are quite fast.
embedding-shape 2 days ago [-]
Which one is the fastest and how fast is it? Even the "fast" ones doesn't seem to even reach close to consumer NVIDIA GPUs released years ago when it comes to prompt processing.
anonreplier 2 days ago [-]
large model and slow, or speed and a teeny model, make your choice
embedding-shape 2 days ago [-]
Or you get N of RTX Pro 6000 and get large and fast models :) Make your choice, and do it before the prices go up even more.
larodi 2 days ago [-]
indeed. make up your mind, and also "slow/fast" mean very little given plethora of models to choose from. fast for what, slow for what. fast with which harness, etc...
anonreplier 2 days ago [-]
well yes if money is no object, why not
ekianjo 2 days ago [-]
none of them are Nvidia GPU FAST
thecolorblue 2 days ago [-]
But more memory is available so a larger model can be loaded. It depends on the use case which is better. Any chat or voice model will have better UX with nvidia but document or code generation will be better with apple.
ekianjo 7 minutes ago [-]
it's not a binary thing. At some point Apple becomes too slow with very large models. If you can just run a model at 1 token per second and it takes 30 mins to process a long context, it's useless
CoastalCoder 2 days ago [-]
> Prompt processing on Macs is VERY slow...
That sounds like a contradiction.
/J
menaerus 2 days ago [-]
I wonder what the target audience for this price range is? At that price point I would rather go for a single H100.
lukeschlather 2 days ago [-]
I would rather take this option than a single H100:
Though I don't think there's anything stopping you from trying to stuff an H100 into this machine if you want to BYO.
menaerus 2 days ago [-]
For that price you get the H100. The machine is overpriced, and I'd personally rather go for a second-hand workstation with server-grade components.
embedding-shape 2 days ago [-]
Hopper is ~4 years old at this point though, compared to Blackwell which is ~2 years old, the difference isn't nil.
Depending on your use case, you might prefer native FP4 and FP6 low-precision support and 5th-generation Tensor Cores rather than what Hopper offers.
I think for training Hopper makes sense as it's generally a bit cheaper and the difference isn't that big, but for inference the difference widens a bunch makes a lot more sense to go with Blackwell.
menaerus 2 days ago [-]
You're correct, the difference isn't nil - H100 is data-center grade GPU with loads of bandwidth and compute while RTX PRO is a consumer grade GPU. The difference between the microarchitectures would make sense to point out if we had been comparing apples to apples and not apples to pits.
lukeschlather 2 days ago [-]
I would assume an H100 selling for less than $30k is a scam. I see one for 11,329€, and one for $25k but most are over $30k and I don't know how to judge these random storefronts. You've bought an H100 for $20k that was in working order recently?
menaerus 2 days ago [-]
What kind of comment is that? No, I have not bought the h100 obviously, and, yes, you can buy the new one for 20k. Used one for even less.
rubyn00bie 2 days ago [-]
Anyone know why they chose a consumer grade CPU on this? I’m a little surprised to see a 9950X as the top option. There’s not enough PCIe lanes available to run everything without bifurcation. I imagine the GPUs probably aren’t too bothered generally… but the NVMe drives are likely to slow down as a result.
A Threadripper ain’t cheap, but if you’re spending $45k on a workstation it seems like a weird place to skimp. Not to mention you’d then have the option of shoveling a few more 6000 pros in it when you want to run something larger (assuming your home/office electrical box can support it).
jdboyd 2 days ago [-]
The GPUs might care quite a bit about bifurcation. The RTX Pro 6000s can communicate directly between themselves over PCIe 5.0 16x and saturate the bus doing so. This is used when doing tensor parallel LLM execution for the All-Reduce steps.
Someone can work around this by including a PCIe 5.0 switch like the Broadcom PEX89048. That switch can be used to, for instance, have the two GPUs talk to each other at full PCIe 5.0 16x speeds, while talking to the motherboard at say PCIe 4.0 16x, or even 8x. As an end user, cards using that switch seem to be an AliExpress only item, but I assume an OEM like System76 could get their hands on it to place directly on the motherboard. I imagine the price of adding that to a motherboard could be less than the price of switching to a Threadripper or Xeon 600 platform.
However, I can't find any evidence that System76 has added a switch, and in fact they refer to x8/x8 multi-GPU support, making me think they didn't add a switch. To my mind, this isn't a very satisfactory setup.
dannyw 2 days ago [-]
You need far more absurdly expensive RAM (RDIMM/ECC?) for a threadripper. Like 4x more expensive.
rubyn00bie 2 days ago [-]
Aye but at $45k you’re so deep already is that really worth the savings? It’s so much easier to manage a single system, and the performance will be in absolute terms better.
Truly, I get your point, but $45k is obscene for a workstation (because of the RAM shortage). I can’t fathom it being a reasonable decision[1] unless you’re able to churn an immense profit from it. And… if you can, it’s an expense that saves time, energy, and effort. If it is only profitable when $8k in price difference makes it viable, is that really worth the effort? An $8k increase making a difference, when you’re wholly dependent on the frontier (or near frontier) lab not releasing a model that makes your work(station) irrelevant— seems reckless. Couple that with the doubling of DeepSeek’s parameters for their flash model… and well the math just don’t math for my naive brain.
[1] I’m absolutely incapable of spending that much on a workstation so my opinions may be irrelevant… but I cannot understand the “stepping over quarters to save a penny” mentality[2].
[2] I could be missing the forest for the trees. As a result, I’d love to know how I am being shortsighted. I just can’t fathom a situation where an $8k surcharge in system RAM doesn’t make sense. It’s not about system RAM, it’s about GPU RAM, margins, and useful lifetime of the system.
dannyw 2 days ago [-]
This is a link to their configurable workstation on consumer boards. They offer TR or Xeon based boards too; it's just a different link/SKU: https://system76.com/desktops/thelio-major
So they offer both options.
embedding-shape 2 days ago [-]
> An $8k increase making a difference, when you’re wholly dependent on the frontier (or near frontier) lab not releasing a model that makes your work(station) irrelevant
Hmm, why? I don't understand why better models being available remotely would change the usefulness of what you already have working locally? If it works for you, it works, regardless of what the frontier labs have, and if it doesn't, it doesn't, again regardless.
embedding-shape 2 days ago [-]
Seems they might be selling a combo of dual RTX Pro 6000 workstation cards (non-refundable), but the setup they have, unless they change how the GPUs cooling work, isn't gonna work out thermally if you stack two of those workstation cards on top of each other. But it doesn't say WS or "Workstation", so I suppose could be the server variant too? But also doesn't say...
Hope they have good overall cooling the case, cooling doesn't seem to be mentioned, it'll be a noisy little machine no doubt :)
wmf 2 days ago [-]
They're [Max-Q] blower cards and there's a big gap between them so the cooling looks fine.
embedding-shape 2 days ago [-]
So the two variants they offer is the Max-Q and the Server edition one, but only explicitly mentioned for one?
Edit: The Max-Q is one of the options, what's the other option then?
nightski 2 days ago [-]
Seeing as it adds a second PSU, I'm guessing the workstation one.
embedding-shape 2 days ago [-]
That's bananas, unless they also water cool them, would easily overheat if it's just two bog standard workstation cards on top of each other.
Are they possibly selling setups they haven't actually tested practically in the real world?
rkagerer 2 days ago [-]
Page mentions liquid cooling, for what it's worth.
Tsiklon 2 days ago [-]
That memory speed drop going from single DIMM per channel DDR5 to dual DIMM per channel is very much notable - 3600MT/s is barely any faster than what DDR4 could do in it's final days.
The CPU market seems to be missing the "HEDT" platform we used to enjoy.
tpm 2 days ago [-]
Threadripper (4 or 8 memory channels)
layer8 2 days ago [-]
> Accelerate your AI development with Thelio Mira AI, System76's affordable, GPU-focused workstations
From $3,299 (with just 64 GB RAM and a 4 GB GPU) to over $50,000. Very affordable indeed.
abenga 2 days ago [-]
50K without a proper CPU, mind.
madduci 2 days ago [-]
Insane
pulkas 2 days ago [-]
Thelio Mira AI
$3,299.00
Description
Specs
Warranty
Accelerate your AI development with Thelio Mira AI, System76's affordable, GPU-focused workstations, built for local AI development — so you can train, fine-tune, and iterate challenging AI workloads entirely on your own hardware.
Configure Thelio Mira with up to:
16-core AMD Ryzen 9000 Series CPU
192 GB DDR5 RAM
Dual NVIDIA RTX Pro 6000 GPU
192 GB GPU memory
this is miss leading. actually it is 4gb ram a400 price.
dual RTX6000 price is : $40,538.00
andy99 2 days ago [-]
When I saw the 192GB I first thought they might have beat Framework to market with a Gorgon Halo machine with unified RAM that Framework has teased. This is just a GPU workstation.
halz 2 days ago [-]
I'd really like to just buy some collective shares of a GB300 NVL72 in a datacenter somewhere and have a daily token quota on a shared DeepSeek model or whatever was hot that week. I would just need to find ~100 like-minded folks at this $40k per share price to get started haha.
vitno 2 days ago [-]
I've thought a little bit about this, really I suspect you'd actually want maybe 20 folks for about a 20k buy-in and a dedicated 8×B300. You could get some pretty sweet token/s. Probably a community/coop share structure. It'd be better than the GB300 which is fairly throughput optimized (not needed for 20ish folk)
bitbasher 2 days ago [-]
It only costs $40,000.
jillesvangurp 2 days ago [-]
People routinely buy cars that cost double that. This is a machine that people that typically earn six figure salaries use to do their work on. Exactly the target demographic that would buy/lease expensive cars as well. And then those get used mainly to commute to work.
I don't really need one. But I do have a 4.5K mac book pro for work. Yes it's a bit overpowered. But it's the main vehicle with which I earn a living and it routinely saves me time by blazing through builds and generally doing things quickly for me. I have slightly more GPU on that thing than I need. But it's nice to have the option to experiment with some of the open weight AI models.
embedding-shape 2 days ago [-]
> People routinely buy cars that cost double that.
Maybe it's the "routinely" or the "people" part, but "people" generally don't ever buy a 80K car, it's a small fraction of the population who can afford such luxury cars. And I'm sure the ones who buy cars for 80K, unless they're car enthusiasts or just plain billionaires, don't do so "routinely" either.
coffeebeqn 2 days ago [-]
And you would buy it through your company so it won’t be quite as much net. Surely no one is buying these just for fun
embedding-shape 2 days ago [-]
> Surely no one is buying these just for fun
-_-
Some of us worked hard in life and have spare money, sue me!
mandeepj 2 days ago [-]
the landing page showed "$3,299.00" and I had my candy store moment for a few secs.
jgalt212 2 days ago [-]
5 years ago I could not afford top shelf coders. Now I cannot afford top shelf machines.
tomaytotomato 2 days ago [-]
Can't wait for in 10 years time to find one of these workstations on eBay selling for £100-150
$37,000 in GPUs. Man, I never expected another PC manufacturer to make Apple's top configuration look cheap by comparison.
embedding-shape 2 days ago [-]
Calculate what performance you get per $ spent, and Apple again looks the most expensive out of probably anything else you could buy.
swiftcoder 2 days ago [-]
Are you sure about that? For the price of the top configuration here you can buy a 5-stack of 256GB Mac Studio M5 Ultras.
This dual NVIDIA RTX PRO 6000 setup has 192GB of VRAM at 1.8 TB/s, versus the 5-stack of Macs that collectively have 1.25 TB at 1.2 TB/second... I'm not convinced that equation comes out in Nvidia's favour.
embedding-shape 2 days ago [-]
Right, performance is more than just the aggregated memory speed of the hardware you have. I'm fairly sure, at least last time I looked, maybe Apple launched something new in the last 2-3 months that has completely changed the picture?
swiftcoder 2 days ago [-]
> performance is more than just the aggregated memory speed of the hardware you have
In other fields, sure, but for big LLMs it's a very significant part of the performance picture. That stack of Macs also is going to be able to natively run models 4-5x larger than the dual Blackwells can hold in memory - any model over about 128GB of weights isn't going to fit on the GPUs, and is going to be heavily performance constrained by moving data across the PCIE bus.
> maybe Apple launched something new in the last 2-3 months that has completely changed the picture
Indeed. The M5 Ultra (currently up for pre-order) has 50% higher memory bandwidth than its predecessor, and a claimed 4x improvement in prompt prefill.
embedding-shape 2 days ago [-]
> any model over about 128GB of weights isn't going to fit on the GPUs
Not sure if you misunderstand what GPUs we're talking about, one RTX Pro 6000 has 96GB of VRAM.
> Indeed. The M5 Ultra (currently up for pre-order) has 50% higher memory bandwidth than its predecessor, and a claimed 4x improvement in prompt prefill.
Exciting! Eagerly awaiting the benchmarks and comparisons then. Lets hope "4x improvement" had a good enough baseline so 4x actually ends up useful in practice compared to the current hardware they offer.
> 50% higher memory bandwidth
Seems this lands on ~1.2 TB/s if what Apple claims is correct. For reference, RTX Pro 6000 does 1.8 TB/s, so seems Apple is indeed getting closer incrementally.
swiftcoder 2 days ago [-]
> Not sure if you misunderstand what GPUs we're talking about, one RTX Pro 6000 has 96GB of VRAM.
Right, but the top option here is a pair of RTX Pro 6000s, hence 192 GB of VRAM in total. Should be enough for a 128GB model plus context, cache, etc.
embedding-shape 2 days ago [-]
Yeah, that's why the "any model over about 128GB of weights isn't going to fit on the GPUs" part doesn't make sense, you'll easily be able to run weights in that weight class on two of them. Or did I misunderstand what "on the GPUs" you meant?
swiftcoder 2 days ago [-]
Right, easily run 128-160 GB models, yes. Not easily run models much larger than that. Anything that doesn't fit in the 192 GB (including context, cache, etc) is going to have to be sparse/MoE, and the rather anaemic PCIE bandwidth is going to hurt.
By comparison, the 5x Mac cluster should be able to run a dense ~800GB-1TB model without a drastic slowdown.
rkagerer 2 days ago [-]
What motherboard do they use? (Or do they design their own now?)
xeeeeeeeeeeenu 2 days ago [-]
The ASUS logo is visible in the photo of the innards.
embedding-shape 2 days ago [-]
I'm guessing it'll depend on the GPU (and CPU) choice, putting the cheapest that is sufficient for the choice.
waterTanuki 2 days ago [-]
I find it hard to believe any enterprise willing to drop over $40k on a workstation PC would choose `Pop!_OS 24.04 LTS with the COSMIC Desktop Environment` over Ubuntu.
brendanmc6 2 days ago [-]
Why? It is built from Ubuntu[1]. I am a very happy full-time daily user. Anecdotally it is more stable and productive than Windows ever was for me, and plenty of enterprises use windows…
I bought a spark instead of a motorcycle so it’s somewhat relative and it’s also somewhat relative.
whartung 2 days ago [-]
There was a time when I sold my computer and bought a motorcycle.
Can honestly say that was one of the best trades I’ve ever done. I didn’t buy another computer for 3 more years.
embedding-shape 2 days ago [-]
Sold my computer + DLSR camera back in the day to afford moving countries. Can't imagine what life would have been if I didn't, very happy I did. Lived with a netbook for 2-3 years, but was before I was a professional programmer.
andsoitis 2 days ago [-]
Second-hand car is more rational than new car.
LoganDark 2 days ago [-]
Is it more powerful than Apple silicon, or why would anyone buy this? Just for Linux?
Oh, it's System76, that's why. Open hardware. Makes sense.
Edit: Also Nvidia
kelvie 2 days ago [-]
The prefill rates are way faster on these cards compared to apple silicon, which affects TTFT and therefore usability for a lot of coding tasks, unless you fire and forget most of the time.
numpad0 2 days ago [-]
Mac GPUs supposedly being fastest thing on Earth obsoleting everything NVIDIA is just pure marketing. It was just faster once than some midrange laptop NVIDIA, which conveniently wasn't explicitly marked as different chip sharing the branding with the desktop variant(oof).
The real benefits to going Mac is its quiet and discreet, high wife/CEO acceptance design, and their huge GPU-assignable shared RAM.
If you're okay with your desk being the wing top of a flying airplane, and 3M Peltor or David-Clark is your favorite working time headphone brand anyway, then your build will be faster and cheaper than a maxed out Mac Studio.
LoganDark 2 days ago [-]
For me the operating system (and ecosystem in general) is one of the biggest reasons I use a Mac, but the unified memory pool is a benefit too. I am still distraught about the gaming situation though.
wmf 2 days ago [-]
It is more powerful than Apple.
LoganDark 2 days ago [-]
I wonder why Apple has not really pushed their memory bandwidth numbers. They're starting to catch up to 2020 at this point -- cool, but still struggling to run years-old models. (FLOPS is even further behind by a year or two)
Maybe they're waiting on HBM? Given that they're skipping M6 Max to go straight for M7, I'd assume they're doing a redesign for it.
wmf 2 days ago [-]
GDDR has higher latency and much lower capacity. HBM requires an interposer and also has lower capacity. It's not an easy choice. The M7 family will probably double memory bandwidth using LPDDR6.
LoganDark 2 days ago [-]
So the MacBooks might get roughly the M5 Ultra's bandwidth in a couple years. Not bad -- my M4 Max is currently stuck in 2016
ethin 2 days ago [-]
I absolutely love system76. Never owned one of their desktop machines, but I have a couple of their older laptops and they work fantastically for me. As in I can easily get a decade out of them and replace the batteries every few years. One of them is almost a decade old ironically enough.
cherioo 2 days ago [-]
This makes Mac Studio with 256GB memory look cheap in comparison. More memory to store weight at a quarter of the price!
colordrops 2 days ago [-]
It's gonna have WAY faster token rates than that Mac Studio though, enough to be a qualitative rather than quantitative difference.
GeekyBear 2 days ago [-]
At that price, it's competing against a cluster of M5 Mac Studios, which we don't have test data for yet.
b112 2 days ago [-]
Can't it be both?
EG As I gazed at Janice, joy with equal part contentment washed through my soul.
nubinetwork 2 days ago [-]
Nobody seems to have noticed that they dropped their ampere-based thelio.
thom 2 days ago [-]
I remember losing sleep when I bought my RTX Pro 6000, but somehow they keep going up in price, and waddya know I’ve even done real work that appreciates the VRAM size.
embedding-shape 2 days ago [-]
Same. Had to convince the wife, initially aimed to get two of them, ended up with one. Now my wife is berating me for not convincing her that I should have bought four and then sell two later... Can't win :)
fsuts 2 days ago [-]
Speaking of AMD CPU’s, how is Strix Halo coming along?
As that’s unified memory à la Apple and I think up to 128gb
chaostheory 2 days ago [-]
How is this interesting compared to either a Mac Studio or Nvidia Spark? Did I miss something?
wmf 2 days ago [-]
This is a tier above the best Mac Studio and two tiers above the Spark.
coffeebeqn 2 days ago [-]
Big GPUs
arjie 2 days ago [-]
Haha, bloody hell, mate. $17k a pop? I bought these for $8k-ish. That's bonkers to buy this when newer flash models won't fit on this any more. DeepSeek V4 Flash seems the last in its line, and then you have to count on Qwen Next. Seems like a waste honestly.
heronbank 2 days ago [-]
k is single H100 money. The workstation form factor only makes sense if you're locked out of cloud GPU allocation.
Artoooooor 2 days ago [-]
Damn, what a misleading price.
senectus1 2 days ago [-]
wouldnt you be better off maxing out the new Mac hardware? the memory bandwidth is insane on those jobbies
rubyn00bie 2 days ago [-]
There’s roughly 50% more memory bandwidth on an RTX 6000 pro versus the M5 Ultra. I imagine it really comes down to what you’re doing, I know my 5090 runs laps around the M5 Pro I have when it comes to local LLM performance (until it runs out of memory).
If it wasn’t for the fact the 6000 Pro is selling for like 100% ($9,000) over MSRP it would probably look a lot more reasonable.
wmf 2 days ago [-]
Two RTX 6000 have more FLOPS and more memory bandwidth than a M5 Ultra (but they also cost more).
bigyabai 2 days ago [-]
If your goal is pure GPU compute, you should have got a 5090. It can bench comparable to the RTX 6000 Pro cards and is near guaranteed to outperform M5 Ultra.
fsuts 2 days ago [-]
Depends on your use case
If running models then Apple is fine, if training or tuning or large number of users then Nvidia is king
protocolture 2 days ago [-]
Do I want one? Yes.
Can I afford it? Absolutely not.
poppafuze 2 days ago [-]
if it goes over pcie to get to each other, it's not really "gpu memory"
rrgok 2 days ago [-]
Is this just a marketing stunt? From a completely ignorant user, doesn't AI reeuqires high bandwidth ram (like the GPU)? Is really DDR5’s bandwidth enough?
zepearl 2 days ago [-]
> 192 GB GPU memory
rrgok 2 days ago [-]
How did I miss that? I was still sleeping I guess
zepearl 2 days ago [-]
Hehe - yeah, I was pretty sure that you just overlooked it - pointing that out was irresistible :o)
That's a bitter pill to swallow. Most of this is the cost of the Nvidia card, and the RAM.
Still, you're right, I have a 1200W PSU for a single RTX Pro 6000, cost ~500 something, so still the PSU shouldn't be that large part of the total budget.
But you'll find most household power sockets aren't actually rated for the kind of power these things draw at peak. If you go for a dual GPU you're going to want to sustain a higher current. Think washer / drier / fridge kind of hookups for the UPS and that rig.
You spent $42K on the rig, you're also going to need to get an electrician in and have the electrical hookup seen to. The single 4090 I'm running put me right up to that edge where I could use a standard electrical hookup with a 1000w UPS.
So another $1K for the UPS, and $500 worth of electrical work (or more if they have to up the switch to 200 amps, for example).
Not for the faint of wallet.
So you're right. If the $42K price tag didn't make you blink, the additional $2700 in tax, and the $2K to $3K for the work in your house are going to financial hiccups. But it means the cost to start is $50K, not just the price tag.
It would be great if China had more lithography machine....
Wish I built this a few years ago!
That sounds like a contradiction.
/J
> 96 GB Dual NVIDIA RTX PRO 5000 w/ 1000W PSU (non-refundable) +$19,169
Though I don't think there's anything stopping you from trying to stuff an H100 into this machine if you want to BYO.
Depending on your use case, you might prefer native FP4 and FP6 low-precision support and 5th-generation Tensor Cores rather than what Hopper offers.
I think for training Hopper makes sense as it's generally a bit cheaper and the difference isn't that big, but for inference the difference widens a bunch makes a lot more sense to go with Blackwell.
A Threadripper ain’t cheap, but if you’re spending $45k on a workstation it seems like a weird place to skimp. Not to mention you’d then have the option of shoveling a few more 6000 pros in it when you want to run something larger (assuming your home/office electrical box can support it).
Someone can work around this by including a PCIe 5.0 switch like the Broadcom PEX89048. That switch can be used to, for instance, have the two GPUs talk to each other at full PCIe 5.0 16x speeds, while talking to the motherboard at say PCIe 4.0 16x, or even 8x. As an end user, cards using that switch seem to be an AliExpress only item, but I assume an OEM like System76 could get their hands on it to place directly on the motherboard. I imagine the price of adding that to a motherboard could be less than the price of switching to a Threadripper or Xeon 600 platform.
However, I can't find any evidence that System76 has added a switch, and in fact they refer to x8/x8 multi-GPU support, making me think they didn't add a switch. To my mind, this isn't a very satisfactory setup.
Truly, I get your point, but $45k is obscene for a workstation (because of the RAM shortage). I can’t fathom it being a reasonable decision[1] unless you’re able to churn an immense profit from it. And… if you can, it’s an expense that saves time, energy, and effort. If it is only profitable when $8k in price difference makes it viable, is that really worth the effort? An $8k increase making a difference, when you’re wholly dependent on the frontier (or near frontier) lab not releasing a model that makes your work(station) irrelevant— seems reckless. Couple that with the doubling of DeepSeek’s parameters for their flash model… and well the math just don’t math for my naive brain.
[1] I’m absolutely incapable of spending that much on a workstation so my opinions may be irrelevant… but I cannot understand the “stepping over quarters to save a penny” mentality[2].
[2] I could be missing the forest for the trees. As a result, I’d love to know how I am being shortsighted. I just can’t fathom a situation where an $8k surcharge in system RAM doesn’t make sense. It’s not about system RAM, it’s about GPU RAM, margins, and useful lifetime of the system.
So they offer both options.
Hmm, why? I don't understand why better models being available remotely would change the usefulness of what you already have working locally? If it works for you, it works, regardless of what the frontier labs have, and if it doesn't, it doesn't, again regardless.
Hope they have good overall cooling the case, cooling doesn't seem to be mentioned, it'll be a noisy little machine no doubt :)
Edit: The Max-Q is one of the options, what's the other option then?
Are they possibly selling setups they haven't actually tested practically in the real world?
The CPU market seems to be missing the "HEDT" platform we used to enjoy.
From $3,299 (with just 64 GB RAM and a 4 GB GPU) to over $50,000. Very affordable indeed.
Configure Thelio Mira with up to:
this is miss leading. actually it is 4gb ram a400 price.dual RTX6000 price is : $40,538.00
I don't really need one. But I do have a 4.5K mac book pro for work. Yes it's a bit overpowered. But it's the main vehicle with which I earn a living and it routinely saves me time by blazing through builds and generally doing things quickly for me. I have slightly more GPU on that thing than I need. But it's nice to have the option to experiment with some of the open weight AI models.
Maybe it's the "routinely" or the "people" part, but "people" generally don't ever buy a 80K car, it's a small fraction of the population who can afford such luxury cars. And I'm sure the ones who buy cars for 80K, unless they're car enthusiasts or just plain billionaires, don't do so "routinely" either.
-_-
Some of us worked hard in life and have spare money, sue me!
Then I will do a video of me playing Crysis on it
This dual NVIDIA RTX PRO 6000 setup has 192GB of VRAM at 1.8 TB/s, versus the 5-stack of Macs that collectively have 1.25 TB at 1.2 TB/second... I'm not convinced that equation comes out in Nvidia's favour.
In other fields, sure, but for big LLMs it's a very significant part of the performance picture. That stack of Macs also is going to be able to natively run models 4-5x larger than the dual Blackwells can hold in memory - any model over about 128GB of weights isn't going to fit on the GPUs, and is going to be heavily performance constrained by moving data across the PCIE bus.
> maybe Apple launched something new in the last 2-3 months that has completely changed the picture
Indeed. The M5 Ultra (currently up for pre-order) has 50% higher memory bandwidth than its predecessor, and a claimed 4x improvement in prompt prefill.
Not sure if you misunderstand what GPUs we're talking about, one RTX Pro 6000 has 96GB of VRAM.
> Indeed. The M5 Ultra (currently up for pre-order) has 50% higher memory bandwidth than its predecessor, and a claimed 4x improvement in prompt prefill.
Exciting! Eagerly awaiting the benchmarks and comparisons then. Lets hope "4x improvement" had a good enough baseline so 4x actually ends up useful in practice compared to the current hardware they offer.
> 50% higher memory bandwidth
Seems this lands on ~1.2 TB/s if what Apple claims is correct. For reference, RTX Pro 6000 does 1.8 TB/s, so seems Apple is indeed getting closer incrementally.
Right, but the top option here is a pair of RTX Pro 6000s, hence 192 GB of VRAM in total. Should be enough for a 128GB model plus context, cache, etc.
By comparison, the 5x Mac cluster should be able to run a dense ~800GB-1TB model without a drastic slowdown.
[1]https://system76.com/support/difference-between-pop-ubuntu
Can honestly say that was one of the best trades I’ve ever done. I didn’t buy another computer for 3 more years.
Oh, it's System76, that's why. Open hardware. Makes sense.
Edit: Also Nvidia
The real benefits to going Mac is its quiet and discreet, high wife/CEO acceptance design, and their huge GPU-assignable shared RAM.
If you're okay with your desk being the wing top of a flying airplane, and 3M Peltor or David-Clark is your favorite working time headphone brand anyway, then your build will be faster and cheaper than a maxed out Mac Studio.
Maybe they're waiting on HBM? Given that they're skipping M6 Max to go straight for M7, I'd assume they're doing a redesign for it.
EG As I gazed at Janice, joy with equal part contentment washed through my soul.
As that’s unified memory à la Apple and I think up to 128gb
If it wasn’t for the fact the 6000 Pro is selling for like 100% ($9,000) over MSRP it would probably look a lot more reasonable.
If running models then Apple is fine, if training or tuning or large number of users then Nvidia is king
Can I afford it? Absolutely not.