deleted by creator
It makes sense to me; AI needs GPUs, and there’s an AI boom.
Probably won’t be as bad as the crypto situation, but I imagine it will have an effect on GPU availability at some point.
The AI boom started last year, it’s the reason prices are already high. So I’m skeptical that they are going to get higher, seeing as this whole AI thing feels like a bubble.
true, if they are high, it’s right now, a lot of news debunking AI, regulations, chat gpt geting worse etc are appearing
People are skeptical about AI, but companies seem to be less so.
They see an opportunity to save money and they’re gonna go for it, with the ship steered by shareholders who like the “AI” buzzword.
Maybe it won’t have any effect on GPU prices though. Crypto was a much more accessible market for anyone who wanted to have a go at it, whereas you do need to actually have an idea to make use of AI.
AI needs GPUs with a huge amount of vram. Nvidia moves to put criminally low amount of VRAM in their low and mid range gaming GPUs will ensure those GPUs won’t be hoarded by companies for AI stuff. Whether gamers actually want to buy those GPUs with such low amount of VRAM is a different matter though.
Ai cards need to be efficient so they need tsmc 3n and 5n fabs. Desktop cards don’t so they could use Samsung or older cheap tsmc fabs. When we had the shortages before it was the smaller components like vrm stages and capacitors that had a shortage. Those are now over supplied. There is no reason for the price hikes other than Nvidia seeing what people paid to scalpers and wanting it for themselves.
Nvidia has continued to push for more clock speed on lower end parts and charging higher prices. There is no reason a mid range die should be clocked to 3ghz on a super expensive pcb like the 4080 has or a low end part doing it on the 4070. Those both have pcb that cost more than the die for a die that would historically be used in $150-400 parts when adjusted for inflation. They also both should be clocked around 1.8-2ghz as that has a 60-70% reduction their power consumption for a 30% performance loss (see the mobile parts for what those parts should be close to for their base sku.)
Why would AI cards need to be more efficient?
Data centers care a lot about power. The ai products run around 2ghz in the sweet spot. Consumer cards target 3ghz this gen and use 3-4x the power that they do at ~2ghz. The die in the 4080 is a mid range size. It is what used to be in things like the 60 series or maybe a 70 series card. They have been overclocking the snot out of them stock and putting them on massively expensive pcb instead of giving us the larger dies we used to get. That shifts the costs to the board partners and lets them get away with selling the dies at a huge profit compared to their older products.
Back to data centers. You pay a lot for your spot based on power and location. If they stay efficient and pack lots of chips in, that is the cheapest way over the life of the server. If you save 10 or 20% power due to using a new node that is worth a huge reduction in data center fees. On the consumer desktop side, they can overclock to double the power instead of using a larger more expensive die and pocket the difference with no one really caring.
Tsmc 4n is a 6nm process based on improving their 7nm. 3n and 5n are both experimental process. 5n is smaller than 4n at lower density. The consumer cards are 7nm then 4n (the cheap ones). The data center cards are 5n and 3n (the high end expensive processes.) Ordering more consumer or data center do not conflict with each other. Doing more of the workstation cards could since they are full feature consumer dies, but those are not the ai cards.
AI is much more taxing than gaming. Machine learning will peg a gpu at a flat 100% constant use, while gaming fluctuates up and down depending on what’s going on on screen. So being more power efficient while running a card at 100% 24/7 saves money on power costs, and corporations love saving money.
Guess I’m keeping my 1070 for another couple years
1070 gang
deleted by creator
Then I will play retro games until the universe falls to darkness.
Any day now
again?
deleted by creator
Not sure if you’re asking what they mean by “again”, but the prices were just starting to come down after the cryptobros began hording them.
Nvidia is shifting alottment to AI, they know as long as they hold mind share, people will buy Nvidia over AMD and Intel. AMD adjusts their pricing to Nvidia, but is more prone on getting price drops after launch (e.g how popular the 7900xt was when it price dropped to 700)
What’s the point of really expensive hardware and hard to run games if only a small percentage of people can play them? It’s not just that PC parts are getting better, but devs are getting lazier regarding optimizations because they have more headroom than ever. We keep running into new problems like running out of VRAM or shader compilation stuttering. Long live the source engine that has spent more development time on optimizing than actually producing games.
Remember when PC gaming used to be hands down better than console?
Anyone who still believes that is stuck in the past.
PC certainly has its benefits and is my platform of choice, but if somebody was getting into gaming or just casually interested then a PS5 is a much better choice these days.
Yeah when a single component can cost up to 3x the price of a console then it’s no longer worth it. I’d like to upgrade my GPU to get stable 80+fps on all games max graphics but when that would cost me $1000-$1500, I’d rather just play on medium graphics.
But aren’t the GPUs used by AI different than the GPUs used by gamers? 8GB of RAM isn’t enough to run even the smaller LLMs, you need specialized GPUs with 80+GB like A100s and H100s.
The top-tier consumer models like the 3090 and 4090 have 32GB, with them you can train and run smaller LLMs locally. But there still isn’t much demand to do that because you can rent GPUs on the cloud for cheap; enough that the point where renting exceeds the cost of buying is very far off. For consumers it’s still too expensive to fine-tune your own model, and startups and small businesses have enough money to rent the more expensive, specialized GPUs.
Right now GPU prices aren’t extremely low, but you can actually but them from retailers at market price. That wasn’t the case when crypto-mining was popular
They’re not that different, really. CUDA processing cores are the most used in AI training, and those are the main processors used in both Nvidia’s consumer desktop cards and machine learning enterprise cards. As “AI” is on the rise, more and more of the supply of CUDA processors and VRAM chips will be diverted to enterprise solutions that will fetch a higher price from deals with corporations. Meaning there will be less materials available for the consumer-level GPU supply, which will drive prices up for normal consumers. NVIDIA has been banking on this for a long time; that’s why they don’t care about overpricing the consumer market and have been trying to push people towards cloud-based GeForce Now subscription models where you don’t even own the hardware and just basically rent the processing power to play games.
Also just to be anal, the 3090 and 4090 have 24Gb of vram, not 32Gb. And unlike gaming nowadays you can distribute the workload to multiple GPU’s in one system, or over a network of machines.
Not all applications of it are as demanding as LLMs though. The company referenced in the article as buying likely upwards of twenty million dollars worth of graphics cards seems to work mostly on driver assistance (presumably with the goal of full self-driving at some point, but I haven’t looked at them thoroughly). It seems quite possible that however their product works is within the capacity of a GPU that would otherwise be popular for gaming, and it being installed in cars means that it has to have the capabilities in the physical device that’s in the car
