![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Nvidia making 1,000% profit on H100 accelerators
- Thread starter Agent_N
- Start date
Or maybe the Dutch company AMSL. Only issue for TSMC right now is their location in Taiwan. What happens when (not if) China invades?
I can only hope that the long-range planners at TSMC are making plans to move both personnel and equipment to another country. US or Canada? Or somewhere in Europe not too close to Russia.
And exactly who pays Dell for this 2TB?
Depends on who’s buying really.
I don’t actually remember the last time I paid the advertised price on those sorts of things, so certainly not me.
But if you were to dissect that 2TB drive it’s probably actually 3TB and there is a controller there doing internal hash verification and storing checksums.
So just calling them a 2TB drive is a bit of an understatement.
Twisted Kidney
Supreme [H]ardness
- Joined
- Mar 18, 2013
- Messages
- 4,112
Very true, I think 5nm was developed in a Texas university or some such as well. But the lack of diversity in the actual supply chain of higher end technology is pretty scary. There's a real winner-takes-all thing happening in the tech sector, I have to wonder if it's hurting us in the long term.
4saken
[H]F Junkie
- Joined
- Sep 14, 2004
- Messages
- 13,204
How to say you don't understand enterprise tech/architecture w/o saying it.
Last edited:
And that’s where the L40 comes in.
In an interesting twist of fate, NVIDIA has flipped from being a graphics company to a compute company so quickly that their revenues are now imbalanced in the other direction. 77% of revenue is compute & networking; only 23% comes from graphics
https://twitter.com/RyanSmithAT/status/1694475294021984697?s=20
NVIDIA Reports Q2 FY2024 Earnings: $13B Revenue Blows Past Records On Absurd Data Center Demand
In the span of just a year, NVIDIA has gone from earning $6 billion a quarter in revenue to keeping $6 billion in revenue as profits
https://www.anandtech.com/show/2002...ows-past-records-on-absurd-data-center-demand
this is a quarter for the record books, as NVIDIA has set new records for everything from revenue to net income.
Driven by their highly profitable, high-margin data center products, NVIDIA achieved a GAAP gross margin of 70.1% for the quarter. Coupled with their record revenue, this has resulted in NVIDIA booking a blistering $6.1B in net income, an 843% improvement over Q2’22, and even more than trebling their net income versus just the previous quarter.
The company, handily beating their already very bullish $11B revenue projection for Q2, is projecting a further 18%+ jump in revenue for Q3, to $16B in revenue. Which, if NVIDIA’s projections pan out, would afford a 71.5% GAAP gross margin.
https://twitter.com/RyanSmithAT/status/1694475294021984697?s=20
NVIDIA Reports Q2 FY2024 Earnings: $13B Revenue Blows Past Records On Absurd Data Center Demand
In the span of just a year, NVIDIA has gone from earning $6 billion a quarter in revenue to keeping $6 billion in revenue as profits
https://www.anandtech.com/show/2002...ows-past-records-on-absurd-data-center-demand
this is a quarter for the record books, as NVIDIA has set new records for everything from revenue to net income.
NVIDIA Q2 FY2024 Financial Results (GAAP) | |||||
Q2 FY2024 | Q1 FY2024 | Q2 FY2023 | Q/Q | Y/Y | |
| Revenue | $13.5B | $7.2B | $6.7B | +88% | +101% |
| Gross Margin | 70.1% | 64.6% | 43.5% | +5.5ppt | +26.6ppt |
| Operating Income | $6.8B | $2.1B | $499M | +218% | +1263% |
| Net Income | $6.1B | $2.0B | $656M | +203% | +843% |
| EPS | $2.48 | $0.82 | $0.26 | +202% | +854% |
Driven by their highly profitable, high-margin data center products, NVIDIA achieved a GAAP gross margin of 70.1% for the quarter. Coupled with their record revenue, this has resulted in NVIDIA booking a blistering $6.1B in net income, an 843% improvement over Q2’22, and even more than trebling their net income versus just the previous quarter.
The company, handily beating their already very bullish $11B revenue projection for Q2, is projecting a further 18%+ jump in revenue for Q3, to $16B in revenue. Which, if NVIDIA’s projections pan out, would afford a 71.5% GAAP gross margin.
Last edited:
They were rumours yesterday that they would beat the absurdly high record expectation (apparently gaming card a la 4060 are selling really well in Asia) but by that much ! Data center alone almost did the $11B they were talking about.
$16 billions in Q3.... that would be %270 of the last Q3 of 5.9B.....
$16 billions in Q3.... that would be %270 of the last Q3 of 5.9B.....
NVIDIA Market Segment Results
NVIDIA Market Platform Results, Q2 FY2024 (GAAP) | |||||
Q2 FY2024 | Q1 FY2024 | Q2 FY2023 | Q/Q | Y/Y | |
| Data Center | $10,323M | $4,284M | $3,806M | +141% | +171% |
| Gaming | $2,486M | $2,240M | $2,042M | +11% | +21% |
| Professional Visualization | $379M | $295M | $496M | +28% | -24% |
| Automotive | $253M | $296M | $220M | -15% | +15% |
| OEM & IP | $66M | $77M | $140M | -14% | -53% |
Diving into the performance of NVIDIA’s individual market segments, the bellwether of NVIDIA’s product portfolio remains their data center segment. That segment posted $10.3B in revenue for Q2, not just setting a new segment record, but smashing the old record in the process.
"Q2 is the first quarter where NVIDIA has been shipping Hopper architecture products in large volumes – meaning that Q2 is essentially the start of the Hopper architecture era from and NVIDIA sales perspective.
And should NVIDIA beat their own projections by more than a fraction, then the company will book more revenue in Q3’24 than they did in all of FY2021."
I cannot help but laugh at stock analysts that interpret Nvidia against most other companies where they think that Nvidia is due for a correction and their P/E is so high blah blah blah. Nvidia is just getting started and companies are beating down Nvidia's door and throwing money at them for AI accelerators. When Jensen sets his mind to do something he goes all out.
the gaming space is so small compared to that enterprise space, a lot of people in this forum make me chuckle when they think nvidia will go down due to bad card sales or when they release terrible overpriced cards (everything other than 4090 has been junk lately)
MLID claims that even the gaming GPUs of Nvidia are being grabbed up by China (for AI purposes) before Biden/USA wise up to it !!!
You are thinking like stock price is really a matter of business fundamentals. These days it's mostly about predicting where the money will flock to and flee from, and that seldom has much to do with solid business fundamentals. Infinite price run up requires infinite suckers, finite suckers with infinite money, or helicopter money from the feds. Hell, it could just be that they know that, and are very concerned about being the bag holders on other tech stocks, so they are just spreading some fud to try to stem the bleeding of something else they are holding.
However, it honestly wouldn't take much to cause a correction, even just a healthier overall market would likely do that. But in addition to that nvidia's revenue at this point is basically just an indicator of their ability to get sufficient HBM supplies. Nvidia can be doing everything right, and rationally their price should be heavily based on the health and decisions of their HBM suppliers unless they pivot to a new bottleneck. Their order queue is insanely deep at this point, and that's a risk, because the math for a competitor isn't is brand X better than nvidia, it's is brand X better than nothing. Then there's the secondary risk in that queue depth, which is how many companies can wait out the queue having nothing and still be able to pay when their slot comes due given current market conditions, interest rates, etc. I mean look at very high end off the shelf networking hardware circa 1999-2005 ish. That stuff was a money press because internet and data centers. And then all the really big names just started rolling their own because the cost and margins and optimizations. Literally every large customer of nvidia for machine learning has joined up in various consortiums to design their own chips and optimizer core software for not just nvidia, that's ignoring the blood in the water for competition that a 10,000% margin will cause. Anything that cuases their growth to be checked will have serious money taking a payday even if they come right back in at a lower price.
The gaming gpus are just as capable as the enterprise ones, but for significant cost difference. I do this work daily and I run it on a variety of hardware from dgx down to laptop gpu. It doesn't surprise me China would bypass enterprise restrictions by snapping up consumer gpus.
Bankie
2[H]4U
- Joined
- Jul 27, 2004
- Messages
- 2,476
Way way back in the late 90s I had just graduated high school and my dad asked me, "If I gave you $1500 to invest what would you put it in to?". I replied with, "I don't know; Nvidia I guess. But I don't want your money dad." That would be worth $350,000+ now. 
Furious_Styles
Supreme [H]ardness
- Joined
- Jan 16, 2013
- Messages
- 4,581
Assuming it was $1/share back then (maybe less) it would be more like 700k++Way way back in the late 90s I had just graduated high school and my dad asked me, "If I gave you $1500 to invest what would you put it in to?". I replied with, "I don't know; Nvidia I guess. But I don't want your money dad." That would be worth $350,000+ now.
I'm not thinking Nvidias stock price is a matter of business fundamentals, the analysts might be, which might explain the articles prior to Q2 earnings where some thought Nvidia was going to tank in stock price becuse their P/E ratio is so high. I am saying "no", they will grow into their P/E ratio until they get some serious competition from other companies producing AI hardware.
It can depend on how the dividend were handled over time, but $1500 of Nvidia bought in january 1999 could be around 1.7 million now, a quarter back than was like 40 millions in revenues. I think the stock splitted 4 time since, you would have 12 of them for every one you bought in 99.
Even in 2014, in a drip that re-invest dividend automatically, you have 1 millions for each $10,000
It is both
1) how much goes into the stock market and
2) who in the stock market get it
Look at the top 5 market valuation, look at the bottom 5 market valuation of the S&p 500
Top 5: Apple, Microsoft, Amazon, Nvidia, Google
bottom 5: advanced auto pats, Lincoln Naitonal Corps, Newell brans, New corp, Fortrea Holdings
If I look at my day to day interaction with the world and look at those company revenues, profit, margin, moat and other business fundamental I am sure we would see a giant link between their respective valuation.
Which one will be the biggest company of the world in 20 years, way better chance to be one of the top 5 than one of the bottom 5.
If you invest with a 20+ years horizon, you do not have to think or try to predict the money flock-flee fluctuation, picking Nvidia instead of Matrox or Apple instead of HP in 1999 will be a much bigger factor (say 1000:1) than the current treasury yield has (say 2:1) and will be very much based a lot on those company difference in their business fundamentals.
Last edited:
NamelessPFG
Gawd
- Joined
- Oct 16, 2016
- Messages
- 893
That is some insane profit margin, and if they're actually getting it from companies with deep pockets for GPU compute, I can't really blame them.
NVIDIA now really reminds me of where SiliconGraphics was in the early-mid '90s, the one company that everyone had eyes on for when you needed the absolute most performance at any cost (and cost you it did, anywhere from $30,000~ to $200,000~ to literal millions in '90s dollars), and people gladly paid it to get that competitive advantage that they couldn't have anywhere else.
NVIDIA now really reminds me of where SiliconGraphics was in the early-mid '90s, the one company that everyone had eyes on for when you needed the absolute most performance at any cost (and cost you it did, anywhere from $30,000~ to $200,000~ to literal millions in '90s dollars), and people gladly paid it to get that competitive advantage that they couldn't have anywhere else.
staknhalo
Supreme [H]ardness
- Joined
- Jun 11, 2007
- Messages
- 6,924
Oh I just posted this in the Finance thread in Genmay lol:
https://www.semianalysis.com/p/google-gemini-eats-the-world-gemini
https://www.semianalysis.com/p/google-gemini-eats-the-world-gemini
They post this like using the latest and greatest of hardware hasn't been a major recruiting tactic for every R&D facility for the past 200 years.
staknhalo
Supreme [H]ardness
- Joined
- Jun 11, 2007
- Messages
- 6,924
Yeah, but this time it's Nvidia lol that's what we're talking about
In my early days, it was Sun Ultra SPARC clusters and electron microscopes... The more things change the more they stay the same.
staknhalo
Supreme [H]ardness
- Joined
- Jun 11, 2007
- Messages
- 6,924
Nah like the guy above, SGI will always be the one I think about
1) Purple servers edit: "workstations"
2) They both made the dinosaurs for the movie Jurassic Park and were good enough for Jurassic Park the fake company to use in the movie
What's not to love?
1) Purple servers edit: "workstations"
2) They both made the dinosaurs for the movie Jurassic Park and were good enough for Jurassic Park the fake company to use in the movie
What's not to love?
Blade-Runner
Supreme [H]ardness
- Joined
- Feb 25, 2013
- Messages
- 4,410
Unsurprisingly there is fuckery afoot....
View: https://youtu.be/K6xALgeaSZM
Some unknown company lent $2.3 billion (including from Nvidia) to buy chips used as collateral for the very same loan. Looks like a giant pump and dump scam.
View: https://youtu.be/K6xALgeaSZM
Some unknown company lent $2.3 billion (including from Nvidia) to buy chips used as collateral for the very same loan. Looks like a giant pump and dump scam.
Blade-Runner
Supreme [H]ardness
- Joined
- Feb 25, 2013
- Messages
- 4,410
I think where he get wrong is about how well H100 old their value right now, for a while can probably resale them at a better price on the used market than new, same for the value of having a direct line with Nvidia, the only having 10 of thousands of quasi-impossible to buy Nvidia GPU being thrown as if it was nothing or something other company with money could do.
Last edited:
Blade-Runner
Supreme [H]ardness
- Joined
- Feb 25, 2013
- Messages
- 4,410
A company with no existing history, business dealings, infrastructure, employees or logistical chains is now going to offload 2.3 BILLION in inventory, which also happens to the collateral for the loan used to buy those chips....yes lets see how they manage to do that.
Would it not be the bank job to sell them ?
Maybe it is not 100% similar to Inflection AI because that led by a well know superstar, but that a 2022 company that got 1.3 billions, capital in AI venture can be wild.
Those loans are probably considered not that risky with how well those gpu can be resold, but with not that much upside either with how much of the funding/spending of those startup goes in hardware.
People involved in it has vast history too (CEO or very high level exec from github, apple), Nvidia and they have a contract with Microsoft to power openAI needs:
https://www.cnbc.com/2023/06/01/microsoft-inks-deal-with-coreweave-to-meet-openai-cloud-demand.html
I can imagine them simply buying them (at the gpu raw price) if needed, could be some pumping, put could be the classic putting a lot of expense/low margin business in made up tierce entity to help boost margin on the main one that just happen to pump as well.
Last edited:
ManofGod
[H]F Junkie
- Joined
- Oct 4, 2007
- Messages
- 12,880
Would this number include long term hardware and software support? If not, that would make the profit far less than 1000%.
This is raw hardware margin on sale I think, does not include amortized R&D/software cost that went into them before sale either.
H100 Gpus come with an 5 years enterprise support.
Come with is a strong word, depending on what you are doing with them you get into their various licenses after that for specific support, and woof, those are not small numbers.
This was my first thought if we include everything, including paid support, access to their cloud service, etc... margin could end up higher than basic provided customer service cost.
Like I pay about $800 per server GPU per year extra for my Grid licenses.
And another $200 or so per GPU per year for Nvidia VMWare support which is above the VMWare Enterprise licensing.
Dell/HP/Lenovo also tack on extra support things on there as well for the servers when they ship them with those classifications of GPUs and how much of that goes to them specifically or gets attached back to Nvidia for other things I can't say.
Microsoft could replace Nvidia's GPUs with custom designed hardware, in its data center servers currently powering cutting-edge LLMs for cloud customers, including OpenAI and Intuit, as well as for AI features in Microsoft’s productivity etc.
A report from Reuters details OpenAI's plans to solve the hardware gridlock the AI industry faces.
According to Reuters, if OpenAI increased its query volume to 1/10th of Google's over time, it would require $48 billion in GPUs to scale to that level. Beyond that, it would need to spend $16 billion annually just to keep up with demand. This is an existential problem for the company and the industry at large. It's also good news for Nvidia, which reportedly earns up to 1,000% margins on every H100 it sells.
The path being examined by OpenAI is also not new, as Amazon, Google, and Meta use custom chips designed for their specific needs. Its biggest partner, Microsoft, is reportedly working on its own custom silicon. However, it would take OpenAI years to get new silicon designed, manufactured, and installed, so it'll still have to buy chips from Nvidia or AMD in the interim.
https://www.extremetech.com/computing/openai-mulls-manufacturing-its-own-ai-chips-chatgpt
Red Falcon
[H]ard DCOTM December 2023
- Joined
- May 7, 2007
- Messages
- 12,471
Except the gaming GPUs are lacking further FP64, ECC, more than 24GB VRAM, etc.
NVIDIA isn't going to cannibalize their enterprise and AI market with their GeForce line, and this isn't 2005 where a simple BIOS flash is going to change a GeForce into a Quadro.
So other than potentially FP32, in what ways are gaming GPUs just as capable as the enterprise ones?