One of the most famous leakers for NVIDIA GPUs has just confirmed a key detail about the upcoming next-generation Ada Lovelace GPUs from the company. The brand new architecture will be built on TSMC's 5nm process. This is massive news if true - and let's talk about that for a second. Kopite has leaked pretty much all the details of the Turing and Ampere architectures and has historically been extremely accurate. That said, as always with leaks and before multiple confirmations, a grain of salt never hurt anyone.
NVIDIA Ada Lovelace GPUs will "come out a little bit earlier" according to industry insider
Kopite had already previously confirmed that the Ada Lovelace architecture would be built on the 5nm process, but it was assumed that this would be a Samsung node. NVIDIA had made the (correct) decision to transition to Samsung for foundry services because it had (accurately) forecasted the demand that would be hitting TSMC. Having to share resources with Apple and AMD was a gamble and NVIDIA's decision to go with Samsung appears to have been mostly validated.
of course TSMC N5
— kopite7kimi (@kopite7kimi) August 26, 2021
Yet, if this leak is correct then NVIDIA is going back to TSMC for their 5nm process - and likely means at least one flagship GPU series will be produced at TSMC - which is business that Samsung Foundries just lost out on. Even if they build the high end chips on TSMC and keep the low end at Samsung (which is where the volume usually lies) it is still interesting because it means that Jensen has lost faith in Samsung's ability to keep pace with the bleeding edge nodes.
Ada Lovelace will come out a little bit earlier. Keep patient.
— kopite7kimi (@kopite7kimi) August 26, 2021
Kopite has also stated that Ada Lovelace GPU will be arriving sooner than expected, although since they have not confirmed the timeline in the first place, we still don't have a concrete date. Rumors have put the arrival of Ada Lovelace GPUs between Q2 2022 and Q4 2022 so if Kopite is expecting NVIDIA to push back the production and commence mass production a bit early, the Q2 2022 timeframe might be the one to keep in mind going forward.
Recap on NVIDIA ADA GPU architecture
In many ways, Ada Lovelace can be thought of as the world's first computer enthusiast. She is the first person to have realized that the Analytical Engine proposed by Charles Babbage had applications beyond pure calculation and also published what is thought to be the first algorithm (becoming the first computer programmer) intended to be carried by such a machine. This was almost half a century before Alan Turing would finish their work and invent the general-purpose computer during the world war.
NVIDIA has been known to base its architectures on prominent physicists, mathematicians, and scientists and Ada Lovelace is no different. Videocardz actually managed to find a major hint in NVIDIA's own merchandise store that appears to confirm this rumor about Lovelace architecture being the next generation of GPUs from the company. If you look at the heroes showcased during GTC's 2018 keynote you find not only Ada Lovelace but what are potentially all future architectural codenames from NVIDIA. Jensen might have sneakily left the entire future roadmap (as far as codenames go) in the GTC'18 keynote.
NVIDIA CUDA GPU (RUMORED) Preliminary:
| GPU | TU102 | GA102 | AD102 |
|---|---|---|---|
| Flagship SKU | RTX 2080 Ti | RTX 3090 Ti | RTX 4090? |
| Architecture | Turing | Ampere | Ada Lovelace |
| Process | TSMC 12nm NFF | Samsung 8nm | TSMC 4N? |
| Die Size | 754mm2 | 628mm2 | ~600mm2 |
| Graphics Processing Clusters (GPC) | 6 | 7 | 12 |
| Texture Processing Clusters (TPC) | 36 | 42 | 72 |
| Streaming Multiprocessors (SM) | 72 | 84 | 144 |
| CUDA Cores | 4608 | 10752 | 18432 |
| L2 Cache | 6 MB | 6 MB | 96 MB |
| Theoretical TFLOPs | 16 TFLOPs | 40 TFLOPs | ~90 TFLOPs? |
| Memory Type | GDDR6 | GDDR6X | GDDR6X |
| Memory Capacity | 11 GB (2080 Ti) | 24 GB (3090 Ti) | 24 GB (4090?) |
| Memory Speed | 14 Gbps | 21 Gbps | 24 Gbps? |
| Memory Bandwidth | 616 GB/s | 1.008 GB/s | 1152 GB/s? |
| Memory Bus | 384-bit | 384-bit | 384-bit |
| PCIe Interface | PCIe Gen 3.0 | PCIe Gen 4.0 | PCIe Gen 4.0 |
| TGP | 250W | 350W | 600W? |
| Release | Sep. 2018 | Sept. 20 | 2H 2022 (TBC) |
The NVIDIA AD102 "ADA GPU" appears to have 18432 CUDA Cores based on the preliminary specs (which can change) provided by Kopite. This is almost twice the cores present in Ampere which was already a massive step up from Turing. The only way this is even possible is because NVIDIA is apparently building this on the 5nm process which has significant die area and power reduction. Interestingly, if you assume a clock speed of 1.75 GHz you can also get the peak single-precision performance of the ADA 102 GPU: 64 TFLOPs.
According to Kopite the ADA architecture will feature a much bigger L2 Cache (both Turing and Ampere have 6 MB of cache) which means this might be a major architectural revision (as Turing was to Pascal and Pascal was to Fermi/Kepler) instead of just the usual process shrink.
