NVidia Titan V - observations

A

AndyE

Limp Gawd
Joined
May 13, 2013
Messages
276
Hi there.
Haven't been around for many years - too many other things in life .... :)
Please feel free to add your own observations to this thread.


Here my initial FAH observations:
I got a Titan V and a 1080 Ti. Put them in one of my older folding systems and just restarted the rig to see what happens. No further optimizations, just watching.

Config:
Win 10
FahClient 7.4.4.
Geforce Driver 388.71

Observation:
Both cards started with 9431 simultaneously.
1080 Ti : 970.000 ppd
Titan V: 940.000 ppd

1080 Ti (data from GPU-Z):
Clock: 1870 MHz
Mem: 1260 Mhz
GPU Temp: 81C
Fan 60%
GPU load 82%
Power consumption 85%

Titan V:
Clock: 1342 MHz
Mem Speed: 212 Mhz
GPU Temp: 57C
Fan 50%
GPU load 79%
Power consumption 38%

CPU i7-3770K:
CPU load 35%, no folding
Both cards run at PCI 3.0 x8


In a nutshell:
1) The Titan V delivers about the same ppd performance as the 1080 TI with half the power consumption
2) something in the FAH chain is not properly working with the Titan V, looks like it is just "idling" around. Need to look for further details and potential root causes.


Started also FAHBench:

Implicit solvent / single precision:
1080 Ti: 385.81 ns/day
Titan V: 532.24 ns/day (+38%)

Implicit solvent / double precision:
1080 Ti: 23.99 ns / day
Titan V: 242.13 ns / day (~ 10x)


Andy
 
What PCI-E version are you running on? If it's V1 or V2, you could have a bandwidth bottleneck, or if it's a X8 slot or smaller. The Titan should be getting more PPD, albeit not much more from what I've seen. The 1080 Ti is most definitely the PPD/$ winner in this comparison. I don't think energy efficiency, even if it does use that much less power, would counter the initial cost of the card. The CPU could also be a bottleneck from some experiences I've seen. Each F@H process seems to use a single core. So turn off Hyperthreading if you have it on, it could change those numbers.

Anybody else run into these kinds of numbers? I can verify that my 1080 Ti tends to get just under 1M PPD with no OC.
 
I do not intent to use the TitanV as a folding card, I was just using an "old" (and well known) application to get a first hands-on experience.

1st change:
I dropped the 1080Ti from the system, letting the TitanV use the full x16 PCIe 3.0 bus bandwidth. Verified by devicebandwidth.exe, the card moves from 6 GB/s to the expected 12 GB/s. The CPU load drops to 16%, as only one card need to be supplied. This change doesn't impact the ppd number - looks like the PCI bandwidth is not the bottleneck.

2nd change:
I used MSI afterburner, increased the frequency by 100 MHz (no other changes). With the card running at 1537 MHz, ppd of unit 9431 increased to 1160k (energy load is still at 40%).

Changing to Linux:
I don't have time to set up a new OS with Ubuntu to verify, but another Titan V folder reports an increase from 950k to 1150k (I assume, all other things being equal)

My current working assumption
1) The last compile of the Core21 application by the F@H team happened before CUDA 9.0 (including Volta knowledge) was released in early December 2017 (current version is CUDA 9.1, released Dec 19th, 2017)
2) F@H code relies on a CUDA feature called "fat binary". (I don't want to go deeper into the PTX to .cubin compilation process all CUDA GPU applications rely on.)
3) The microarchitecture of Volta is *significant" different from the previous GPU architectures (Fermi, Kepler, Maxwell, Pascal), for instance having many native instructions available be the 4 previous generations removed from the Volta instruction set (and possible replaced by other/more modern instructions - tbd)
4) If the compiled F@H code doesn't include the Volta version of the binary, the GPU driver need to emulate those missing (i.e. Pascal based) instructions in software (possible be the Intel CPU) when the driver need to accomodate a Volta-based card. This would explain the low power load on the Volta GPU.
5) If the F@H team recompiles the Core21 application with a Volta aware CUDA version (i.e. v9.1) and the performance does change significantly, then my working assumption was right. If it doesn't change, then my assumption was wrong.

Andy
 
not sure if they would fundamentally change 0x21 and risking impact on running projects; but hey, maybe a future variant (0x22 ?) could go that way ...

PS: welcome back AndyE; still have you green-colored external power supply ? Still remember that pic ... :ROFLMAO:
 
Hi Christian, yes, everything is still in the "lab". Didn't throw anything away. :)

To challenge my own working assumption: One user with a "decent" rig with 8x Tesla V100s seem to enjoy his Volta based folding.
How to get 15mio PPD from one System

Now the question is: Is there a FAH relevant difference between the Titan and Tesla Cards, or is just my setup screwed up ?

The Tesla system seems to work fine, the Titan users are still scratching their head....


Andy
 
You must log in or register to reply here.
Back
Top