Comments by "Vitaly L" (@vitalyl1327) on "comparing GPUs to CPUs isn't fair" video.

@SillySussySally this is exactly what I said - GPUs issue instructions from another threads (in NVidia parlance, warps), while OoO CPUs issue instructions from the same thread that it knows do not have a dependency on anything that's currently being stalled. So, yes, OoO CPUs have higher latency. Simpler CPUs (such as ones you'll find in microcontrollers) have a much lower latency, and, more importantly, predictable latency. GPUs have lower latency (in terms of cycle count, not time - they run on lower clock frequency normally) just by a virtue of being much simpler cores and featuring shorter and simpler pipelines. Keep in mind that the exact NVidia microarchitecture is not a public knowledge, so we can only assume here. There are other GPU designs that are far more open and well documented though, so we can extrapolate that knowledge. I personally worked on two mobile GPU cores, ARM Mali 6xx and Broadcom VC5, both are wildly different from each other. Latencies in both were (in clock cycles) still smaller than in high performance Intel cores and high end ARM cores (but higher than in the in-order ARM cores).
2
@SillySussySally Branch prediction (and in GPU context, divergence) is a totally different story here. I'm talking about hazards - the register dependencies between instructions. In an OoO architecture instructions are dynamically stalled while waiting for the register they depend on to be updated by a previous instruction, while instructions that do not need to wait for anything can be executed out of order. Another mechanism is register renaming - when you have more physical registers than the addressing space allows. In an in-order architecture you either stall everything until the result is ready, or trust the compiler not to put the dependent instructions too close to each other. All GPUs are in-order. And GPUs have a lot of addressable registers, split between threads / warps, so no register renaming here. You can read about, say, Tomasulo algorithm, to see how it's done in OoO.
2
your desktop CPU is also designed for maximising throughput, latency be damned. If you want a CPU core designed for low latency, look at something like ARM Cortex-R range.
1
@SillySussySally confusing latency and throughput, aren't you? And no, no CPU have a 1 cycle latency. And there are dozens of different GPU architectures with different pipelines
1
@SillySussySally How is your response even relevant? We've been talking about those mythical 1-cycle latency instructions of yours (and equally mythical 4-cycle latency GPU instructions). Mind elaborating on that?
1
@SillySussySally comments can get removed if they contain URLs. Mine are always disappearing if there are any links. This is not the full latency - it is the latency between consequent instructions. The actual latency is the number of clock cycles between instruction fetch and instruction retirement, and for a modern OoO CPU it's anything from few dozen cycles to hundreds of clock cycles. For a GPU it's lower, but still above a dozen of cycles (pipeline length + all possible stalls).
1
@SillySussySally the difference here is that in a large OoO core instruction dependencies are handled dynamically, and instructions that are independent from the ones being currently executed or stalled can get issued out of order. This way a high instruction-level parallelism is achieved, with many instructions being processed at the same time, absorbing the high latency of each individual instruction. In GPU cores no such mechanisms exist. It's up to you to handle the dependencies (hazards). E.g., to make sure the instruction that depends on the previous arithmetic instruction result is scheduled after at least 4 clock cycles. Parallelism is achieved by scheduling multiple independent threads into a single execution unit, so it does not have to be stalled. It also results at high latency for individual instructions, but it's absorbed by high thread-level parallelism (instead of sequential instruction level parallelism in OoO cores).
1
@SillySussySally Sure, but when it's executing instructions from the same thread, in most GPU architectures (again, we don't even know the ISA of NVidia, so cannot tell exactly) it's your (or your compiler) responsibility to make sure instructions with a direct dependency do not follow each other. You can see a similar design constraint in the early RISC architectures, such as MIPS. GPU cores do not track dependencies and (at least those I know) don't even do forwarding.
1