NVidia Meeting 2015-3-4: Difference between revisions

Revision as of 17:49, 4 March 2015

We discussed the VTK-m strategy for supporting multiple devices from one host (i.e., a single node of Summit). Options presented were:

one MPI task for each device (i.e., multiple MPI tasks per node)
- minus: may be lots of MPI tasks
- minus: may be incongruent with sim code's usage of MPI
- minus: hard boundaries between devices
- plus: easy to implement
one MPI task per node, with (for example) threading to manage access to multiple devices
- plus: less MPI tasks
- plus: more likely to be congruent with sim code's usage of MPI
- minus: hard boundaries between devices (??)
- plus/minus: implementation easier? (depends on details)
one MPI tasks per node, devices are treated as one giant device
- plus: less MPI tasks
- minus: could lend itself to inefficient patterns (reaching across device memories)
one MPI task per node, devices are knowledgable of other devices and can coordinate between each other
- plus: less MPI tasks
- plus: more likely to be congruent with sim code's usage of MPI
- plus: no boundaries between devices
- minus: big implementation, right?

We discussed what use case we were optimizing for. It was observed that optimizing for one may be in tension with the other.

All data located on device, memory never (rarely) transferred from device to host.
- This would be consistent with in situ usage.
Data located on host (with big memory) and is streamed to the device (which has small memory)
- This is consistent with post-processing usage (at least this could be argued).

using EAVL for comparison:
- in EAVL, you could queue up operations into a Plan, then execute the plan which would do all the kernel launches
- in theory, we could split these kernels into streams based on their dependencies
- e.g. if we use arrays a, b, and c in the plan, but c isn't needed until kernel 5, we could send c down while kernels 1-4 execute
- and, if kernel 7 were the final stage of a reduction that used only 1 SM, then if we could put kernel 8 was in a separate stream, the card could start executing kernel 8 before 7 finished
if we could do some batching up of worklets (kernel launches) like this in VTK-m, i.e. put it all into a plan before executing it, then that may give us enough info to do all this awesome async stuff

Steve Parker mentions that we might be able to JIT a 3-line template instantiation to avoid pre-compiling the (e.g. float/double/int===3) 3^numarg variations with concrete types

Steve: cost of a function call on GPU is very low (assuming all threads do same thing) -- only one instruction.
- but: the register cost may be high enough to cause some penalty

@@ Line 49: / Line 49: @@
 ** and, if kernel 7 were the final stage of a reduction that used only 1 SM, then if we could put kernel 8 was in a separate stream, the card could start executing kernel 8 before 7 finished
 * if we could do some batching up of worklets (kernel launches) like this in VTK-m, i.e. put it all into a plan before executing it, then that may give us enough info to do all this awesome async stuff
+==== CUDA JIT for dynamic array handle ====
+* Steve Parker mentions that we might be able to JIT a 3-line template instantiation to avoid pre-compiling the (e.g. float/double/int===3)  3^numarg variations  with concrete types
+==== virtual functions instead of combinatorial code to get to concrete types ====
+* Steve: cost of a function call on GPU is very low (assuming all threads do same thing) -- only one instruction.
+** but: the register cost may be high enough to cause some penalty