I'm getting this error. My suspicion is that _something_ is writing to some part of card memory that it shouldn't, in a way that doesn't immediately cause a segfault.
LaunchError: cuMemcpyDtoH failed: launch timeout
The general behavior is that I can launch my kernel, it will finish, but I can't get my data back to the host. After doing this several times the whole card seems to lock up and become useless.
I'll just keep investigating, but I thought I should put this out there in case someone else sees the same problem.