You can, and I've generally had better luck with unsafe code than with C++ code. Unsafe code creates GC overhead, though, so it can also end up doing more harm than good if you're not careful. It's another spot where I've found that microbenchmarks can be misleading - the performance cost that pinning incurs is insidious and hard to measure.
Usually in the managed world you have references that point to an object. The objects themselves don't live in the same place forever, they may be moved by the GC (to consolidate "holes" in memory). References reflect that movement so you don't notice. However, when using unsafe code (which has pointers and pointer arithmetic) you need to keep the objects in place. That's pinning and it essentially forces the GC to work around those islands of pinned objects.
