It's a leaky abstraction over the underlying system call mechanism. Because the user stack lives in user space and accessing it from the kernel requires the same dance as any other access to user memory, system call arguments are instead passed in registers where they are immediately available to the kernel with no possibility of faults or TOCTTOU holes.
Here 'unsigned long' is just a convenient stand-in for "generic register-sized argument".
A "void * pointing to struct" requires a copy from user space. If a particular prctl() does need a struct, it can certainly stuff a pointer into one of those 'unsigned long' parameters (in the kernel environment, a pointer can be converted to and from an unsigned long without concern).
On all Linux architectures, "unsigned long" and "long" are always register-sized. For 32-bit architectures, "long" is 32 bits (the same as "int" which is always 32 bits), and for 64-bit architectures, "long" is 64 bits (the same as "long long" which is always 64 bits). This is different from Windows, in which "long" is always 32 bits, even on 64-bit architectures (and you have to use "long long" if you want 64 bits).
Therefore, Linux-only code (including the Linux kernel itself) commonly uses "long" or "unsigned long" to mean "register-sized" and "large enough to fit a pointer"; portable code normally uses "size_t" or "uintptr_t" for that.
It's not even just Linux - way back in the mid 90s there was an agreement among UNIX vendors to use the "LP64" model for 64 bit architectures, and ILP32 was already the de facto standard for 32 bit UNIX-likes, so "long can hold a pointer" is somewhat of a tradition in the POSIXy world.
It's not even about how the kernel is built. The userspace-side system call declarations need to be #include-able in C89 user code, so they can't use C99 types.
Here 'unsigned long' is just a convenient stand-in for "generic register-sized argument".
A "void * pointing to struct" requires a copy from user space. If a particular prctl() does need a struct, it can certainly stuff a pointer into one of those 'unsigned long' parameters (in the kernel environment, a pointer can be converted to and from an unsigned long without concern).