Yeah you could but wouldn’t it defeat the purpose of being basically a secret knock before you can give the password? The password should be the ssh password.
This would be just to allow you to connect to the server. If there was a vulnerable sshd on port 22, an adversary would have to know the port knocking sequence to connect to sshd and run the exploit.
