The usual saying is along the lines that a poison is something that harms when you ingest, while a venom harms when you inject.
The normal application of a venom is to go from the venom gland to the bloodstream pretty directly (possibly via the mouth).
The evolution of venom has thus usually prioritised efficacy, with little need for resilience. So venoms are not resilient to environmental factors (stomach acid, digestive enzymes, they often denature quickly just in air), nor will they be adapted to going through epithelial tissue (skin, gut lining, ...). The odds that they would be able to enter the bloodstream through the digestive system are limited to non-existent.
There's a caveat if you suffer from ulcers or open wounds in your digestive tract (or on your skin for contact), as those are openings into the organism which would allow venom through much more easily, but barring that you can literally drink venom and suffer no significant effects (well the gut flora might be affected which is not great). Though don't go and try it obviously, even if the risk is nigh non-existent, it's also 100% not worth it.
Now it should be noted that some products are both e.g. a few snakes will "spit", and their venom can affect mucous membranes or go through them (so it's poisonous). Also injecting a poison into your bloodstream directly is generally a very bad idea, if something burns / breaks down skin on contact, it's not going to be milder if it starts inside the organism.
