If you have a penis and have ever gone swimming in cold water, you may have noticed that your penis looks similar to the head of a frightened turtle (apologies to Jerry Seinfeld). This is a real thing with some real scientific explanations.
When we are talking about the actual shaft itself, the main explanation is all about blood flow. According to Dr. Abraham Morgentaler, a urologist, clinical professor of urology at Harvard Medical School, and author of Why Men Fake It: The Totally Unexpected Truth About Men and Sex, cold causes a narrowing of the blood vessels (aka vasoconstriction), "so the spongy erectile chambers called the corpora cavernosa become less filled with blood, making them smaller."
A second and related phenomenon happens to the balls. A dude's noble scrotum plays a role in both holding your balls and keeping them at the right temperature. Ideal sperm production occurs at temperatures a bit colder than your body temperature, so it's important that your boys are kept away from your body heat.
"Although the penis essentially functions as a big blood vessel, the same is not true for the scrotum," Morgentaler told BuzzFeed Science. "Rather, its size is determined largely by muscular contraction or relaxation. When it is warm, the scrotum expands, allowing the testicles to hang further away from the body, which is a heat source. When it’s cold, the scrotum contracts, bringing them closer to the heat source."