I can make two general categories for horror. The first is cheap horror.
Cheap horror is running through the woods in your nightgown. It's hearing a strange noise when you know some of the other campers have disappeared, and going to investigate on your own. It's something you would never do but for some reason you're doing it anyway. The reason is that the writer knows we feel suspense to see someone do something Wrong, something that will probably get them injured or killed. So the writer makes you sit there for an entire scene knowing the monster is going to grab her, and then it finally does. It doesn't matter that real people wouldn't put their hand into the dark hole with a trail of blood leading into it to look for their car keys (or that if they would, that they're so stupid we're glad they're out of the gene pool). There is a kind of rudimentary suspense achieved by any predator-and-prey situation. If you're sick of looking at screaming girls in torn nightgowns, you can always turn to the Discovery channel and watch a crocodile go after a wildebeest. It's the same damn thing.
It's cheap because after the book/movie is over, you don't care anymore. You sat on the edge of your seat because you were watching an idiot do stupid things, but you know you're not an idiot and you wouldn't do those stupid things. You know you're not a wildebeest. You sleep well that night.
I call the second kind real horror. It's not about reducing a thinking, feeling human into a frightened animal and then letting the inevitability of Nature's laws crush this poor but stupid prey. It's about taking an intelligent, sympathetic character and having them make choices the reader or the viewer would make. They're not going to stick their hand in the blood-smeared hole to get their car keys--they're going to try fishing with a wire coathanger, which gets yanked out of their hand and disappears. Rightfully terrified, they make a molotov cocktail and throw that in the hole, because if they can't drive out of this cabin in the middle of nowhere, they're not going to share it with a monster. It kills it all right--but it turns out that was just the monster's beloved firstborn. And the eighteen-foot-tall alien-dinosaur-ninja-dentist-Republican*
Real horror is real because you've managed to soundly remove all avenues for escape from a person with whom we identify. I don't think I'm better than this character--I admire her. So when awful things happen to her, and I literally cannot think of another way she should have acted, her plight becomes gripping and personal. I start wondering what else I could have done--you know, if I was there--but the monster is still everywhere. Even if she finally triumphs, it's a relief, but it can't dissipate the horror of knowing she did everything right and still barely made it.
I also agree with this post of jimvanpelt's, so if anyone has told you that your horror isn't as effective as it could be, check out what he has to say.
My way can't be the only way, and ticket sales to deplorable films prove it. So what makes it scary for you?
* I'm KIDDING. Sort of.