Voyager had the more interesting premise. The Starship Voyager is sucked into a wormhole or something and dumped light years out in space, deep in unexplored territories. Scary, right? Not really, it's like every other Star Trek, only with even more bad acting and a crew that bicker all the time. Enterprise, we don't really talk about, because it's the worst thing to have Trek in the title since Star Trek vs The X-Men. It's a prequel set on the first Enterprise, and features yet more bad acting and bickering.
Jeffrey Combs makes a few cameos through these series (most notably, Enterprise and Deep Space Nine), playing different characters. This bodes well, because Jeffrey Combs is Re-Animator, right? He's a scary chap. Well, no, because they choose to make him look like this:
But is there any scary amongst the Trek? Very occasionally. DS9 is too concerned with politics to even try, but it has its moments. Empok Nor does a visit to a sort-of haunted space station, whereupon the crew's pet Cardassian loses control of his faculties and tries to kill everyone. Well, he is played by Andrew J. Robinson, from off've Hellraiser (true story, trivia fans: Hellraiser 3's Terry Farrell plays a main character too). The Cardassians are sucky villains, but their history takes in lots of cruel experiments and several moments of grotesquerie that make DS9 vaguely worthwhile. For the most part though, it's a dull, self-important series in which everyone is boring and everyone talks about politics.
Voyager is known to many as the worst of the Trek series. I always had a soft spot for it myself. Darkling is my favourite episode. The ship's holographic doctor has himself a funny turn and tries to kill just a few of his crewmates. It's a fun little play on Jekyll & Hyde, and one of Voyager's more interesting episodes. There's also The Haunting Of Deck Twelve, in which sillly cook Neelix (him off've True Blood) tells ghost stories to the Borg children. Yes, they have Borg children. No, I'm not sure where they came from.
There might be scary episodes of Enterprise, but I never watched it much beyond the first series, due to it being hidden on television channels I don't have. The soft-rock bullshit theme tune is pretty scary though. Whatever happened to "these are the voyages...?" There's an episode with zombie Cardassians, and The Borg make a return (or a first entrance, even) but nothing to write home about. Unless you like writing home about how terrible things are.
In summary, and back to the original question, why is Star Trek scary? At its best, Star Trek plays on that fear of being far away from home, under attack from forces unknown. There are dangerous alien planets, salt-sucking shape shifters and William Shatner trying to rape things. Star Trek is scary both ironically and unironically. It may not have aged too well, but who can forget Deanna Troi being turned into a cake? Or Jeffrey Combs with blue penises stuck to his head? Star Trek probably isn't scary, not really, but I do so enjoy writing about it. Sorry about that.
NOT THE LEAST BECAUSE IT FEATURES AN ANDROID WHO WRITES POETRY ABOUT HIS CAT.