Nah, I liked Season 4. Especially one the best holy shit moments in TV the mid way finale of season 4, and how the crew responds to it in the episodes after that. Probably the highlight of the show that. I also really liked the ending, but I know it was one that might not be to everyones taste.
Season 1 and first half of season 2 are the weakest, but that's the same for most big story arc shows.
Am I right in saying that there are pilot episodes before the first season? (seem to remember reading that) and if so, is it important to watch those first?