To answer your question: "If i use a pll to divide, will the derived clocks drift too far over the run?"
By definition, PLLs do not drift - their goal is always to keep edges phase aligned. However, they do have some level of jitter, which is dependent upon many factors and could be 100's of ps or more depending upon their configuration and source clock.
At 5MHz, you have a period of 200nS. With a period this large, the overall jitter will account for a very small percentage of phase difference between the clocks. How much phase mis-match can you tolerate in your design?