timer interrupts

have a timer built in SOPC that generates Interrupts every 250usec. This periodic timer is driving an alarm at a 10ms rate. I flip an LED on the alarm and the frequency is very precise (50.0Hz). When the system is hit from another interrupt in logic every 1ms (and the ISR measures 100usec long) the 10ms alarm is extremely erratic...? I don't understand why. The clock speed is 65MHz. How can it not keep up? Its behavior is as if the timer misses ticks when another interrupt is happening. The timer is priority 0 (highest).

This would seemingly never produce a stable system.

Any ideas...????????