I have a suggestion about handling inputs that need quick responses. Try using TriMatrix arrays set up to do logical functions.
The attached sketch shows how a small statemachine merged with the inputs could produce an address input to a second array which then drives the output.
The 2 stage pipeline first stage generates code, then uses code to generate output.
If further processing is required, then use it more for processing, not as a controller, offload the critical timing to arrays.
The second attachment is a start at using arrays for a microprogrammmed computer.
The possibilities for TriMatrix seem endless.
Thanks to all for your time to read this.