custom instruction not working in 13.1.3

Hmmm that's very odd indeed. It might be caused by the register file index (b) being passed in without datab being present. Maybe try adding datab to the .tcl and VHDL implementation to see if that helps, but your first post suggests you may have that test case already.

Another workaround would be to send 'b' using 'datab' instead but I suspect the reason why you want to use 'b' is to avoid having a Nios II holding register for the datab value. If you don't use up all 256 'n' locations perhaps you could use the 'n' field to send the value for 'b' instead while this bug is being fixed.