Some glitches in the simualtion waveforms

As a first point, an edge_sensitive process shouldn't have another condition like if(cali_done = 0) then enclosing it, except for an asynchronous input, as said. When used as in the present code, they must be supposed to prevent regular inference of synchronous FF's from the behavioural description.

You should translate your overall component structure to a regular edge sensitive design structure, as shown in the Quartus VHDL templates and cali_done to a state variable inside the edge sensitive block.

Another possible reason for glitches is a timing violation caused by inputs to the design. If they are unrelated, they should be synchronized to the clock before entering the logic. In simulation, stimulating signals, that are clock synchronous in real operation, can cause apparent timing violations.