Video IP Cusom ST Block SOP, EOP

Not in that particular scenario, a hold-off would work in that case. However, I may have another scenario where, for example, I would need to buffer up some of the incoming data, do some processing, then send it out. For the sake of simplicity, lets say a custom frame buffer. On the output side, how would I know how to form the control packet and when to provide the startofpacket/endofpacket signals.