This happens every once in a while with me. The problem is the mill will bind for whatever reason and throw the G-Code out of sequence vs. the correct position of zero.
My recent double sided project, the material was 3-5/8" thick. On the rough cut I used 1/2" mill going a total depth of 60 mm. I forgot to edit the tool information because I was going so deep with the material I was using, which turned out to be too fast. Half way through the cutting sequence, I stopped and checked the Z-height. Turned out it dropped 14 mm. On the backside, at a slower speed, I did not have any problems.
However, on the finish past, things didn't look right, so I stopped it again and check the Z-height. Again the mill dropped about 5 mm. It turned out it slipped in the collet. That collet now is on the way to the city dump.
Two more ideas to check of you're having Z-height problems.