Oak is difficult to work with and not just because it is hard. It is brittle and chips out very easily.
I find Maple to be a GREAT wood for this stuff. It is about as hard as oak, but a lot more stable.
Poplar is an OK choice of woods
Cherry is my favorite wood, not quite as stable as Maple, but it is an awesome wood to work with.
For just testing Poplar is relatively inexpensive.
In my opinion the softwoods like pine are not a good choice to test with as it chips out really easy and is just too soft.
Looking at your V-Carve file I see the same thing as Adrian sees. It all looks OK. Saying that - you then need to focus on other things like machine, slides, holders for the spindle, screws and lots of mechanical things. Seems like something is loose
Imagine the Possibilities of a Creative mind