Well after several more hours and a lot of cussing I finally got it installed correctly. Long story short RV closet flanges are made of ABS and typical stick and brick flanges are PVC. Well, I pulled the flange out to try and replace with a low profile metal flange. Nope, ABS vs PVC.
After putting the flange back in place a rotated it a little more to get a better angle on it and redrilled the holes for the screws to mount to the floor and everything worked out great.