This paper presents a method for synthesizing novel views of moving objects in airborne video. The object of interest is tracked using an appearance based visual tracking method. The on-object point correspondence is then built and used to estimate the homography induced by the ground plane. With known camera focal length, the surface normal to the ground plane and the camera motion between two views are factored out from the homography. In order to assure robustness of surface normal estimation, a rank one constraint is applied to decompose a matrix which contains the homographies from multiple frame pairs. Given a desired viewing direction, the novel image of the object is generated by warping the reference frame using the new homography between the desired viewpoint and the reference frame. Experimental results show that the method is robust and errors due to small depth variations of the object is negligible.