This paper presents a real-time tracking technique for constructing spherical image mosaics from sequences of images. The main contribution is a tracking algorithm able to estimate in real-time camera rotations during the sequence, assuming that the optical center of the camera remain fixed. This assumption allows us to consider a sphere centered at this particular point and having its radius equal to the focal length of the camera. The problem is to estimate for each image of the sequence the position of the image plane relatively to this sphere and to back project the image to create the global spherical mosaic image. The proposed algorithm allows, during a first stage, to track in real time (less than 12ms for each image) the camera rotations around the optical center. This is done by using a real-time template matching algorithm, using the formalism described in [2], adapted in order to estimate only 3D camera rotations. After this stage, each one of the images of the sequence are located relatively to the spherical image. In a second step, the spherical image is build by back projecting on the sphere the images of the sequence. This paper is mainly devoted to the description of the first stage. Experimental results proving the efficiency of the proposed approach are given.