In this work we present a method for accurate estimation of temperature at various locations on a chip considering the inaccuracies in thermal sensor readings due to limitations in thermal sensor placement and sensor noise. We utilize Kalman filter for temperature estimation and for elimination of sensing inaccuracies as well. The computational complexity is reduced by using steady state Kalman filter during normal operation of the chip and reducing the order of the thermal model by a projection based model order reduction method. Our experimental results show that this technique typically reduces the standard deviation and maximum value of temperature estimation errors by about an order of magnitude. Another important aspect of this technique is enabling the estimation of temperature at different locations on the chip with only a limited number of sensors in an efficient way.