Low-rank modeling has achieved great success in tensor completion. However, the low-rank prior is not sufficient for the recovery of the underlying tensor, especially when the sampling rate (SR) is extremely low. Fortunately, many real world data exhibit the piecewise smoothness prior along both the spatial and the third modes (e.g., the temporal mode in video data and the spectral mode in hyperspectral data). Motivated by this observation, we propose a novel low-rank tensor completion model using smooth matrix factorization (SMF-LRTC), which exploits the piecewise smoothness prior along all modes of the underlying tensor by introducing smoothness constraints on the factor matrices. An efficient block successive upper-bound minimization (BSUM)-based algorithm is developed to solve the proposed model. The developed algorithm converges to the set of the coordinate-wise minimizers under some mild conditions. Extensive experimental results demonstrate the superiority of the proposed method over the compared ones.