We propose a scheme to predict human mobility in this paper. First, a hierarchical interest model is introduced to organize the semantic category of locations in human mobility logs as well as representing their personalized mobility patterns. Then, by combining the interest models of different people, a 3-modes tensor with the features of person identity, time, and the semantic category of location is constructed. Tensor factorization is utilized to reveal people's mobility interest on different kinds of locations. Finally, personalized interest models are recovered from cumulative tensor to predict human mobility in a person-by-person way. Extensive evaluation results based on a large scale check-in dataset from real location-based social networks have validated that our proposal achieves better recall, precision, and F-Score in human mobility prediction as compared with the state-of-art approach.