System identification from noisy data is challenging in many science and engineering fields. In current work, we present an approach of system identification by sparse Bayesian learning methods. The key idea is to determine the sparse relevant weights from a constructed library by learning from noisy data. The sparse promoting prior is used to regularize the learning process. Furthermore, to identify a parsimonious system, the sequential threshold training is incorporated into sparse Bayesian learning. It is especially helpful when the learned data has large noise. Furthermore, we extend our approach to learn a parametric system by using group sparsity. Several explicit and implicit ODE/PDE systems are used to demonstrate the effectiveness of this method.