This paper studies panel data models with unobserved group factor structures. The group membership of each unit and the number of groups are left unspecified. We estimate the model by minimizing the sum of least squared errors with a shrinkage penalty. The number of explanatory variables can be large. The regressions coefficients can be homogeneous or group specific. The consistency and asymptotic normality of the estimator are established. We also introduce new Cp-type criteria for selecting the number of groups, the numbers of group-specific common factors and relevant regressors. Monte Carlo results show that the proposed method works well. We apply the method to the study of US mutual fund returns and to the study of individual stock returns of the China mainland stock markets.