To meet the demand of various high-speed data rate services as well as serving an exponential increment of mobile devices, millimeter-Wave (mmWave) communication in ultra-dense networks (UDNs) has been considered as a promising technology for future wireless communication systems. By deploying multiple small-cell base stations (SBSs), the dense topology combining high frequency mmWave is expected to grow not only the users (UEs) throughput but also the energy efficiency (EE) of the whole networks. Exploiting the benefits from mmWave and UDNs, in this paper, we introduce a new approach for jointly optimizing SBS-UE association and power allocation to maximize the system EE while guaranteeing the quality of service (QoS) constraints for each UE. Specifically, the throughput fairness among UEs is also taken into account by formulating UE throughput maxmin problem. Unfortunately, such SBS-UE association problem poses a new challenge since it reflects as a complex mixed-integer non-convex problem. On the other hand, the power allocation problem is in nonconvexity structure, which is impossible to handle with the association problem concurrently. Tackling those issues, an alternating descent method is proposed to separate the primal optimization problem into two subproblems and handle one-by-one at each iteration. In particular, the SBS-UE association problem is reformulated using the penalty approach. Then, successive convex programming is developed to convert non-convex problem into the simple convex quadratic functions at each iteration. Numerical results are provided to demonstrate the convergence and low-complexity of our proposed schemes, where the increment of the objective function is guaranteed at each iteration.