In this work, we propose an adaptive phase field method (PFM) to simulate quasi-static brittle fracture problems. The phase field equations are solved using the scaled boundary finite element method (SBFEM). The adaptive refinement strategy is based on an error indicator evaluated directly from the solutions of the SBFEM without any need for stress recovery techniques. Quadtree meshes are adapted to perform mesh refinement. The polygons with hanging nodes in the quadtree decomposition are treated as n−sided polygons within the framework of the SBFEM and do not require any special treatment in contrast to the conventional finite element method. Several benchmark problems are used to demonstrate the robustness and the efficacy of the proposed technique. The adaptive refinement strategy reduces the mesh burden when adopting the PFM to model fracture. Numerical results show an improvement in the computational efficiency in terms of the number of elements required in the standard PFM without compromising the accuracy of the solution.