The polygon-based scaled boundary finite element method is applied to two finite fracture mechanics based failure criteria to predict the crack initiation from stress concentrations, i.e. notches and holes. The stress and displacement fields are modelled by the scaled boundary finite element method through semi-analytical expressions that resemble asymptotic expansions around cracks and notches. Important fracture parameters, i.e. energy release rate and stress, are accurately and conveniently computed from the solutions of stresses and displacements via analytical integration. One distinguished advantage of applying the scaled boundary finite element method to finite fracture mechanics is that the required changes in the mesh are easily accommodated by shifting the crack tip within the cracked polygon without changing the global mesh structure. The developed framework is validated using four numerical examples. The crack initiation predictions obtained from the scaled boundary finite element method agree well with the reference finite element results.