The semi-classical discrete truncated Wigner approximation has recently been proposed as a simulation method for spin-1/2 systems. While it appears to provide a powerful approach which shows promising results in higher dimensions and for systems with long-range interactions, its performance is still not well understood in general. Here we perform a systematic benchmarking on the one-dimensional transverse-field Ising model and point to limitations of the approximation arising after sudden quenches into the quantum critical regime. Our procedure allows to identify the limitations of the semi-classical simulations and with that to determine the regimes and questions where quantum simulators can provide information which is inaccessible to semi-classics.