The thing is, it's almost always clear when ML will outperform and when it wont. It isn't magic. ML systems are just compressed aggregations of their input datasets.
The question is then, (1) do we have datasets that are highly representative of the solutions to our problems? and (2) are our current systems sensitive to the relevant variations in these datasets?
If (1) is NO, then ML is impossible. If (2) is YES, then it's unlikely to provide a big ROI.
When ML replaces human decisions or very strict old software, there's also a more fundamental problem: are we enabling new mistakes that weren't possible before? How catastrophic?
For example, processing images according to some trained model instead of fixed rules and formulas introduces the risk of mismatched models (e.g. landscape photographs treated as line art from anime). Cases like self-driving cars not seeing obstacles are more obvious and more tragic.
The question is then, (1) do we have datasets that are highly representative of the solutions to our problems? and (2) are our current systems sensitive to the relevant variations in these datasets?
If (1) is NO, then ML is impossible. If (2) is YES, then it's unlikely to provide a big ROI.