It does matter, because determinism generally (but not always) depends on causation. People making (determined) predictions would still influence the (determined) future actions of other people. Punishing people who acted in a (determined) way would still affect the (determined) actions of other people. Lived experience allows for the possibility of a number of different ontological theories, which is a large part of why there are so many different and conflicting ones.