Wouldn't there be some safety concerns to letting the AI run overnight with access to run any command?

Maybe if it can run sandboxed, with no internet access (but if the LLM is not local, it does require internet access).