No real modulation or switching occurs. If you start a new chat, your “explicit” memories will pretty much be injected right into the system prompt (I almost think of it as compile time memory). The other memories can sort of thought of as “runtime” memory: your message will be queried against the embeddings of your chat memories and if a strong match is made, the model will use the embedding data it matches against.