PEX (and polyethylene in general) is one of the least concerning plastics, it’s not worth worrying about on that front. (I wouldn’t want to drink out of PVC though since it often contains phthalates as a plasticiser).
Rigid PVC has few if any plasticizers. (This makes sense: it isn't very plastic!) This is the type usually used for water pipe, at least around here. It's probably not very dangerous.
