I'm not from the UK or a native speaker, but I do speak fluent (British) English, and I've never made the connection either. To me, "loo" and "
Lou" are pronounced a bit differently as well; "
Lou" has a "thicker" sound, both in consonant and vowel.
Chronic thinker.