With Unicode, why aren't the two Turkish Is just treated as if they have nothing to do with the normal Latin I? The fact that the glyph for uppercase dotless I resembles the glyph for uppercase Latin I should be irrelevant, surely. It's a kind of typographic false friend situation.
Maybe there's a missing level of indirection in Unicode that prevents it from doing this, but I can't see how there could be.
One answer is that unicode had to import existing documents; I suspect that a lot of documents are written in a Turkish codepage that would have been an 8-bit encoding with the lower half as ASCII, that wouldn't have bothered with a different codepoint for "Turkish" I. As I said, you can't rely on upper/lowercasing roundtripping correctly in general.
(I was about to give the example of ß, which is usually uppercased to SS. But interestingly Unicode has now adopted a codepoint for the (disputed, and currently lacking a typographic consensus) capital version, ẞ. So maybe a codepoint for "uppercase Turkish I" is on the way. Turkish users will still expect to be able to lowercase "I" to a dotless lowercase i though, since a lot of existing documents will have "I"s in)
I did a bit of research on this and you're right, legacy encodings are one problem. More seriously there seems to be no established way to manage multilingual text which includes homoglyphs (say by using colour coding) so you would really be replacing one problem with another.
It does seem like this Turkish I problem is the most conspicuous situation, maybe unique, where changing locale changes the behaviour of toupper/tolower. Unicode, on the other hand, has many homoglyphs and duplicate characters which all need to be dealt with.
Yeah, turkish I is bit of a red herring due it being a quirk in Unicode specifically. From a previous comment of mine:
> They [Unicode consortium] should have specified Turkish alphabet to use ı and a diacritic to make the dotted one. That would have made (in this case) capitalization locale-independent. [...] I dislike the common usage of turkish i as a example because it is such a obviously fixable (if legacy stuff wasn't concern) flaw in Unicode rather than fundamental issue.
That's helpful. Wikipedia page for "glyph" seems to concur:
"For example, in most languages written in any variety of the Latin alphabet the dot on a lower-case "i" is not a glyph because it does not convey any distinction, and an i in which the dot has been accidentally omitted is still likely to be read as an "i". In Turkish, however, it is a glyph because that language has two distinct versions of the letter "i", with and without a dot."
Maybe there's a missing level of indirection in Unicode that prevents it from doing this, but I can't see how there could be.