[RFC] Statistics of non-ASCII characters in strings

[Wilco Dijkstra]

> Alfred M. Szmidt wrote:
>    Does anyone have statistics of how often strings contain non-ASCII
>    characters? I'm asking because it's feasible to make many string
>    functions faster if they are predominantly ASCII by using a
>    different check for the null byte. So if say 80-90% of strings in
>    strcpy/strlen are ASCII then it would be well worth optimizing for
>    it.
> 
> Not the whole world is ASCII...

Of course, but few people use native-language directory names or identifiers in their code etc, so I
bet most strings will still be pure ASCII. Anyway I was hoping for something more concrete, such as
what is the likelihood that a block of N characters contains a non-ASCII character if the previous
block did?

Wilco