Systems without IPv6 connectivity may still run software which uses AF_UNSPEC lookups. Such systems may want to skip the AAAA lookup in getaddrinfo.
Similarly, IPv6-only systems might want to avoid the A lookups.
Therefore, configuration options like “af_unspec both“ (the default), “af_unspec af_inet” and “af_unspec af_inet6” make a lot of sense to me.
Seeing that this is still an issue and this bug is referenced here and there, I'd like to shamelessly plug nss-dns4only  which uses the nsswitch subsystem to replace AF_UNSPEC queries.
You may find it useful.