10.3 Incomplete lineage sorting
More rarely, the tree constructed for a specific SNP is discordant with the species tree. This is usually due to incomplete lineage sorting (ILS) – when genetic variation is present in the ancestor of two populations (e.g., the modern human–Neanderthal ancestor), then randomly fixes in a fashion that happens to conflict with the populations’ evolutionary relationships.
ILS can create discordant ABBA or BABA trees for a given SNP. Under ILS, we would expect to see approximately equal numbers of ABBA and BABA trees.
Fig. 3. ABBA and BABA allele patterns can be formed by incomplete lineage sorting.