Yes African human groups that migrated outside of Africa have interbreeding with Neanderthals, which for racial pursuits then means people in Africa are actually the "purest" human group (lol, don't think the racial supremacists like that though).
Still our genetic variance is very low. Neanderthal genes are common to people in Asia and Africa too, it's not like it was limited to Europe. The genetic contribution of Neanderthals to the human genetic code is still also measured at about 1-4% so it's not a ton either. Likely some small group of our distant ancestors mated with Neanderthals out of need (no suitable partners to fuck) or probably more likely were raped or did the raping, lol. I'm guessing our mating rituals/habits back then weren't quite so posh.