@noelle This happens a lot in many cultures.
Cagots are similar to Japanese burakumin, Korean baekjeong, and India's dalits (often doing work considered "unclean").
In the case of the burakumin and dalits, they *still* face discrimination despite government protection. They differ in no way from their neighbours in language, but are still ostracised.
Chinese tankas (although they're possibly a separate ethnic group, no one knows where they come from) are also outcasts who live on boats.
@noelle (specifically in the case of the Tankas, it's known they come from China, but it's not exactly known what ethnic group they come from; the leading theory as far as I know is that they were the original inhabitants of Guandong before the Han Chinese displaced them)