Why had no one taken settlement in Vinyamar after Turgon and his people left?

I'm wondering how a whole Elven city is left abandoned. I'm guessing it's because of Ulmo, maybe combined with everyone moving farther south.

I've read the Silmarillion and am reading Unfinished tales for the first time.

Thanks for the help!