WD Blue SN570 crazy latency spikes and extremely slow Steam downloads when write caching policy (Windows 10) is ENABLED? Is my NVMe SSD messed up?

I just picked up a WD Blue SN570 not long ago and noticed issues where my Steam downloads would just go to 0b/s and a 60GB game/update would go from taking 25 mins based on my internet speed, to more like 1-2 hours. Latency would appear like 200ms, 500ms, 1100ms, etc randomly.

ODDLY, if I disable the "enable write caching to this device" in Device Manager, my games will download at my full internet speed and NEVER drops to 0b/s again on my SSD. I am also noticing a general improvement to speed overall. I tried reinstalling Windows and such already but unfortunately it appears the write to cache needs to be disabled still for decent performance.

Questions I have:

  1. Why is this happening?
  2. Is it safe to keep the write to cache feature off long term?