It's all in the prep
One of the great things about swimming outdoors is it's free. You don't need any special kit -- in fact, if you're discreet you don't need any kit at all! But preparation is everything, and that means arriving knowing your swim spot and with everything you need to warm up afterwards.
Here are some tips:
Know your venue: be aware of nuances like currents, tides and entry/exit points
Have an idea of the temperature: be aware of how cold the water is and use it to guide you
Don't swim alone: make sure you have a swim buddy or somebody to spot from the side
Don't stay in for too long
Take warming up seriously: have extra layers and a warm drink for afterwards
Here's a handy kit list to download
But remember, you don't have to buy new stuff -- you can improvise with what you've got.