Arrival or first half day: keep it to the town core
If the first stretch is short, keep it simple. Walk the harbor, look up toward the castle, take in Panagia from the waterfront and use Kryoneri only if you want the easiest possible swim without turning the whole day into a transfer.