At what age is my baby old enough to be away from me overnight?

I'm having a baby next month and his father and I aren't together. We're trying to work out a visitation schedule and I'm wondering at what point the baby will be okay to stay at his dad's without me? I'm also going to breastfeeding and plan on keeping frozen milk at his dad's