I am so frustrated, should I let ditch my guy or be patient?

I have been dating a divorcee with a child who's been in and out of court due to visitation rights. Just when he thought it was over, the ex dropped the bomb that she wasn't agreeing with the schedule and now they are back in court. I have been dating him for eight months now and I sense his apprehension to get close which is normal considering his circumstances. I truly care about him and we have a very nice relationship but lately things have been a little messed up. He went away to visit his buddy for the weekend and did not tell me, postponed a date cause he was too tired from running after his daughter one night and today just cancelled a breakfast date we had scheduled very early in the morning--his excuse too tired from the exhausted weekend with his daughter. I told him already we ought to make time for each other and we both initiated these dates but did not happened because of him. I am frustrated, I don't know if I should stop answering his calls/texts and stop seeing him all together. Or do I continue being the supportive, patient girlfriend. Help!!