Duke Ravenguard - spoilers act 3 - eviltoast

I just finished saving Duke Ravenguard and the first thing he did was complain about his son. Dude is kind of an ass. Is it too late to put him back in the prison? I kinda wish I hadn’t saved him.