If both of you have parental responsibility and there are no residence orders or other restrictions in place, then neither of you can take your children on holiday outside the UK without the written consent of the other (or any other party with parental responsibility). If consent is refused, an application to the Court can be made for permission.
If you have a residence order in your favour then you do not need your ex-partner’s written permission to take your children out of the country provided it is for less than a month.
If only one of you has parental responsibility and there are no residence orders in place, then permission is not strictly needed to take your children abroad on holiday. However, it is advisable to consult and agree with your ex-partner. Of course this does not stop a partner without parental responsibility applying for parental responsibility and then objecting.