Moving a DNS Only Server

I have 2 DNS Only cPanel servers (ns1 & ns2), running in a cluster for a single main cPanel server. I have dozens of domains whose nameservers are set to ns1 and ns2.

I'd like to move ns1 from Rackspace to Linode and am looking for advice on the best way to do this, avoiding any downtime for my domains.

i.e. can I just delete the ns1 server on Rackspace, and create a new one on Linode called ns1 and setup the A record? Or is there a better way?

Thanks