Adam Spencer and Wil Anderson are set to leave Triple J Breakfast at the end of the year.