"I feel very badly actually for Secretary Acosta because I've known him as someone who has worked so hard and has done such a good job," President Trump said.