"While the choice to withdraw from the campaign is President Biden's alone, I believe it is time for him to pass the torch,” Schiff said Wednesday in a statement released by his office.