As Dr. Fauci prepares to exit, he reflects on his legacy and COVID decisions he would change

After 54 years at the National Institutes of Health and 38 years as the director of the National Institute of Allergy and Infectious Diseases, Dr. Anthony Fauci will be stepping down from public service at the end of the year.

"I have been driving onto that campus every single day, every single weekend for the last 54 years," Fauci told ABC News' chief Washington correspondent and "This Week" co-anchor Jonathan Karl in an interview that aired Sunday. "So I don't even want to think about what it's going to be like when I drive off the campus for the last time ... That idea just gives me chills just thinking about that."

In an intimate interview at his home, Fauci sat down with ABC News to talk about his tenure in public service, the COVID-19 pandemic during which he became perhaps the country's most famous doctor and the controversies that have consumed the last two and a half years -- and sometimes ensnared him.

Fauci has lived in the same home since 1977. Pictures hang next to the banister stairwell, dozens of framed photos sit atop a bookshelf and the floor is scuffed from years of use, the carpet worn down too. The mismatched red and brown chairs in the living room are cozy; on one sits an overstuffed pillow that has Fauci's face on one side and, on the other, a quote reading "'It is what it is.' - Anthony S. Fauci, M.D."

"You became an icon," Karl told him. "It was kind of wild to see. There were Fauci bobbleheads. People had Fauci shirts that said 'In Fauci We Trust.' You became somebody the whole country was turning to. What was that like?"

"I was pretty well known among my peers in science, but certainly not to the extent it is now," Fauci answered. "You know, I actually think both extremes, Jon, are aberrations of a reflection of the divisiveness in our country."

As much as Fauci -- who served under presidents of both parties as a nonpartisan health official -- was respected by many amid the pandemic, he was lambasted and even despised by others. Conservatives on Capitol Hill have criticized his recommendations on COIVD, called for investigations into him, he's received death threats and at a rally just days before the 2020 election, supporters of former President Donald Trump chanted "Fire Fauci! Fire Fauci!"

"When did it all get so political?" Karl asked.

"It got political very, very quickly," Fauci responded. "Because we had the misfortune of an outbreak, and a double misfortune of an outbreak in a divided society, and the triple misfortune of a divided society in an election year. I mean, you couldn't get more -- getting the cards stacked against you, than right there. It was a triple whammy."

Fauci said he has remained dedicated to his work, despite the threats of violence against him and his family.

"I look upon the country, in many respects, as my patient," he went on to say. "And when you -- if you're a really good physician, you are concerned and worry about every element of your patient."

"Including how your patient is going to react to something you said?" Karl asked.

"Exactly," Fauci responded. "Exactly. And even if the patient is somebody who's not the most attractive person in the world in the sense of personality, you still got to treat them the way you would treat anybody else. We learned that in medical school."

While Fauci said he hasn't communicated with Trump since Trump left office, he did praise the former administration on Operation Warp Speed, the program that developed the COVID-19 vaccines in record time.

"Just as he takes the blame for things in the administration, he should take the credit for things in the administration," Fauci said. "That was a positive thing, Operation Warp Speed. And they should take credit for that."

COVID-19 has killed more than a million Americans, a death toll higher than any war in which the U.S. has fought. And Fauci was one of the faces of the government's response. For a time, he appeared nearly constantly at White House briefings and in the media to share the government's latest, sometimes shifting, pandemic guidance.

"There were a lot of dark days, obviously a lot of deaths," Karl said. "Was there a day that sticks out to you or a time period that sticks out to you as the darkest?"

"It wasn't a day," Fauci answered. "It was a period. I've trained a lot of Italian scientists in my lab in the arena of infectious diseases, many of whom went back to Italy and were in the epicenter of the northern Italy disaster there."

"And when I got on the phone and heard them describe what was going on in the ward, where they were having people packed up in the hallways -- who they had to decide who to give a ventilator to, or who to take care of," Fauci later added. "I knew these people. So I knew what effect it would have on them. And then I said, 'Whoa, we got a real problem here. We have a real, real problem.'"

For months, cities were locked down. Schools in many areas were closed even longer.

"Obviously, these are local decisions. But was it a mistake in so many states, in so many localities, to see schools closed as long as they were?" Karl asked.

"I think in some -- I don't want to use the word 'mistake,' Jon, because if I do, it gets taken out of the context that you're asking me the question on," Fauci said. "And I don't want to do that because that's just happened too many times over the last years with me."

"Did we pay too high a price?" Karl pressed.

"Yeah, I would say that what we should realize, and have realized, that there will be deleterious collateral consequences when you do something like that," Fauci answered.

"That's the reason why I continually would say on any media appearances I've had: We've got to do everything we can to keep the schools open," Fauci said. "The most important thing is to protect the children."

As the evidence on how the virus changed, the medical advice changed, too. At the very beginning of the pandemic, Fauci told the public that there was no need to wear masks. But that guideline was soon reversed.

"If you are true to the data and the evidence, if something is evolving, means it isn't the same as it was before and therefore the data are going to allow you to upgrade and update -- whether it's a recommendation, whether it's a guideline, whether it's the communication to the public," Fauci explained.

"Would you take back what you said about masks?" Karl later asked.

"Yeah," Fauci answered. "I mean, sure, if I had to do it over again. Of course. Again, if I tell you why we did it, it would be interpreted as making an excuse, and I don't want to go there because that creates nothing but backlash. If I had to do it over again, I would have analyzed it a little bit better."

Fauci has been the national leading expert on infectious diseases longer than many Americans have been alive. And for 38 years, he hasn't even changed desks, telling Karl with a laugh that he "didn't want to ruin taxpayers' money."

But as he reaches the final months of his tenure as a public servant, he reflected on how he wants to be remembered.

"I want to be remembered as someone who gave everything they had for the public health of the American public and indirectly for the rest of the world, because we're such a leader in science and public health," Fauci said. "I mean, I just want people to know that I gave it everything I had and didn't leave anything on the field. I was all there."