In the South, Confederate monuments aren't simply a black-and-white issue

They're the faces in communities that can hide in broad daylight. Many residents admit that before George Floyd's death, they never even noticed the chiseled and controversial figures.

But for those who have, monuments -- especially Confederate monuments -- can spark a range of emotions from honor to hate.

Over the weekend in New Orleans, a group ripped down a statue of John McDonogh, a wealthy pre-Civil War slave owner who helped fund the city's public school system.

The protesters then rolled the bust into the Mississippi River.

"It's time for the statues to go," said Malcolm Suber, a protester with Take 'Em Down NOLA. "The death of George Floyd and what it really represented was the straw that broke the camel's back."

Some cities, including Jacksonville, Florida, have preemptively started taking their monuments down. But many statues are still standing, including a massive monument of a Confederate soldier in Jasper, Alabama.

In Gulfport, Mississippi, a monument of a Confederate soldier sits next to the Harrison County Courthouse with the words, "Lest We Forget." The statue was unveiled in 1911, 46 years after the end of the Civil War.

1911 was a big year for putting up Confederate monuments in a number of states; across the South, numerous statues were erected during the Jim Crow era. Then more were erected while the KKK flourished, during the civil rights era.

Critics say monuments were meant to intimidate, while supporters believe they were built to remember.

"I do believe that those monuments were put up in good faith by people who wanted to honor what they perceive to be the sacrifice of their own flesh and blood," said Bruce Roberts, a Harrison County resident.

Roberts, like many other whites in the South, isn't shy about his Confederate pride. While talking to us, he was holding the state flag of Mississippi -- the only state flag in the United States that still incorporates the Confederate battle flag.

"I still think that this should be our state flag. I don't think it should be offensive," Roberts said.

But Gulfport city leaders, and some residents, disagree. On Tuesday, the Gulfport City Council voted to take down the Mississippi flag and replace it with a neutral magnolia flag. The magnolia is the state flower of Mississippi.

"If somebody says that they are offended, people of goodwill, if they are offended, and they don't like it, I mean I do have to be cognizant of that," Roberts said.

Supporters like Roberts believe the legacy of the Confederacy is being turned into one of hate and racism, even though the facts are clear that the Confederacy fought to maintain the institution of slavery and its economics, resulting in a war that split up families and left at least 618,000 dead.

"If we just can't agree then at some point, with the democratic process then OK ... if we're going to change things, let's change them, but let's do it in an orderly fashion," Roberts said. "And we can do it without mob violence. We can do it without disrespecting one another."

Down the street, Spencer Schultz and his brother, Jared, were getting haircuts. Spencer's heading into the army after graduating from a high school not far from the Confederate monument that sits outside the Harrison County courthouse.

"I think states kind of idolize those figures," said Spencer, who is white. "And I don't think we should idolize people who owned slaves."

"[The monument] shouldn't really be outside. It should be somewhere to memorialize at least soldiers but not what they stood for," Jared added.

But proving this isn't simply a black-and-white issue. Avery Sullivan, a black Gulfport resident, said he's fine with the monument. What matters to him is racial justice for all.

"To me, it's just a part of our history, it really doesn't bother me that much," he says. "It's not about that, it's about the people."

It's a debate striking cities across the country. A number of Southern states have passed laws which make it difficult to remove or alter Confederate monuments.

In Mobile, Alabama, however, city leaders ignored their state law when they decided to take down a statue of a Confederate naval officer, sparking anger from members of the Sons of Confederate Veterans, a nonprofit made up of descendants of Confederate soldiers.

Arnold, who did not want to give his last name, is a member of the Sons of Confederate Veterans and a veteran himself. "If you would speak to veterans of different wars, not that they would have the same political or social views or opinions, but most of us, I do believe, I'm not speaking for anyone but myself, but I believe most of us respect the dignity and honor of the other's service," he said.

The Confederate symbol is no stranger to New Orleans, a city known for its diverse culture, rich food, and, of course, Mardi Gras. Once the home of the largest slave market in the United States, New Orleans erected Confederate monuments across the city following the Civil War. But in 2017, at the direction of then-Mayor Mitch Landrieu, the city took down a handful.

"There's a difference between reverence and remembrance," Landrieu told ABC News. "We should always remember, so that we never repeat. That's a whole lot different than using a prominent public space to revere people for doing something that, in my opinion, should not be revered."

"When these statues, Confederate statues particularly, but when others went up as well ... it wasn't a community decision," said Molly Mitchell, a history professor at University of New Orleans. "It was a decision by certain groups of people, and those were the white people in power."

In New Orleans, some groups are now calling for more statues to come down, like the monument to President Andrew Jackson in the French Quarter's Jackson Square -- sparking more conflict in the Big Easy. Critics say the pre-Civil War president was a brutal slave owner who brought some of them to White House.

"I just view it as an attack on our history," said Jeff Crouere, a radio show host for WGSO 990 AM. When asked if he thinks it's an ugly history, Crouere, who is white, said, "I would say it's certainly a history that's got some terrible things about some of these people, and some things that were pretty exemplary."

Crouere points out that after the Civil War, General P.G.T. Beauregard, whose statue was taken down in New Orleans in 2017, advocated for voting rights for former slaves. Beauregard, though, according to historians, thought blacks were "ignorant" and could be persuaded to vote his way.

"Let's look at that history," said Professor Mitchell. "It's ugly -- it's very ugly. It's state violence, and it's intimidation, and it's denial of the political power control process."

And now, some are taking history into their own hands.

Remember that bust of John McDonogh that sank to the bottom of the Mississippi in New Orleans?

That statue was later fished out by a group of what appeared on tape to be white men. It's unclear where the statue is now.