I think that any statue that glorifies hurting and enslaving people are not symbols that should be present in the public sphere. They act to threaten those who try to live peacefully. In the South, they are reminders of oppression, maintaining old power structures.
Keeping these statues up or fighting for them to stay up can only be done by those who disregard the pain and hardship of slavery and systematic racism.