It is important to remember that the statues aren't just about the civil war, but that they were largely erected after reconstruction, by segregationist governments. They are part of the creation of a myth which justified discrimination and race based terror.
The heroes of the rebellion should have counted themselves lucky that they weren't hanged. Magnanimity in victory is generally a good and wise thing, but I'll be damned if we are called upon to honor those who waged war on the United States. That is "too much of a good thing."
Why not put up statues to Benedict Arnold? I mean "you can't erase history" and the American Revolution was certainly a civil war.
Gravemarkers, cemetery monuments, and historical markers on battlefields should remain. Historic homes should be preserved. Statues of special artistic or historic value should be removed from their pedestals and appropriately preserved in museums.
Stone Mountain, Georgia will be hard to move, but it is a thumb in the eye and deeply offensive. Historical markers putting it in "context" doesn't quite cut it. It was clearly intended to preserve and honor "the lost cause" forever, as the Southern "Mount Rushmore."
What undisputed artistic and historical value it has is outweighed by the human tragedy it glorifies. It is still a rallying point for bigots. Personally, I say blast it. Some might blanch at the idea, but were it a Nazi monument, would you preserve it? Photograph it and destroy it. That will cost. I'll contribute.