CHARLESTON, S.C. (WCBD) – Local civil rights groups are demanding a statue dedicated to one of South Carolina’s most famous politicians be taken down.

The John C. Calhoun statue has towered over Marion Square for years, but some say it’s time for it to be removed.

John C. Calhoun was the Vice President, a U.S. Senator and staunch proponent of slavery. He died in 1850.

More than 100 people gathered at White Point Garden at the Battery over the weekend after violent clashes in Virginia between white supremacists and protesters.

Members of the National Action Network, pastors along with State Representative Wendell Gilliard (D-Charleston) explained how they want to make sure scenes like what played out in Charlottesville doesn’t come to Charleston.

“When the flag shows up, call it what it is,” said Nelson B. Rivers, “and when John C. Calhoun is lifted to a place he does not deserve, remind us who he is.”

NAN is demanding the city of Charleston intervene, but the city does not own the park and officials say the Calhoun monument is protected by the Heritage Act.

Under the Heritage Act, it takes two thirds majority vote from lawmakers in Columbia to change a monument’s status.

Lawmakers voted to take the Confederate battle flag down in 2015 after the Mother Emanuel Massacre.

The land the Calhoun statue is on is owned by the Washington Light Infantry and Sumter Guard.