MILLEDGEVILLE, Ga. (WMGT) - The sign outside of the Society of Ten Commandments Church reads, "Gays and lesbians are disgraces to humanity." And that has some Milledgeville residents complaining to code enforcement officers.

Pastor Robert T. Lee is standing behind his sign and says county officials abused their power and his right to free speech when they asked him to take it down.

"They're willing to work for the homosexuals and their agenda...and their power of government to silence me," he said.

Lee says a code enforcement officer from Baldwin County told him that if he didn't take the sign down within 24 hours, he would lose his business license. The county had a change of heart, he says, once media got word of the story.

A county attorney couldn't be reached for comment over whether the officer violated the pastor's first amendment right, or the sign complies with the county's sign ordinance. Brian West, of the county's code enforcement office admitted that he went to the church to see the sign first-hand. He says he asked Pastor Lee to take the sign down because "some people might find it offensvie." Joan Minton, the county manager said an attorney will review the incident when they return to work Tuesday.

"If you want to make a difference, you've got to be different," said Pastor Lee. The church changes the message about every 2 weeks. It now reads, "Most churches are gateways to hell!"