Many won't even know it exists -- but the old stone quarry mill, otherwise known as Seneca Quarry in Gaithersburg, Maryland, is on the National Register of Historic Places and was important in DC's early days.

Now, it's covered in graffiti and falling into ruin.

Mike Bailey lives close to the site and takes a walk through it each year.

He said he was shocked when he came back this year to find graffiti everywhere.

“I think that the graffiti is basically young people,” he said, “coming out here, having a good time, partying, not recognizing the historical value.”

The ruins date back to 1830 when rock from the nearby slopes was cut there and sent down the Potomac to DC. This rock was used to build the Smithsonian Castle, among other notable buildings in DC.

There used to be a plaque and a metal barrier protecting the site. Now, these are covered in graffiti and have been left to fall down.

Bailey fears vandals may have also taken away a 3,000-pound grinding wheel that used to be inside the old mill.

"It's disheartening because it has such rich history,” said Bailey. “To see it falling apart and knowing it's on the historic register and knowing what it used to do, it's kind of sad."

It's not clear who the caretakers of the site are. The site is located near Seneca Creek State Park.

Officials with the National Park Service and the Chesapeake and Ohio Canal Historical Park Service, both say the site isn’t their responsibility.

Officials with Maryland’s Department of Natural Resources also say the site doesn’t fall under their jurisdiction.