Sometimes working with stone can be a battle. It's almost like the stones have a personality. There's always a few rebels that resist your every effort to put them in their place.
It's not always best to confront them face to face or try to show them who's boss. Better to walk away and just accept that sometimes there needs to be a balance of power.