Well for one it would stop people from getting things they don't like removed because of false and fraud copyright claims.
You're going to have people abusing the system no matter what. Taking the system out entirely doesn't help anyone; people may not be able to falsely take down videos, but the legitimate claims wouldn't hold any ground either.