Depends! If it's like a Dark Souls kind of game I keep all armors just because of fashion.
If it impedes movement or something of the like, then i probably won't use it.
Speed is better than fashion but once your equipment load is up it's all fashion from there.
If we're talking weapons, I usually keep them for play styles. Find out what i like.