FACT #1: 150 billion pieces of garments made a year.
FACT #2: 7.5 billion world population.
The 2 number should put things into perspective clear enough; we are simply producing too much. Much of this is due to the illusion of the low cost of garment. Many of the cheap garment that we see are manufactured with low cost synthetic fibers. These fibers are essentially plastic which are not biodegradable and will stay with us for a very long time.
The article by Jill Di Donato in the below link is worth reading if you care anything about the place we live in.