Like many other limits, none are known precisely. This is not like running out of a tank of gasoline.
However, it is known that recycling aluminum takes about 5% of the energy required to obtain bauxite and refine it. If you drink a cola from a can, just think of what must be done to put another full can back in your hand. Imagine how much energy to return it to a plant, burn off printing, melt, roll, reform, fill, and return it to your hand? Not much in total, but you are still slurping from an energy intensive container.
Big-energy commodity materials are one issue. Another is the chemistry of all kinds of rare earths and other substances that until the mid-20th century were seldom present in a household.
And organic materials pose different kinds of problems. For example, what happens to discarded antibiotics -- flushed down the toilet, often?The feeding of sub-medicinal dosages of antibiotics to food animals boosts growth and reduced loss to disease. But this has long been considered a major factor in increasing resistance of bacteria to antibiotics. The whole environment winds up carrying a minute level of antibiotics not present. 50 years ago. (Resolving this dispute between the medical community and animal husbandry community has been gridlocked in the present political system for years.)
And that's before beginning to investigate chemicals that might be endocrine disruptors, or which can have long-delayed effects not obvious for decades.