Non-belief in the arbitrary is not arbitrary
It is arbitrary to believe in arbitrary things, such as the Abramic god, Russell's Celestial Teapot, or the Flying Spaghetti Monster. Not believing in such arbitrary things is not itself arbitrary, unless one arbitrarily chooses which arbitrary things to not believe in.