Boltzmann's Brain objection
Given that the probability of low entropy states diminishes exponentially, we would expect to find ourselves in the minimum entropy fluctuation consistent with conscious experience. This would not be the appearance of a whole universe - just a minimal 'Boltzmann brain'.
The case of a Bolzmann brain is depicted by point C in the diagram. Points A and B are the much less probable states in which a conscious brain exists in a whole universe - like the one we (like to think) we live in.