Retreats and like minds

What is the basis for the desire to retreat from the world and seek out like minds? What is so wrong with facing the world as it is and seeking out those whose minds are unlike our own? Should we withdraw from life or meet it head –on?