Greek life is at Cal Poly if you want it, and it’s not if you don’t. Roughly one in ten students on campus is a member of a Greek organization on campus. This means that students who do choose to participate in Greek life have a community of about 2,000 people with whom they can hang out, study, and do philanthropic work. This also means that the other 17,000 students on campus aren't completely overwhelmed by the Greek community. Greek housing is available, but not in huge mansions. What fraternities and sororities do is purchase an apartment complex either in lieu of or adjacent to a smaller house. This means that members of the organization can still reap the benefits of living in a Greek house (being near friends, etc.) without having to share a kitchen with 100-plus other people. The houses/complexes are located within close proximity to both campus and each other, but by no means do they constitute a Greek row.
There is no reason to feel pressured into Greek life. If you like parties, there are plenty going on outside of the Greek system. If you like networking, join a career-oriented group. If you want friends, join an interest club or simply introduce yourself to the person sitting next to you in class. With all of that in mind, remember that the men and women of Greek life are just that: men and women. Greeks are extremely well-integrated into the Cal Poly community, so even if you don't rush, you'll still know people who have gone Greek.