When I first openly called myself polyamorous I faced a common accusation… that it was just a way to justify having sex with multiple people. I objected pretty strongly to this accusation, afterall, I was already sexually open and saw no reason to hide my actions behind the veil of relationship. In fact, I stated that polyamory had nothing to do with sex and everything to do with love, but now I think I protested too much.
Now, I still believe that polyamory is more about love than sex, but sex is a small part of it because sex as a way of expressing your love and intimacy for a person. There are many people I love who I am not intimate with and because there is no intimacy (or desire for such) I don’t consider my relationship with them to be polyamorous. Sex can be a form of exercise, a way to blow off steam, explore curiosities, live in the moment, or it can be a way to become more intimate with a partner. That last option is where polyamory and sex link.
Polyamory is the realization that some people can romantically love multiple people at the same time. With that romantic love comes a desire for intimacy, which monogamy forbids. That desire for intimacy is a drive to be physically, as well as emotionally, close to someone and there are few (if any) better ways to be physically intimate than sex. So I guess polyamory isn’t about sex, but it certainly includes sex eventually.