What society or tradition or religion or our own conscience says is right and wrong are good but not completely accurate guides.  In the end we have to think about it, not letting our emotions and biases get in the way, and deduce right and wrong from more fundamental principles.  One of these is that every person is a "person" with innate rights to not be harmed and to pursue their life so long as they don't infringe on the same rights of others.

I suppose one could take this as a pragmatic necessity for modern living, if there is to be peace in the world.  I tend instead to take it as derived from even more fundamental principles, those of "love" in Western terms and wise compassion in Buddhism.