@thekitmalone (Answering my own post here but a metaphor has been rattling in my brain all day):

Are these people saying you shouldn't have the right to deny entry to your home if someone you don't want to visit you ring your doorbell?