To the best of our knowledge, the present study is the first to explore the role of consumers’ societal concerns in their decision to shop online. It determines the nature of these concerns and assesses their impact on consumers’ attitude and choice to purchase online versus offline. Based on a thorough literature review, seven focus group discussions, and ten in-depth interviews, an analytical model to measure consumers’ perceived negative effects of e-commerce on society (PNEES) is developed. The validity of the construct is assessed by means of a quantitative study measuring its impact on consumers’ affective and cognitive attitudinal as well as behavioral responses. PLS-SEM results highlight the validity and reliability of the proposed measurement. The impact-performance map results of the driver analysis reveal that consumers’ concerns regarding a potential reduction of the product offer, the degrading attractiveness of their cities, and the decreasing wealth of their fellow citizen are three aspects of importance to understand their behavioral responses against online shopping. At a higher analytical level, this study also highlights a discrepancy between consumers’ affective and cognitive attitudinal responses to these negative societal consequences of e-commerce. It shows the predominance of the affective construct to explain consumers’ behavioral responses. These findings provide relevant implications for online and offline retailers, particularly regarding the content of their CSR strategies as well as the tone and topics of future communication campaigns.