Artificial intelligence (AI) technologies enable conversational agents (CAs) to perform highly complex tasks in a human-like manner. For example, CAs may help people cope with anxiety and thus can improve mental health and well-being. In order to achieve this and support patients in an authentic way, it is needed to imbue CAs with human-like behavior, such as personality. However, with today’s powerful AI capabilities, critical voices regarding AI ethics are becoming increasingly loud to carefully consider potential consequences of designing CAs that appear too human-like. Personality adaptive conversational agents (PACAs) that automatically infer users’ personality traits and adapt accordingly to their personality, fall into this category and need to be investigated regarding their benefits and caveats in mental health care. The results of our conducted qualitative study show that PACAs can be beneficial for mental health support, however it also raises concerns among participants about trust and privacy issues.