Thinking about causality (i.e. what circumstances and conditions it takes to support our food chain) doesn't provide easy answers, sorry… But that doesn't mean it is smart to avoid the conversation.
It seems vegan diet is a wholesome idea, however plant-based diet may be quite destructive: irrigated plants drink some water but, more importantly, they often grow in hot climate hence they lead to a lot of evaporation of clean water… Genetically-modified (e.g. drought-resistant) plants may prove as destructive to biodiversity as other destructive activities. As said, no easy answer, but this is a reason to enquire more, not less! Plant-based seems better, but this does not mean just any plant…
Denial doesn't change facts.
If you're not a parent (or are, but are also considering extending your family further), possibly the one biggest contributions you can make to a sustainable environment is actually not having a child… If you have children, avoiding 'disposable' children-related whatever is a gigantic contribution you can make: second-hand clothes and toys exist, washable diapers exist (and no, don't go stupid in washing products that destroy more than using disposable diapers, in the name of 'hygiene')… To see anything 'new' as automatically 'better' for the child is an aberration, since it contributes to destroying the very environment the child will be left with!
Basically, be mindful of causality. There are no easy answers (I know, vegans would want you to believe there are); there is no answer which is not context-dependent! But giving in to desires "just because I want it" is not the wholesome path; this applies in relation to food, to family choices, to everything… and this doesn't mean the solution is in reclusion and austerities: if you alone disappear from the world, there is virtually no measurable impact! The solution might only arise in engaging others and getting a change at a much wider level than the individual. There is no ultimate solution, but neither doing nothing nor denying reality is "responding appropriately"… If the solution is context-dependent, then we need to keep adjusting our response (and this means 'now').