Yes, because its not making sense. This would make you still a vegetarian. While this behaviour might fit your own views it does not really fit the definition veganism holds. Again, id say it is akin to saying you are an atheist and a christian at the same time. It is nonsensical. Veganism is closer to a believe than a diet. If you do not hold the ethical believe most people will feel it to be actually offensive to their ethical stance ( if you call yourself a vegan while getting animal produtcs from time to time). Why not say you mostly eat plant based?