What language should we speak when the family is all together?
It is important that you decide as a family what works best for you! Have you read Why, What, Who, When, Where? Start with that since it is important that as a family you agree on who speaks which langauge(s) when. Being that more and more of us are becoming multilingual, we have the luxury (and confusion) to decide which languages to speak when.
As you children get older, they will become good at switching between languages as they speak to different individuals in the family (e.g. speaking French with dad, Chinese with mom and English with siblings).
After a while you will be so used to switching yourself that you don’t have to actively think about what language to speak to whom when the family gathers at the dinner table. Note that this doesn’t necessarily mean that families “mix” languages in sentences (yet, that may be what your family decides to do if everyone is multilingual!).
However, if you find switching it too strenuous to be speaking two, three or four different languages at the dinner table, then you can consider adopting a family language for when the family is together. You could decide that whenever the family sits down together at the dinner table, that you all speak French. When interacting on a one-on-one basis, you could revert to your individual languages again. Again here, do what is most natural for you.