I understand your concern as I am in a similar situation. My husband works in China. His company has rented an apartment for the employees. He has to live with other colleagues all the time including female and male colleagues.
I think the main problem here is not who lives with your husband, the problem is that you two do not stay together all the time. When you two stay together, you would not probably not too worry about it.
For me, what I can do is to go back to China more often to stay with him. I can do this because I am a full time mum. It is probably harder if you are a working mum. I think it is a social problem, but not a family problem for many HK ppl nowadays.
Sorry typo in the last sentense,
"I think the main problem here is not who lives with your husband, the problem is that you two do not stay together all the time. When you two stay together, you would not probably not too worry about it."
Sorry typo in the last sentense, it should be:
"I think the main problem here is not who lives with your husband, the problem is that you two do not stay together all the time. When you two stay together, you would probably not too worry about it."