Hi Ana, I am Italian and somehow my story has something similar to yours. My Chinese husband and I ended in divorce one year ago and then he decided to go back to China. Once he left, I moved to my parents house for a while together with our 23 months old kid.
Our kid enjoys kindergarden in the morning and then he spends about 4 hours with my parents in the afternoon, constantly hearing only Italian language, from monday to friday. Back from work, I am his only Chinese language resource.
My chinese is quite fluent since i used to live in China for 4 years before returning to Italy.
For a long time I asked myself wether our kid would have felt bad in hearing his mum speaking a language different from the one spoken by the rest of the family. I was also scared as you that speaking Chinese in presence of my family would have let them feel uncomfortable. But in the end, I realized that all these fears were in my mind, and that the way a kid learns a language when he is so young is just the same as he learns to cycle. They will not feel confused. It is just a natural process which does not mind anybody around us. Today, I have set some goals and I feel more spontaneous in speaking chinese to our kid every time i can. I also found out that many useful information on the bilingualism, like those offered on multilingualchildren.org, helps you to take a breath and start with more energy than before in speaking another language to your kid.
Don't be afraid of speaking English to your kids Ana, you are giving them a big chance. Be positive!