I'm a Spanish native speaker raising my 2 yo child in Spain and I've spoken English to him from the very beginning. My first plan was to use stories, dvds and so on, but I asked native friends and colleagues (I'm an English teacher) and all of them recommended me to try to speak to him in English. After much hesitation and extensive reading I made up my mind and went for it. I won't say it's easy, but it really pays. One of the universal truths known to any language teacher is that meaning needs context, when I read books to Eloy he reacts to 'ice-cream' because he knows what and ice-cream is, I have gone with him to the shop, speaking all the way there about the fantastic ice-cream we are going to buy and asking all the way back about how he likes it. He had heard the word many times before and he had seen it on pictures, but it's not the same.
You have lived for eight years in the US, my longest visit to England lasted a month; I am sure your command of the language is much better than mine, I have very good grammar and good pronunciation but my vocabulary is not as good as I thought. Gosh! the amount of vocabulary you need to know!
Anyway, I used to worry a lot about my accent and kept going on about it with my friend until one them told me: "Don't worry so much about the accent. Accent is the easiest thing to pick, innit?"
One more thing, when Eloy was a new-born and up to his first year it felt really awful talking to him in another language, especially in front of the family, but you should see them now, they are so proud of their grandson who can understand English.
Stories, videos and DVDs are certainly very helpful, but if you are moderately fluent in the language I think you could give it a try, start from the beginning, you'll need some months to get used to it and to learn all the new vocabulary you are going to use:'babygro, nappy, dummy...' :-)