In general, more practice and exposure is better. It seems very unlikely to me that your imperfect English would have a negative effect on a child. It is possible that if the child only has your input they would only learn to speak English as well as you do, or possibly that over time they would not really learn English if they don't speak it with others and you don't use it very often, but I don't think it would actually make learning English more difficult.

There can be a slight developmental delay when children are learning multiple languages, but that doesn't mean there is any real problem, just that it might take a little time for them to understand fully that there are two or more distinct languages and then to start using them in different contexts. Overall multilingualism has been shown to have cognitive benefits into adulthood, even if they might be a few months or a year behind when they first start school, simply because they've started learning two things. But I don't think it's always even noticeable.

The short answer is that any time/exposure/input is good. More is better.

Listening to English music or watching English tv and movies could help, as well as eventually talking with native speakers. But it is very unlikely that your pronunciation will be harmful.