No studies have been done on damiana and pregnancy or nursing. Most herbalists say it's fine. The real caution is that it can have a diuretic effect-- it can dehydrate you. That may reduce your supply.
Also, it can increase fertility. It's an herb that I wouldn't take unless I was willing to get pregnant. I would have to consider birth spacing before taking it.