Not about being a SM, but being the second wife?
For me, it's the validity that this past relationship is given simply because kids were produced. My husband married his ex and started a family because "that's what you do". It was never a "you're my soul mate and I can't live without you" kind of scenario. At the time she seemed like a good option because she had a job and was more responsible then his previous girlfriend (he was 23 at the time - he's now 41).
They divorced 7 years ago. We met 2 years ago. We are soul mates. But there are so many witnesses to this first relationship, it sometimes feels like a shadow I can never escape. I understand that his kids think the relationship was significant, since they were the product of it. And I don't try to dissuade them otherwise, I would never dream of it.
It's the other stuff. His family. BM herself. Fighting the feeling of being "the second wife" and the stigma that comes along with it, when I know that I am the real deal and she is just a mistake he made.
I had a first husband too. No kids were produced. So the relationship remains where it should - in the past. I never realized, until I met my husband, how much making kids with a person changes peoples' perception of the importance of that person.