Should I trust my gut and?

We've been trying for almost a year now but not consecutive months. I have tried unsuccessfully for 11months with a few months (that ARE NOT counted in the 11months) where my husband was out of town. Plus, we did not use condoms or BC for YEARS before we officially said we are "trying". I believe we may be having issues conceiving but I know what the doctors are going to say..."wait for 12 solid months". But a woman knows her own body. Also, my mom (God rest her soul) had problems conceiving with me and my brother. Her and my dad tried for 10 years and then she had to have surgery before having us. I'm also worried because my husband and I had a lot of issues with sex (I had feminine issues where it was painful to have sex at all...kept getting diagnosed with yeast infections but it was my hormone levels that was causing the pain/itching). Im super worried that the meds they put me on may have altered my hormones too much or something but Im not taking them anymore.... smh. Either way, my question is should I trust my gut and go to the fertility specialist given my history. For the total amount of time (we've barely ever used condoms in the 5 years we've been together...not married but since we started dating 5 years ago) and still no pregnancy at all...And I have taken prenatal vitamins for years now! Its depressing....day in, day out taking vitamins and....no baby.