So after only one quick test I believed the comcast guy that there was some sort of weird “interference” between my specific router model and the modem they gave me. He was wrong!
After replacing my router with a new and very common linksys router, the error was still happening. So I did a ton of more debugging and it turns out that the people upstairs were torrenting!!! Why didn’t I think of that before! So I banned the guilty computer from my wireless router and things are working excellent =) . I am SOOO happy!!