Your numbers are only showing up as SPAM because you haven't done the proper registrations. Most numbers you buy from Twillio will more than likely be marked spam. It's the nature of the beast. But, if you go register with Hiya your spam problems will go away almost instantly (at least that's been the case for me on every number I had that was going to spam)
Hiya is the entity that controls whether your number shows up as SPAM or not on Android. 70% of the global smartphone market consists of Android users. I can almost guarantee this will fix 99% of calls that are being marked spam. (considering you're not actually spamming of course)
The one catch is you have to have a business. There's no sole prop registration that I know of.