If you took a trach aspirate every week you'd grow a different bacteria every time. Trachs are bacteria breeding grounds.
Unless the baby is seriously immunocompromised I say take the baby out and about! You will eventually lose the trach, but you can't get back those lost years--developmentally.
My son had funky colored secretions the entire 3 1/2 years he was trached. We only treated him if he had a fever higher than 100 and acted sick. Each doctor is different. Some will hospitalize trach kids and prescribe antibiotics for any growth, and some do not.
I'd go with your gut but try not to stress out too much. You will know if he is truly sick. Keep him out in the world around people. You will both benefit