I think I just figured it out. I had device notifications set to "direct messages" instead of "on" which seems to override the track feature