Pushing stargazing to the limits with the world’s largest gamma-ray observatory