First, though, not everyone seems to have it preinstalled. When I search for the game on my upgraded laptop, the app is in the Windows App Store and says that I own the game and can install it if I wish. Others, however, find it on their system automatically. (Microsoft announced it would be automatically installed for those who upgraded or downloaded Windows 10 during launch.)
If you don't want the app on your system, you can remove it in PowerShell. StackOverflow user Doktor J offers these instructions, adapted from an article on GHacks.net:
There are other default, built-in apps in Windows 10 that you can remove, including Xbox, Groove Music and Film & TV, and Money, Sports, News, and Weather. You'll find the commands for these at Thomas Vanhoutte's site.
Removing apps should be easier than this!