Drink the water.
I had written a righteous screed about how all guidebooks are just covering their asses when they tell you not to drink the water, and of course you can drink it, if normal middle-class people drink it too.
Then I went to Fes, Morocco, where everyone drinks tap water…and I got sick.
But even so, I believe that tap water is often not so horrible. If people who could afford to buy bottled water drink from the tap, you can certainly brush your teeth with it. You can even swig a bit in the night, when you realize you’ve run out of bottled water. You can have a little ice in your drink.
It’s with cumulative exposure that your system freaks out (or mine does; yours may be different–that’s my CYA). I didn’t get sick in Fes until about a week in. My threshold for Cairo tap water is about four days.
Contrary to logic, the worse the water is, the better off you are. If all the restaurants use bottled water, this means your ice is almost certainly made from purified water. Basically, there are very few situations in which you have to do that prissy “no ice, please” thing.
The reason I’m even being so macho about tap water is that plastic bottles are the world’s third-largest evil, after plastic bags and Halliburton, and I feel like a failure every time I buy bottled water. If you’re not feeling like risking it, I really recommend a Steripen. I just got one this summer–it’s fantastic. It has cut down on my water-risk-taking and makes me feel like a magician every time I use it. (But I recommend rechargeable batteries–it was due to battery fail that I was in the unfortunate Fes situation.)