I have never heard a German get angry because you spoke English to them.
They will just say that they cannot really speak English, and then proceed to explain in perfect, yet slightly-accented English how to get to wherever you need to go.
BTW, they understood me, but they refused to talk to me back in English (not everyone)
One vendor especially, whom I purchased a slice of pizza, asked something in German, I said "sorry? I don't speak german", and she repeated it louder in a pissed off way... like talking louder will make me magically understand German...