The fire drills aren't too much of a pain - usually I'm only curious as to whether it happens to be a scheduled drill or an actual alarm (valid or not). Of course, the elevators are all recalled to the 1st floor right away - so if you are in one at the time it can be a little alarming when the car no longer responds to the buttons you push.
I happened to be down at the snack counter today during one, and it was entertaining to watch people's reactions as a crowd slowly built up by the security magnetometer - they won't let anyone through during the drill/alarm, of course, and where would they go even if they did? Most people are okay with it - they'll chat with the guards or go have a smoke - but occasionally you'll get someone who is determined to be upset about something. First they're annoyed at having to go through the search and metal detector at all; then they're upset because the guards won't run them through right now. These people often seem congenitally unable to follow directions - it's not much of a wonder they are coming in to the IRS walk-in office to talk about their tax returns.
So there it goes: another day, another green, blue, electric yellow dollar...