Usually at the client site – you typically get your own badges – after all if you could only be at the client site when the client was actually there – the whole team would have to leave around 5pm everyday, right? We can't have that.:).
But it really varies by client, the urgency and, most of all, your seniors/managers. It's all up to them. Although if the team is working full blown weekends, it would be VERY rare for everyone to be doing it from their homes/apartments – auditing is a team sport.
For my teams, if it was just a few hours of work needed, it'd be done from home. As the audit ramps up, full day Saturdays and a few hours on sundays start becoming necessary – then Saturdays are usually at the client and Sunday's on your own from home. …to eventually 7 days a week at the client site.
Although I had one client where the senior got sick of the audit room so demanded weekends would be worked from a conference room at the office – although the norm is definitely the client site because you have hardcopies of support there.