If your Tav is a monk, they get a couple of special dialogue options that can end this without a fight. Essentially, you point out that there's no reason to keep collecting gold when the town's gone and the Toll Collector vanishes in a puff of logic.
Otherwise, I've talked to a lot of people who've had some luck using spells like Repelling Blast to punt the Toll Collector off the tower to her death.