Unlocked by beating all 5 Harbingers.
Famine appears in chapter 1
Pestilence appears in chapter 2
War appears in chapter 3
Death appears in chapter 4
Conquest also appears in chapter 4, but is unlocked by taking 10 items from Angel Rooms
Using the Book of Revelations item greatly increases the chances of a Harbinger appearing as the boss for that floor, except they will not replace Mom or Mom's Heart/It Lives.