Only worry about picking up items you need to progress through the game, and discard them as soon as they have served their purpose. You may need to decide whether to backtrack to use certain story items so you can discard them or to carry them all the way to the end of the game. Use the map to your advantage, so you can see the locations of items your character has spotted but hasn't picked up. The side pouches will obviously help you a lot here too. I did this with only the pistol and shotgun, using mainly knives in the boss fights and only shooting zombies to stun them so I could run past. You can't use any bonus items like infinite weapons or preorder bonuses as those start off in the item box.