To destroy Dark Swirls pick up a branch (there are plenty of them lying on the ground), get close and throw it with the right mouse button. Then, you can pick up the same branch, and use it on another Dark Swirl.
After getting rid of 3-4 of them, they will stop spawning. You can either go on with the game, and wait until the next chapter, or reload the chapter.
Rinse and repeat.