Ghost fish can only be caught on Graveyard Island. They have an added quirk that the controls go in the opposite direction (you'll know when it happens, the game pauses to let you know that something is weird). The easiest way to do this is to turn your controller upside down and play it that way. It can be a tough battle, but with the controller upside down it's more about patience than relearning all of the directional issues.
If using a mouse, you could try to turn it around...not sure if that will work as easily.