In addition to the rules about animals on land and in the sky, there are rules for everything that is in the water. Fish need to have fins and scales (both!) to fall into the category of kosher. No other fish or water animals are allowed. These rules come directly from the Torah:
These ye may eat of all that are in the waters: whatsoever hath fins and scales may ye eat; and whatsoever hath not fins and scales ye shall not eat; it is unclean unto you.
(Deuteronomy / Devarim 14:9-10, there’s a similar part in Leviticus / Vayikra 11:9-12)
This includes only the "stereotypical fish", i.e., what a child will draw if you say "draw a fish". In particular it means the following is NOT kosher:
- Shrimp, crab, lobster and all other crustaceans
- Oysters, clams, mussels, and all other molluscs
- Squid, sepia, octopus and similar
- Sea cucumbers, starfish, sea urchin and similar
- Sea turtles
- Sea snakes
- Frogs and all other water-dwelling amphibians
- Crocodiles and all other water-dwelling reptiles
- Whale, dolphin and all other "fish-like" mammals (no scales)
I live very far from the sea and I do not eat fish. So I have no idea what the different sorts of fish look like. Also the halachic definitions of fins and scales are somewhat different from the scientific definitions. So for a complete list of kosher fish I’ll just link you to the CRC list of kosher fish. I’ve also seen such a list in many books or how-to-s on kashrut. When in doubt, ask your local rabbi or kashrut organisation!
There is no prohibition about fish blood, so you may kill fish any way you like. You do not have to remove the blood like you have to for mammals and fowl. So you can just go into any supermarket and buy any fresh fish as long as you can be certain that it is from a kosher species (or you buy it whole and check fins and scales). With canned or frozen fish there might be problems with additional ingredients, e.g., the oil, or the processing pipeline being used for non-kosher food also, so it’s better to rely on some supervision.
The general rules are that "what comes from a kosher animal is kosher, what comes from a non-kosher animal is non-kosher". So caviar or fish roe is only kosher if the fish it comes from is kosher. Wikipedia claims that "most caviar consists of sturgeon eggs", which might be problematic as there is controversy about whether sturgeon is kosher. The same issues apply to fish gelatine, it may be problematic if made out of non-kosher fish like shark or catfish.