Fish can of course be caught almost anywhere in Norway.
In the fjords and at sea, in all the salt water, fishing is free.
If you want to fish in freshwater, in lakes and the like, you need a permit. They are not the same everywhere. Each municipality or region has its own. To check which permit you need and what rules you must adhere to, visit the website of Inatur https://www.inatur.no/ . Here you select the desired location and date, and you can pay immediately.
You are allowed to keep the fish you catch. Although often the small ones are thrown back so they can grow a bit more.
Tip. Take a small disposable BBQ with you. You can immediately fry the little fish. Delicious with a large piece of baguette.