Throughout the game, you are likely to find various PokéBalls floating around the aquarium. This is treasure for you to collect and there are two different kinds of treasure.
The most common treasure that will be seen is the one in the Golden Poké Balls. For every 5 levels your Magikarp grows, a Gold Poké Ball will arrive in the tank. This Gold Poké Ball will only give one gift, coins. The amount of coins you get is dependant upon your Magikarp's level, so Level 5 will only yield a small amount of coins while Level 55 will provide a much larger amount.
This amount can be increased by having the Lampent Lamp decoration.
The other kind of treasure comes in the Red Poké Ball. This treasure is rare and only occasionally appears in the tank, though you can trigger it every 7 hours with Popplio's Support Skill. This treasure gives one of a few random possibilities.