In theory, it's defenitely possible. Let's imagine a person who inherited 1 billion dollars. How their parents made that wealth has no impact on the morality of the person who inherited it. The only thing that matters is what they do with the wealth afterward.
While your argument is that a good person would give it away, that's a rather short-sighted approach. If instead, they invest that wealth at a 10% return and donate 99% of the returns to charity after 11 years, they have contributed more than if they would have immediately given away their entire wealth. But they are still considered a billionaire. And they are still making 1 million a year for themselves.
So just because someone has a high net worth doesn’t prevent them from being good. And you don’t have to play any “games” or exploit anyone to stay rich. Obviously, the exact numbers might not match the oversimplified version I presented. So instead of 11 years, it might take 15-20 years to give more than donating everything at once. But if set up correctly this could pay out money to people in need forever. Adjust the payments so that the initial sum keeps up with inflation and you could in theory pay out around 70 million in today's money a year for eternity.