I mean, I think defining a budget is important in order to properly answer this question for the asker because what I consider reasonable may be different from what others think. Clearly they’re wrong, but let’s just sidestep that for a moment.
I think, dollar for dollar, the best device is a refurbished Steam Deck. That may feel like too much money for emulating a 25 year old game console for some people, and I respect that.
I don’t keep up with all the ARM based handhelds anymore precisely because by spending a little more on a Steam Deck, I no longer had to care about the next Anbernic/Powkiddy/Retroid/etc. being able to emulate some specific platform well enough to play that one game the last handheld model just couldn’t quite play. The Deck has enough power that it plays up through Wii U very well and Switch pretty well.
Emulated platforms combined with all the games you have available on PC means the Deck can cover the vast majority of the history of video games.
So I thought it was reasonable to spend the money one time on a Steam rather than to buy multiple, cheaper handhelds over years.