What video game system would be the hardest to make?

I think difficulty depends more on the size of the team. The larger the corporation, the more that can be invested into building video game systems and doing research and development.

But aside from team size, I think video game systems are more dependent on how the games should function. Every developer has their own vision. For instance, Sony and Microsoft focus a lot on power for their consoles, so people could play games with little lag, best graphics, etc. But Nintendo, on the other hand, focuses on improving gameplay by adding gimmicks like HD Rumble, hybrid gameplay (being able to play at home or on-the-go), detachable controllers, etc.

But theoretically speaking, the video game system that would be the hardest to make is a system that caters to everyone, and trying to bundle thousands of features would be a daunting task, especially when you have other considerations (making the system lightweight, compact, etc.).