I looked at 10G Ethernet but it's still a bit too expensive and I couldn't see much benefit as it wouldn't be utilised much. I just upgraded my main fibre router to 2.5Gbit last year, the main distribution switch in the garage, and the one under my desk. Everything is still connected by 1Gbit switches as that's all they need.

For servers of any description, even home servers, Linux is the solution. Windows IMO is not even a contender. With drives all formatted to exFAT for unlimited file sizes. Only issue sometimes can be getting Linux to see and be seen on a Windows workgroup network.

I am also thinking like you that I might not go with a dedicated NAS moving forward, but instead just build a new Linux server with a drive array. You don't need a really powerful CPU for a server. Number of cores is more useful if anything, and built in graphics is fine too.