The desktop OS which people call "Linux" actually consists of a lot of components added on top of the Linux kernel itself. You can choose among many desktop environments, including KDE, GNOME, Xfce, Unity and more. There are even alternatives for various lower level packages. This is nice in terms of how it gives the user a lot of choice, but it leads to a terrible mess in terms of configuration. The same setting is affected by multiple configuration options. If you change desktop environments, you have to re-do some settings. Lower level packages have settings which conflict with desktop environment settings, leading to various results. Setting a lower-level configuration setting may cause the desktop environment to disallow changes, or it may set its own settings when it starts, overriding the lower level setting.
Another issue in Linux is that things change a lot. GNOME made a drastic change with GNOME 3, and KDE made a pretty big change with Plasma 5. I find that desktop environments are best when they intelligently evolve and improve over time. These drastic changes throw away progress and even features, and introduce bugs. Because of them I switched desktop environments several times. I used GNOME 2, then hated GNOME 3 when it came out, tried Xfce for a bit, and then switched to KDE. Then Plasma 5 sucked when it came out, and I switched to Cinnamon. Now I may be switching back to Xfce because it has improved. Lower level things also change. A lot changed when Ubuntu switched to systemd. Various settings and scripts from before needed to be moved and changed. Some packages still install scripts which are ignored since the switch to systemd.
Linux is also quite buggy. Even basic functionality like display of battery level in Cinnamon can be broken. Often there are ways around the problem, but it's still additional work you need to do to get things to work. Linux has excellent driver support in terms of the number of drivers, but bugs can be a problem there also.
In terms of performance, Linux had a much more noticeable improvement when going from 2 GB to 6 GB RAM, and when installing an SSD. It seemed slower than Windows before, and those upgrades helped it catch up. Maybe it is even a bit faster now. The fact it was slower before probably means that Linux is less efficient in terms of memory use and caching, but now that doesn't matter much anymore,
I used to judge Linux harshly based on its shortcomings. The only way to like it is to like the process of fixing these problems and customizing things to improve my experience. Because of the way things change fairly often, I don't feel that I'm learning knowledge that's valuable in a long-term way. It's more like I'm playing a video game designed by the developers. That seems okay for now.
The main factor that's driving me toward Linux is the direction Windows has taken recently. With Windows 8, Microsoft did something similar to GNOME 3. It's was a big change which made the desktop experience worse. Even the Windows 10 previews didn't seem like an improvement over Windows 7. Though I tried a recent preview to explore the Ubuntu in Windows feature, and found that Windows 10 is pretty good. I'm still not sure if it is an improvement in terms of desktop experience compared to Windows 7, but it is okay. There is still the decrease in user freedom and increased sending of data to Microsoft. So, I expect to continue using Linux as my primary OS.