Edited - It has nothing to do with the crown. I am keeping what I wrote below but it is that wearing 5 pieces of azerials actually cause the four piece buff to disapear. -
So I am playing the special edition with no further mods and am at the point where I have alchemy and enchanting at 100 and going back and forth to get the bonuses to diminishing returns. Anyway I was on the second round and my enchanted gear was a percent down. it was driving me nuts and I could not figure it out but then recalled I was wearing the aetherial crown with the steed stone before as I was doing stuff in between and had carried a lot (my main use of the aetherial crown is to hold the steed stone) and you only need four pieces of azerials (im sure im butchering the name). So I swapped to it and low and behold I went a few percent ahead. No idea why this is but I have not seen a mention anywhere about it. Im not sure if having five pieces of azerials maybe messes it up or if its the aetherial crown or I guess the steed stone.