How are the parasitic brain worms supposed to be scary?
Do I have it wrong?
Someone puts a thing in your brain. This thing is not sentient. If it is removed from your brain, it will not be sentient. Therefore... it is not sentient!
In fact, it offers the possibility of life beyond the death of one's physical body. The brain worm can take your memories and skills and bring them to it's next host.
It would be different if the brain worms were sentient and they just used human bodies. But that isn't how the setting is described at all.
I understand it may be disgusting, a wormy buggy thingie eating part of your brain and then living in your skull doing what that part of your brain was doing, but you are then able to continue to live unharmed. Your brain continues to live with your memories and values. The worm did not have any decision making ability or memories of its own prior to implantation. And when you die, most of you memories and values can be placed in another brain, a human that would like to inherit your experiences or a less sentient brain.
There is a Terran parasite that eats part of its host and then continues to live as one of the host's organs. And yep, it's pretty disgusting, but the host does not suffer any ill health as a result. It's a bug that attaches to the inside of a fish's mouth and functions as the fishes tongue.
So yeah, icky, but so tremendously useful that I could easily see human society overcoming the ick factor to universally adopt such a useful creature.