I have written a simple compiler in Haskell using parser combinators for parsing. And I couldn't disagree more with you. It was super straightforward writing the parser and it worked immediately. Why do say they are tricky?
Unfortunately this was years ago so I'm not sure I can dig that far into specifics.
Overall, combinators (almost by definition) have the property that you can plug anything into anything else. Haskell encourages a "zero-point" style where you don't even name the parameters (this is sort of the point of parser combinator libraries) and provides a bunch of fancy operators to glue them together in various ways.
I think the mistakes I was making were mainly in just gluing the combinators together in the wrong way. It's not that the grammar was ambiguous, I just messed it up.
You could say some of the same things about CFGs, and I guess that's true at some level. But in my opinion the syntax is more obvious and you don't usually spend time squinting at the syntax to remember how $ works or what order things execute in. And like I said, with parser combinators, there is no feedback (because everything compiles, more or less), so the only way to tell if it's working is to run it, which is a very un-Haskell-like experience.