Idris,
Try the following, or a combination:
1) Make sure you shoot the forward cast high enough. Practice it on grass and see how the height allows a gentle fall of the fly. That leads too...
2) ... trying the pile cast. A high forward cast followed by dropping the rod tip. This allows the fly to fall very gently and with a series of wiggles in the leader to combat drag.
3) Make sure you are using fine enough tippet. Depends on the size of fly you're using and the size of expected fish, but 7X on small streams is my 'go to'.
4) Make sure you are using supple enough tippet.
5) Make sure you have along enough leader. A short, stout leader may add to your troubles.
As for it being thr reason your catch rates are lower, don't forget to degrease the leader - the last 24" would be fine.
Also, watch your fly size. Many fish a fly that is simply too large.