If the pain is occurring even when not doing dips the doing dips are not the cause of the pain.
Have you had anyone look at your technique when performing dips?
Yes, you may need to consciously be away of your positioning when making a change with posture. As you are consciously working on it it will become less work and just your posture. Creating new neuro pathways for the changed positioning can take some time. One impacting factor is how engrained the old position is. It is the same as breaking a bad habit and creating a new habit.
You may have a rub out.
Episode 271: Full Over Head Test and Super Couch

Episode 209: Free Your Ribs, Your Mind Will Follow