This is a known issue in the Video PHY Controller v2.0 (Rev. 5) and earlier.
When the output is generated for HDMI, the register bits in the following registers (used for setting the polarity) are not connected, so the register settings are not applied in hardware.
This issue will be resolved in the Video PHY Controller v2.0 (Rev. 6) and later.