FFmpeg: Retry sw pixel formats for software video encoders too


enumerateTargetSwFormats() only tried more than one candidate pixel
format for the hw-frames-context path. Some software encoders reject
certain pixel formats (or require -strict experimental), so try to
open with different formats according to their score, similar to hw
codecs.

Pick-to: 6.12
Change-Id: I65527dcfddfb24ef74ae6b8f295974c75ff51236
Reviewed-by: Artem Dyomin <artem.dyomin@qt.io>
commit039c0cc58802c6ca1bfd6835bcb6fe01302714a4
parent9e16b640f5bc86aff245729b1e98a64c82e5e933
authorTim Blechmann <tim.blechmann@qt.io>
committerTim Blechmann <tim.blechmann@qt.io>
commit timeTue, 15 Sep 2026 00:55:33 +0000