VectorCAN: Fix CAN-FD frame reading and sending
When setting the CAN device to flexible data rate mode the correct
sampling point has to be set.
When invalid CAN-FD configuration is set, respond with correct
error instead of always complain about open port.
When the bit rate switch is set for the sending frame, also set the
corresponding Vector flag.
Also set it to QCanBusFrame when receiving from Vector backend.
When an payload higher than 8 is used, the DLC has to be set to the
correct indexes.
Fixes: QTBUG-99259
Change-Id: I07e91c74078be86de1c27f8c41a9b7709c9f9c48
Reviewed-by: André Hartmann <aha_1980@gmx.de>