Note: They are not yet used but in the long term the permutation should be
handled here since it is about input/output setup.

Note:

- direct_is_possible true/false could probably be handled in an upper/lower
  vectorization way.

- Vectorization of SF kernels should be based on cost permuted matrix sequence.