[SYCL][Matrix spec] Remove deleted folder that came back by accident in a merge (#11015)

The last merge in #7964 brought back
accidentally a deleted folder that caused conflicts during the merge.
I am removing it again here.

Author: dkhaldi