Disable source build cache on pocl/nvidia (#749)

* Disabled source build cache on pocl/nvidia

Co-authored-by: Matt Smith <mjsmith6@illinois.edu>

* Force pocl cache directories to be local, if they have not been specified

* partially revert 4e2ab3e942888562e25689570b7e624b0b465b7d

* empty commit to bump CI for new pocl version, run 5

---------

Co-authored-by: Matt Smith <mjsmith6@illinois.edu>