fixed QLoRA CPU issue due to internal API change. (#1503)

Signed-off-by: Ye, Xinyu <[email protected]>
Co-authored-by: VincyZhang <[email protected]>

Author: XinyuYe-Intel

intel_extension_for_transformers/transformers/llm/finetuning/finetuning.py

@@ -303,6 +303,13 @@ def finetune(self):
                     bnb_4bit_use_double_quant=finetune_args.double_quant,
                     bnb_4bit_quant_type=finetune_args.quant_type,
                 )
+            elif training_args.device.type == "cpu":
+                self.device_map = "cpu"
+            else:
+                raise NotImplementedError(
+                    f"Unsupported device type {training_args.device.type}, only support cpu and cuda now."
+                )
+
             if finetune_args.bits not in [4, 8]:
                 raise NotImplementedError(
                     f"Unsupported bits {finetune_args.bits}, only support 4 and 8 now."