[Feature]: Request for NPU Support on New Surface Devices to Enhance AI Model Execution

[behroozbc]

Background & Description

LLamaSharp currently supports efficient execution of AI models on CPU and GPU. With the introduction of Neural Processing Units (NPUs) in new Surface devices (e.g., Surface Pro 11 and Surface Laptop 7 powered by Snapdragon X Elite), adding NPU support could significantly improve the performance and speed of AI model execution.

Objective:

Leverage NPU hardware acceleration for faster AI model inference.
Enhance user experience for Surface device owners with NPU capabilities.
Technical Details:

Target Platform: New Surface devices with NPU (e.g., Qualcomm Snapdragon X Elite).
Benefits: NPUs are optimized for matrix operations and deep learning tasks, offloading compute from CPU/GPU.

API & Usage

No response

How to implement

No response

[martindevans]

Is this something llama.cpp already has support for?

[behroozbc]

I found out this issue related to it,
ggml-org/llama.cpp#7772

[martindevans]

It looks like there's a lot of work left to do in llama.cpp before we can support anything on our end.