Welcome to Intel® NPU Acceleration Library’s documentation! The Intel® NPU Acceleration Library is a Python library designed to boost the efficiency of your applications by leveraging the power of the Intel Neural Processing Unit (NPU) to perform high-speed computations on compatible hardware
Basic usage — Intel® NPU Acceleration Library documentation Basic usage # For implemented examples, please check the examples folder Run a single MatMul in the NPU # from intel_npu_acceleration_library backend import MatMul import numpy as np inC, outC, batch =
intel_npu_acceleration_library package Submodules # intel_npu_acceleration_library bindings module # intel_npu_acceleration_library compiler module # class intel_npu_acceleration_library compiler CompilerConfig(use_to: bool = False, dtype: dtype | NPUDtype = torch float16, training: bool = False) # Bases: object Configuration class to store the compilation configuration of a model for the NPU intel_npu_acceleration_library
Quick overview of Intel’s Neural Processing Unit (NPU) Quick overview of Intel’s Neural Processing Unit (NPU) # The Intel NPU is an AI accelerator integrated into Intel Core Ultra processors, characterized by a unique architecture comprising compute acceleration and data transfer capabilities
Decoding LLM performance — Intel® NPU Acceleration Library documentation Static shapes allows the NN graph compiler to improve memory management, schedule and overall network performance For a example implementation, you can refer to the intel_npu_acceleration_library nn llm generate_with_static_shape or transformers library StaticCache Conclusions #
Advanced Setup — Intel® NPU Acceleration Library documentation To build the package you need a compiler in your system (Visual Studio 2019 suggested for Windows build) MacOS is not yet supported For development packages use (after cloning the repo)
Developer Guide — Intel® NPU Acceleration Library documentation It is suggested to install the package locally by using pip install -e [dev] Git hooks # All developers should install the git hooks that are tracked in the githooks directory We use the pre-commit framework for hook management The recommended way of installing it is using pip: