IPEX-LLM Docker Container User Guides

In this section, you will find guides related to using IPEX-LLM with Docker, covering how to:

• Overview of IPEX-LLM Containers

• Inference in Python/C++

  • GPU Inference in Python with IPEX-LLM

  • VSCode LLM Development with IPEX-LLM on Intel GPU

  • llama.cpp/Ollama/Open-WebUI with IPEX-LLM on Intel GPU

• Serving

  • FastChat with IPEX-LLM on Intel GPU

  • vLLM with IPEX-LLM on Intel GPU

  • vLLM with IPEX-LLM on Intel CPU