Add Windows (CPU) build support

[doringeman]

Implement full Windows build support for AMD64 and ARM64 architectures. The build now uses LLVM/clang toolchain with Vulkan GPU acceleration support on AMD64, and CPU-only builds on ARM64.

Backend (terminal 1)

make -C llamacpp/ build

MODEL_RUNNER_PORT=8080 make run LOCAL_LLAMA=1

CLI (terminal 2):

make -C cmd/cli install

$ MODEL_RUNNER_HOST=http://localhost:8080 docker model ls
MODEL NAME  PARAMETERS  QUANTIZATION    ARCHITECTURE  MODEL ID      CREATED       CONTEXT  SIZE
smollm2     361.82 M    IQ2_XXS/Q4_K_M  llama         354bf30d0aa3  8 months ago           256.35 MiB

$ MODEL_RUNNER_HOST=http://localhost:8080 docker model run smollm2 hi
Hi, I'm happy to help. How can I assist you today?

[gemini-code-assist]

Summary of Changes

Hello @doringeman, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the project's build capabilities by introducing comprehensive support for Windows. It enables building on both AMD64 and ARM64 architectures, leveraging the LLVM/clang toolchain. AMD64 builds benefit from Vulkan GPU acceleration, while ARM64 builds are CPU-only. The changes streamline the build process across different operating systems by centralizing OS detection and adapting build scripts to handle platform-specific requirements, including automated dependency management for Windows.

Highlights

• Windows Build Support: Full support for building the project on Windows has been implemented, targeting both AMD64 and ARM64 architectures.
• LLVM/Clang Toolchain: The build process for Windows now consistently utilizes the LLVM/clang toolchain.
• Vulkan GPU Acceleration: AMD64 Windows builds now include support for Vulkan GPU acceleration, enhancing performance.
• Centralized OS Detection: A new Makefile.common has been introduced to centralize operating system detection logic, which is then included by other Makefiles for consistent behavior.
• Makefile Adaptations: The cmd/cli/Makefile and llamacpp/Makefile have been updated to handle Windows-specific binary extensions (.exe), plugin installation paths (USERPROFILE), and detailed build configurations for different Windows architectures.
• Automated Dependency Installation: The llamacpp/Makefile now includes automated dependency installation for Windows AMD64, leveraging Chocolatey for tools like Ninja, CMake, LLVM, and a script for the Vulkan SDK.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds Windows build support for both AMD64 and ARM64 architectures by introducing a common Makefile for OS detection and updating existing Makefiles to handle Windows-specific commands and paths. The changes are well-structured, particularly the abstraction of OS-specifics. My review focuses on improving robustness and maintainability by addressing hardcoded paths and code duplication in the Makefiles. I've suggested using variables for toolchain and SDK paths and refactoring repeated command blocks.

[gemini-code-assist]

The path to the LLVM toolchain is hardcoded. This can cause issues if LLVM is installed in a different location. It's better to define a variable for the LLVM path at the top of the file (e.g., LLVM_PATH ?= C:/Program Files/LLVM), which can be easily overridden if needed.

PATH="$(LLVM_PATH)/bin;$$PATH" cmake -B $(BUILD_DIR) \
		-DCMAKE_BUILD_TYPE=Release \
		-DGGML_NATIVE=OFF \
		-DGGML_OPENMP=OFF \
		-DDDLLAMA_BUILD_UTILS=ON \
		-GNinja \
		-DLLAMA_CURL=OFF \
		-DBUILD_SHARED_LIBS=ON \
		-DGGML_BACKEND_DL=ON \
		-DGGML_CPU_ALL_VARIANTS=ON \
		-DGGML_VULKAN=ON \
		"-DCMAKE_C_COMPILER=$(LLVM_PATH)/bin/clang.exe" \
		"-DCMAKE_CXX_COMPILER=$(LLVM_PATH)/bin/clang++.exe" \
		"-DCMAKE_RC_COMPILER=$(LLVM_PATH)/bin/llvm-rc.exe" \
		-S $(NATIVE_DIR)
	@echo "Building..."
	PATH="$(LLVM_PATH)/bin;$$PATH" cmake --build $(BUILD_DIR) --config Release

[gemini-code-assist]

Similar to the amd64 build, the path to the LLVM toolchain is hardcoded here. This should be replaced with a variable (e.g., LLVM_PATH) to make the build script more robust and configurable.

cmake -B $(BUILD_DIR) \
		-G Ninja \
		-DCMAKE_C_COMPILER="$(LLVM_PATH)/bin/clang.exe" \
		-DCMAKE_CXX_COMPILER="$(LLVM_PATH)/bin/clang++.exe" \
		-DCMAKE_BUILD_TYPE=Release \
		-DGGML_NATIVE=OFF \
		-DLLAMA_CURL=OFF \
		-S $(NATIVE_DIR)

[gemini-code-assist]

The paths for LLVM and Vulkan SDK are hardcoded. This can lead to problems if the user has installed them in non-default locations. It would be more robust to use variables for these paths, with sensible defaults. For example:

LLVM_INSTALL_PATH ?= C:\\Program Files\\LLVM
VULKAN_INSTALL_PATH ?= C:\\VulkanSDK

Then use $(LLVM_INSTALL_PATH) and $(VULKAN_INSTALL_PATH) in these commands.

[gemini-code-assist]

The current OS detection logic doesn't handle cases where uname -s returns something other than Linux or Darwin (e.g., FreeBSD). In such cases, DETECTED_OS will be empty, which could lead to confusing "Unsupported OS: " messages later. It would be better to use an if-else-if structure and set a default value for DETECTED_OS to provide a more informative message for unsupported systems.

else
    UNAME_S := $(shell uname -s)
    ifeq ($(UNAME_S),Linux)
        DETECTED_OS := Linux
    else ifeq ($(UNAME_S),Darwin)
        DETECTED_OS := macOS
    else
        DETECTED_OS := $(UNAME_S)
    endif
endif

[gemini-code-assist]

This block of code for cleaning the install directory is duplicated in the arm64 build section (lines 105-110). To improve maintainability and reduce redundancy, you could define a multi-line variable using define for these commands and call it from both places. For example:

define WINDOWS_POST_INSTALL_CLEAN
	@echo "Cleaning install directory..."
	rm -f $(INSTALL_DIR)/bin/*.py
	rm -rf $(INSTALL_DIR)/lib/cmake
	rm -rf $(INSTALL_DIR)/lib/pkgconfig
	rm -rf $(INSTALL_DIR)/include
	@echo "Build complete! Binaries are in $(INSTALL_DIR)"
endef

Then use $(call WINDOWS_POST_INSTALL_CLEAN) in both build sections.

[gemini-code-assist]

This is a duplicate of the cleanup code in the amd64 build section (lines 82-87). Consider refactoring this into a reusable block to avoid code duplication, as suggested in the comment on that section.