2025-01-29
in my case here. a single m1 mac mini 8gb.
can run only the smallest deepseek model.
using llama.cpp.
but without homebrew.
is easy still. (if the toolchain on your mac isn't all a mess—of brew packages & compiled code, like a debian linux.)
but on a brand-new m1 mac, say.
u need xcode first. app-store search & download. then agree to xcode terms.
sudo xcodebuild -license
then Command Line Tools.
xcode-select --install
at the time of writing this. (dated above.) u need cmake in order to compile llama.cpp.
a fresh new cmake tarball. (cmake*tar.gz)
but first. u need a new make binary.
(in order to compile cmake.)
if u check
make --version
u see. that the make that apple ships on your arm64 mac is for i386 architectures actually. which is no good. your make command needs to match your architecture.
so u need to download a make tarball too. (make*.tar.gz)
tar -xf make.tar && cd make
and u can read the INSTALL or README. but to compile, it's your typical
./configure && make && make install
probably quit & reopen your shell then.
and check your make command now.
make --version
is the one u just compiled. good.
now u can compile cmake.
tar -xf cmake.tar && cd cmake
again the INSTALL & or README. a little bit different here.
./bootstrap && make && make install
good.
now you're ready to compile llama.cpp.
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build && cd build
cmake ..
make
congratulations.
u downloaded that deepseek .gguf file from hugging face. move it.
mv deepseek.gguf /path/to/llama.cpp/models/
ready to run the model now. (from llama.cpp/bin.) there's different commands. but one that'll get u started.
./llama-cli -m /path/to/llama.cpp/models/deepseek.guff -p "is this thing on?" -n 150
the -m flag...am assuming is for model
the -p...prompt
the -n...believe it's number of tokens output
enjoy.
comment