add CPU version of the deployment
This commit is contained in:
parent
2f7813c591
commit
3bb1117361
11
README.md
11
README.md
|
@ -38,7 +38,16 @@ A quick prototype to self-host [LibreChat](https://github.com/danny-avila/LibreC
|
||||||
6. Browse http://localhost:3080/
|
6. Browse http://localhost:3080/
|
||||||
7. Create an admin account and start chatting!
|
7. Create an admin account and start chatting!
|
||||||
|
|
||||||
The API along with the APIDoc will be available at http://localhost:8000/
|
### Steps for NO GPU (use CPU)
|
||||||
|
|
||||||
|
**Warning: This may be very slow depending on your CPU and may us a lot of RAM depending on the model**
|
||||||
|
|
||||||
|
1. Make sure your drivers are up to date.
|
||||||
|
2. Clone the repo.
|
||||||
|
3. Copy the CPU compose spec to select it. `cp docker-compose.cpu.yml docker.compose.yml`
|
||||||
|
4. Run `docker compose up`. Wait for a few minutes for the model to be downloaded and served.
|
||||||
|
5. Browse http://localhost:3080/
|
||||||
|
6. Create an admin account and start chatting!
|
||||||
|
|
||||||
## Configuring additional models
|
## Configuring additional models
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,17 @@
|
||||||
|
include:
|
||||||
|
- docker-compose.base.yml
|
||||||
|
|
||||||
|
services:
|
||||||
|
# Begin Ollama service
|
||||||
|
ollama:
|
||||||
|
image: ollama/ollama:0.1.23
|
||||||
|
restart: unless-stopped
|
||||||
|
entrypoint: /bootstrap.sh
|
||||||
|
command: mistral
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
ports:
|
||||||
|
- 11434:11434
|
||||||
|
volumes:
|
||||||
|
- ./ollama/bootstrap.sh:/bootstrap.sh:ro
|
||||||
|
- ./ollama:/root/.ollama
|
Reference in New Issue