Ollama Cloud is a Highly Scalable Cloud-native Stack for Ollama. Help us shine by turning a ★ into a ⭐!

This is not a Code repository, please read Components

Development usually kicks off on your local machine, comfy and controlled.

But moving to production? That’s a huge leap — hello, delay, inconsistency, and dependence.

Ollama Cloud steps in with some solid perks:

• Consistency: If it works locally, it'll work at scale with no change to your code.
• Simplicity: Can be deployed at scale with no DevOps or SRE skills.
• A Cloud Platform: Empowers developers with self-service capabilities.

What you see locally is what you get in production. Basically, "Local Is Production". Maybe we should call it LIP 🫦, lol.

Notice the use of host calls over REST APIs, which significantly speeds up communication, enhances reliability, and ensures privacy.

Now you can focus on building amazing AI applications without worrying about the usual steps involved in taking them to production, such as DevOps or service dependencies. You can relax, as now it's a mere git push away!

Ollama Cloud is based on the following components:

• tau: The implementation of taubyte, a solution to build autonomous cloud computing platforms.
• ollama plugin: A plugin that allows calling ollama from WebAssembly.
• dreamland: A tool that allows running a taubyte based cloud on your computer for local development and E2E automated testing.

First, prepare your local environment with dreamland, tau-cli, docker, and the necessary tools to build the ollama plugin.

If you have Node.js, just run:

npm i @taubyte/dream @taubyte/cli

If not, check dreamland and tau-cli for other installation methods.

To install Docker simply use:

curl -fsSL https://get.docker.com | sh

Building the plugin requires go, but ollama itself needs:

• cmake version 3.24 or higher
• go version 1.22 or higher
• gcc version 11.4.0 or higher

On Ubuntu Linux, install gcc & cmake with:

sudo apt install build-essential

For Go, follow the steps at Go's official installation guide.

On macOS, you can install these using:

brew install go cmake gcc

If you're unsure whether you have the necessary development libraries for your acceleration hardware, please refer to https://github.com/ollama/ollama/blob/main/docs/development.md.

To build the plugin, first clone the repository:

git clone https://github.com/samyfodil/ollama.git

Then, compile the C++ backend:

cd ollama
go generate ./...

Finally, build the plugin itself:

cd tau
go build . -o plugin

That's it! You should now have a plugin executable.

With the plugin built, it's time to start a local cloud. Run:

dream new multiverse

Wait until it indicates that the universe is ready.

The Cloud you've just started lacks LLM capabilities. To add these, load the plugin by running:

dream inject attach-plugin -p /path/to/plugin

If you haven't changed directories, it should be:

dream inject attach-plugin -p $(pwd)/plugin

Now, you're all set to kick ass!

• Create your first application

This project stands on the shoulders of the projects mentioned in the introduction, as well as ollama itself. Please show your support by leaving a ⭐ if you find them useful.