Feature Request: Self-hosted tts server (openai compatible) configuration

[Neptoos]

Hi,

thanks for your effort putting this together. I ordered my echokit a few days ago and I already started to setup the echokit server. So far so good. :)
But I did not see a way to configure a custom openai compatible tts service.
I tried it with this:

[tts]
platform = "Groq"
url = "http://tts-webui:7778/v1/audio/speech"
model = "global_preset"
api_key = "test"
voice = "chatterbox_optimus_prime_de"

But as it seems the url part is not used. Are you planning to add the possibility to add a way to do so? Is there a workaround I can use in the meantime?

[L-jasmine]

Thank you very much for your purchase. I hadn’t initially considered the need to use the Groq format to access other URLs. However, if this would meet your expectations, I can update the server’s code to support setting a new URL in the configuration.

[Neptoos]

I think it could work as the groq API is openai compatible and matches the parameters typical for openai compatible tts apis.

Would be great if you coud implement it.

[L-jasmine]

@Neptoos
I’ve updated the code, and you can now set the Groq URL.
If you encounter any further issues or anything doesn’t work as expected, please feel free to continue the discussion in this issue.

[Neptoos]

Ok, got it working. Was a bit of a hassle, because the base image for building the rust application was outdated and caused build issues (thanks gemini for suggesting such an old image).
Thanks for you fast reply and code changes. It indeed works with the groq config type. For the future, I think an extra config type for custom tts would be more clean.

For future reference. This is my Dockerfile that builds from source:

# syntax=docker/dockerfile:1.6

# =============================================================================
# Step 1: "Builder"
# =============================================================================
FROM rust:latest as builder

RUN apt-get update && apt-get install -y --no-install-recommends \
    git \
    build-essential \
    pkg-config \
    libssl-dev \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

RUN git clone https://github.com/second-state/echokit_server.git

WORKDIR /app/echokit_server

RUN cargo build --release


# =============================================================================
# Step 2: "Final Image"
# =============================================================================
FROM debian:bookworm-slim

WORKDIR /app

ARG TARGETPLATFORM
ARG VAD_VERSION=0.1.0

RUN set -eux; \
    apt-get update; \
    apt-get install -y --no-install-recommends \
        bash \
        ca-certificates \
        curl \
        unzip \
        libgomp1 \
        libssl3; \
    if [ "${TARGETPLATFORM}" = "linux/arm64" ]; then \
        apt-get install -y --no-install-recommends \
            libopenblas-dev \
            libgomp1; \
    fi; \
    rm -rf /var/lib/apt/lists/*

RUN set -eux; \
    CURRENT_ARCH="${TARGETPLATFORM}"; \
    if [ -z "${CURRENT_ARCH}" ]; then \
        CURRENT_ARCH="linux/amd64"; \
    fi; \
    case "${CURRENT_ARCH}" in \
    "linux/amd64") \
        curl -fsSL "https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.4.0%2Bcpu.zip" -o /tmp/libtorch.zip; \
        unzip /tmp/libtorch.zip -d /usr/local/lib; \
        rm /tmp/libtorch.zip; \
        ;; \
    "linux/arm64") \
        curl -fsSL "https://github.com/second-state/libtorch-releases/releases/download/v2.4.0/libtorch-cxx11-abi-aarch64-2.4.0.tar.gz" -o /tmp/libtorch.tar.gz; \
        tar -xzf /tmp/libtorch.tar.gz -C /usr/local/lib; \
        mv /usr/local/lib/torch /usr/local/lib/libtorch; \
        rm /tmp/libtorch.tar.gz; \
        ;; \
    *) echo "Unsupported TARGETPLATFORM: ${CURRENT_ARCH}" >&2; exit 1 ;; \
    esac; \
    if [ -f /usr/local/lib/libtorch/lib/libnvrtc-builtins.so ]; then \
        ln -sf /usr/local/lib/libtorch/lib/libnvrtc-builtins.so /usr/local/lib/libtorch/lib/libnvrtc-builtins.so.12.4; \
    fi

RUN set -eux; \
    CURRENT_ARCH="${TARGETPLATFORM}"; \
    if [ -z "${CURRENT_ARCH}" ]; then \
        CURRENT_ARCH="linux/amd64"; \
    fi; \
    case "${CURRENT_ARCH}" in \
    "linux/amd64") ARCH_SUFFIX="x86_64-unknown-linux-gnu" ;; \
    "linux/arm64") ARCH_SUFFIX="aarch64-unknown-linux-gnu" ;; \
    *) echo "Unsupported TARGETPLATFORM: ${CURRENT_ARCH}" >&2; exit 1 ;; \
    esac; \
    tmpdir="$(mktemp -d)"; \
    archive="silero_vad_server-v${VAD_VERSION}-${ARCH_SUFFIX}.tar.gz"; \
    url="https://github.com/second-state/silero_vad_server/releases/download/v${VAD_VERSION}/${archive}"; \
    curl -fsSL -o /tmp/silero_vad_server.tar.gz "${url}"; \
    tar -xzf /tmp/silero_vad_server.tar.gz -C "$tmpdir"; \
    vad_bin="$(find "$tmpdir" -type f -name 'silero_vad_server' -print -quit)"; \
    test -n "$vad_bin"; \
    install -m 0755 "$vad_bin" /usr/local/bin/silero_vad_server; \
    rm -rf "$tmpdir" /tmp/silero_vad_server.tar.gz


COPY --from=builder /app/echokit_server/target/release/echokit_server /usr/local/bin/echokit_server


RUN curl -fsSL "https://github.com/second-state/silero_vad_server/raw/refs/tags/v${VAD_VERSION}/silero_vad.jit" -o /app/silero_vad.jit


ENV LD_LIBRARY_PATH=/usr/local/lib/libtorch/lib:$LD_LIBRARY_PATH
ENV LIBTORCH=/usr/local/lib/libtorch

COPY config.toml .
COPY hello.wav .

RUN cat <<'EOF' > /usr/local/bin/start_servers.sh \
    && chmod +x /usr/local/bin/start_servers.sh
#!/usr/bin/env bash
set -euo pipefail

vad_pid=""
server_pid=""

terminate() {
    if [[ -n "$vad_pid" ]]; then
        kill -TERM "$vad_pid" 2>/dev/null || true
    fi
    if [[ -n "$server_pid" ]]; then
        kill -TERM "$server_pid" 2>/dev/null || true
    fi
}

silero_vad_server &
vad_pid=$!

echokit_server config.toml &
server_pid=$!

trap terminate SIGINT SIGTERM EXIT

wait -n "$vad_pid" "$server_pid" || true
terminate
wait || true
EOF

ENV RUST_LOG=info

CMD ["/usr/local/bin/start_servers.sh"]

This is my config.toml:

addr = "0.0.0.0:8080"
hello_wav = "hello.wav"

[tts]
platform = "Groq"
url = "http://my-tts:7778/v1/audio/speech"
model = "global_preset"
api_key = "*"
voice = "my-chatterbox_voice"

[asr]
url = "http://faster-whisper:8000/v1/audio/transcriptions"
api_key = "*"
model = "Systran/faster-whisper-large-v3"
lang = "en"
prompt = "Hello\n你好\n(noise)\n(bgm)\n(silence)\n"
# Requires a local Silero VAD server: https://github.com/second-state/silero_vad_server
# Default port is 8000, but some setups may use 9093. Update the port below if needed.
vad_url = "http://localhost:9093/v1/audio/vad"
vad_realtime_url = "ws://localhost:8000/v1/audio/realtime_vad"

[llm]
llm_chat_url = "http://ollama:11434/v1/chat/completions"
api_key = "*"
model = "ibm/granite4:3b"
history = 5

[[llm.sys_prompts]]
role = "system"
content = """
You are a helpful assistant. Answer truthfully and concisely.
"""