Commit 04e15037 authored by tashfeenahmed's avatar tashfeenahmed

Initial release of FreeLLMAPI

Self-hosted OpenAI-compatible proxy that aggregates the free tiers of
fourteen LLM providers — Google, Groq, Cerebras, SambaNova, NVIDIA,
Mistral, OpenRouter, GitHub Models, Hugging Face, Cohere, Cloudflare,
Zhipu, Moonshot, MiniMax — behind a single /v1/chat/completions endpoint.

Server:
- Express + SQLite, per-provider adapters with streaming and non-streaming
  support, automatic fallover on 429/5xx, per-key RPM/RPD/TPM/TPD tracking,
  sticky sessions for multi-turn, AES-256-GCM encrypted key storage,
  unified bearer-token auth, periodic health checks.

Client:
- React + Vite + shadcn/ui admin dashboard: keys, fallback chain (drag
  to reorder, color-coded per-provider monthly token budget), playground,
  analytics with per-provider breakdowns.

Tooling:
- GitHub Actions CI (server tests + client build), MIT license,
  README with provider-by-provider ToS review.

For personal experimentation, not production.
parents
# Server encryption key for API key storage (generate with: node -e "console.log(require('crypto').randomBytes(32).toString('hex'))")
ENCRYPTION_KEY=your-64-char-hex-key-here
# Server port (default: 3001)
PORT=3001
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test:
name: Test & build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm install
- name: Run server tests
run: npm test -w server
- name: Build server
run: npm run build -w server
- name: Build client
run: npm run build -w client
node_modules/
dist/
server/data/
*.db
*.db-wal
*.db-shm
.env
.env.local
.DS_Store
# Personal deployment scripts (contain keys/credentials — kept local)
deploy-pi.sh
update-hermes.sh
MIT License
Copyright (c) 2026 Tashfeen Ahmed
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
This diff is collapsed.
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
# React + TypeScript + Vite
This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
Currently, two official plugins are available:
- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Oxc](https://oxc.rs)
- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/)
## React Compiler
The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
## Expanding the ESLint configuration
If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules:
```js
export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
// Other configs...
// Remove tseslint.configs.recommended and replace with this
tseslint.configs.recommendedTypeChecked,
// Alternatively, use this for stricter rules
tseslint.configs.strictTypeChecked,
// Optionally, add this for stylistic rules
tseslint.configs.stylisticTypeChecked,
// Other configs...
],
languageOptions: {
parserOptions: {
project: ['./tsconfig.node.json', './tsconfig.app.json'],
tsconfigRootDir: import.meta.dirname,
},
// other options...
},
},
])
```
You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules:
```js
// eslint.config.js
import reactX from 'eslint-plugin-react-x'
import reactDom from 'eslint-plugin-react-dom'
export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
// Other configs...
// Enable lint rules for React
reactX.configs['recommended-typescript'],
// Enable lint rules for React DOM
reactDom.configs.recommended,
],
languageOptions: {
parserOptions: {
project: ['./tsconfig.node.json', './tsconfig.app.json'],
tsconfigRootDir: import.meta.dirname,
},
// other options...
},
},
])
```
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "base-nova",
"rsc": false,
"tsx": true,
"tailwind": {
"config": "",
"css": "src/index.css",
"baseColor": "neutral",
"cssVariables": true,
"prefix": ""
},
"iconLibrary": "lucide",
"rtl": false,
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
"ui": "@/components/ui",
"lib": "@/lib",
"hooks": "@/hooks"
},
"menuColor": "default",
"menuAccent": "subtle",
"registries": {}
}
import js from '@eslint/js'
import globals from 'globals'
import reactHooks from 'eslint-plugin-react-hooks'
import reactRefresh from 'eslint-plugin-react-refresh'
import tseslint from 'typescript-eslint'
import { defineConfig, globalIgnores } from 'eslint/config'
export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
js.configs.recommended,
tseslint.configs.recommended,
reactHooks.configs.flat.recommended,
reactRefresh.configs.vite,
],
languageOptions: {
ecmaVersion: 2020,
globals: globals.browser,
},
},
])
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>FreeLLMAPI · Unified LLM Router</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
{
"name": "@freellmapi/client",
"private": true,
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc -b && vite build",
"lint": "eslint .",
"preview": "vite preview"
},
"dependencies": {
"@base-ui/react": "^1.3.0",
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@fontsource-variable/geist": "^5.2.8",
"@fontsource-variable/geist-mono": "^5.2.7",
"@tailwindcss/vite": "^4.2.2",
"@tanstack/react-query": "^5.97.0",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"lucide-react": "^1.8.0",
"react": "^19.2.4",
"react-dom": "^19.2.4",
"react-router-dom": "^7.14.0",
"recharts": "^3.8.1",
"shadcn": "^4.2.0",
"tailwind-merge": "^3.5.0",
"tailwindcss": "^4.2.2",
"tw-animate-css": "^1.4.0"
},
"devDependencies": {
"@eslint/js": "^9.39.4",
"@types/node": "^24.12.2",
"@types/react": "^19.2.14",
"@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^6.0.1",
"eslint": "^9.39.4",
"eslint-plugin-react-hooks": "^7.0.1",
"eslint-plugin-react-refresh": "^0.5.2",
"globals": "^17.4.0",
"typescript": "~6.0.2",
"typescript-eslint": "^8.58.0",
"vite": "^8.0.4"
}
}
<svg xmlns="http://www.w3.org/2000/svg" width="48" height="46" fill="none" viewBox="0 0 48 46"><path fill="#863bff" d="M25.946 44.938c-.664.845-2.021.375-2.021-.698V33.937a2.26 2.26 0 0 0-2.262-2.262H10.287c-.92 0-1.456-1.04-.92-1.788l7.48-10.471c1.07-1.497 0-3.578-1.842-3.578H1.237c-.92 0-1.456-1.04-.92-1.788L10.013.474c.214-.297.556-.474.92-.474h28.894c.92 0 1.456 1.04.92 1.788l-7.48 10.471c-1.07 1.498 0 3.579 1.842 3.579h11.377c.943 0 1.473 1.088.89 1.83L25.947 44.94z" style="fill:#863bff;fill:color(display-p3 .5252 .23 1);fill-opacity:1"/><mask id="a" width="48" height="46" x="0" y="0" maskUnits="userSpaceOnUse" style="mask-type:alpha"><path fill="#000" d="M25.842 44.938c-.664.844-2.021.375-2.021-.698V33.937a2.26 2.26 0 0 0-2.262-2.262H10.183c-.92 0-1.456-1.04-.92-1.788l7.48-10.471c1.07-1.498 0-3.579-1.842-3.579H1.133c-.92 0-1.456-1.04-.92-1.787L9.91.473c.214-.297.556-.474.92-.474h28.894c.92 0 1.456 1.04.92 1.788l-7.48 10.471c-1.07 1.498 0 3.578 1.842 3.578h11.377c.943 0 1.473 1.088.89 1.832L25.843 44.94z" style="fill:#000;fill-opacity:1"/></mask><g mask="url(#a)"><g filter="url(#b)"><ellipse cx="5.508" cy="14.704" fill="#ede6ff" rx="5.508" ry="14.704" style="fill:#ede6ff;fill:color(display-p3 .9275 .9033 1);fill-opacity:1" transform="matrix(.00324 1 1 -.00324 -4.47 31.516)"/></g><g filter="url(#c)"><ellipse cx="10.399" cy="29.851" fill="#ede6ff" rx="10.399" ry="29.851" style="fill:#ede6ff;fill:color(display-p3 .9275 .9033 1);fill-opacity:1" transform="matrix(.00324 1 1 -.00324 -39.328 7.883)"/></g><g filter="url(#d)"><ellipse cx="5.508" cy="30.487" fill="#7e14ff" rx="5.508" ry="30.487" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(89.814 -25.913 -14.639)scale(1 -1)"/></g><g filter="url(#e)"><ellipse cx="5.508" cy="30.599" fill="#7e14ff" rx="5.508" ry="30.599" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(89.814 -32.644 -3.334)scale(1 -1)"/></g><g filter="url(#f)"><ellipse cx="5.508" cy="30.599" fill="#7e14ff" rx="5.508" ry="30.599" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="matrix(.00324 1 1 -.00324 -34.34 30.47)"/></g><g filter="url(#g)"><ellipse cx="14.072" cy="22.078" fill="#ede6ff" rx="14.072" ry="22.078" style="fill:#ede6ff;fill:color(display-p3 .9275 .9033 1);fill-opacity:1" transform="rotate(93.35 24.506 48.493)scale(-1 1)"/></g><g filter="url(#h)"><ellipse cx="3.47" cy="21.501" fill="#7e14ff" rx="3.47" ry="21.501" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(89.009 28.708 47.59)scale(-1 1)"/></g><g filter="url(#i)"><ellipse cx="3.47" cy="21.501" fill="#7e14ff" rx="3.47" ry="21.501" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(89.009 28.708 47.59)scale(-1 1)"/></g><g filter="url(#j)"><ellipse cx=".387" cy="8.972" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(39.51 .387 8.972)"/></g><g filter="url(#k)"><ellipse cx="47.523" cy="-6.092" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(37.892 47.523 -6.092)"/></g><g filter="url(#l)"><ellipse cx="41.412" cy="6.333" fill="#47bfff" rx="5.971" ry="9.665" style="fill:#47bfff;fill:color(display-p3 .2799 .748 1);fill-opacity:1" transform="rotate(37.892 41.412 6.333)"/></g><g filter="url(#m)"><ellipse cx="-1.879" cy="38.332" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(37.892 -1.88 38.332)"/></g><g filter="url(#n)"><ellipse cx="-1.879" cy="38.332" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(37.892 -1.88 38.332)"/></g><g filter="url(#o)"><ellipse cx="35.651" cy="29.907" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(37.892 35.651 29.907)"/></g><g filter="url(#p)"><ellipse cx="38.418" cy="32.4" fill="#47bfff" rx="5.971" ry="15.297" style="fill:#47bfff;fill:color(display-p3 .2799 .748 1);fill-opacity:1" transform="rotate(37.892 38.418 32.4)"/></g></g><defs><filter id="b" width="60.045" height="41.654" x="-19.77" y="16.149" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="7.659"/></filter><filter id="c" width="90.34" height="51.437" x="-54.613" y="-7.533" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="7.659"/></filter><filter id="d" width="79.355" height="29.4" x="-49.64" y="2.03" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="e" width="79.579" height="29.4" x="-45.045" y="20.029" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="f" width="79.579" height="29.4" x="-43.513" y="21.178" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="g" width="74.749" height="58.852" x="15.756" y="-17.901" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="7.659"/></filter><filter id="h" width="61.377" height="25.362" x="23.548" y="2.284" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="i" width="61.377" height="25.362" x="23.548" y="2.284" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="j" width="56.045" height="63.649" x="-27.636" y="-22.853" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="k" width="54.814" height="64.646" x="20.116" y="-38.415" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="l" width="33.541" height="35.313" x="24.641" y="-11.323" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="m" width="54.814" height="64.646" x="-29.286" y="6.009" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="n" width="54.814" height="64.646" x="-29.286" y="6.009" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="o" width="54.814" height="64.646" x="8.244" y="-2.416" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter><filter id="p" width="39.409" height="43.623" x="18.713" y="10.588" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="4.596"/></filter></defs></svg>
\ No newline at end of file
<svg xmlns="http://www.w3.org/2000/svg">
<symbol id="bluesky-icon" viewBox="0 0 16 17">
<g clip-path="url(#bluesky-clip)"><path fill="#08060d" d="M7.75 7.735c-.693-1.348-2.58-3.86-4.334-5.097-1.68-1.187-2.32-.981-2.74-.79C.188 2.065.1 2.812.1 3.251s.241 3.602.398 4.13c.52 1.744 2.367 2.333 4.07 2.145-2.495.37-4.71 1.278-1.805 4.512 3.196 3.309 4.38-.71 4.987-2.746.608 2.036 1.307 5.91 4.93 2.746 2.72-2.746.747-4.143-1.747-4.512 1.702.189 3.55-.4 4.07-2.145.156-.528.397-3.691.397-4.13s-.088-1.186-.575-1.406c-.42-.19-1.06-.395-2.741.79-1.755 1.24-3.64 3.752-4.334 5.099"/></g>
<defs><clipPath id="bluesky-clip"><path fill="#fff" d="M.1.85h15.3v15.3H.1z"/></clipPath></defs>
</symbol>
<symbol id="discord-icon" viewBox="0 0 20 19">
<path fill="#08060d" d="M16.224 3.768a14.5 14.5 0 0 0-3.67-1.153c-.158.286-.343.67-.47.976a13.5 13.5 0 0 0-4.067 0c-.128-.306-.317-.69-.476-.976A14.4 14.4 0 0 0 3.868 3.77C1.546 7.28.916 10.703 1.231 14.077a14.7 14.7 0 0 0 4.5 2.306q.545-.748.965-1.587a9.5 9.5 0 0 1-1.518-.74q.191-.14.372-.293c2.927 1.369 6.107 1.369 8.999 0q.183.152.372.294-.723.437-1.52.74.418.838.963 1.588a14.6 14.6 0 0 0 4.504-2.308c.37-3.911-.63-7.302-2.644-10.309m-9.13 8.234c-.878 0-1.599-.82-1.599-1.82 0-.998.705-1.82 1.6-1.82.894 0 1.614.82 1.599 1.82.001 1-.705 1.82-1.6 1.82m5.91 0c-.878 0-1.599-.82-1.599-1.82 0-.998.705-1.82 1.6-1.82.893 0 1.614.82 1.599 1.82 0 1-.706 1.82-1.6 1.82"/>
</symbol>
<symbol id="documentation-icon" viewBox="0 0 21 20">
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="m15.5 13.333 1.533 1.322c.645.555.967.833.967 1.178s-.322.623-.967 1.179L15.5 18.333m-3.333-5-1.534 1.322c-.644.555-.966.833-.966 1.178s.322.623.966 1.179l1.534 1.321"/>
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="M17.167 10.836v-4.32c0-1.41 0-2.117-.224-2.68-.359-.906-1.118-1.621-2.08-1.96-.599-.21-1.349-.21-2.848-.21-2.623 0-3.935 0-4.983.369-1.684.591-3.013 1.842-3.641 3.428C3 6.449 3 7.684 3 10.154v2.122c0 2.558 0 3.838.706 4.726q.306.383.713.671c.76.536 1.79.64 3.581.66"/>
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="M3 10a2.78 2.78 0 0 1 2.778-2.778c.555 0 1.209.097 1.748-.047.48-.129.854-.503.982-.982.145-.54.048-1.194.048-1.749a2.78 2.78 0 0 1 2.777-2.777"/>
</symbol>
<symbol id="github-icon" viewBox="0 0 19 19">
<path fill="#08060d" fill-rule="evenodd" d="M9.356 1.85C5.05 1.85 1.57 5.356 1.57 9.694a7.84 7.84 0 0 0 5.324 7.44c.387.079.528-.168.528-.376 0-.182-.013-.805-.013-1.454-2.165.467-2.616-.935-2.616-.935-.349-.91-.864-1.143-.864-1.143-.71-.48.051-.48.051-.48.787.051 1.2.805 1.2.805.695 1.194 1.817.857 2.268.649.064-.507.27-.857.49-1.052-1.728-.182-3.545-.857-3.545-3.87 0-.857.31-1.558.8-2.104-.078-.195-.349-1 .077-2.078 0 0 .657-.208 2.14.805a7.5 7.5 0 0 1 1.946-.26c.657 0 1.328.092 1.946.26 1.483-1.013 2.14-.805 2.14-.805.426 1.078.155 1.883.078 2.078.502.546.799 1.247.799 2.104 0 3.013-1.818 3.675-3.558 3.87.284.247.528.714.528 1.454 0 1.052-.012 1.896-.012 2.156 0 .208.142.455.528.377a7.84 7.84 0 0 0 5.324-7.441c.013-4.338-3.48-7.844-7.773-7.844" clip-rule="evenodd"/>
</symbol>
<symbol id="social-icon" viewBox="0 0 20 20">
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="M12.5 6.667a4.167 4.167 0 1 0-8.334 0 4.167 4.167 0 0 0 8.334 0"/>
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="M2.5 16.667a5.833 5.833 0 0 1 8.75-5.053m3.837.474.513 1.035c.07.144.257.282.414.309l.93.155c.596.1.736.536.307.965l-.723.73a.64.64 0 0 0-.152.531l.207.903c.164.715-.213.991-.84.618l-.872-.52a.63.63 0 0 0-.577 0l-.872.52c-.624.373-1.003.094-.84-.618l.207-.903a.64.64 0 0 0-.152-.532l-.723-.729c-.426-.43-.289-.864.306-.964l.93-.156a.64.64 0 0 0 .412-.31l.513-1.034c.28-.562.735-.562 1.012 0"/>
</symbol>
<symbol id="x-icon" viewBox="0 0 19 19">
<path fill="#08060d" fill-rule="evenodd" d="M1.893 1.98c.052.072 1.245 1.769 2.653 3.77l2.892 4.114c.183.261.333.48.333.486s-.068.089-.152.183l-.522.593-.765.867-3.597 4.087c-.375.426-.734.834-.798.905a1 1 0 0 0-.118.148c0 .01.236.017.664.017h.663l.729-.83c.4-.457.796-.906.879-.999a692 692 0 0 0 1.794-2.038c.034-.037.301-.34.594-.675l.551-.624.345-.392a7 7 0 0 1 .34-.374c.006 0 .93 1.306 2.052 2.903l2.084 2.965.045.063h2.275c1.87 0 2.273-.003 2.266-.021-.008-.02-1.098-1.572-3.894-5.547-2.013-2.862-2.28-3.246-2.273-3.266.008-.019.282-.332 2.085-2.38l2-2.274 1.567-1.782c.022-.028-.016-.03-.65-.03h-.674l-.3.342a871 871 0 0 1-1.782 2.025c-.067.075-.405.458-.75.852a100 100 0 0 1-.803.91c-.148.172-.299.344-.99 1.127-.304.343-.32.358-.345.327-.015-.019-.904-1.282-1.976-2.808L6.365 1.85H1.8zm1.782.91 8.078 11.294c.772 1.08 1.413 1.973 1.425 1.984.016.017.241.02 1.05.017l1.03-.004-2.694-3.766L7.796 5.75 5.722 2.852l-1.039-.004-1.039-.004z" clip-rule="evenodd"/>
</symbol>
</svg>
import { useEffect, useState } from 'react'
import { BrowserRouter, Routes, Route, Navigate, NavLink } from 'react-router-dom'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { Button } from '@/components/ui/button'
import KeysPage from '@/pages/KeysPage'
import PlaygroundPage from '@/pages/PlaygroundPage'
import FallbackPage from '@/pages/FallbackPage'
import AnalyticsPage from '@/pages/AnalyticsPage'
const queryClient = new QueryClient()
function NavItem({ to, children }: { to: string; children: React.ReactNode }) {
return (
<NavLink
to={to}
className={({ isActive }) =>
`relative text-sm px-1 py-4 transition-colors ${
isActive
? 'text-foreground after:absolute after:inset-x-0 after:-bottom-px after:h-px after:bg-foreground'
: 'text-muted-foreground hover:text-foreground'
}`
}
>
{children}
</NavLink>
)
}
function DarkModeToggle() {
const [dark, setDark] = useState(() =>
typeof window !== 'undefined' && document.documentElement.classList.contains('dark')
)
useEffect(() => {
const stored = localStorage.getItem('theme')
if (stored === 'dark' || (!stored && window.matchMedia('(prefers-color-scheme: dark)').matches)) {
document.documentElement.classList.add('dark')
setDark(true)
}
}, [])
function toggle() {
const next = !dark
setDark(next)
document.documentElement.classList.toggle('dark', next)
localStorage.setItem('theme', next ? 'dark' : 'light')
}
return (
<Button variant="ghost" size="sm" onClick={toggle} aria-label="Toggle theme">
{dark ? (
<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><circle cx="12" cy="12" r="4"/><path d="M12 2v2"/><path d="M12 20v2"/><path d="m4.93 4.93 1.41 1.41"/><path d="m17.66 17.66 1.41 1.41"/><path d="M2 12h2"/><path d="M20 12h2"/><path d="m6.34 17.66-1.41 1.41"/><path d="m19.07 4.93-1.41 1.41"/></svg>
) : (
<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M12 3a6 6 0 0 0 9 9 9 9 0 1 1-9-9Z"/></svg>
)}
</Button>
)
}
function Brand() {
return (
<div className="flex items-center gap-2">
<span className="inline-block size-2 rounded-full bg-foreground" />
<span className="font-semibold tracking-tight text-sm">FreeLLMAPI</span>
</div>
)
}
function App() {
return (
<QueryClientProvider client={queryClient}>
<BrowserRouter basename={import.meta.env.BASE_URL}>
<div className="min-h-screen bg-background">
<header className="sticky top-0 z-40 bg-background/80 backdrop-blur border-b">
<div className="max-w-6xl mx-auto px-6 flex items-center">
<Brand />
<nav className="flex items-center gap-6 ml-10">
<NavItem to="/playground">Playground</NavItem>
<NavItem to="/keys">Keys</NavItem>
<NavItem to="/fallback">Fallback</NavItem>
<NavItem to="/analytics">Analytics</NavItem>
</nav>
<div className="ml-auto py-2">
<DarkModeToggle />
</div>
</div>
</header>
<main className="max-w-6xl mx-auto px-6 py-8">
<Routes>
<Route path="/" element={<Navigate to="/playground" replace />} />
<Route path="/playground" element={<PlaygroundPage />} />
<Route path="/keys" element={<KeysPage />} />
<Route path="/fallback" element={<FallbackPage />} />
<Route path="/analytics" element={<AnalyticsPage />} />
<Route path="/test" element={<Navigate to="/playground" replace />} />
<Route path="/health" element={<Navigate to="/keys" replace />} />
</Routes>
</main>
</div>
</BrowserRouter>
</QueryClientProvider>
)
}
export default App
<svg xmlns="http://www.w3.org/2000/svg" width="77" height="47" fill="none" aria-labelledby="vite-logo-title" viewBox="0 0 77 47"><title id="vite-logo-title">Vite</title><style>.parenthesis{fill:#000}@media (prefers-color-scheme:dark){.parenthesis{fill:#fff}}</style><path fill="#9135ff" d="M40.151 45.71c-.663.844-2.02.374-2.02-.699V34.708a2.26 2.26 0 0 0-2.262-2.262H24.493c-.92 0-1.457-1.04-.92-1.788l7.479-10.471c1.07-1.498 0-3.578-1.842-3.578H15.443c-.92 0-1.456-1.04-.92-1.788l9.696-13.576c.213-.297.556-.474.92-.474h28.894c.92 0 1.456 1.04.92 1.788l-7.48 10.472c-1.07 1.497 0 3.578 1.842 3.578h11.376c.944 0 1.474 1.087.89 1.83L40.153 45.712z"/><mask id="a" width="48" height="47" x="14" y="0" maskUnits="userSpaceOnUse" style="mask-type:alpha"><path fill="#000" d="M40.047 45.71c-.663.843-2.02.374-2.02-.699V34.708a2.26 2.26 0 0 0-2.262-2.262H24.389c-.92 0-1.457-1.04-.92-1.788l7.479-10.472c1.07-1.497 0-3.578-1.842-3.578H15.34c-.92 0-1.456-1.04-.92-1.788l9.696-13.575c.213-.297.556-.474.92-.474H53.93c.92 0 1.456 1.04.92 1.788L47.37 13.03c-1.07 1.498 0 3.578 1.842 3.578h11.376c.944 0 1.474 1.088.89 1.831L40.049 45.712z"/></mask><g mask="url(#a)"><g filter="url(#b)"><ellipse cx="5.508" cy="14.704" fill="#eee6ff" rx="5.508" ry="14.704" transform="rotate(269.814 20.96 11.29)scale(-1 1)"/></g><g filter="url(#c)"><ellipse cx="10.399" cy="29.851" fill="#eee6ff" rx="10.399" ry="29.851" transform="rotate(89.814 -16.902 -8.275)scale(1 -1)"/></g><g filter="url(#d)"><ellipse cx="5.508" cy="30.487" fill="#8900ff" rx="5.508" ry="30.487" transform="rotate(89.814 -19.197 -7.127)scale(1 -1)"/></g><g filter="url(#e)"><ellipse cx="5.508" cy="30.599" fill="#8900ff" rx="5.508" ry="30.599" transform="rotate(89.814 -25.928 4.177)scale(1 -1)"/></g><g filter="url(#f)"><ellipse cx="5.508" cy="30.599" fill="#8900ff" rx="5.508" ry="30.599" transform="rotate(89.814 -25.738 5.52)scale(1 -1)"/></g><g filter="url(#g)"><ellipse cx="14.072" cy="22.078" fill="#eee6ff" rx="14.072" ry="22.078" transform="rotate(93.35 31.245 55.578)scale(-1 1)"/></g><g filter="url(#h)"><ellipse cx="3.47" cy="21.501" fill="#8900ff" rx="3.47" ry="21.501" transform="rotate(89.009 35.419 55.202)scale(-1 1)"/></g><g filter="url(#i)"><ellipse cx="3.47" cy="21.501" fill="#8900ff" rx="3.47" ry="21.501" transform="rotate(89.009 35.419 55.202)scale(-1 1)"/></g><g filter="url(#j)"><ellipse cx="14.592" cy="9.743" fill="#8900ff" rx="4.407" ry="29.108" transform="rotate(39.51 14.592 9.743)"/></g><g filter="url(#k)"><ellipse cx="61.728" cy="-5.321" fill="#8900ff" rx="4.407" ry="29.108" transform="rotate(37.892 61.728 -5.32)"/></g><g filter="url(#l)"><ellipse cx="55.618" cy="7.104" fill="#00c2ff" rx="5.971" ry="9.665" transform="rotate(37.892 55.618 7.104)"/></g><g filter="url(#m)"><ellipse cx="12.326" cy="39.103" fill="#8900ff" rx="4.407" ry="29.108" transform="rotate(37.892 12.326 39.103)"/></g><g filter="url(#n)"><ellipse cx="12.326" cy="39.103" fill="#8900ff" rx="4.407" ry="29.108" transform="rotate(37.892 12.326 39.103)"/></g><g filter="url(#o)"><ellipse cx="49.857" cy="30.678" fill="#8900ff" rx="4.407" ry="29.108" transform="rotate(37.892 49.857 30.678)"/></g><g filter="url(#p)"><ellipse cx="52.623" cy="33.171" fill="#00c2ff" rx="5.971" ry="15.297" transform="rotate(37.892 52.623 33.17)"/></g></g><path d="M6.919 0c-9.198 13.166-9.252 33.575 0 46.789h6.215c-9.25-13.214-9.196-33.623 0-46.789zm62.424 0h-6.215c9.198 13.166 9.252 33.575 0 46.789h6.215c9.25-13.214 9.196-33.623 0-46.789" class="parenthesis"/><defs><filter id="b" width="60.045" height="41.654" x="-5.564" y="16.92" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="7.659"/></filter><filter id="c" width="90.34" height="51.437" x="-40.407" y="-6.762" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="7.659"/></filter><filter id="d" width="79.355" height="29.4" x="-35.435" y="2.801" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="e" width="79.579" height="29.4" x="-30.84" y="20.8" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="f" width="79.579" height="29.4" x="-29.307" y="21.949" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="g" width="74.749" height="58.852" x="29.961" y="-17.13" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="7.659"/></filter><filter id="h" width="61.377" height="25.362" x="37.754" y="3.055" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="i" width="61.377" height="25.362" x="37.754" y="3.055" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="j" width="56.045" height="63.649" x="-13.43" y="-22.082" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="k" width="54.814" height="64.646" x="34.321" y="-37.644" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="l" width="33.541" height="35.313" x="38.847" y="-10.552" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="m" width="54.814" height="64.646" x="-15.081" y="6.78" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="n" width="54.814" height="64.646" x="-15.081" y="6.78" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="o" width="54.814" height="64.646" x="22.45" y="-1.645" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter><filter id="p" width="39.409" height="43.623" x="32.919" y="11.36" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17286" stdDeviation="4.596"/></filter></defs></svg>
import type { ReactNode } from 'react'
export function PageHeader({
title,
description,
actions,
}: {
title: string
description?: string
actions?: ReactNode
}) {
return (
<div className="flex items-end justify-between gap-6 pb-6 mb-6 border-b">
<div className="min-w-0">
<h1 className="text-2xl font-semibold tracking-tight">{title}</h1>
{description && (
<p className="text-sm text-muted-foreground mt-1">{description}</p>
)}
</div>
{actions && <div className="flex items-center gap-2 shrink-0">{actions}</div>}
</div>
)
}
import { mergeProps } from "@base-ui/react/merge-props"
import { useRender } from "@base-ui/react/use-render"
import { cva, type VariantProps } from "class-variance-authority"
import { cn } from "@/lib/utils"
const badgeVariants = cva(
"group/badge inline-flex h-5 w-fit shrink-0 items-center justify-center gap-1 overflow-hidden rounded-4xl border border-transparent px-2 py-0.5 text-xs font-medium whitespace-nowrap transition-all focus-visible:border-ring focus-visible:ring-[3px] focus-visible:ring-ring/50 has-data-[icon=inline-end]:pr-1.5 has-data-[icon=inline-start]:pl-1.5 aria-invalid:border-destructive aria-invalid:ring-destructive/20 dark:aria-invalid:ring-destructive/40 [&>svg]:pointer-events-none [&>svg]:size-3!",
{
variants: {
variant: {
default: "bg-primary text-primary-foreground [a]:hover:bg-primary/80",
secondary:
"bg-secondary text-secondary-foreground [a]:hover:bg-secondary/80",
destructive:
"bg-destructive/10 text-destructive focus-visible:ring-destructive/20 dark:bg-destructive/20 dark:focus-visible:ring-destructive/40 [a]:hover:bg-destructive/20",
outline:
"border-border text-foreground [a]:hover:bg-muted [a]:hover:text-muted-foreground",
ghost:
"hover:bg-muted hover:text-muted-foreground dark:hover:bg-muted/50",
link: "text-primary underline-offset-4 hover:underline",
},
},
defaultVariants: {
variant: "default",
},
}
)
function Badge({
className,
variant = "default",
render,
...props
}: useRender.ComponentProps<"span"> & VariantProps<typeof badgeVariants>) {
return useRender({
defaultTagName: "span",
props: mergeProps<"span">(
{
className: cn(badgeVariants({ variant }), className),
},
props
),
render,
state: {
slot: "badge",
variant,
},
})
}
export { Badge, badgeVariants }
import { Button as ButtonPrimitive } from "@base-ui/react/button"
import { cva, type VariantProps } from "class-variance-authority"
import { cn } from "@/lib/utils"
const buttonVariants = cva(
"group/button inline-flex shrink-0 items-center justify-center rounded-lg border border-transparent bg-clip-padding text-sm font-medium whitespace-nowrap transition-all outline-none select-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50 active:not-aria-[haspopup]:translate-y-px disabled:pointer-events-none disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4",
{
variants: {
variant: {
default: "bg-primary text-primary-foreground [a]:hover:bg-primary/80",
outline:
"border-border bg-background hover:bg-muted hover:text-foreground aria-expanded:bg-muted aria-expanded:text-foreground dark:border-input dark:bg-input/30 dark:hover:bg-input/50",
secondary:
"bg-secondary text-secondary-foreground hover:bg-secondary/80 aria-expanded:bg-secondary aria-expanded:text-secondary-foreground",
ghost:
"hover:bg-muted hover:text-foreground aria-expanded:bg-muted aria-expanded:text-foreground dark:hover:bg-muted/50",
destructive:
"bg-destructive/10 text-destructive hover:bg-destructive/20 focus-visible:border-destructive/40 focus-visible:ring-destructive/20 dark:bg-destructive/20 dark:hover:bg-destructive/30 dark:focus-visible:ring-destructive/40",
link: "text-primary underline-offset-4 hover:underline",
},
size: {
default:
"h-8 gap-1.5 px-2.5 has-data-[icon=inline-end]:pr-2 has-data-[icon=inline-start]:pl-2",
xs: "h-6 gap-1 rounded-[min(var(--radius-md),10px)] px-2 text-xs in-data-[slot=button-group]:rounded-lg has-data-[icon=inline-end]:pr-1.5 has-data-[icon=inline-start]:pl-1.5 [&_svg:not([class*='size-'])]:size-3",
sm: "h-7 gap-1 rounded-[min(var(--radius-md),12px)] px-2.5 text-[0.8rem] in-data-[slot=button-group]:rounded-lg has-data-[icon=inline-end]:pr-1.5 has-data-[icon=inline-start]:pl-1.5 [&_svg:not([class*='size-'])]:size-3.5",
lg: "h-9 gap-1.5 px-2.5 has-data-[icon=inline-end]:pr-2 has-data-[icon=inline-start]:pl-2",
icon: "size-8",
"icon-xs":
"size-6 rounded-[min(var(--radius-md),10px)] in-data-[slot=button-group]:rounded-lg [&_svg:not([class*='size-'])]:size-3",
"icon-sm":
"size-7 rounded-[min(var(--radius-md),12px)] in-data-[slot=button-group]:rounded-lg",
"icon-lg": "size-9",
},
},
defaultVariants: {
variant: "default",
size: "default",
},
}
)
function Button({
className,
variant = "default",
size = "default",
...props
}: ButtonPrimitive.Props & VariantProps<typeof buttonVariants>) {
return (
<ButtonPrimitive
data-slot="button"
className={cn(buttonVariants({ variant, size, className }))}
{...props}
/>
)
}
export { Button, buttonVariants }
import * as React from "react"
import { cn } from "@/lib/utils"
function Card({
className,
size = "default",
...props
}: React.ComponentProps<"div"> & { size?: "default" | "sm" }) {
return (
<div
data-slot="card"
data-size={size}
className={cn(
"group/card flex flex-col gap-4 overflow-hidden rounded-xl bg-card py-4 text-sm text-card-foreground ring-1 ring-foreground/10 has-data-[slot=card-footer]:pb-0 has-[>img:first-child]:pt-0 data-[size=sm]:gap-3 data-[size=sm]:py-3 data-[size=sm]:has-data-[slot=card-footer]:pb-0 *:[img:first-child]:rounded-t-xl *:[img:last-child]:rounded-b-xl",
className
)}
{...props}
/>
)
}
function CardHeader({ className, ...props }: React.ComponentProps<"div">) {
return (
<div
data-slot="card-header"
className={cn(
"group/card-header @container/card-header grid auto-rows-min items-start gap-1 rounded-t-xl px-4 group-data-[size=sm]/card:px-3 has-data-[slot=card-action]:grid-cols-[1fr_auto] has-data-[slot=card-description]:grid-rows-[auto_auto] [.border-b]:pb-4 group-data-[size=sm]/card:[.border-b]:pb-3",
className
)}
{...props}
/>
)
}
function CardTitle({ className, ...props }: React.ComponentProps<"div">) {
return (
<div
data-slot="card-title"
className={cn(
"font-heading text-base leading-snug font-medium group-data-[size=sm]/card:text-sm",
className
)}
{...props}
/>
)
}
function CardDescription({ className, ...props }: React.ComponentProps<"div">) {
return (
<div
data-slot="card-description"
className={cn("text-sm text-muted-foreground", className)}
{...props}
/>
)
}
function CardAction({ className, ...props }: React.ComponentProps<"div">) {
return (
<div
data-slot="card-action"
className={cn(
"col-start-2 row-span-2 row-start-1 self-start justify-self-end",
className
)}
{...props}
/>
)
}
function CardContent({ className, ...props }: React.ComponentProps<"div">) {
return (
<div
data-slot="card-content"
className={cn("px-4 group-data-[size=sm]/card:px-3", className)}
{...props}
/>
)
}
function CardFooter({ className, ...props }: React.ComponentProps<"div">) {
return (
<div
data-slot="card-footer"
className={cn(
"flex items-center rounded-b-xl border-t bg-muted/50 p-4 group-data-[size=sm]/card:p-3",
className
)}
{...props}
/>
)
}
export {
Card,
CardHeader,
CardFooter,
CardTitle,
CardAction,
CardDescription,
CardContent,
}
import * as React from "react"
import { Input as InputPrimitive } from "@base-ui/react/input"
import { cn } from "@/lib/utils"
function Input({ className, type, ...props }: React.ComponentProps<"input">) {
return (
<InputPrimitive
type={type}
data-slot="input"
className={cn(
"h-8 w-full min-w-0 rounded-lg border border-input bg-transparent px-2.5 py-1 text-base transition-colors outline-none file:inline-flex file:h-6 file:border-0 file:bg-transparent file:text-sm file:font-medium file:text-foreground placeholder:text-muted-foreground focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50 disabled:pointer-events-none disabled:cursor-not-allowed disabled:bg-input/50 disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 md:text-sm dark:bg-input/30 dark:disabled:bg-input/80 dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40",
className
)}
{...props}
/>
)
}
export { Input }
import * as React from "react"
import { cn } from "@/lib/utils"
function Label({ className, ...props }: React.ComponentProps<"label">) {
return (
<label
data-slot="label"
className={cn(
"flex items-center gap-2 text-sm leading-none font-medium select-none group-data-[disabled=true]:pointer-events-none group-data-[disabled=true]:opacity-50 peer-disabled:cursor-not-allowed peer-disabled:opacity-50",
className
)}
{...props}
/>
)
}
export { Label }
"use client"
import * as React from "react"
import { Select as SelectPrimitive } from "@base-ui/react/select"
import { cn } from "@/lib/utils"
import { ChevronDownIcon, CheckIcon, ChevronUpIcon } from "lucide-react"
const Select = SelectPrimitive.Root
function SelectGroup({ className, ...props }: SelectPrimitive.Group.Props) {
return (
<SelectPrimitive.Group
data-slot="select-group"
className={cn("scroll-my-1 p-1", className)}
{...props}
/>
)
}
function SelectValue({ className, ...props }: SelectPrimitive.Value.Props) {
return (
<SelectPrimitive.Value
data-slot="select-value"
className={cn("flex flex-1 text-left", className)}
{...props}
/>
)
}
function SelectTrigger({
className,
size = "default",
children,
...props
}: SelectPrimitive.Trigger.Props & {
size?: "sm" | "default"
}) {
return (
<SelectPrimitive.Trigger
data-slot="select-trigger"
data-size={size}
className={cn(
"flex w-fit items-center justify-between gap-1.5 rounded-lg border border-input bg-transparent py-2 pr-2 pl-2.5 text-sm whitespace-nowrap transition-colors outline-none select-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50 disabled:cursor-not-allowed disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 data-placeholder:text-muted-foreground data-[size=default]:h-8 data-[size=sm]:h-7 data-[size=sm]:rounded-[min(var(--radius-md),10px)] *:data-[slot=select-value]:line-clamp-1 *:data-[slot=select-value]:flex *:data-[slot=select-value]:items-center *:data-[slot=select-value]:gap-1.5 dark:bg-input/30 dark:hover:bg-input/50 dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4",
className
)}
{...props}
>
{children}
<SelectPrimitive.Icon
render={
<ChevronDownIcon className="pointer-events-none size-4 text-muted-foreground" />
}
/>
</SelectPrimitive.Trigger>
)
}
function SelectContent({
className,
children,
side = "bottom",
sideOffset = 4,
align = "center",
alignOffset = 0,
alignItemWithTrigger = true,
...props
}: SelectPrimitive.Popup.Props &
Pick<
SelectPrimitive.Positioner.Props,
"align" | "alignOffset" | "side" | "sideOffset" | "alignItemWithTrigger"
>) {
return (
<SelectPrimitive.Portal>
<SelectPrimitive.Positioner
side={side}
sideOffset={sideOffset}
align={align}
alignOffset={alignOffset}
alignItemWithTrigger={alignItemWithTrigger}
className="isolate z-50"
>
<SelectPrimitive.Popup
data-slot="select-content"
data-align-trigger={alignItemWithTrigger}
className={cn("relative isolate z-50 max-h-(--available-height) w-(--anchor-width) min-w-36 origin-(--transform-origin) overflow-x-hidden overflow-y-auto rounded-lg bg-popover text-popover-foreground shadow-md ring-1 ring-foreground/10 duration-100 data-[align-trigger=true]:animate-none data-[side=bottom]:slide-in-from-top-2 data-[side=inline-end]:slide-in-from-left-2 data-[side=inline-start]:slide-in-from-right-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 data-open:animate-in data-open:fade-in-0 data-open:zoom-in-95 data-closed:animate-out data-closed:fade-out-0 data-closed:zoom-out-95", className )}
{...props}
>
<SelectScrollUpButton />
<SelectPrimitive.List>{children}</SelectPrimitive.List>
<SelectScrollDownButton />
</SelectPrimitive.Popup>
</SelectPrimitive.Positioner>
</SelectPrimitive.Portal>
)
}
function SelectLabel({
className,
...props
}: SelectPrimitive.GroupLabel.Props) {
return (
<SelectPrimitive.GroupLabel
data-slot="select-label"
className={cn("px-1.5 py-1 text-xs text-muted-foreground", className)}
{...props}
/>
)
}
function SelectItem({
className,
children,
...props
}: SelectPrimitive.Item.Props) {
return (
<SelectPrimitive.Item
data-slot="select-item"
className={cn(
"relative flex w-full cursor-default items-center gap-1.5 rounded-md py-1 pr-8 pl-1.5 text-sm outline-hidden select-none focus:bg-accent focus:text-accent-foreground not-data-[variant=destructive]:focus:**:text-accent-foreground data-disabled:pointer-events-none data-disabled:opacity-50 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4 *:[span]:last:flex *:[span]:last:items-center *:[span]:last:gap-2",
className
)}
{...props}
>
<SelectPrimitive.ItemText className="flex flex-1 shrink-0 gap-2 whitespace-nowrap">
{children}
</SelectPrimitive.ItemText>
<SelectPrimitive.ItemIndicator
render={
<span className="pointer-events-none absolute right-2 flex size-4 items-center justify-center" />
}
>
<CheckIcon className="pointer-events-none" />
</SelectPrimitive.ItemIndicator>
</SelectPrimitive.Item>
)
}
function SelectSeparator({
className,
...props
}: SelectPrimitive.Separator.Props) {
return (
<SelectPrimitive.Separator
data-slot="select-separator"
className={cn("pointer-events-none -mx-1 my-1 h-px bg-border", className)}
{...props}
/>
)
}
function SelectScrollUpButton({
className,
...props
}: React.ComponentProps<typeof SelectPrimitive.ScrollUpArrow>) {
return (
<SelectPrimitive.ScrollUpArrow
data-slot="select-scroll-up-button"
className={cn(
"top-0 z-10 flex w-full cursor-default items-center justify-center bg-popover py-1 [&_svg:not([class*='size-'])]:size-4",
className
)}
{...props}
>
<ChevronUpIcon
/>
</SelectPrimitive.ScrollUpArrow>
)
}
function SelectScrollDownButton({
className,
...props
}: React.ComponentProps<typeof SelectPrimitive.ScrollDownArrow>) {
return (
<SelectPrimitive.ScrollDownArrow
data-slot="select-scroll-down-button"
className={cn(
"bottom-0 z-10 flex w-full cursor-default items-center justify-center bg-popover py-1 [&_svg:not([class*='size-'])]:size-4",
className
)}
{...props}
>
<ChevronDownIcon
/>
</SelectPrimitive.ScrollDownArrow>
)
}
export {
Select,
SelectContent,
SelectGroup,
SelectItem,
SelectLabel,
SelectScrollDownButton,
SelectScrollUpButton,
SelectSeparator,
SelectTrigger,
SelectValue,
}
import { Separator as SeparatorPrimitive } from "@base-ui/react/separator"
import { cn } from "@/lib/utils"
function Separator({
className,
orientation = "horizontal",
...props
}: SeparatorPrimitive.Props) {
return (
<SeparatorPrimitive
data-slot="separator"
orientation={orientation}
className={cn(
"shrink-0 bg-border data-horizontal:h-px data-horizontal:w-full data-vertical:w-px data-vertical:self-stretch",
className
)}
{...props}
/>
)
}
export { Separator }
import { Switch as SwitchPrimitive } from "@base-ui/react/switch"
import { cn } from "@/lib/utils"
function Switch({
className,
size = "default",
...props
}: SwitchPrimitive.Root.Props & {
size?: "sm" | "default"
}) {
return (
<SwitchPrimitive.Root
data-slot="switch"
data-size={size}
className={cn(
"peer group/switch relative inline-flex shrink-0 items-center rounded-full border border-transparent transition-all outline-none after:absolute after:-inset-x-3 after:-inset-y-2 focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 data-[size=default]:h-[18.4px] data-[size=default]:w-[32px] data-[size=sm]:h-[14px] data-[size=sm]:w-[24px] dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40 data-checked:bg-primary data-unchecked:bg-input dark:data-unchecked:bg-input/80 data-disabled:cursor-not-allowed data-disabled:opacity-50",
className
)}
{...props}
>
<SwitchPrimitive.Thumb
data-slot="switch-thumb"
className="pointer-events-none block rounded-full bg-background ring-0 transition-transform group-data-[size=default]/switch:size-4 group-data-[size=sm]/switch:size-3 group-data-[size=default]/switch:data-checked:translate-x-[calc(100%-2px)] group-data-[size=sm]/switch:data-checked:translate-x-[calc(100%-2px)] dark:data-checked:bg-primary-foreground group-data-[size=default]/switch:data-unchecked:translate-x-0 group-data-[size=sm]/switch:data-unchecked:translate-x-0 dark:data-unchecked:bg-foreground"
/>
</SwitchPrimitive.Root>
)
}
export { Switch }
"use client"
import * as React from "react"
import { cn } from "@/lib/utils"
function Table({ className, ...props }: React.ComponentProps<"table">) {
return (
<div
data-slot="table-container"
className="relative w-full overflow-x-auto"
>
<table
data-slot="table"
className={cn("w-full caption-bottom text-sm", className)}
{...props}
/>
</div>
)
}
function TableHeader({ className, ...props }: React.ComponentProps<"thead">) {
return (
<thead
data-slot="table-header"
className={cn("[&_tr]:border-b", className)}
{...props}
/>
)
}
function TableBody({ className, ...props }: React.ComponentProps<"tbody">) {
return (
<tbody
data-slot="table-body"
className={cn("[&_tr:last-child]:border-0", className)}
{...props}
/>
)
}
function TableFooter({ className, ...props }: React.ComponentProps<"tfoot">) {
return (
<tfoot
data-slot="table-footer"
className={cn(
"border-t bg-muted/50 font-medium [&>tr]:last:border-b-0",
className
)}
{...props}
/>
)
}
function TableRow({ className, ...props }: React.ComponentProps<"tr">) {
return (
<tr
data-slot="table-row"
className={cn(
"border-b transition-colors hover:bg-muted/50 has-aria-expanded:bg-muted/50 data-[state=selected]:bg-muted",
className
)}
{...props}
/>
)
}
function TableHead({ className, ...props }: React.ComponentProps<"th">) {
return (
<th
data-slot="table-head"
className={cn(
"h-10 px-2 text-left align-middle font-medium whitespace-nowrap text-foreground [&:has([role=checkbox])]:pr-0",
className
)}
{...props}
/>
)
}
function TableCell({ className, ...props }: React.ComponentProps<"td">) {
return (
<td
data-slot="table-cell"
className={cn(
"p-2 align-middle whitespace-nowrap [&:has([role=checkbox])]:pr-0",
className
)}
{...props}
/>
)
}
function TableCaption({
className,
...props
}: React.ComponentProps<"caption">) {
return (
<caption
data-slot="table-caption"
className={cn("mt-4 text-sm text-muted-foreground", className)}
{...props}
/>
)
}
export {
Table,
TableHeader,
TableBody,
TableFooter,
TableHead,
TableRow,
TableCell,
TableCaption,
}
import * as React from "react"
import { cn } from "@/lib/utils"
function Textarea({ className, ...props }: React.ComponentProps<"textarea">) {
return (
<textarea
data-slot="textarea"
className={cn(
"flex field-sizing-content min-h-16 w-full rounded-lg border border-input bg-transparent px-2.5 py-2 text-base transition-colors outline-none placeholder:text-muted-foreground focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50 disabled:cursor-not-allowed disabled:bg-input/50 disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 md:text-sm dark:bg-input/30 dark:disabled:bg-input/80 dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40",
className
)}
{...props}
/>
)
}
export { Textarea }
@import "tailwindcss";
@import "tw-animate-css";
@import "shadcn/tailwind.css";
@import "@fontsource-variable/geist";
@import "@fontsource-variable/geist-mono";
@custom-variant dark (&:is(.dark *));
@theme inline {
--font-heading: var(--font-sans);
--font-sans: 'Geist Variable', ui-sans-serif, system-ui, -apple-system, sans-serif;
--font-mono: 'Geist Mono Variable', ui-monospace, SFMono-Regular, Menlo, monospace;
--color-sidebar-ring: var(--sidebar-ring);
--color-sidebar-border: var(--sidebar-border);
--color-sidebar-accent-foreground: var(--sidebar-accent-foreground);
--color-sidebar-accent: var(--sidebar-accent);
--color-sidebar-primary-foreground: var(--sidebar-primary-foreground);
--color-sidebar-primary: var(--sidebar-primary);
--color-sidebar-foreground: var(--sidebar-foreground);
--color-sidebar: var(--sidebar);
--color-chart-5: var(--chart-5);
--color-chart-4: var(--chart-4);
--color-chart-3: var(--chart-3);
--color-chart-2: var(--chart-2);
--color-chart-1: var(--chart-1);
--color-ring: var(--ring);
--color-input: var(--input);
--color-border: var(--border);
--color-destructive: var(--destructive);
--color-accent-foreground: var(--accent-foreground);
--color-accent: var(--accent);
--color-muted-foreground: var(--muted-foreground);
--color-muted: var(--muted);
--color-secondary-foreground: var(--secondary-foreground);
--color-secondary: var(--secondary);
--color-primary-foreground: var(--primary-foreground);
--color-primary: var(--primary);
--color-popover-foreground: var(--popover-foreground);
--color-popover: var(--popover);
--color-card-foreground: var(--card-foreground);
--color-card: var(--card);
--color-foreground: var(--foreground);
--color-background: var(--background);
--radius-sm: calc(var(--radius) * 0.6);
--radius-md: calc(var(--radius) * 0.8);
--radius-lg: var(--radius);
--radius-xl: calc(var(--radius) * 1.4);
--radius-2xl: calc(var(--radius) * 1.8);
--radius-3xl: calc(var(--radius) * 2.2);
--radius-4xl: calc(var(--radius) * 2.6);
}
:root {
--background: oklch(0.995 0 0);
--foreground: oklch(0.18 0 0);
--card: oklch(1 0 0);
--card-foreground: oklch(0.18 0 0);
--popover: oklch(1 0 0);
--popover-foreground: oklch(0.18 0 0);
--primary: oklch(0.22 0 0);
--primary-foreground: oklch(0.98 0 0);
--secondary: oklch(0.97 0 0);
--secondary-foreground: oklch(0.22 0 0);
--muted: oklch(0.97 0 0);
--muted-foreground: oklch(0.52 0 0);
--accent: oklch(0.96 0 0);
--accent-foreground: oklch(0.22 0 0);
--destructive: oklch(0.577 0.245 27.325);
--border: oklch(0.92 0 0);
--input: oklch(0.92 0 0);
--ring: oklch(0.55 0 0);
--chart-1: oklch(0.28 0 0);
--chart-2: oklch(0.45 0 0);
--chart-3: oklch(0.6 0 0);
--chart-4: oklch(0.75 0 0);
--chart-5: oklch(0.87 0 0);
--radius: 0.5rem;
--sidebar: oklch(0.98 0 0);
--sidebar-foreground: oklch(0.18 0 0);
--sidebar-primary: oklch(0.22 0 0);
--sidebar-primary-foreground: oklch(0.98 0 0);
--sidebar-accent: oklch(0.96 0 0);
--sidebar-accent-foreground: oklch(0.22 0 0);
--sidebar-border: oklch(0.92 0 0);
--sidebar-ring: oklch(0.55 0 0);
}
.dark {
--background: oklch(0.135 0 0);
--foreground: oklch(0.98 0 0);
--card: oklch(0.175 0 0);
--card-foreground: oklch(0.98 0 0);
--popover: oklch(0.175 0 0);
--popover-foreground: oklch(0.98 0 0);
--primary: oklch(0.93 0 0);
--primary-foreground: oklch(0.18 0 0);
--secondary: oklch(0.24 0 0);
--secondary-foreground: oklch(0.98 0 0);
--muted: oklch(0.22 0 0);
--muted-foreground: oklch(0.68 0 0);
--accent: oklch(0.24 0 0);
--accent-foreground: oklch(0.98 0 0);
--destructive: oklch(0.704 0.191 22.216);
--border: oklch(1 0 0 / 10%);
--input: oklch(1 0 0 / 14%);
--ring: oklch(0.6 0 0);
--chart-1: oklch(0.92 0 0);
--chart-2: oklch(0.78 0 0);
--chart-3: oklch(0.6 0 0);
--chart-4: oklch(0.45 0 0);
--chart-5: oklch(0.32 0 0);
--sidebar: oklch(0.175 0 0);
--sidebar-foreground: oklch(0.98 0 0);
--sidebar-primary: oklch(0.93 0 0);
--sidebar-primary-foreground: oklch(0.18 0 0);
--sidebar-accent: oklch(0.24 0 0);
--sidebar-accent-foreground: oklch(0.98 0 0);
--sidebar-border: oklch(1 0 0 / 10%);
--sidebar-ring: oklch(0.6 0 0);
}
@layer base {
* {
@apply border-border outline-ring/50;
}
html {
@apply font-sans antialiased;
font-feature-settings: 'ss01', 'cv11';
text-rendering: optimizeLegibility;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}
body {
@apply bg-background text-foreground;
font-feature-settings: 'ss01', 'cv11', 'tnum';
}
code, pre, kbd, samp {
font-family: var(--font-mono);
font-feature-settings: 'ss02', 'ss03';
}
/* Tabular numerals on any element that opts in */
.tabular-nums {
font-variant-numeric: tabular-nums;
}
/* Tighter focus ring across inputs */
input, textarea, select {
font-feature-settings: 'ss01';
}
}
const BASE = import.meta.env.BASE_URL.replace(/\/$/, '');
export async function apiFetch<T>(path: string, options?: RequestInit): Promise<T> {
const res = await fetch(`${BASE}${path}`, {
headers: { 'Content-Type': 'application/json', ...options?.headers },
...options,
});
if (!res.ok) {
const body = await res.json().catch(() => ({ error: { message: res.statusText } }));
throw new Error(body.error?.message ?? `HTTP ${res.status}`);
}
return res.json();
}
import { clsx, type ClassValue } from "clsx"
import { twMerge } from "tailwind-merge"
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs))
}
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import './index.css'
import App from './App'
createRoot(document.getElementById('root')!).render(
<StrictMode>
<App />
</StrictMode>,
)
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
import { useState, useRef, useEffect } from 'react'
import { useQuery } from '@tanstack/react-query'
import { apiFetch } from '@/lib/api'
import { Button } from '@/components/ui/button'
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select'
import { PageHeader } from '@/components/page-header'
interface FallbackEntry {
modelDbId: number
priority: number
enabled: boolean
platform: string
modelId: string
displayName: string
sizeLabel: string
keyCount: number
}
interface ChatMessage {
role: 'user' | 'assistant'
content: string
meta?: {
platform?: string
model?: string
latency?: number
fallbackAttempts?: number
}
}
export default function PlaygroundPage() {
const [messages, setMessages] = useState<ChatMessage[]>([])
const [input, setInput] = useState('')
const [loading, setLoading] = useState(false)
const [selectedModel, setSelectedModel] = useState<string>('auto')
const messagesEndRef = useRef<HTMLDivElement>(null)
const inputRef = useRef<HTMLTextAreaElement>(null)
const { data: keyData } = useQuery<{ apiKey: string }>({
queryKey: ['unified-key'],
queryFn: () => apiFetch('/api/settings/api-key'),
})
const { data: fallbackEntries = [] } = useQuery<FallbackEntry[]>({
queryKey: ['fallback'],
queryFn: () => apiFetch('/api/fallback'),
})
const availableModels = fallbackEntries.filter(e => e.keyCount > 0 && e.enabled)
useEffect(() => {
messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' })
}, [messages])
const handleSend = async () => {
const text = input.trim()
if (!text || loading) return
const userMsg: ChatMessage = { role: 'user', content: text }
const newMessages = [...messages, userMsg]
setMessages(newMessages)
setInput('')
setLoading(true)
inputRef.current?.focus()
try {
const headers: Record<string, string> = { 'Content-Type': 'application/json' }
if (keyData?.apiKey) headers['Authorization'] = `Bearer ${keyData.apiKey}`
const body: any = {
messages: newMessages.map(m => ({ role: m.role, content: m.content })),
}
if (selectedModel !== 'auto') body.model = selectedModel
const base = import.meta.env.BASE_URL.replace(/\/$/, '')
const start = Date.now()
const res = await fetch(`${base}/v1/chat/completions`, {
method: 'POST',
headers,
body: JSON.stringify(body),
})
const latency = Date.now() - start
const routedVia = res.headers.get('X-Routed-Via')
const fallbackAttempts = res.headers.get('X-Fallback-Attempts')
if (!res.ok) {
const err = await res.json().catch(() => ({ error: { message: `HTTP ${res.status}` } }))
setMessages([...newMessages, {
role: 'assistant',
content: `Error: ${err.error?.message ?? 'Unknown error'}`,
}])
return
}
const data = await res.json()
const content = data.choices?.[0]?.message?.content ?? JSON.stringify(data, null, 2)
const via = data._routed_via ?? (routedVia ? {
platform: routedVia.split('/')[0],
model: routedVia.split('/').slice(1).join('/'),
} : undefined)
setMessages([...newMessages, {
role: 'assistant',
content,
meta: {
platform: via?.platform,
model: via?.model,
latency,
fallbackAttempts: fallbackAttempts ? parseInt(fallbackAttempts) : undefined,
},
}])
} catch (err: any) {
setMessages([...newMessages, {
role: 'assistant',
content: `Error: ${err.message}`,
}])
} finally {
setLoading(false)
setTimeout(() => inputRef.current?.focus(), 0)
}
}
const handleKeyDown = (e: React.KeyboardEvent) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault()
handleSend()
}
}
const handleClear = () => {
setMessages([])
inputRef.current?.focus()
}
const activeModelLabel = selectedModel === 'auto'
? 'Auto (fallback chain)'
: availableModels.find(m => m.modelId === selectedModel)?.displayName ?? selectedModel
return (
<div className="flex flex-col h-[calc(100vh-8rem)]">
<PageHeader
title="Playground"
description="Send a chat completion through the router and see which provider serves it."
actions={
<>
<Select value={selectedModel} onValueChange={(v) => setSelectedModel(v ?? 'auto')}>
<SelectTrigger className="w-[260px]">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="auto">Auto (fallback chain)</SelectItem>
{availableModels.map(m => (
<SelectItem key={m.modelDbId} value={m.modelId}>
<span className="flex items-center gap-2">
<span>{m.displayName}</span>
<span className="text-xs text-muted-foreground">{m.platform}</span>
</span>
</SelectItem>
))}
</SelectContent>
</Select>
{messages.length > 0 && (
<Button variant="outline" size="sm" onClick={handleClear}>
Clear
</Button>
)}
</>
}
/>
<div className="flex-1 flex flex-col rounded-lg border bg-card overflow-hidden min-h-0">
<div className="flex-1 overflow-y-auto p-6 space-y-4">
{messages.length === 0 ? (
<div className="flex items-center justify-center h-full text-center">
<div className="space-y-2 max-w-sm">
<p className="text-base font-medium">Send a message to get started.</p>
<p className="text-sm text-muted-foreground">
Using <span className="text-foreground">{activeModelLabel}</span>. Switch models in the selector above.
</p>
</div>
</div>
) : (
<>
{messages.map((msg, i) => (
<div key={i} className={`flex ${msg.role === 'user' ? 'justify-end' : 'justify-start'}`}>
<div
className={`max-w-[78%] rounded-2xl px-4 py-2.5 text-sm leading-relaxed ${
msg.role === 'user'
? 'bg-primary text-primary-foreground'
: 'bg-muted'
}`}
>
<div className="whitespace-pre-wrap">{msg.content}</div>
{msg.meta && (
<div className="flex items-center gap-2 mt-2 flex-wrap text-[11px] opacity-70 tabular-nums">
{msg.meta.platform && <span>{msg.meta.platform}</span>}
{msg.meta.model && <span className="font-mono">· {msg.meta.model}</span>}
{msg.meta.latency != null && <span>· {msg.meta.latency} ms</span>}
{msg.meta.fallbackAttempts != null && msg.meta.fallbackAttempts > 0 && (
<span>· {msg.meta.fallbackAttempts} fallback{msg.meta.fallbackAttempts > 1 ? 's' : ''}</span>
)}
</div>
)}
</div>
</div>
))}
{loading && (
<div className="flex justify-start">
<div className="bg-muted rounded-2xl px-4 py-3">
<div className="flex gap-1">
<span className="size-1.5 rounded-full bg-muted-foreground/50 animate-bounce" style={{ animationDelay: '0ms' }} />
<span className="size-1.5 rounded-full bg-muted-foreground/50 animate-bounce" style={{ animationDelay: '150ms' }} />
<span className="size-1.5 rounded-full bg-muted-foreground/50 animate-bounce" style={{ animationDelay: '300ms' }} />
</div>
</div>
</div>
)}
<div ref={messagesEndRef} />
</>
)}
</div>
<div className="border-t bg-background/50 p-3">
<div className="flex gap-2 items-end">
<textarea
ref={inputRef}
value={input}
onChange={e => setInput(e.target.value)}
onKeyDown={handleKeyDown}
placeholder="Type a message… (⏎ to send, ⇧⏎ for newline)"
rows={1}
className="flex-1 resize-none rounded-md border bg-background px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-ring/50 min-h-[40px] max-h-[160px]"
style={{ height: 'auto', overflow: 'hidden' }}
onInput={e => {
const el = e.target as HTMLTextAreaElement
el.style.height = 'auto'
el.style.height = Math.min(el.scrollHeight, 160) + 'px'
}}
/>
<Button onClick={handleSend} disabled={loading || !input.trim()} size="default">
{loading ? 'Sending…' : 'Send'}
</Button>
</div>
</div>
</div>
</div>
)
}
{
"compilerOptions": {
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
"target": "es2023",
"lib": ["ES2023", "DOM", "DOM.Iterable"],
"module": "esnext",
"types": ["vite/client"],
"skipLibCheck": true,
/* Bundler mode */
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"verbatimModuleSyntax": true,
"moduleDetection": "force",
"noEmit": true,
"jsx": "react-jsx",
/* Linting */
"noUnusedLocals": true,
"noUnusedParameters": true,
"erasableSyntaxOnly": true,
"noFallthroughCasesInSwitch": true,
"paths": {
"@/*": ["./src/*"]
}
},
"include": ["src"]
}
{
"files": [],
"references": [
{ "path": "./tsconfig.app.json" },
{ "path": "./tsconfig.node.json" }
],
"compilerOptions": {
"baseUrl": ".",
"paths": {
"@/*": ["./src/*"]
}
}
}
{
"compilerOptions": {
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
"target": "es2023",
"lib": ["ES2023"],
"module": "esnext",
"types": ["node"],
"skipLibCheck": true,
/* Bundler mode */
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"verbatimModuleSyntax": true,
"moduleDetection": "force",
"noEmit": true,
/* Linting */
"noUnusedLocals": true,
"noUnusedParameters": true,
"erasableSyntaxOnly": true,
"noFallthroughCasesInSwitch": true
},
"include": ["vite.config.ts"]
}
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import tailwindcss from '@tailwindcss/vite'
import path from 'path'
export default defineConfig({
plugins: [react(), tailwindcss()],
base: process.env.VITE_BASE ?? '/',
resolve: {
alias: {
'@': path.resolve(__dirname, './src'),
},
},
server: {
proxy: {
'/api': 'http://localhost:3001',
'/v1': 'http://localhost:3001',
},
},
})
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
{
"name": "freellmapi",
"private": true,
"workspaces": [
"shared",
"server",
"client"
],
"scripts": {
"dev": "concurrently \"npm run dev -w server\" \"npm run dev -w client\"",
"test": "npm run test -w server && npm run test -w client",
"build": "npm run build -w server && npm run build -w client",
"build:server": "npm run build -w server"
},
"devDependencies": {
"concurrently": "^9.1.2"
}
}
{
"name": "@freellmapi/server",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"dev": "tsx watch src/index.ts",
"build": "tsc",
"start": "node dist/index.js",
"test": "vitest run",
"test:watch": "vitest"
},
"dependencies": {
"@freellmapi/shared": "*",
"better-sqlite3": "^11.8.1",
"cors": "^2.8.5",
"drizzle-orm": "^0.44.2",
"express": "^5.1.0",
"helmet": "^8.1.0",
"zod": "^3.24.4"
},
"devDependencies": {
"@types/better-sqlite3": "^7.6.13",
"@types/cors": "^2.8.17",
"@types/express": "^5.0.2",
"@types/node": "^22.15.3",
"drizzle-kit": "^0.31.1",
"tsx": "^4.19.4",
"typescript": "^5.8.3",
"vitest": "^3.1.3"
}
}
import { describe, it, expect, beforeAll, vi } from 'vitest';
import type { Express } from 'express';
import { createApp } from '../../app.js';
import { initDb, getDb } from '../../db/index.js';
async function req(app: Express, method: string, path: string, body?: any) {
const server = app.listen(0);
const addr = server.address() as any;
const url = `http://127.0.0.1:${addr.port}${path}`;
const res = await fetch(url, {
method,
headers: body ? { 'Content-Type': 'application/json' } : {},
body: body ? JSON.stringify(body) : undefined,
});
const data = await res.text();
server.close();
let json: any = null;
try { json = JSON.parse(data); } catch {}
return { status: res.status, body: json, headers: res.headers, raw: data };
}
describe('Full Integration Flow', () => {
let app: Express;
beforeAll(() => {
process.env.ENCRYPTION_KEY = '0'.repeat(64);
initDb(':memory:');
app = createApp();
// Clean
const db = getDb();
db.prepare('DELETE FROM api_keys').run();
db.prepare('DELETE FROM requests').run();
});
it('Step 1: Verify models are seeded', async () => {
const { status, body } = await req(app, 'GET', '/api/models');
expect(status).toBe(200);
expect(body.length).toBeGreaterThanOrEqual(14);
expect(body[0]).toHaveProperty('modelId');
expect(body[0]).toHaveProperty('hasProvider');
// All should have providers
for (const m of body) {
expect(m.hasProvider).toBe(true);
}
});
it('Step 2: Verify fallback chain is populated', async () => {
const { status, body } = await req(app, 'GET', '/api/fallback');
expect(status).toBe(200);
expect(body.length).toBeGreaterThanOrEqual(14);
expect(body[0]).toHaveProperty('priority');
expect(body[0]).toHaveProperty('enabled');
});
it('Step 3: Proxy returns 429 with no keys', async () => {
const { status, body } = await req(app, 'POST', '/v1/chat/completions', {
messages: [{ role: 'user', content: 'hello' }],
});
// 429 (all exhausted) or 502 (provider error) or 503 (no route)
expect([429, 502, 503]).toContain(status);
expect(body.error).toBeDefined();
});
it('Step 4: Add a Groq key', async () => {
const { status, body } = await req(app, 'POST', '/api/keys', {
platform: 'groq',
key: 'gsk_integration_test_key',
label: 'Integration Test',
});
expect(status).toBe(201);
expect(body.platform).toBe('groq');
expect(body.maskedKey).toContain('...');
});
it('Step 5: Proxy routes to Groq and handles provider error gracefully', async () => {
// Mock fetch to simulate a Groq API error
const origFetch = global.fetch;
vi.spyOn(global, 'fetch').mockImplementation(async (url, init) => {
const urlStr = typeof url === 'string' ? url : url.toString();
// If it's calling the Groq API, return an error
if (urlStr.includes('api.groq.com')) {
return {
ok: false,
status: 401,
statusText: 'Unauthorized',
json: () => Promise.resolve({ error: { message: 'Invalid API Key' } }),
} as any;
}
// Otherwise pass through (for our test server)
return origFetch(url, init);
});
const { status, body } = await req(app, 'POST', '/v1/chat/completions', {
messages: [{ role: 'user', content: 'hello' }],
});
// 502 (provider error) or 429 (all exhausted after retries)
expect([502, 429]).toContain(status);
expect(body.error).toBeDefined();
vi.restoreAllMocks();
});
it('Step 6: Error was logged in analytics', async () => {
const { status, body } = await req(app, 'GET', '/api/analytics/summary?range=24h');
expect(status).toBe(200);
// May or may not have logged depending on retry behavior
expect(body.totalRequests).toBeGreaterThanOrEqual(0);
});
it('Step 7: Sort fallback by speed', async () => {
const { status } = await req(app, 'POST', '/api/fallback/sort/speed');
expect(status).toBe(200);
const { body } = await req(app, 'GET', '/api/fallback');
expect(body[0].speedRank).toBe(1);
});
it('Step 8: Health endpoint works', async () => {
const { status, body } = await req(app, 'GET', '/api/health');
expect(status).toBe(200);
expect(body).toHaveProperty('platforms');
expect(body).toHaveProperty('keys');
});
it('Step 9: Delete a key if any exist', async () => {
// Add a fresh key to ensure we have one to delete
await req(app, 'POST', '/api/keys', {
platform: 'groq', key: 'gsk_delete_test', label: 'delete-test',
});
const { body: keys } = await req(app, 'GET', '/api/keys');
const target = keys.find((k: any) => k.label === 'delete-test');
expect(target).toBeDefined();
const { status } = await req(app, 'DELETE', `/api/keys/${target.id}`);
expect(status).toBe(200);
});
it('Step 10: Validate request schema', async () => {
const { status } = await req(app, 'POST', '/v1/chat/completions', {
messages: [], // empty
});
expect(status).toBe(400);
const { status: s2 } = await req(app, 'POST', '/v1/chat/completions', {
// missing messages entirely
});
expect(s2).toBe(400);
});
});
import { describe, it, expect, beforeAll } from 'vitest';
import { initDb } from '../../db/index.js';
import { encrypt, decrypt, maskKey } from '../../lib/crypto.js';
describe('Crypto', () => {
beforeAll(() => {
process.env.ENCRYPTION_KEY = '0'.repeat(64);
initDb(':memory:');
});
it('should encrypt and decrypt a key round-trip', () => {
const original = 'gsk_test1234567890abcdef';
const { encrypted, iv, authTag } = encrypt(original);
const decrypted = decrypt(encrypted, iv, authTag);
expect(decrypted).toBe(original);
});
it('should produce different ciphertext for same input (random IV)', () => {
const original = 'same-key';
const a = encrypt(original);
const b = encrypt(original);
expect(a.encrypted).not.toBe(b.encrypted);
expect(a.iv).not.toBe(b.iv);
});
it('should fail to decrypt with wrong auth tag', () => {
const { encrypted, iv } = encrypt('test-key');
expect(() => decrypt(encrypted, iv, 'a'.repeat(32))).toThrow();
});
describe('maskKey', () => {
it('should mask long keys', () => {
expect(maskKey('gsk_test1234567890abcdef')).toBe('gsk_...cdef');
});
it('should mask short keys', () => {
expect(maskKey('abcd')).toBe('****abcd');
});
});
});
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { CerebrasProvider } from '../../providers/cerebras.js';
describe('CerebrasProvider', () => {
let provider: CerebrasProvider;
beforeEach(() => {
provider = new CerebrasProvider();
});
it('should have correct platform and name', () => {
expect(provider.platform).toBe('cerebras');
expect(provider.name).toBe('Cerebras');
});
it('should call Cerebras API with OpenAI-compatible format', async () => {
const mockResponse = {
id: 'chatcmpl-456',
object: 'chat.completion',
created: 1234567890,
model: 'qwen3-235b',
choices: [{
index: 0,
message: { role: 'assistant', content: 'Response from Cerebras' },
finish_reason: 'stop',
}],
usage: { prompt_tokens: 8, completion_tokens: 4, total_tokens: 12 },
};
let capturedUrl = '';
vi.spyOn(global, 'fetch').mockImplementation(async (url, _init) => {
capturedUrl = url as string;
return {
ok: true,
json: () => Promise.resolve(mockResponse),
} as any;
});
const result = await provider.chatCompletion(
'csk_test456',
[{ role: 'user', content: 'Hello' }],
'qwen3-235b',
);
expect(capturedUrl).toContain('api.cerebras.ai');
expect(result.choices[0].message.content).toBe('Response from Cerebras');
expect(result._routed_via?.platform).toBe('cerebras');
expect(result._routed_via?.model).toBe('qwen3-235b');
});
it('should validate key', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({ ok: true } as any);
expect(await provider.validateKey('valid')).toBe(true);
});
});
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { CloudflareProvider } from '../../providers/cloudflare.js';
describe('CloudflareProvider', () => {
let provider: CloudflareProvider;
beforeEach(() => {
provider = new CloudflareProvider();
});
it('should have correct platform and name', () => {
expect(provider.platform).toBe('cloudflare');
expect(provider.name).toBe('Cloudflare Workers AI');
});
it('should parse account_id:token key format', async () => {
let capturedUrl = '';
let capturedHeaders: Record<string, string> = {};
vi.spyOn(global, 'fetch').mockImplementation(async (url, init) => {
capturedUrl = url as string;
capturedHeaders = (init as any).headers;
return {
ok: true,
json: () => Promise.resolve({ result: { response: 'Hello from CF!' } }),
} as any;
});
const result = await provider.chatCompletion(
'abc123:my-token-here',
[{ role: 'user', content: 'Hi' }],
'@cf/meta/llama-3.1-70b-instruct',
);
expect(capturedUrl).toContain('abc123');
expect(capturedUrl).toContain('@cf/meta/llama-3.1-70b-instruct');
expect(capturedHeaders['Authorization']).toBe('Bearer my-token-here');
expect(result.choices[0].message.content).toBe('Hello from CF!');
});
it('should throw if key format is wrong', async () => {
await expect(
provider.chatCompletion('no-colon-here', [{ role: 'user', content: 'Hi' }], 'model')
).rejects.toThrow(/account_id:api_token/);
});
});
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { CohereProvider } from '../../providers/cohere.js';
describe('CohereProvider', () => {
let provider: CohereProvider;
beforeEach(() => {
provider = new CohereProvider();
});
it('should have correct platform and name', () => {
expect(provider.platform).toBe('cohere');
expect(provider.name).toBe('Cohere');
});
it('should translate response to OpenAI format', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({
ok: true,
json: () => Promise.resolve({
id: 'cohere-123',
message: { content: [{ type: 'text', text: 'Hello from Cohere!' }] },
finish_reason: 'COMPLETE',
usage: { tokens: { input_tokens: 10, output_tokens: 5 } },
}),
} as any);
const result = await provider.chatCompletion(
'test-key',
[{ role: 'user', content: 'Hi' }],
'command-r-plus-08-2024',
);
expect(result.object).toBe('chat.completion');
expect(result.choices[0].message.content).toBe('Hello from Cohere!');
expect(result.usage.prompt_tokens).toBe(10);
expect(result.usage.completion_tokens).toBe(5);
expect(result._routed_via?.platform).toBe('cohere');
});
it('should validate key', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({ ok: true } as any);
expect(await provider.validateKey('valid')).toBe(true);
});
});
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { GoogleProvider } from '../../providers/google.js';
describe('GoogleProvider', () => {
let provider: GoogleProvider;
beforeEach(() => {
provider = new GoogleProvider();
});
it('should have correct platform and name', () => {
expect(provider.platform).toBe('google');
expect(provider.name).toBe('Google AI Studio');
});
it('should call Gemini API and return OpenAI-compatible response', async () => {
const mockResponse = {
candidates: [{
content: { parts: [{ text: 'Hello from Gemini!' }] },
finishReason: 'STOP',
}],
usageMetadata: {
promptTokenCount: 10,
candidatesTokenCount: 5,
totalTokenCount: 15,
},
};
vi.spyOn(global, 'fetch').mockResolvedValueOnce({
ok: true,
json: () => Promise.resolve(mockResponse),
} as any);
const result = await provider.chatCompletion(
'test-key',
[{ role: 'user', content: 'Hi' }],
'gemini-2.5-pro',
);
expect(result.object).toBe('chat.completion');
expect(result.choices[0].message.content).toBe('Hello from Gemini!');
expect(result.choices[0].message.role).toBe('assistant');
expect(result.usage.prompt_tokens).toBe(10);
expect(result.usage.completion_tokens).toBe(5);
expect(result._routed_via?.platform).toBe('google');
});
it('should throw on API error', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({
ok: false,
status: 429,
statusText: 'Too Many Requests',
json: () => Promise.resolve({ error: { message: 'Rate limit exceeded' } }),
} as any);
await expect(
provider.chatCompletion('test-key', [{ role: 'user', content: 'Hi' }], 'gemini-2.5-pro')
).rejects.toThrow(/Rate limit exceeded/);
});
it('should validate key via models endpoint', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({ ok: true } as any);
expect(await provider.validateKey('valid-key')).toBe(true);
vi.spyOn(global, 'fetch').mockResolvedValueOnce({ ok: false, status: 401 } as any);
expect(await provider.validateKey('invalid-key')).toBe(false);
});
it('should translate system messages to systemInstruction', async () => {
let capturedBody: any;
vi.spyOn(global, 'fetch').mockImplementation(async (_url, init) => {
capturedBody = JSON.parse((init as any).body);
return {
ok: true,
json: () => Promise.resolve({
candidates: [{ content: { parts: [{ text: 'ok' }] }, finishReason: 'STOP' }],
usageMetadata: { promptTokenCount: 1, candidatesTokenCount: 1, totalTokenCount: 2 },
}),
} as any;
});
await provider.chatCompletion(
'test-key',
[
{ role: 'system', content: 'You are helpful' },
{ role: 'user', content: 'Hi' },
],
'gemini-2.5-pro',
);
expect(capturedBody.systemInstruction).toEqual({ parts: [{ text: 'You are helpful' }] });
expect(capturedBody.contents).toHaveLength(1);
expect(capturedBody.contents[0].role).toBe('user');
});
});
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { GroqProvider } from '../../providers/groq.js';
describe('GroqProvider', () => {
let provider: GroqProvider;
beforeEach(() => {
provider = new GroqProvider();
});
it('should have correct platform and name', () => {
expect(provider.platform).toBe('groq');
expect(provider.name).toBe('Groq');
});
it('should call Groq API with OpenAI-compatible format', async () => {
const mockResponse = {
id: 'chatcmpl-123',
object: 'chat.completion',
created: 1234567890,
model: 'llama-3.3-70b-versatile',
choices: [{
index: 0,
message: { role: 'assistant', content: 'Hello!' },
finish_reason: 'stop',
}],
usage: { prompt_tokens: 5, completion_tokens: 2, total_tokens: 7 },
};
let capturedHeaders: Record<string, string> = {};
vi.spyOn(global, 'fetch').mockImplementation(async (_url, init) => {
capturedHeaders = Object.fromEntries(
Object.entries((init as any).headers)
);
return {
ok: true,
json: () => Promise.resolve(mockResponse),
} as any;
});
const result = await provider.chatCompletion(
'gsk_test123',
[{ role: 'user', content: 'Hi' }],
'llama-3.3-70b-versatile',
);
expect(capturedHeaders['Authorization']).toBe('Bearer gsk_test123');
expect(result.choices[0].message.content).toBe('Hello!');
expect(result._routed_via?.platform).toBe('groq');
});
it('should throw on API error', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({
ok: false,
status: 401,
statusText: 'Unauthorized',
json: () => Promise.resolve({ error: { message: 'Invalid API key' } }),
} as any);
await expect(
provider.chatCompletion('bad-key', [{ role: 'user', content: 'Hi' }], 'llama-3.3-70b-versatile')
).rejects.toThrow(/Invalid API key/);
});
it('should validate key', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({ ok: true } as any);
expect(await provider.validateKey('valid')).toBe(true);
vi.spyOn(global, 'fetch').mockResolvedValueOnce({ ok: false } as any);
expect(await provider.validateKey('invalid')).toBe(false);
});
});
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { OpenAICompatProvider } from '../../providers/openai-compat.js';
describe('OpenAICompatProvider', () => {
let provider: OpenAICompatProvider;
beforeEach(() => {
provider = new OpenAICompatProvider({
platform: 'groq',
name: 'TestProvider',
baseUrl: 'https://api.test.com/v1',
extraHeaders: { 'X-Custom': 'test' },
});
});
it('should set platform and name from config', () => {
expect(provider.platform).toBe('groq');
expect(provider.name).toBe('TestProvider');
});
it('should call API with correct URL and headers', async () => {
let capturedUrl = '';
let capturedHeaders: Record<string, string> = {};
vi.spyOn(global, 'fetch').mockImplementation(async (url, init) => {
capturedUrl = url as string;
capturedHeaders = (init as any).headers;
return {
ok: true,
json: () => Promise.resolve({
id: 'test-id',
object: 'chat.completion',
created: 123,
model: 'test-model',
choices: [{ index: 0, message: { role: 'assistant', content: 'hi' }, finish_reason: 'stop' }],
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
}),
} as any;
});
await provider.chatCompletion('my-key', [{ role: 'user', content: 'test' }], 'test-model');
expect(capturedUrl).toBe('https://api.test.com/v1/chat/completions');
expect(capturedHeaders['Authorization']).toBe('Bearer my-key');
expect(capturedHeaders['X-Custom']).toBe('test');
});
it('should throw on error response', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({
ok: false,
status: 429,
statusText: 'Rate Limited',
json: () => Promise.resolve({ error: { message: 'Too many requests' } }),
} as any);
await expect(
provider.chatCompletion('key', [{ role: 'user', content: 'hi' }], 'model')
).rejects.toThrow(/Too many requests/);
});
it('should validate key using models endpoint', async () => {
vi.spyOn(global, 'fetch').mockResolvedValueOnce({ ok: true } as any);
expect(await provider.validateKey('valid')).toBe(true);
});
});
describe('OpenAICompatProvider - platform instances', () => {
const platforms = [
{ platform: 'sambanova', name: 'SambaNova', baseUrl: 'https://api.sambanova.ai/v1' },
{ platform: 'nvidia', name: 'NVIDIA NIM', baseUrl: 'https://integrate.api.nvidia.com/v1' },
{ platform: 'mistral', name: 'Mistral', baseUrl: 'https://api.mistral.ai/v1' },
{ platform: 'openrouter', name: 'OpenRouter', baseUrl: 'https://openrouter.ai/api/v1' },
{ platform: 'github', name: 'GitHub Models', baseUrl: 'https://models.inference.ai.azure.com' },
{ platform: 'fireworks', name: 'Fireworks AI', baseUrl: 'https://api.fireworks.ai/inference/v1' },
] as const;
for (const p of platforms) {
it(`${p.name} provider should make requests to ${p.baseUrl}`, async () => {
const provider = new OpenAICompatProvider(p as any);
let capturedUrl = '';
vi.spyOn(global, 'fetch').mockImplementation(async (url) => {
capturedUrl = url as string;
return {
ok: true,
json: () => Promise.resolve({
id: 'id', object: 'chat.completion', created: 1, model: 'm',
choices: [{ index: 0, message: { role: 'assistant', content: 'ok' }, finish_reason: 'stop' }],
usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 },
}),
} as any;
});
const result = await provider.chatCompletion('key', [{ role: 'user', content: 'hi' }], 'model');
expect(capturedUrl).toContain(p.baseUrl);
expect(result._routed_via?.platform).toBe(p.platform);
});
}
});
import { describe, it, expect, beforeAll } from 'vitest';
import type { Express } from 'express';
import { createApp } from '../../app.js';
import { initDb } from '../../db/index.js';
async function request(app: Express, method: string, path: string, body?: any) {
const server = app.listen(0);
const addr = server.address() as any;
const url = `http://127.0.0.1:${addr.port}${path}`;
const res = await fetch(url, {
method,
headers: body ? { 'Content-Type': 'application/json' } : {},
body: body ? JSON.stringify(body) : undefined,
});
const data = await res.json().catch(() => null);
server.close();
return { status: res.status, body: data };
}
describe('Fallback API', () => {
let app: Express;
beforeAll(() => {
process.env.ENCRYPTION_KEY = '0'.repeat(64);
initDb(':memory:');
app = createApp();
});
it('GET /api/fallback returns fallback chain', async () => {
const { status, body } = await request(app, 'GET', '/api/fallback');
expect(status).toBe(200);
expect(Array.isArray(body)).toBe(true);
expect(body.length).toBeGreaterThan(0);
// Should be sorted by priority
for (let i = 1; i < body.length; i++) {
expect(body[i].priority).toBeGreaterThanOrEqual(body[i - 1].priority);
}
});
it('GET /api/fallback entries have expected fields', async () => {
const { body } = await request(app, 'GET', '/api/fallback');
const first = body[0];
expect(first).toHaveProperty('modelDbId');
expect(first).toHaveProperty('priority');
expect(first).toHaveProperty('enabled');
expect(first).toHaveProperty('platform');
expect(first).toHaveProperty('displayName');
expect(first).toHaveProperty('intelligenceRank');
});
it('PUT /api/fallback updates order', async () => {
const { body: original } = await request(app, 'GET', '/api/fallback');
// Reverse the order
const reversed = original.map((e: any, i: number) => ({
modelDbId: e.modelDbId,
priority: original.length - i,
enabled: e.enabled,
}));
const { status } = await request(app, 'PUT', '/api/fallback', reversed);
expect(status).toBe(200);
// Verify order changed
const { body: after } = await request(app, 'GET', '/api/fallback');
expect(after[0].modelDbId).toBe(original[original.length - 1].modelDbId);
// Restore original order
const restore = original.map((e: any, i: number) => ({
modelDbId: e.modelDbId,
priority: i + 1,
enabled: e.enabled,
}));
await request(app, 'PUT', '/api/fallback', restore);
});
it('POST /api/fallback/sort/intelligence sorts by intelligence', async () => {
const { status } = await request(app, 'POST', '/api/fallback/sort/intelligence');
expect(status).toBe(200);
const { body } = await request(app, 'GET', '/api/fallback');
// Should be sorted ascending by intelligence rank
for (let i = 1; i < body.length; i++) {
expect(body[i].intelligenceRank).toBeGreaterThanOrEqual(body[i - 1].intelligenceRank);
}
});
it('POST /api/fallback/sort/speed sorts by speed', async () => {
const { status } = await request(app, 'POST', '/api/fallback/sort/speed');
expect(status).toBe(200);
const { body } = await request(app, 'GET', '/api/fallback');
// Should be sorted ascending by speed rank
for (let i = 1; i < body.length; i++) {
expect(body[i].speedRank).toBeGreaterThanOrEqual(body[i - 1].speedRank);
}
});
it('POST /api/fallback/sort/invalid returns 400', async () => {
const { status } = await request(app, 'POST', '/api/fallback/sort/invalid');
expect(status).toBe(400);
});
});
import { describe, it, expect, beforeAll, beforeEach } from 'vitest';
import type { Express } from 'express';
import { createApp } from '../../app.js';
import { initDb, getDb } from '../../db/index.js';
async function request(app: Express, method: string, path: string, body?: any) {
const server = app.listen(0);
const addr = server.address() as any;
const url = `http://127.0.0.1:${addr.port}${path}`;
const res = await fetch(url, {
method,
headers: body ? { 'Content-Type': 'application/json' } : {},
body: body ? JSON.stringify(body) : undefined,
});
const data = await res.json().catch(() => null);
server.close();
return { status: res.status, body: data };
}
describe('Keys API', () => {
let app: Express;
beforeAll(() => {
process.env.ENCRYPTION_KEY = '0'.repeat(64);
initDb(':memory:');
app = createApp();
});
beforeEach(() => {
const db = getDb();
db.prepare('DELETE FROM api_keys').run();
});
it('GET /api/keys returns empty array initially', async () => {
const { status, body } = await request(app, 'GET', '/api/keys');
expect(status).toBe(200);
expect(body).toEqual([]);
});
it('POST /api/keys creates a new key', async () => {
const { status, body } = await request(app, 'POST', '/api/keys', {
platform: 'groq',
key: 'gsk_test123456789',
label: 'My Groq Key',
});
expect(status).toBe(201);
expect(body.platform).toBe('groq');
expect(body.label).toBe('My Groq Key');
expect(body.maskedKey).toContain('...');
});
it('GET /api/keys returns the created key', async () => {
// First create a key
await request(app, 'POST', '/api/keys', {
platform: 'groq',
key: 'gsk_test123456789',
});
const { status, body } = await request(app, 'GET', '/api/keys');
expect(status).toBe(200);
expect(body).toHaveLength(1);
expect(body[0].platform).toBe('groq');
});
it('POST /api/keys rejects invalid platform', async () => {
const { status } = await request(app, 'POST', '/api/keys', {
platform: 'invalid_platform',
key: 'test',
});
expect(status).toBe(400);
});
it('POST /api/keys rejects missing key', async () => {
const { status } = await request(app, 'POST', '/api/keys', {
platform: 'groq',
});
expect(status).toBe(400);
});
it('DELETE /api/keys/:id removes a key', async () => {
const { body: created } = await request(app, 'POST', '/api/keys', {
platform: 'groq',
key: 'gsk_test123456789',
});
const { status } = await request(app, 'DELETE', `/api/keys/${created.id}`);
expect(status).toBe(200);
const { body: after } = await request(app, 'GET', '/api/keys');
expect(after).toHaveLength(0);
});
it('DELETE /api/keys/:id returns 404 for nonexistent key', async () => {
const { status } = await request(app, 'DELETE', '/api/keys/99999');
expect(status).toBe(404);
});
});
import { describe, it, expect, beforeEach } from 'vitest';
import {
canMakeRequest,
canUseTokens,
recordRequest,
recordTokens,
getRateLimitStatus,
} from '../../services/ratelimit.js';
describe('Rate Limiter', () => {
// Use unique identifiers per test to avoid cross-contamination
let testId: number;
beforeEach(() => {
testId = Math.floor(Math.random() * 1_000_000);
});
describe('canMakeRequest', () => {
it('should allow request when under RPM limit', () => {
expect(canMakeRequest('groq', 'llama-70b', testId, {
rpm: 30, rpd: null, tpm: null, tpd: null,
})).toBe(true);
});
it('should deny request when RPM limit reached', () => {
const limits = { rpm: 2, rpd: null, tpm: null, tpd: null };
recordRequest('groq', 'llama-70b', testId);
recordRequest('groq', 'llama-70b', testId);
expect(canMakeRequest('groq', 'llama-70b', testId, limits)).toBe(false);
});
it('should deny request when RPD limit reached', () => {
const limits = { rpm: null, rpd: 1, tpm: null, tpd: null };
recordRequest('google', 'gemini', testId);
expect(canMakeRequest('google', 'gemini', testId, limits)).toBe(false);
});
it('should allow request when limits are null (unlimited)', () => {
expect(canMakeRequest('nvidia', 'nemotron', testId, {
rpm: null, rpd: null, tpm: null, tpd: null,
})).toBe(true);
});
});
describe('canUseTokens', () => {
it('should allow tokens when under TPM limit', () => {
expect(canUseTokens('groq', 'llama-70b', testId, 500, {
tpm: 6000, tpd: null,
})).toBe(true);
});
it('should deny tokens when TPM limit would be exceeded', () => {
recordTokens('cerebras', 'qwen3', testId, 50000);
expect(canUseTokens('cerebras', 'qwen3', testId, 20000, {
tpm: 60000, tpd: null,
})).toBe(false);
});
it('should allow when limit is null', () => {
expect(canUseTokens('nvidia', 'nemotron', testId, 100000, {
tpm: null, tpd: null,
})).toBe(true);
});
});
describe('getRateLimitStatus', () => {
it('should return current usage counts', () => {
const limits = { rpm: 30, rpd: 1000, tpm: 6000, tpd: null };
recordRequest('groq', 'test-model', testId);
recordRequest('groq', 'test-model', testId);
recordTokens('groq', 'test-model', testId, 500);
const status = getRateLimitStatus('groq', 'test-model', testId, limits);
expect(status.rpm.used).toBe(2);
expect(status.rpm.limit).toBe(30);
expect(status.rpd.used).toBe(2);
expect(status.tpm.used).toBe(500);
});
});
});
import { describe, it, expect, beforeAll, beforeEach } from 'vitest';
import { initDb, getDb } from '../../db/index.js';
import { encrypt } from '../../lib/crypto.js';
import { routeRequest } from '../../services/router.js';
describe('Router', () => {
beforeAll(() => {
process.env.ENCRYPTION_KEY = '0'.repeat(64);
initDb(':memory:');
});
beforeEach(() => {
const db = getDb();
db.prepare('DELETE FROM api_keys').run();
// Reset fallback order to intelligence ranking
const models = db.prepare('SELECT id, intelligence_rank FROM models ORDER BY intelligence_rank ASC').all() as any[];
const update = db.prepare('UPDATE fallback_config SET priority = ? WHERE model_db_id = ?');
for (let i = 0; i < models.length; i++) {
update.run(i + 1, models[i].id);
}
});
it('should throw when no keys are configured', () => {
expect(() => routeRequest()).toThrow(/exhausted/i);
});
it('should route to highest priority model with available key', () => {
const db = getDb();
const { encrypted, iv, authTag } = encrypt('test-groq-key');
db.prepare(`
INSERT INTO api_keys (platform, label, encrypted_key, iv, auth_tag, status, enabled)
VALUES (?, ?, ?, ?, ?, ?, ?)
`).run('groq', 'test', encrypted, iv, authTag, 'healthy', 1);
const result = routeRequest();
expect(result.platform).toBe('groq');
expect(result.apiKey).toBe('test-groq-key');
});
it('should prefer higher-priority model when keys exist for multiple platforms', () => {
const db = getDb();
const googleKey = encrypt('test-google-key');
db.prepare(`
INSERT INTO api_keys (platform, label, encrypted_key, iv, auth_tag, status, enabled)
VALUES (?, ?, ?, ?, ?, ?, ?)
`).run('google', 'test', googleKey.encrypted, googleKey.iv, googleKey.authTag, 'healthy', 1);
const groqKey = encrypt('test-groq-key');
db.prepare(`
INSERT INTO api_keys (platform, label, encrypted_key, iv, auth_tag, status, enabled)
VALUES (?, ?, ?, ?, ?, ?, ?)
`).run('groq', 'test', groqKey.encrypted, groqKey.iv, groqKey.authTag, 'healthy', 1);
const result = routeRequest();
expect(result.platform).toBe('google');
});
it('should skip disabled keys', () => {
const db = getDb();
const googleKey = encrypt('test-google-key');
db.prepare(`
INSERT INTO api_keys (platform, label, encrypted_key, iv, auth_tag, status, enabled)
VALUES (?, ?, ?, ?, ?, ?, ?)
`).run('google', 'disabled', googleKey.encrypted, googleKey.iv, googleKey.authTag, 'healthy', 0);
const groqKey = encrypt('test-groq-key');
db.prepare(`
INSERT INTO api_keys (platform, label, encrypted_key, iv, auth_tag, status, enabled)
VALUES (?, ?, ?, ?, ?, ?, ?)
`).run('groq', 'test', groqKey.encrypted, groqKey.iv, groqKey.authTag, 'healthy', 1);
const result = routeRequest();
expect(result.platform).toBe('groq');
});
it('should skip invalid keys', () => {
const db = getDb();
const invalidKey = encrypt('invalid-key');
db.prepare(`
INSERT INTO api_keys (platform, label, encrypted_key, iv, auth_tag, status, enabled)
VALUES (?, ?, ?, ?, ?, ?, ?)
`).run('google', 'invalid', invalidKey.encrypted, invalidKey.iv, invalidKey.authTag, 'invalid', 1);
const groqKey = encrypt('test-groq-key');
db.prepare(`
INSERT INTO api_keys (platform, label, encrypted_key, iv, auth_tag, status, enabled)
VALUES (?, ?, ?, ?, ?, ?, ?)
`).run('groq', 'test', groqKey.encrypted, groqKey.iv, groqKey.authTag, 'healthy', 1);
const result = routeRequest();
expect(result.platform).toBe('groq');
});
});
import express from 'express';
import cors from 'cors';
import helmet from 'helmet';
import path from 'path';
import { fileURLToPath } from 'url';
import { keysRouter } from './routes/keys.js';
import { modelsRouter } from './routes/models.js';
import { proxyRouter } from './routes/proxy.js';
import { fallbackRouter } from './routes/fallback.js';
import { analyticsRouter } from './routes/analytics.js';
import { healthRouter } from './routes/health.js';
import { settingsRouter } from './routes/settings.js';
import { errorHandler } from './middleware/errorHandler.js';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
export function createApp() {
const app = express();
app.use(helmet({ contentSecurityPolicy: false, hsts: false }));
app.use(cors());
app.use(express.json({ limit: '1mb' }));
// API routes
app.use('/api/keys', keysRouter);
app.use('/api/models', modelsRouter);
app.use('/api/fallback', fallbackRouter);
app.use('/api/analytics', analyticsRouter);
app.use('/api/health', healthRouter);
app.use('/api/settings', settingsRouter);
// OpenAI-compatible proxy
app.use('/v1', proxyRouter);
// Health check
app.get('/api/ping', (_req, res) => {
res.json({ status: 'ok', timestamp: new Date().toISOString() });
});
// Error handler (for API routes)
app.use(errorHandler);
// Serve client static files (after API error handler)
const clientDist = path.resolve(__dirname, '../../client/dist');
app.use(express.static(clientDist));
// SPA fallback — serve index.html for non-API routes
app.use((req, res, next) => {
if (req.path.startsWith('/api/') || req.path.startsWith('/v1/')) {
next();
return;
}
res.sendFile(path.join(clientDist, 'index.html'));
});
return app;
}
This diff is collapsed.
import { createApp } from './app.js';
import { initDb } from './db/index.js';
import { startHealthChecker } from './services/health.js';
const PORT = process.env.PORT ?? 3001;
async function main() {
initDb();
const app = createApp();
app.listen(Number(PORT), '0.0.0.0', () => {
console.log(`Server running on http://0.0.0.0:${PORT}`);
console.log(`Proxy endpoint: http://0.0.0.0:${PORT}/v1/chat/completions`);
startHealthChecker();
});
}
main().catch(console.error);
import crypto from 'crypto';
import Database from 'better-sqlite3';
const ALGORITHM = 'aes-256-gcm';
let cachedKey: Buffer | null = null;
/**
* Initialize encryption key from env, DB, or generate a new one.
* Must be called after DB is initialized.
*/
export function initEncryptionKey(db: Database.Database): void {
// 1. Check env var
const envKey = process.env.ENCRYPTION_KEY;
if (envKey && envKey !== 'your-64-char-hex-key-here') {
cachedKey = Buffer.from(envKey, 'hex');
return;
}
// 2. Check DB for persisted key
const row = db.prepare("SELECT value FROM settings WHERE key = 'encryption_key'").get() as { value: string } | undefined;
if (row) {
cachedKey = Buffer.from(row.value, 'hex');
return;
}
// 3. Generate and persist
cachedKey = crypto.randomBytes(32);
db.prepare("INSERT INTO settings (key, value) VALUES ('encryption_key', ?)").run(cachedKey.toString('hex'));
}
function getEncryptionKey(): Buffer {
if (!cachedKey) {
throw new Error('Encryption key not initialized. Call initEncryptionKey() first.');
}
return cachedKey;
}
export function encrypt(text: string): { encrypted: string; iv: string; authTag: string } {
const key = getEncryptionKey();
const iv = crypto.randomBytes(16);
const cipher = crypto.createCipheriv(ALGORITHM, key, iv);
let encrypted = cipher.update(text, 'utf8', 'hex');
encrypted += cipher.final('hex');
const authTag = cipher.getAuthTag().toString('hex');
return {
encrypted,
iv: iv.toString('hex'),
authTag,
};
}
export function decrypt(encrypted: string, iv: string, authTag: string): string {
const key = getEncryptionKey();
const decipher = crypto.createDecipheriv(ALGORITHM, key, Buffer.from(iv, 'hex'));
decipher.setAuthTag(Buffer.from(authTag, 'hex'));
let decrypted = decipher.update(encrypted, 'hex', 'utf8');
decrypted += decipher.final('utf8');
return decrypted;
}
export function maskKey(key: string): string {
if (key.length <= 8) return '****' + key.slice(-4);
return key.slice(0, 4) + '...' + key.slice(-4);
}
import type { Request, Response, NextFunction } from 'express';
export function errorHandler(err: Error, _req: Request, res: Response, _next: NextFunction) {
console.error('[Error]', err.message);
const status = (err as any).status ?? 500;
res.status(status).json({
error: {
message: err.message,
type: err.name ?? 'server_error',
},
});
}
import type {
ChatMessage,
ChatCompletionResponse,
ChatCompletionChunk,
Platform,
} from '@freellmapi/shared/types.js';
export interface CompletionOptions {
model?: string;
temperature?: number;
max_tokens?: number;
top_p?: number;
}
export abstract class BaseProvider {
abstract readonly platform: Platform;
abstract readonly name: string;
abstract chatCompletion(
apiKey: string,
messages: ChatMessage[],
modelId: string,
options?: CompletionOptions,
): Promise<ChatCompletionResponse>;
abstract streamChatCompletion(
apiKey: string,
messages: ChatMessage[],
modelId: string,
options?: CompletionOptions,
): AsyncGenerator<ChatCompletionChunk>;
abstract validateKey(apiKey: string): Promise<boolean>;
protected async fetchWithTimeout(
url: string,
init: RequestInit,
timeoutMs = 15000,
): Promise<Response> {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
return await fetch(url, { ...init, signal: controller.signal });
} finally {
clearTimeout(timeout);
}
}
protected makeId(): string {
return `chatcmpl-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
}
}
import type {
ChatMessage,
ChatCompletionResponse,
ChatCompletionChunk,
} from '@freellmapi/shared/types.js';
import { BaseProvider, type CompletionOptions } from './base.js';
const API_BASE = 'https://api.cerebras.ai/v1';
export class CerebrasProvider extends BaseProvider {
readonly platform = 'cerebras' as const;
readonly name = 'Cerebras';
async chatCompletion(
apiKey: string,
messages: ChatMessage[],
modelId: string,
options?: CompletionOptions,
): Promise<ChatCompletionResponse> {
const res = await this.fetchWithTimeout(`${API_BASE}/chat/completions`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: modelId,
messages,
temperature: options?.temperature,
max_tokens: options?.max_tokens,
top_p: options?.top_p,
}),
});
if (!res.ok) {
const err = await res.json().catch(() => ({}));
throw new Error(`Cerebras API error ${res.status}: ${(err as any).error?.message ?? res.statusText}`);
}
const data = await res.json() as ChatCompletionResponse;
data._routed_via = { platform: 'cerebras', model: modelId };
return data;
}
async *streamChatCompletion(
apiKey: string,
messages: ChatMessage[],
modelId: string,
options?: CompletionOptions,
): AsyncGenerator<ChatCompletionChunk> {
const res = await this.fetchWithTimeout(`${API_BASE}/chat/completions`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: modelId,
messages,
temperature: options?.temperature,
max_tokens: options?.max_tokens,
top_p: options?.top_p,
stream: true,
}),
});
if (!res.ok) {
const err = await res.json().catch(() => ({}));
throw new Error(`Cerebras API error ${res.status}: ${(err as any).error?.message ?? res.statusText}`);
}
const reader = res.body?.getReader();
if (!reader) throw new Error('No response body');
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split('\n');
buffer = lines.pop() ?? '';
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed || !trimmed.startsWith('data: ')) continue;
const data = trimmed.slice(6);
if (data === '[DONE]') return;
yield JSON.parse(data) as ChatCompletionChunk;
}
}
}
async validateKey(apiKey: string): Promise<boolean> {
try {
const res = await this.fetchWithTimeout(`${API_BASE}/models`, {
method: 'GET',
headers: { 'Authorization': `Bearer ${apiKey}` },
}, 10000);
return res.ok;
} catch {
return false;
}
}
}
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
import { Router } from 'express';
import type { Request, Response } from 'express';
import { getDb } from '../db/index.js';
import { hasProvider } from '../providers/index.js';
export const modelsRouter = Router();
// List all models with availability info
modelsRouter.get('/', (_req: Request, res: Response) => {
const db = getDb();
const models = db.prepare(`
SELECT m.*, fc.priority, fc.enabled as fallback_enabled
FROM models m
LEFT JOIN fallback_config fc ON fc.model_db_id = m.id
ORDER BY COALESCE(fc.priority, m.intelligence_rank) ASC
`).all() as any[];
// Count keys per platform
const keyCounts = db.prepare(`
SELECT platform, COUNT(*) as count
FROM api_keys
WHERE enabled = 1
GROUP BY platform
`).all() as { platform: string; count: number }[];
const keyCountMap = new Map(keyCounts.map(k => [k.platform, k.count]));
const result = models.map(m => ({
id: m.id,
platform: m.platform,
modelId: m.model_id,
displayName: m.display_name,
intelligenceRank: m.intelligence_rank,
speedRank: m.speed_rank,
sizeLabel: m.size_label,
rpmLimit: m.rpm_limit,
rpdLimit: m.rpd_limit,
tpmLimit: m.tpm_limit,
tpdLimit: m.tpd_limit,
monthlyTokenBudget: m.monthly_token_budget,
contextWindow: m.context_window,
enabled: m.enabled === 1,
priority: m.priority,
fallbackEnabled: m.fallback_enabled === 1,
hasProvider: hasProvider(m.platform),
keyCount: keyCountMap.get(m.platform) ?? 0,
}));
res.json(result);
});
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment