Skip to content

C
canifa_note
Unverified Commit 334dfef7 authored by memoclaw's avatar memoclaw Committed by GitHub
Browse files
Options

docs: streamline AGENTS.md and simplify CLAUDE.md (#5685) 

Co-authored-by: 's avatarClaude Opus 4.6 <noreply@anthropic.com>
parent 5a17614a
Show whitespace changes
Inline Side-by-side
Showing with 76 additions and 596 deletions
AGENTS.md
View file @ 334dfef7
# Memos Codebase Guide for AI Agents
This document provides comprehensive guidance for AI agents working with the Memos codebase. It covers architecture, workflows, conventions, and key patterns.
Self-hosted knowledge management platform. Go 1.25 backend (Echo v5, Connect RPC + gRPC-Gateway), React 18 + TypeScript 5.9 + Vite 7 frontend, Protocol Buffers API, SQLite/MySQL/PostgreSQL.
## Project Overview
## Commands
Memos is a self-hosted knowledge management platform built with:
- **Backend:** Go 1.25 with gRPC + Connect RPC
- **Frontend:** React 18.3 + TypeScript + Vite 7
- **Databases:** SQLite (default), MySQL, PostgreSQL
- **Protocol:** Protocol Buffers (v2) with buf for code generation
- **API Layer:** Dual protocol - Connect RPC (browsers) + gRPC-Gateway (REST)
```bash
# Backend
go run ./cmd/memos --port 8081 # Start dev server
go test ./... # Run all tests
golangci-lint run # Lint (v2)
goimports -w . # Format imports
# Frontend (cd web)
pnpm install # Install deps
pnpm dev # Dev server (:3001, proxies API to :8081)
pnpm lint # Type check + Biome lint
pnpm lint:fix # Auto-fix lint issues
pnpm format # Format code
pnpm build # Production build
pnpm release # Build to server/router/frontend/dist
# Protocol Buffers (cd proto)
buf generate # Regenerate Go + TypeScript + OpenAPI
buf lint # Lint proto files
```
## Architecture
### Backend Architecture
```
cmd/memos/ # Entry point
└── main.go # Cobra CLI, profile setup, server initialization
cmd/memos/main.go # Cobra CLI + Viper config, server init
server/
├── server.go # Echo HTTP server, healthz, background runners
├── auth/ # Authentication (JWT, PAT, session)
├── server.go # Echo v5 HTTP server, background runners
├── auth/ # JWT access (15min) + refresh (30d) tokens, PAT
├── router/
│ ├── api/v1/ # gRPC service implementations
│ │ ├── v1.go # Service registration, gateway & Connect setup
│ ├── api/v1/ # 9 gRPC services (Connect + Gateway)
│ │ ├── acl_config.go # Public endpoints whitelist
│ │ ├── connect_services.go # Connect RPC handlers
│ │ ├── connect_interceptors.go # Auth, logging, recovery
│ │ └── *_service.go # Individual services (memo, user, etc.)
│ ├── frontend/ # Static file serving (SPA)
│ ├── fileserver/ # Native HTTP file serving for media
│ └── rss/ # RSS feed generation
└── runner/
├── memopayload/ # Memo payload processing (tags, links, tasks)
└── s3presign/ # S3 presigned URL management
store/ # Data layer with caching
├── driver.go # Driver interface (database operations)
├── store.go # Store wrapper with cache layer
├── cache.go # In-memory caching (instance settings, users)
├── migrator.go # Database migrations
├── db/
│ ├── db.go # Driver factory
│ ├── sqlite/ # SQLite implementation
│ ├── mysql/ # MySQL implementation
│ └── postgres/ # PostgreSQL implementation
└── migration/ # SQL migration files (embedded)
proto/ # Protocol Buffer definitions
├── api/v1/ # API v1 service definitions
└── gen/ # Generated Go & TypeScript code
```
### Frontend Architecture
```
web/
├── src/
│ ├── components/ # React components
│ ├── contexts/ # React Context (client state)
│ │ ├── AuthContext.tsx # Current user, auth state
│ │ ├── ViewContext.tsx # Layout, sort order
│ │ └── MemoFilterContext.tsx # Filters, shortcuts
│ ├── hooks/ # React Query hooks (server state)
│ │ ├── useMemoQueries.ts # Memo CRUD, pagination
│ │ ├── useUserQueries.ts # User operations
│ │ ├── useAttachmentQueries.ts # Attachment operations
│ │ └── ...
│ ├── lib/ # Utilities
│ │ ├── query-client.ts # React Query v5 client
│ │ └── connect.ts # Connect RPC client setup
│ ├── pages/ # Page components
│ └── types/proto/ # Generated TypeScript from .proto
├── package.json # Dependencies
└── vite.config.mts # Vite config with dev proxy
plugin/ # Backend plugins
├── scheduler/ # Cron jobs
├── email/ # Email delivery
├── filter/ # CEL filter expressions
├── webhook/ # Webhook dispatch
├── markdown/ # Markdown parsing & rendering
├── httpgetter/ # HTTP fetching (metadata, images)
└── storage/s3/ # S3 storage backend
```
## Key Architectural Patterns
### 1. API Layer: Dual Protocol
**Connect RPC (Browser Clients):**
- Protocol: `connectrpc.com/connect`
- Base path: `/memos.api.v1.*`
- Interceptor chain: Metadata → Logging → Recovery → Auth
- Returns type-safe responses to React frontend
- See: `server/router/api/v1/connect_interceptors.go:177-227`
**gRPC-Gateway (REST API):**
- Protocol: Standard HTTP/JSON
- Base path: `/api/v1/*`
- Uses same service implementations as Connect
- Useful for external tools, CLI clients
- See: `server/router/api/v1/v1.go:52-96`
**Authentication:**
- JWT Access Tokens (V2): Stateless, 15-min expiration, verified via `AuthenticateByAccessTokenV2`
- Personal Access Tokens (PAT): Stateful, long-lived, validated against database
- Both use `Authorization: Bearer <token>` header
- See: `server/auth/authenticator.go:17-166`
### 2. Store Layer: Interface Pattern
All database operations go through the `Driver` interface:
```go
type Driver interface {
GetDB() *sql.DB
Close() error
IsInitialized(ctx context.Context) (bool, error)
CreateMemo(ctx context.Context, create *Memo) (*Memo, error)
ListMemos(ctx context.Context, find *FindMemo) ([]*Memo, error)
UpdateMemo(ctx context.Context, update *UpdateMemo) error
DeleteMemo(ctx context.Context, delete *DeleteMemo) error
// ... similar methods for all resources
}
```
**Three Implementations:**
- `store/db/sqlite/` - SQLite (modernc.org/sqlite)
- `store/db/mysql/` - MySQL (go-sql-driver/mysql)
- `store/db/postgres/` - PostgreSQL (lib/pq)
**Caching Strategy:**
- Store wrapper maintains in-memory caches for:
- Instance settings (`instanceSettingCache`)
- Users (`userCache`)
- User settings (`userSettingCache`)
- Config: Default TTL 10 min, cleanup interval 5 min, max 1000 items
- See: `store/store.go:10-57`
### 3. Frontend State Management
**React Query v5 (Server State):**
- All API calls go through custom hooks in `web/src/hooks/`
- Query keys organized by resource: `memoKeys`, `userKeys`, `attachmentKeys`
- Default staleTime: 30s, gcTime: 5min
- Automatic refetch on window focus, reconnect
- See: `web/src/lib/query-client.ts`
**React Context (Client State):**
- `AuthContext`: Current user, auth initialization, logout
- `ViewContext`: Layout mode (LIST/MASONRY), sort order
- `MemoFilterContext`: Active filters, shortcut selection, URL sync
### 4. Database Migration System
**Migration Flow:**
1. `preMigrate`: Check if DB exists. If not, apply `LATEST.sql`
2. `checkMinimumUpgradeVersion`: Reject pre-0.22 installations
3. `applyMigrations`: Apply incremental migrations in single transaction
4. Demo mode: Seed with demo data
**Schema Versioning:**
- Stored in `system_setting` table
- Format: `major.minor.patch`
- Migration files: `store/migration/{driver}/{version}/NN__description.sql`
- See: `store/migrator.go:21-414`
### 5. Protocol Buffer Code Generation
**Definition Location:** `proto/api/v1/*.proto`
**Regeneration:**
```bash
cd proto && buf generate
```
**Generated Outputs:**
- Go: `proto/gen/api/v1/` (used by backend services)
- TypeScript: `web/src/types/proto/api/v1/` (used by frontend)
**Linting:** `proto/buf.yaml` - BASIC lint rules, FILE breaking changes
## Development Commands
### Backend
```bash
# Start dev server
go run ./cmd/memos --port 8081
# Run all tests
go test ./...
# Run tests for specific package
go test ./store/...
go test ./server/router/api/v1/test/...
# Lint (golangci-lint)
golangci-lint run
# Format imports
goimports -w .
# Run with MySQL/Postgres
DRIVER=mysql go run ./cmd/memos
DRIVER=postgres go run ./cmd/memos
```
### Frontend
```bash
# Install dependencies
cd web && pnpm install
# Start dev server (proxies API to localhost:8081)
pnpm dev
# Type checking
pnpm lint
# Auto-fix lint issues
pnpm lint:fix
# Format code
pnpm format
# Build for production
pnpm build
# Build and copy to backend
pnpm release
```
### Protocol Buffers
```bash
# Regenerate Go and TypeScript from .proto files
cd proto && buf generate
# Lint proto files
cd proto && buf lint
# Check for breaking changes
cd proto && buf breaking --against .git#main
```
## Key Workflows
### Adding a New API Endpoint
1. **Define in Protocol Buffer:**
- Edit `proto/api/v1/*_service.proto`
- Add request/response messages
- Add RPC method to service
2. **Regenerate Code:**
```bash
cd proto && buf generate
```
3. **Implement Service (Backend):**
- Add method to `server/router/api/v1/*_service.go`
- Follow existing patterns: fetch user, validate, call store
- Add Connect wrapper to `server/router/api/v1/connect_services.go` (optional, same implementation)
4. **If Public Endpoint:**
- Add to `server/router/api/v1/acl_config.go:11-34`
5. **Create Frontend Hook (if needed):**
- Add query/mutation to `web/src/hooks/use*Queries.ts`
- Use existing query key factories
### Database Schema Changes
1. **Create Migration Files:**
```
store/migration/sqlite/0.28/1__add_new_column.sql
store/migration/mysql/0.28/1__add_new_column.sql
store/migration/postgres/0.28/1__add_new_column.sql
```
2. **Update LATEST.sql:**
- Add change to `store/migration/{driver}/LATEST.sql`
3. **Update Store Interface (if new table/model):**
- Add methods to `store/driver.go:8-71`
- Implement in `store/db/{driver}/*.go`
4. **Test Migration:**
- Run `go test ./store/test/...` to verify
### Adding a New Frontend Page
1. **Create Page Component:**
- Add to `web/src/pages/NewPage.tsx`
- Use existing hooks for data fetching
2. **Add Route:**
- Edit `web/src/App.tsx` (or router configuration)
3. **Use React Query:**
```typescript
import { useMemos } from "@/hooks/useMemoQueries";
const { data, isLoading } = useMemos({ filter: "..." });
```
4. **Use Context for Client State:**
```typescript
import { useView } from "@/contexts/ViewContext";
const { layout, toggleSortOrder } = useView();
```
## Testing
### Backend Tests
**Test Pattern:**
```go
func TestMemoCreation(t *testing.T) {
ctx := context.Background()
store := test.NewTestingStore(ctx, t)
// Create test user
user, _ := createTestUser(ctx, store, t)
// Execute operation
memo, err := store.CreateMemo(ctx, &store.Memo{
CreatorID: user.ID,
Content: "Test memo",
// ...
})
require.NoError(t, err)
assert.NotNil(t, memo)
}
│ │ ├── sse_hub.go # Server-Sent Events (live updates)
│ │ └── mcp/ # MCP server for AI assistants
│ ├── frontend/ # SPA static file serving
│ ├── fileserver/ # Native HTTP file server (thumbnails, range requests)
│ └── rss/ # RSS feeds
└── runner/ # Background: memo payload processing, S3 presign refresh
store/
├── driver.go # Database driver interface
├── store.go # Store wrapper + in-memory cache (TTL 10min, max 1000)
├── migrator.go # Migration logic (LATEST.sql for fresh, incremental for upgrades)
└── db/{sqlite,mysql,postgres}/ # Driver implementations
proto/
├── api/v1/ # Service definitions
├── store/ # Internal storage messages
└── gen/ # Generated Go, TypeScript, OpenAPI
plugin/ # scheduler, cron, email, filter (CEL), webhook,
# markdown (Goldmark), httpgetter, idp (OAuth2), storage/s3
web/src/
├── connect.ts # Connect RPC client + auth interceptor + token refresh
├── auth-state.ts # Token storage (localStorage + BroadcastChannel cross-tab)
├── contexts/ # AuthContext, InstanceContext, ViewContext, MemoFilterContext
├── hooks/ # React Query hooks (useMemoQueries, useUserQueries, etc.)
├── lib/query-client.ts # React Query v5 (staleTime: 30s, gcTime: 5min)
├── router/index.tsx # Route definitions
├── components/ # UI components (Radix UI primitives, MemoEditor, Settings, etc.)
├── themes/ # CSS themes (default, dark, paper) — OKLch color tokens
└── pages/ # 14 page components
```
**Test Utilities:**
- `store/test/store.go:22-35` - `NewTestingStore()` creates isolated DB
- `store/test/store.go:37-77` - `resetTestingDB()` cleans tables
- Test DB determined by `DRIVER` env var (default: sqlite)
**Running Tests:**
```bash
# All tests
go test ./...
# Specific package
go test ./store/...
go test ./server/router/api/v1/test/...
# With coverage
go test -cover ./...
```
### Frontend Testing
**TypeScript Checking:**
```bash
cd web && pnpm lint
```
**No Automated Tests:**
- Frontend relies on TypeScript checking and manual validation
- React Query DevTools available in dev mode (bottom-left)
## Code Conventions
## Conventions
### Go
**Error Handling:**
- Use `github.com/pkg/errors` for wrapping: `errors.Wrap(err, "context")`
- Return structured gRPC errors: `status.Errorf(codes.NotFound, "message")`
**Naming:**
- Package names: lowercase, single word (e.g., `store`, `server`)
- Interfaces: `Driver`, `Store`, `Service`
- Methods: PascalCase for exported, camelCase for internal
**Comments:**
- Public exported functions must have comments (godot enforces)
- Use `//` for single-line, `/* */` for multi-line
**Imports:**
- Grouped: stdlib, third-party, local
- Sorted alphabetically within groups
- Use `goimports -w .` to format
### TypeScript/React
**Components:**
- Functional components with hooks
- Use `useMemo`, `useCallback` for optimization
- Props interfaces: `interface Props { ... }`
**State Management:**
- Server state: React Query hooks
- Client state: React Context
- Avoid direct useState for server data
**Styling:**
- Tailwind CSS v4 via `@tailwindcss/vite`
- Use `clsx` and `tailwind-merge` for conditional classes
**Imports:**
- Absolute imports with `@/` alias
- Group: React, third-party, local
- Auto-organized by Biome
## Important Files Reference
### Backend Entry Points
| File | Purpose |
|------|---------|
| `cmd/memos/main.go` | Server entry point, CLI setup |
| `server/server.go` | Echo server initialization, background runners |
| `store/store.go` | Store wrapper with caching |
| `store/driver.go` | Database driver interface |
### API Layer
| File | Purpose |
|------|---------|
| `server/router/api/v1/v1.go` | Service registration, gateway setup |
| `server/router/api/v1/acl_config.go` | Public endpoints whitelist |
| `server/router/api/v1/connect_interceptors.go` | Connect interceptors |
| `server/auth/authenticator.go` | Authentication logic |
### Frontend Core
| File | Purpose |
|------|---------|
| `web/src/lib/query-client.ts` | React Query client configuration |
| `web/src/contexts/AuthContext.tsx` | User authentication state |
| `web/src/contexts/ViewContext.tsx` | UI preferences |
| `web/src/contexts/MemoFilterContext.tsx` | Filter state |
| `web/src/hooks/useMemoQueries.ts` | Memo queries/mutations |
### Data Layer
| File | Purpose |
|------|---------|
| `store/memo.go` | Memo model definitions, store methods |
| `store/user.go` | User model definitions |
| `store/attachment.go` | Attachment model definitions |
| `store/migrator.go` | Migration logic |
| `store/db/db.go` | Driver factory |
| `store/db/sqlite/sqlite.go` | SQLite driver implementation |
## Configuration
### Backend Environment Variables
| Variable | Default | Description |
|----------|----------|-------------|
| `MEMOS_DEMO` | `false` | Enable demo mode |
| `MEMOS_PORT` | `8081` | HTTP port |
| `MEMOS_ADDR` | `` | Bind address (empty = all) |
| `MEMOS_DATA` | `~/.memos` | Data directory |
| `MEMOS_DRIVER` | `sqlite` | Database: `sqlite`, `mysql`, `postgres` |
| `MEMOS_DSN` | `` | Database connection string |
| `MEMOS_INSTANCE_URL` | `` | Instance base URL |
### Frontend Environment Variables
| Variable | Default | Description |
|----------|----------|-------------|
| `DEV_PROXY_SERVER` | `http://localhost:8081` | Backend proxy target |
## CI/CD
### GitHub Workflows
**Backend Tests** (`.github/workflows/backend-tests.yml`):
- Runs on `go.mod`, `go.sum`, `**.go` changes
- Steps: verify `go mod tidy`, golangci-lint, all tests
**Frontend Tests** (`.github/workflows/frontend-tests.yml`):
- Runs on `web/**` changes
- Steps: pnpm install, lint, build
**Proto Lint** (`.github/workflows/proto-linter.yml`):
- Runs on `.proto` changes
- Steps: buf lint, buf breaking check
### Linting Configuration
**Go** (`.golangci.yaml`):
- Linters: revive, govet, staticcheck, misspell, gocritic, etc.
- Formatter: goimports
- Forbidden: `fmt.Errorf`, `ioutil.ReadDir`
**TypeScript** (`web/biome.json`):
- Linting: Biome (ESLint replacement)
- Formatting: Biome (Prettier replacement)
- Line width: 140 characters
- Semicolons: always
## Common Tasks
### Debugging API Issues
1. Check Connect interceptor logs: `server/router/api/v1/connect_interceptors.go:79-105`
2. Verify endpoint is in `acl_config.go` if public
3. Check authentication via `auth/authenticator.go:133-165`
4. Test with curl: `curl -H "Authorization: Bearer <token>" http://localhost:8081/api/v1/...`
### Debugging Frontend State
1. Open React Query DevTools (bottom-left in dev)
2. Inspect query cache, mutations, refetch behavior
3. Check Context state via React DevTools
4. Verify filter state in MemoFilterContext
### Running Tests Against Multiple Databases
```bash
# SQLite (default)
DRIVER=sqlite go test ./...
# MySQL (requires running MySQL server)
DRIVER=mysql DSN="user:pass@tcp(localhost:3306)/memos" go test ./...
# PostgreSQL (requires running PostgreSQL server)
DRIVER=postgres DSN="postgres://user:pass@localhost:5432/memos" go test ./...
```
## Plugin System
Backend supports pluggable components in `plugin/`:
| Plugin | Purpose |
|--------|----------|
| `scheduler` | Cron-based job scheduling |
| `email` | SMTP email delivery |
| `filter` | CEL expression filtering |
| `webhook` | HTTP webhook dispatch |
| `markdown` | Markdown parsing (goldmark) |
| `httpgetter` | HTTP content fetching |
| `storage/s3` | S3-compatible storage |
Each plugin has its own README with usage examples.
## Performance Considerations
### Backend
- Database queries use pagination (`limit`, `offset`)
- In-memory caching reduces DB hits for frequently accessed data
- WAL journal mode for SQLite (reduces locking)
- Thumbnail generation limited to 3 concurrent operations
- **Errors:** `errors.Wrap(err, "context")` from `github.com/pkg/errors`. Never `fmt.Errorf` (lint-enforced).
- **gRPC errors:** `status.Errorf(codes.X, "message")` from service methods.
- **Imports:** stdlib → third-party → local (`github.com/usememos/memos`). Run `goimports -w .`.
- **Comments:** All exported functions must have doc comments (godot enforced).
### Frontend
- **Imports:** Use `@/` alias for absolute imports.
- **Formatting:** Biome — 140 char lines, double quotes, always semicolons, 2-space indent.
- **State:** Server data via React Query hooks (`hooks/`). Client state via React Context (`contexts/`).
- **Styling:** Tailwind CSS v4 (`@tailwindcss/vite`), `cn()` utility (clsx + tailwind-merge), CVA for variants.
- React Query reduces redundant API calls
- Infinite queries for large lists (pagination)
- Manual chunks: `utils-vendor`, `mermaid-vendor`, `leaflet-vendor`
- Lazy loading for heavy components
### Database & Proto
- **DB changes:** Migration files for all 3 drivers + update `LATEST.sql`.
- **Proto changes:** Run `buf generate`. Generated code: `proto/gen/` and `web/src/types/proto/`.
- **Public endpoints:** Add to `server/router/api/v1/acl_config.go`.
## Security Notes
## CI/CD
- JWT secrets must be kept secret (generated on first run in production mode)
- Personal Access Tokens stored as SHA-256 hashes in database
- CSRF protection via SameSite cookies
- CORS enabled for all origins (configure for production)
- Input validation at service layer
- SQL injection prevention via parameterized queries
- **backend-tests.yml:** Go 1.25.7, golangci-lint v2.4.0, tests parallelized by group (store tests all 3 DBs via TestContainers)
- **frontend-tests.yml:** Node 22, pnpm 10, lint + build
- **proto-linter.yml:** buf lint + format check
- **Docker:** Multi-stage (`scripts/Dockerfile`), Alpine 3.21, non-root user, port 5230, multi-arch (amd64/arm64/arm/v7)
CLAUDE.md
View file @ 334dfef7
# CLAUDE.md
Memos — self-hosted knowledge management platform. Go 1.25 backend, React 18 + TypeScript + Vite 7 frontend, Protocol Buffers API.
See `AGENTS.md` for full architecture, workflows, and patterns.
## Essential Commands
### Backend
```bash
go run ./cmd/memos --port 8081 # Start dev server
go test ./... # Run all tests
golangci-lint run # Lint
goimports -w . # Format imports
```
### Frontend
```bash
cd web && pnpm install # Install deps
pnpm dev # Dev server (proxies to :8081)
pnpm lint # Type check + Biome lint
pnpm lint:fix # Auto-fix lint issues
pnpm format # Format code
pnpm build # Production build
```
### Protocol Buffers
```bash
cd proto && buf generate # Regenerate Go + TypeScript
cd proto && buf lint # Lint proto files
```
## Key Rules
- **Go errors:** Use `errors.Wrap(err, "context")` from `github.com/pkg/errors`. Never use `fmt.Errorf`.
- **Go imports:** stdlib → third-party → local (`github.com/usememos/memos`). Run `goimports -w .`.
- **Go comments:** All exported functions must have doc comments (godot enforced).
- **Go gRPC errors:** Return `status.Errorf(codes.X, "message")` from service methods.
- **Frontend imports:** Use `@/` alias for absolute imports.
- **Frontend formatting:** Biome — 140 char lines, double quotes, always semicolons, 2-space indent.
- **Frontend state:** Server data via React Query hooks (`web/src/hooks/`). Client state via React Context.
- **Database changes:** Must provide migration files for all 3 drivers (sqlite, mysql, postgres) and update `LATEST.sql`.
- **Proto changes:** Run `buf generate` after editing `.proto` files. Generated code is in `proto/gen/` and `web/src/types/proto/`.
- **Public endpoints:** Add to `server/router/api/v1/acl_config.go`.
See `AGENTS.md` for full architecture, workflows, conventions, and patterns.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment