Architecture Overview

This document serves as a critical, living template designed to equip new developers with a rapid and comprehensive understanding of the codebase’s architecture, enabling efficient navigation and effective contribution from day one. Update this document as the codebase evolves.

1. Project Structure

This section provides a high-level overview of the project’s directory and file structure, categorised by architectural layer or major functional area. It is essential for quickly navigating the codebase, locating relevant files, and understanding the overall organization and separation of concerns.

[Project Root]/
├── crates/                      # All Rust workspace crates
│   ├── api/                     # API layer
│   │   ├── api/                 #   Core API handlers (lemmy_api)
│   │   ├── api_crud/            #   CRUD operations (lemmy_api_crud)
│   │   ├── api_common/          #   Shared API types (lemmy_api_common)
│   │   ├── api_utils/           #   Utilities (JWT, context, notifications, plugins)
│   │   ├── routes/              #   API v4 route definitions (lemmy_api_routes)
│   │   └── routes_v3/           #   Legacy API v3 routes (lemmy_api_routes_v3)
│   ├── apub/                    # ActivityPub federation layer
│   │   ├── apub/                #   Core federation logic (lemmy_apub)
│   │   ├── activities/          #   Activity types (lemmy_apub_activities)
│   │   ├── objects/             #   Object types (lemmy_apub_objects)
│   │   └── send/                #   Outgoing activity sending (lemmy_apub_send)
│   ├── db_schema/               # Database models, source definitions, newtypes (lemmy_db_schema)
│   ├── db_schema_file/          # Auto-generated schema.rs + enums.rs + joins.rs (lemmy_db_schema_file)
│   ├── db_views/                # 24 query-view crates (read models with joins, filtering, pagination)
│   │   ├── post/                #   PostView (lemmy_db_views_post)
│   │   ├── comment/             #   CommentView (lemmy_db_views_comment)
│   │   ├── community/           #   CommunityView (lemmy_db_views_community)
│   │   ├── person/              #   PersonView (lemmy_db_views_person)
│   │   ├── site/                #   SiteView (lemmy_db_views_site)
│   │   ├── local_user/          #   LocalUserView (lemmy_db_views_local_user)
│   │   ├── person_content_combined/  # Profile content (lemmy_db_views_person_content_combined)
│   │   ├── notification/        #   NotificationView (lemmy_db_views_notification)
│   │   └── ... (16 more)
│   ├── diesel_utils/            # DB connection pool, migrations, pagination (lemmy_diesel_utils)
│   ├── email/                   # Email sending via SMTP, i18n templates (lemmy_email)
│   ├── routes/                  # Non-API HTTP routes (RSS, images, nodeinfo, webfinger, metrics)
│   ├── server/                  # Binary entry point (lemmy_server)
│   └── utils/                   # Shared utilities (error types, rate limiting, settings/config, caching)
├── migrations/                  # ~340 Diesel SQL migration directories
├── config/                      # HJSON config files (defaults.hjson, config.hjson)
├── docker/                      # Docker deployment files (Dockerfile, compose, nginx, postgres config)
├── api_tests/                   # TypeScript/Jest integration tests
├── scripts/                     # Shell scripts for dev, testing, deployment
├── diesel.toml                  # Diesel ORM configuration (schema output path)
├── rust-toolchain.toml          # Rust toolchain pinned to 1.95
├── Cargo.toml                   # Workspace root with 39 crate members
├── .woodpecker.yml              # CI configuration (Woodpecker CI)
├── .github/                     # GitHub issue templates, SECURITY.md, FUNDING.yml
└── ARCHITECTURE.md              # This document

2. High-Level System Diagram

[Browser/App] <--REST/JSON--> [lemmy_server (actix-web)]
                                     |
                          +----------+----------+
                          |                     |
                     [PostgreSQL]        [pictrs (images)]
                          |
                     [ActivityPub] <--> [Other Fediverse Instances]
                          |
                     [SMTP Email]

The server exposes:

• REST API (/api/v4, /api/v3) for clients (web UI, mobile apps, third-party)
• ActivityPub endpoints for federation with other instances
• Non-API HTTP routes for RSS/Atom feeds, image proxying, nodeinfo, webfinger, and Prometheus metrics

3. Core Components

3.1. HTTP Server & Routing

Name: lemmy_server + lemmy_routes

Description: Binary entry point (crates/server/src/main.rs) that initializes actix-web HttpServer with full middleware stack (compression, CORS, tracing, session/JWT extraction, idempotency, federation signatures, rate limiting). lemmy_routes provides non-API endpoints: RSS/Atom feeds, image proxy, nodeinfo, webfinger, Prometheus metrics.

Technologies: Rust (actix-web 4.13, tokio 1.50, rustls)

Deployment: Docker (multi-stage build) or native binary. Published to dessalines/lemmy on Docker Hub.

3.2. API Handlers

Name: lemmy_api (core handlers) + lemmy_api_crud (CRUD) + lemmy_api_common (shared types) + lemmy_api_utils (utilities)

Description: Two API versions (v4 current, v3 legacy). Handlers organized by entity (post, comment, community, person, site, admin, account, reports). lemmy_api_utils provides JWT auth, authorization checks, rate limiting configuration, ActivityPub activity sending channel, WASM plugin execution via Extism, and notification dispatching.

Technologies: Rust, actix-web, serde, jsonwebtoken, extism (WASM plugins)

3.3. Database Views Layer

Name: lemmy_db_views_* (24 crate workspace)

Description: Read-model layer that implements all query logic (joins, filtering, sorting, pagination, visibility rules). Each view corresponds to a primary entity (PostView, CommentView, CommunityView, PersonView, SiteView, etc.). Implements content visibility rules: private community filtering, unlisted/hidden community filtering, blocked user/community/instance filtering, and individual post hiding.

Technologies: Rust, diesel 2.3.7, diesel-async 0.8, PostgreSQL

3.4. Database Schema Layer

Name: lemmy_db_schema + lemmy_db_schema_file

Description: Foundation layer defining all database models (41 source files in source/), type-safe newtype IDs (PostId, CommentId, PersonId, etc.), Diesel insert/update forms, enums via diesel-derive-enum, and reusable traits (Crud, Blockable, Followable, Likeable, Bannable). lemmy_db_schema_file contains the auto-generated schema.rs from diesel print-schema, plus enums.rs and joins.rs.

Technologies: Rust, diesel 2.3.7, diesel-derive-enum, diesel-derive-newtype, diesel_ltree

3.5. ActivityPub Federation

Name: lemmy_apub + lemmy_apub_activities + lemmy_apub_objects + lemmy_apub_send

Description: Complete ActivityPub protocol implementation using activitypub_federation crate. Handles incoming activities via inbox/shared_inbox endpoints, outgoing activities via SendManager with retry logic and exponential backoff. Supports horizontal scaling of federation work across multiple processes. Protocol types: Create, Update, Delete, Follow, Like, Dislike, Block, Announce, Undo, Accept, Reject, Report, Flag.

Technologies: Rust, activitypub_federation 0.7.0-beta.11, reqwest 0.13, serde

3.6. Shared Infrastructure

Name: lemmy_utils + lemmy_diesel_utils + lemmy_email

Description: Utilities layer providing error types (~350 variants in LemmyErrorType with i18n support), rate limiting configuration, global settings from HJSON files (SETTINGS lazy static), database connection pool (deadpool), migration runner, pagination utilities, and email sending via SMTP (lettre).

Technologies: Rust, deadpool, lettre, clokwerk (scheduled tasks), moka (caching), prometheus (metrics)

4. Data Stores

4.1. Primary Database

Name: Primary Database

Type: PostgreSQL 18

Purpose: Stores all application data: users, communities, posts, comments, votes, federated objects, site configuration, etc.

Key Schemas/Collections: person, local_user, community, post, comment, community_actions, post_actions, comment_actions, person_actions, instance_actions, site, local_site, mod_log, registration_application, private_message, secret, email_verification, password_reset_request, custom_emoji, tag, multi_community, and more (~90+ tables)

4.2. Image Store

Name: pictrs

Type: Image hosting service (external, bundled via docker-compose)

Purpose: Stores and serves uploaded images. Provides image proxying and caching.

4.3. Cache/Infrastructure

Name: In-memory cache + Scheduled tasks

Type: moka (Rust caching crate) + clokwerk (scheduled task runner)

Purpose: Caching for frequently accessed data (settings, rate limits). Scheduled tasks for cleanup, federation retry, and periodic maintenance.

5. External Integrations / APIs

5.1. pictrs (Image Hosting)

Purpose: Image upload, storage, proxying, and thumbnail generation.

Integration Method: HTTP REST API (configurable URL + API key)

5.2. ActivityPub (Fediverse Federation)

Purpose: Inter-instance communication with other Lemmy instances and compatible platforms (Mastodon, etc.). Enables federated communities, shared content, and user interactions across instances.

Integration Method: ActivityPub protocol over HTTPS with HTTP Signatures (mandatory for production federation)

5.3. SMTP Email

Purpose: Sends verification emails, password resets, and notification digests.

Integration Method: SMTP via lettre crate, configurable connection URI

5.4. WebAssembly Plugins

Purpose: Custom server-side logic (content moderation, custom feeds, etc.) via WASM modules.

Integration Method: Extism runtime, plugin configuration in settings.hjson

6. Deployment & Infrastructure

Cloud Provider: Self-hosted (Docker Compose) or any Docker-capable infrastructure

Key Services Used: PostgreSQL, pictris (image server), nginx (reverse proxy), optionally Prometheus/Grafana for monitoring

CI/CD Pipeline: Woodpecker CI (.woodpecker.yml) with 19 pipeline steps including linting (rustfmt, clippy, prettier, typos, toml, sql), building, testing (Rust workspace + TypeScript API tests), schema consistency checks, and Docker image publishing (arm64 + amd64).

Docker Images: Published to dessalines/lemmy on Docker Hub

7. Security Considerations

Authentication: JWT (JSON Web Tokens) via jsonwebtoken crate. Tokens validated by signature + database lookup (supports revocation). Optional “stay logged in” tokens. TOTP/2FA support via totp-rs.

Authorization: Permission checks per action in API handlers (admin checks, community moderator checks, content ownership checks). Community visibility rules enforced at DB query level (private, unlisted, public).

Password Security: bcrypt hashing, minimum 10 characters, maximum 60 characters.

API Security: Per-endpoint rate limiting (message, register, post, comment, search, image, import), CORS configuration, idempotency middleware for safe retries, URL validation against blocklist/allowlist, internal IP validation (SSRF prevention), proxy detection.

Federation Security: ActivityPub HTTP Signatures mandatory, domain blocklist/allowlist, signed fetch capability.

Data Encryption: TLS in transit (rustls), TLS for SMTP.

Key Security Practices: unwrap_used and expect_used denied at compile time (panics prevented), strict Clippy lints.

8. Development & Testing Environment

Local Setup: Requires PostgreSQL 18, Rust 1.95. Copy config/config.hjson.example to config/config.hjson and adjust. Run cargo run to start. Database migrations run automatically on startup.

Testing Frameworks:

• Rust: #[tokio::test] with #[serial_test::serial] for DB isolation. Tests use a dedicated test DB pool.
• API Integration: TypeScript with Jest + ts-jest in api_tests/
• Assertions: pretty_assertions for Rust tests

CI Test Commands: cargo test --workspace (run twice on failure for flaky tests). Requires LEMMY_DATABASE_URL and LEMMY_CONFIG_LOCATION env vars.

Code Quality Tools: cargo fmt (nightly), cargo clippy (-D warnings), cargo shear (unused deps), typos-cli, shfmt, taplo (TOML), pg_format (SQL), prettier (markdown/yaml).

9. Future Considerations / Roadmap

• Horizontal scaling: Federation processing already supports splitting across processes. Further scaling improvements possible.
• WASM plugin ecosystem: Extism-based plugin system for custom server logic.
• Ongoing API v3 deprecation and v4 stabilization.
• Database migration to newer PostgreSQL versions.
• Potential migration from diesel to a more async-native ORM or SQLx as the ecosystem evolves.

10. Project Identification

Project Name: Lemmy (link-aggregator / federated Reddit alternative)

Repository URL: https://github.com/LemmyNet/lemmy

Primary Contact/Team: LemmyNet (Dessalines, Nutomic, and contributors)

Date of Last Update: 2026-07-14

11. Glossary / Acronyms

ActivityPub: W3C decentralized social networking protocol standard used for federation between instances.

APub: ActivityPub (abbreviated).

CRUD: Create, Read, Update, Delete.

CORS: Cross-Origin Resource Sharing.

HJSON: Human JSON, a configuration format that allows comments and relaxed syntax.

JWT: JSON Web Token, used for API authentication.

pictrs: Image hosting service used by Lemmy for storing uploaded images and avatars.

TOTP: Time-based One-Time Password, used for two-factor authentication.

WASM: WebAssembly, used for server-side plugin execution via Extism.

Instance: A single Lemmy server installation (each instance has its own domain, users, and communities).

Federation: The exchange of content between instances using the ActivityPub protocol.

Local User: A user registered on this instance (as opposed to a federated/remote user).

Community: A subreddit-like topic group within an instance. Can be federated across instances.

ListingType: Enum controlling feed scope (Subscribed, All, Local, ModeratorView, Suggested).

CommunityVisibility: Enum controlling community discoverability (Public, Unlisted, LocalOnlyPublic, LocalOnlyPrivate, Private).