Architecture
Architecture
System Overview
The Leaderboard system follows a build-time data aggregation pattern with static site generation.
Components
1. Plugin Runner (@leaderboard/plugin-runner)
Purpose: Orchestrate data collection and management
Responsibilities:
- Load configuration from
config.yaml - Fetch and validate plugins from URLs
- Import existing data from data-repo
- Execute plugin setup, scrape, and aggregate methods
- Run standard aggregation and badge evaluation
- Export updated data back to data-repo
CLI Commands:
plugin-runner --data-dir=/path/to/data2. API Layer (@ohcnetwork/leaderboard-api)
Purpose: Unified database utilities and plugin type definitions
Technology: LibSQL (SQLite-compatible)
Schema:
contributor- User profiles and metadataactivity_definition- Activity types defined by pluginsactivity- Individual activity records
Storage:
- Default location:
${DATA_DIR}/.leaderboard.db - Persists
activity_definitiontable across runs - Temporary storage for
contributorandactivitytables
Provides:
- Database abstraction layer
- Query builders and helpers
- TypeScript type definitions
- Plugin interface specification
- Context object structure
Example Plugin:
import type { Plugin } from "@ohcnetwork/leaderboard-api";
export default {
name: "my-plugin",
version: "1.0.0",
async setup(ctx) {
// Define activity types
},
async scrape(ctx) {
// Fetch and store activities
},
async aggregate(ctx) {
// Optional: Compute plugin-specific aggregates
},
} satisfies Plugin;4. Next.js Application (apps/leaderboard-web)
Purpose: Generate static website
Features:
- Server-Side Generation (SSG) at build time
- Reads data directly from LibSQL database
- Markdown documentation via Fumadocs
- Customizable themes via CSS
Pages:
/- Home page with overview/leaderboard- Rankings and leaderboards/people- All contributors/[username]- Individual contributor profiles/badges- Badge definitions and achievements/data- Data Explorer (browser-based SQL REPL)/docs- Documentation
Data Flow
Build Process
-
Import Phase
data-repo/contributors/*.md → LibSQL data-repo/activities/*.jsonl → LibSQL -
Setup Phase
For each plugin: Execute plugin.setup(ctx) → Populate activity_definition table -
Scrape Phase
For each plugin: Execute plugin.scrape(ctx) → Fetch data from API → Insert activities into database -
Aggregation Phase
Calculate standard global aggregates Calculate standard contributor aggregates -
Plugin Aggregation Phase
For each plugin (with aggregate method): Execute plugin.aggregate(ctx) → Compute plugin-specific aggregates -
Export Phase
LibSQL → data-repo/contributors/*.md LibSQL → data-repo/activities/*.jsonl -
Build Phase
Next.js reads LibSQL → Generate static pages → Copy .leaderboard.db to public/data.db → Output static site
Client-Side Data Explorer
In addition to the pre-rendered static pages, the database file is shipped as a static asset (public/data.db). The Data Explorer page loads this database in the browser via sql.js-httpvfs (SQLite compiled to WebAssembly). A Web Worker makes HTTP range requests to fetch only the database pages needed for each query, enabling ad-hoc SQL queries without any backend.
Data Storage Strategies
Contributors (Markdown Files)
Format: Markdown with YAML frontmatter
Rationale:
- Human-editable profiles
- Supports rich bio content
- Version control friendly
Location: data-repo/contributors/<username>.md
Activities (Sharded JSONL)
Format: JSON Lines, one file per contributor
Rationale:
- Efficient for large datasets
- Easy per-user updates
- Fast import/export
Location: data-repo/activities/<username>.jsonl
Activity Definitions (Database Only)
Format: SQLite table
Rationale:
- Managed by plugins
- No manual editing needed
- Avoids sync issues
Location: data-repo/.leaderboard.db
Deployment Architecture
Steps:
- CI/CD clones data repository
- Runs plugin-runner to update data
- Builds Next.js static site
- Deploys to CDN (Netlify, Vercel, etc.)
- Users access static site from CDN
12-Factor App Compliance
The system follows 12-Factor App principles:
- Codebase: Single repo, multiple deployments
- Dependencies: Explicitly declared in
package.json - Config: Environment variables and
config.yaml - Backing Services: LibSQL as attachable resource
- Build/Run/Release: Clear separation of phases
- Processes: Stateless static site
- Port Binding: Not applicable (static export)
- Concurrency: Plugin execution parallelizable
- Disposability: Fast startup, clean shutdown
- Dev/Prod Parity: Same build process everywhere
- Logs: Structured logging in plugin-runner
- Admin Processes: Plugin-runner as separate process
Security Considerations
Plugin Execution
- Plugins fetched from configurable URLs
- Basic validation of plugin structure
- Consider using signed plugins in production
Data Privacy
- All data stored in your infrastructure
- No external data transmission (except plugin API calls)
- Contributor data fully under your control
Static Site
- No server-side code execution
- No authentication required
- Can be deployed behind authentication if needed