feat: Add PostgreSQL database backend support (#439) · basicmachines-co/basic-memory@fb5e9e1 · GitHub
Skip to content

Commit fb5e9e1

Browse files
phernandezclaude
phernandez
and
claude
authored
feat: Add PostgreSQL database backend support (#439)
Signed-off-by: phernandez <paul@basicmachines.co> Co-authored-by: Claude <noreply@anthropic.com>
1 parent 66b91b2 commit fb5e9e1

71 files changed

Lines changed: 3038 additions & 9069 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/commands/spec.md‎

Lines changed: 17 additions & 21 deletions

‎.env.example‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Basic Memory Environment Variables Example
2+
# Copy this file to .env and customize as needed
3+
# Note: .env files are gitignored and should never be committed
4+
5+
# ============================================================================
6+
# PostgreSQL Test Database Configuration
7+
# ============================================================================
8+
# These variables allow you to override the default test database credentials
9+
# Default values match docker-compose-postgres.yml for local development
10+
#
11+
# Only needed if you want to use different credentials or a remote test database
12+
# By default, tests use: postgresql://basic_memory_user:dev_password@localhost:5433/basic_memory_test
13+
14+
# Full PostgreSQL test database URL (used by tests and migrations)
15+
# POSTGRES_TEST_URL=postgresql+asyncpg://basic_memory_user:dev_password@localhost:5433/basic_memory_test
16+
17+
# Individual components (used by justfile postgres-reset command)
18+
# POSTGRES_USER=basic_memory_user
19+
# POSTGRES_TEST_DB=basic_memory_test
20+
21+
# ============================================================================
22+
# Production Database Configuration
23+
# ============================================================================
24+
# For production use, set these in your deployment environment
25+
# DO NOT use the test credentials above in production!
26+
27+
# BASIC_MEMORY_DATABASE_BACKEND=postgres # or "sqlite"
28+
# BASIC_MEMORY_DATABASE_URL=postgresql+asyncpg://user:password@host:port/database

‎.github/workflows/test.yml‎

Lines changed: 56 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -13,30 +13,15 @@ on:
1313
branches: [ "main" ]
1414

1515
jobs:
16-
test:
16+
test-sqlite:
17+
name: Test SQLite (${{ matrix.os }}, Python ${{ matrix.python-version }})
1718
strategy:
1819
fail-fast: false
1920
matrix:
2021
os: [ubuntu-latest, windows-latest]
2122
python-version: [ "3.12", "3.13" ]
2223
runs-on: ${{ matrix.os }}
2324

24-
# Postgres service (only available on Linux runners)
25-
services:
26-
postgres:
27-
image: postgres:17
28-
env:
29-
POSTGRES_DB: basic_memory_test
30-
POSTGRES_USER: basic_memory_user
31-
POSTGRES_PASSWORD: dev_password
32-
options: >-
33-
--health-cmd pg_isready
34-
--health-interval 10s
35-
--health-timeout 5s
36-
--health-retries 5
37-
ports:
38-
- 5433:5432
39-
4025
steps:
4126
- uses: actions/checkout@v4
4227
with:
@@ -83,10 +68,60 @@ jobs:
8368
- name: Run tests (SQLite)
8469
run: |
8570
uv pip install pytest pytest-cov
86-
just test
71+
just test-sqlite
72+
73+
test-postgres:
74+
name: Test Postgres (Python ${{ matrix.python-version }})
75+
strategy:
76+
fail-fast: false
77+
matrix:
78+
python-version: [ "3.12", "3.13" ]
79+
runs-on: ubuntu-latest
80+
81+
# Postgres service (only available on Linux runners)
82+
services:
83+
postgres:
84+
image: postgres:17
85+
env:
86+
POSTGRES_DB: basic_memory_test
87+
POSTGRES_USER: basic_memory_user
88+
POSTGRES_PASSWORD: dev_password
89+
options: >-
90+
--health-cmd pg_isready
91+
--health-interval 10s
92+
--health-timeout 5s
93+
--health-retries 5
94+
ports:
95+
- 5433:5432
96+
97+
steps:
98+
- uses: actions/checkout@v4
99+
with:
100+
submodules: true
101+
102+
- name: Set up Python ${{ matrix.python-version }}
103+
uses: actions/setup-python@v4
104+
with:
105+
python-version: ${{ matrix.python-version }}
106+
cache: 'pip'
107+
108+
- name: Install uv
109+
run: |
110+
pip install uv
111+
112+
- name: Install just
113+
run: |
114+
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin
115+
116+
- name: Create virtual env
117+
run: |
118+
uv venv
119+
120+
- name: Install dependencies
121+
run: |
122+
uv pip install -e .[dev]
87123
88124
- name: Run tests (Postgres)
89-
# Only run on Linux since Postgres service is only available there
90-
if: runner.os == 'Linux'
91125
run: |
92-
just test-postgres
126+
uv pip install pytest pytest-cov
127+
just test-postgres

‎README.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -433,6 +433,57 @@ See the [Documentation](https://memory.basicmachines.co/) for more info, includi
433433
- [Managing multiple Projects](https://docs.basicmemory.com/guides/cli-reference/#project)
434434
- [Importing data from OpenAI/Claude Projects](https://docs.basicmemory.com/guides/cli-reference/#import)
435435

436+
## Development
437+
438+
### Running Tests
439+
440+
Basic Memory supports dual database backends (SQLite and Postgres). Tests are parametrized to run against both backends automatically.
441+
442+
**Quick Start:**
443+
```bash
444+
# Run SQLite tests (default, no Docker needed)
445+
just test-sqlite
446+
447+
# Run Postgres tests (requires Docker)
448+
just test-postgres
449+
```
450+
451+
**Available Test Commands:**
452+
453+
- `just test-sqlite` - Run tests against SQLite only (fastest, no Docker needed)
454+
- `just test-postgres` - Run tests against Postgres only (requires Docker)
455+
- `just test-windows` - Run Windows-specific tests (auto-skips on other platforms)
456+
- `just test-benchmark` - Run performance benchmark tests
457+
- `just test-all` - Run all tests including Windows, Postgres, and benchmarks
458+
459+
**Postgres Testing Requirements:**
460+
461+
To run Postgres tests, you need to start the test database:
462+
```bash
463+
docker-compose -f docker-compose-postgres.yml up -d
464+
```
465+
466+
Tests will connect to `localhost:5433/basic_memory_test`.
467+
468+
**Test Markers:**
469+
470+
Tests use pytest markers for selective execution:
471+
- `postgres` - Tests that run against Postgres backend
472+
- `windows` - Windows-specific database optimizations
473+
- `benchmark` - Performance tests (excluded from default runs)
474+
475+
**Other Development Commands:**
476+
```bash
477+
just install # Install with dev dependencies
478+
just lint # Run linting checks
479+
just typecheck # Run type checking
480+
just format # Format code with ruff
481+
just check # Run all quality checks
482+
just migration "msg" # Create database migration
483+
```
484+
485+
See the [justfile](justfile) for the complete list of development commands.
486+
436487
## License
437488

438489
AGPL-3.0

‎docker-compose-postgres.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Docker Compose configuration for Basic Memory with PostgreSQL
2+
# Use this for local development and testing with Postgres backend
3+
#
4+
# Usage:
5+
# docker-compose -f docker-compose-postgres.yml up -d
6+
# docker-compose -f docker-compose-postgres.yml down
7+
8+
services:
9+
postgres:
10+
image: postgres:17
11+
container_name: basic-memory-postgres
12+
environment:
13+
# Local development/test credentials - NOT for production
14+
# These values are referenced by tests and justfile commands
15+
POSTGRES_DB: basic_memory
16+
POSTGRES_USER: basic_memory_user
17+
POSTGRES_PASSWORD: dev_password # Simple password for local testing only
18+
ports:
19+
- "5433:5432"
20+
volumes:
21+
- postgres_data:/var/lib/postgresql/data
22+
healthcheck:
23+
test: ["CMD-SHELL", "pg_isready -U basic_memory_user -d basic_memory"]
24+
interval: 10s
25+
timeout: 5s
26+
retries: 5
27+
restart: unless-stopped
28+
29+
volumes:
30+
# Named volume for Postgres data
31+
postgres_data:
32+
driver: local
33+
34+
# Named volume for persistent configuration
35+
# Database will be stored in Postgres, not in this volume
36+
basic-memory-config:
37+
driver: local
38+
39+
# Network configuration (optional)
40+
# networks:
41+
# basic-memory-net:
42+
# driver: bridge

‎justfile‎

Lines changed: 67 additions & 5 deletions

0 commit comments

Comments
 (0)