Note Format and Knowledge Representation

Relevant source files

• NOTE-FORMAT.md
• docs/NOTE-FORMAT.md
• docs/specs/SPEC-SCHEMA-IMPL.md
• docs/specs/SPEC-SCHEMA.md
• skills/memory-onboarding/references/schema-guide.md
• skills/memory-schema/SKILL.md
• src/basic_memory/markdown/__init__.py
• src/basic_memory/markdown/entity_parser.py
• src/basic_memory/markdown/markdown_processor.py
• src/basic_memory/markdown/plugins.py
• src/basic_memory/markdown/schemas.py
• src/basic_memory/markdown/utils.py
• src/basic_memory/picoschema/parser.py
• src/basic_memory/picoschema/resolver.py
• src/basic_memory/picoschema/validator.py
• test-int/fixtures/schema/schemas/InvalidOffSchema.md
• test-int/test_picoschema/test_parser_integration.py
• tests/markdown/test_entity_parser.py
• tests/markdown/test_entity_parser_error_handling.py
• tests/markdown/test_markdown_plugins.py
• tests/markdown/test_markdown_processor.py
• tests/markdown/test_observation_edge_cases.py
• tests/markdown/test_parser_edge_cases.py
• tests/markdown/test_relation_edge_cases.py
• tests/picoschema/test_parser.py
• tests/picoschema/test_resolver.py
• tests/picoschema/test_validator.py

Basic Memory uses a structured Markdown format to represent knowledge. This format allows the system to treat plain text files as structured data objects (Entities) while remaining human-readable and compatible with standard Markdown editors like Obsidian or VS Code.

Overview

The core of Basic Memory's knowledge representation is the Entity. An Entity is a single Markdown file that combines metadata, unstructured prose, and structured semantic units called Observations and Relations. The system treats files as the source of truth, where changes to files automatically update the knowledge graph in the database docs/NOTE-FORMAT.md3

Natural Language to Code Entity Space

The following diagram illustrates how Markdown syntax elements are mapped to Python classes and database models during the parsing process.

Knowledge Mapping: Markdown to Code Entities

Sources: src/basic_memory/markdown/entity_parser.py156-180 src/basic_memory/markdown/schemas.py9-95 src/basic_memory/markdown/utils.py17-81

---

YAML Frontmatter

Every note starts with an optional YAML frontmatter block delimited by ---. This block stores metadata that the EntityParser uses to identify and categorize the note.

Key Fields

• title: The display name of the entity. If missing, it defaults to the filename stem docs/NOTE-FORMAT.md38 tests/markdown/test_entity_parser_edge_cases.py161-179
• type: The category of the note (e.g., person, project, component). Defaults to note src/basic_memory/markdown/schemas.py75-79
• permalink: A unique identifier for the note that persists even if the file moves docs/NOTE-FORMAT.md41
• tags: A list or comma-separated string of tags src/basic_memory/markdown/schemas.py66-68
• created/modified: ISO-8601 timestamps. The parser uses dateparser to handle various human-friendly formats like "yesterday" or "2 days ago" src/basic_memory/markdown/entity_parser.py196-212

Normalization

The EntityParser includes a normalize_frontmatter_value function to ensure YAML-parsed types (like datetime.date, bool, or int) are converted to safe strings or ISO format to prevent AttributeError on string operations src/basic_memory/markdown/entity_parser.py30-88 This normalization is applied to all frontmatter metadata via normalize_frontmatter_metadata src/basic_memory/markdown/entity_parser.py107-119

Sources: src/basic_memory/markdown/entity_parser.py30-88 src/basic_memory/markdown/schemas.py40-83 src/basic_memory/markdown/entity_parser.py107-119 docs/NOTE-FORMAT.md32-45

---

Structured Semantic Units

Basic Memory uses specific list-item syntaxes to extract granular data from the Markdown body. These are handled by markdown-it plugins defined in src/basic_memory/markdown/plugins.py.

Observations

Observations capture specific facts or attributes.

• Syntax: - [category] content #tags (context)
• Implementation: The observation_plugin identifies these patterns. It explicitly excludes Markdown tasks (- [ ], - [x], - [-]) to avoid false positives src/basic_memory/markdown/plugins.py52-53
• Transcript Safety: Transcript timecodes (e.g., [00:00:11]) are explicitly rejected as categories to prevent cluttering the graph with time-series markers src/basic_memory/markdown/plugins.py14 src/basic_memory/markdown/plugins.py36-37
• Exclusions: The parser also skips standard Markdown links [text](url) and standalone wikilinks [[text]] from being treated as observations src/basic_memory/markdown/plugins.py55-61
• Tag Extraction: Tags starting with # are extracted. The parser handles concatenated tags like #tag1#tag2 by splitting them into individual elements src/basic_memory/markdown/plugins.py99-107

Relations

Relations define typed edges between entities in the knowledge graph.

• Explicit Syntax: - relation_type [[Target Entity]] (context)
• Implicit Syntax (WikiLinks): Any [[Target]] found in the prose is treated as a links_to relation src/basic_memory/markdown/plugins.py182-202
• Normalization: Target names are normalized using normalize_project_reference to handle cross-project permalinks src/basic_memory/markdown/plugins.py169
• Quoting: Relation labels containing spaces must be quoted (e.g., -"some type" [[Target]]) to be recognized as explicit relations src/basic_memory/markdown/plugins.py130-133

Data Flow: Parsing Observations and Relations

Sources: src/basic_memory/markdown/entity_parser.py156-180 src/basic_memory/markdown/plugins.py42-68 src/basic_memory/markdown/plugins.py140-147 docs/NOTE-FORMAT.md91-186

---

Technical Implementation Details

The MarkdownProcessor

The MarkdownProcessor handles the high-level Read-Modify-Write cycle. It uses EntityParser for reading and frontmatter.Post for writing src/basic_memory/markdown/markdown_processor.py24-40

Feature	Description	File Reference
Atomic Writes	Uses temporary files via write_file_atomic to prevent data corruption.	src/basic_memory/markdown/markdown_processor.py133
Checksums	Verifies file integrity before writing to prevent overwriting concurrent changes.	src/basic_memory/markdown/markdown_processor.py94-98
Formatting	Consistently appends observations and relations at the end of the file.	src/basic_memory/markdown/markdown_processor.py190-204
Null Stripping	Strips null bytes (0x00) to prevent database rejection in PostgreSQL.	tests/markdown/test_parser_edge_cases.py181-200

Picoschema-based Validation

Basic Memory supports note structure validation through "Picoschemas". The schema_to_markdown utility assists in converting these schema models back into Markdown Post objects, merging existing content frontmatter with entity metadata while ensuring fields like type, title, and permalink are handled correctly src/basic_memory/markdown/utils.py84-125

memory:// URLs

Links within notes can use memory:// URLs for unambiguous cross-project references. These URLs are used by the LinkResolver and ContextService to traverse the graph across project boundaries.

URL Resolution: memory:// to Knowledge Graph

Sources: src/basic_memory/markdown/plugins.py6 src/basic_memory/markdown/utils.py48-49 src/basic_memory/markdown/entity_parser.py192-194