Sunbelt Computer Software

PL/B Language Development and Support

rasbt/LLMs-from-scratch · Discussions · GitHub

pos_embedding_layer question
Q&A nicolaleo

Discussions

You must be logged in to vote

Potential bug in ch02/05_bpe-from-scratch/bpe-from-scratch-simple.ipynb

h1yuol started Feb 21, 2026 in General

1
You must be logged in to vote

Question about mask in CausalAttention chapter 3

lehoanglong95 started Apr 19, 2026 in General · Closed

1
You must be logged in to vote

Chapter 2: Position Embeddings

Stampede asked Apr 11, 2026 in Q&A · Unanswered

1
You must be logged in to vote

Whyd I buy this book?

JonHennesseyDFA started Apr 16, 2026 in General · Closed

0
You must be logged in to vote

Tokenizers

DerpLady started Apr 6, 2026 in General

2
You must be logged in to vote

Please help: "IndexError: index out of range in self..." while LLM training with a trained BPE tokenizer,

tmrogo asked Apr 4, 2026 in Q&A · Answered

3
You must be logged in to vote

A latent mistake about the semantic representation of the message within predicted label computation in LLM classification fine-tuning
question Further information is requested
Jack1447 asked Mar 30, 2026 in Q&A · Unanswered

1
You must be logged in to vote

LoRA for Classification

UgurKap asked Dec 5, 2025 in Q&A · Unanswered

2
You must be logged in to vote

Request: Add a git tag for the original book version for better learning

paomian started Mar 7, 2026 in General · Closed

3
You must be logged in to vote

[ch04.ipynb] Question about an inconsistency between text and diagram

jialechan started Mar 4, 2026 in General

1
You must be logged in to vote

Suggestion: supplemental troubleshooting resource for downstream LLM and RAG failure patterns

onestardao started Mar 3, 2026 in Show and tell · Closed

1
You must be logged in to vote

New open weight model: Tiny Aya

rasbt announced Feb 19, 2026 in Announcements

0
You must be logged in to vote

Fine tuning pre-trained LLM for language translation and to build ChatGPT like application

krishnadasar-sudheer-kumar asked Aug 24, 2024 in Q&A · Answered

2
You must be logged in to vote

MultiHeadAttentionWrapper should instantiate CausalSelfAttention with d_out = d_out // num_heads?

henrythe9th asked Apr 9, 2025 in Q&A · Unanswered

5
You must be logged in to vote

Intuition about project input embedding tokens to queries, keys and values

krishnan-duraisamy started Nov 28, 2025 in General

5
You must be logged in to vote

Dot product example on p 58 fig 3.8

richlevinson-design-2 started Jan 14, 2026 in General

5
You must be logged in to vote

Conceptual Question: Why is a large pre-training dataset (10GB+) necessary if I only want the model to specialize in a small domain (100MB)?

tllmmaster started Dec 29, 2025 in General

1
You must be logged in to vote

Process "Killed" (OOM) due to RAM limits with custom dataset – How to implement lazy loading?

tllmmaster started Dec 27, 2025 in General

1
You must be logged in to vote

Feasibility of training a Turkmen LLM from scratch on 100MB data

tllmmaster asked Dec 24, 2025 in Q&A · Answered

2
You must be logged in to vote

No “emergent behavior / aha moment” when retraining GPT-2 on FineWeb; warmup / “warm training” guidance requested

talentJay-ux started Oct 20, 2025 in General

12
You must be logged in to vote

Convering PyTorch (.pth) for use with Ollama

b0bh00d started Dec 20, 2025 in General

0
You must be logged in to vote

Inference-time techniques

talentJay-ux started Dec 20, 2025 in Ideas

1
You must be logged in to vote

Qwen3-Next: Computation of Gated Delta Rule

d-kleine asked Nov 4, 2025 in Q&A · Closed · Answered

30
You must be logged in to vote

Is there a Chinese version?
question Further information is requested
ac6855 asked Dec 9, 2025 in Q&A · Unanswered

3
You must be logged in to vote

Class prediction in chapter 6 depends on the number of paddings added

itdxer started Mar 13, 2025 in Ideas

7