Pull requests

• docs: add Simplified Chinese translation (README.zh.md)

  #317·JasonYeYuhe opened Oct 17, 2026·

  ·
  1

• Score empty documents as doc_not_en=1.0 in language doc score

  #316·shaurya416 opened Sep 25, 2026·

  ·
  1

• Use HTTPS URL for code_composition model and re-enable its tests

  #314·SammyTourani opened Sep 23, 2026·

  ·
  1

• Add some materialization tooling for mixtures

  #313·undfined opened Aug 24, 2026Contributor·

  ·

• Improve throughput with gigatoken and batching

  #312·jopetty opened Jul 31, 2026·

  ·
  3

• docs: clarify temporal deduplication strategies and document types (Issue #267)

  #276·ada-ggf25 opened Nov 27, 2025·

  ·
  1

• Script to produce dolma 2 ablation config

  #275·soldni opened Sep 24, 2025Contributor·

  ·

• Improve WARC processing

  #260·soldni opened Apr 15, 2025Contributor·

  ·

• first

  #240·Whattabatt opened Feb 14, 2025Contributor·

  ·
  6

• [WIP DO NOT MERGE] Learn2Code Feature Branch

  #233·cmwilhelm opened Feb 13, 2025Contributor·

  ·

• simpler logic for calculating code taggers

  #229·kyleclo opened Feb 12, 2025Contributor·

  ·

• Bump openssl from 0.10.66 to 0.10.70 in the cargo group

  dependenciesrust
  #228·dependabot[bot] opened Feb 3, 2025Bot·

  ·

• Fixed ignore_existing flag not working as expected.

  #224·soldni opened Jan 1, 2025Contributor·

  ·
  3

• New language ID

  #223·soldni opened Dec 30, 2024Contributor·

  ·

• Adding support for Classifiers and Search tools

  #219·soldni opened Oct 24, 2024Contributor·

  ·

• DCLM Style Deduplications

  #214·revbucket opened Sep 30, 2024·

  ·
  7

• Mattj/requirements

  #212·revbucket opened Sep 26, 2024·

  ·

• DNM: Patch FT Tagger

  #210·undfined opened Sep 25, 2024Contributor·

  ·

• Allow specifying different bins for visualization and computation.

  #190·soldni opened Aug 27, 2024Contributor·

  ·

• New Progress Bar, Backoff, Batching

  #165·soldni opened May 23, 2024Contributor·

  ·
  14

• Warc Backoff

  #160·soldni opened May 10, 2024Contributor·

  ·

• Text modification config

  #60·rodneykinney opened Oct 19, 2023Member·

  ·
  1