Commit history

1. Commits on Jun 16, 2026

   • Avoid sizing whole distant split sections

     Show description for da32637
     benbrandt

     committed
     da32637View commit details

     Copy full SHA for da32637

     View code at this pointBrowse repository at this point
2. Commits on Mar 26, 2025

   • style: Replace allow attributes with expect attributes

     Show description for 824d2d5
     benbrandt

     committed
     824d2d5View commit details

     Copy full SHA for 824d2d5

     View code at this pointBrowse repository at this point
3. Commits on Mar 25, 2025

   • feat: Replace regex with memchr for linebreak detection

     Show description for a0d1da1
     benbrandt

     committed
     a0d1da1View commit details

     Copy full SHA for a0d1da1

     View code at this pointBrowse repository at this point
4. Commits on Mar 21, 2025

   • feat: use encode_fast

     benbrandt

     committed
     0505991View commit details

     Copy full SHA for 0505991

     View code at this pointBrowse repository at this point

   • Update benchmark output

     benbrandt

     committed
     afe6a52View commit details

     Copy full SHA for afe6a52

     View code at this pointBrowse repository at this point

   • remove rust-tokenizers support

     benbrandt

     committed
     7e42509View commit details

     Copy full SHA for 7e42509

     View code at this pointBrowse repository at this point
5. Commits on Jan 16, 2025

   • Update msrv to 1.80 and remove once_cell dependency

     benbrandt

     committed
     225cdbaView commit details

     Copy full SHA for 225cdba

     View code at this pointBrowse repository at this point
6. Commits on Dec 14, 2024

   • feat!: replace unicode-segmentation with icu_segmenter

     benbrandt

     committed
     f737e16View commit details

     Copy full SHA for f737e16

     View code at this pointBrowse repository at this point

   • Update benchmark output

     benbrandt

     committed
     d790f2bView commit details

     Copy full SHA for d790f2b

     View code at this pointBrowse repository at this point
7. Commits on Sep 2, 2024

   • Simplify codesplitter api

     benbrandt

     committed
     862e47aView commit details

     Copy full SHA for 862e47a

     View code at this pointBrowse repository at this point

   • Updates for pulldown-cmark and tree-sitter

     benbrandt

     committed
     f5c370aView commit details

     Copy full SHA for f5c370a

     View code at this pointBrowse repository at this point
8. Commits on Jul 6, 2024

   • Small performance improvements

     Show description for 883bc2c
     benbrandt

     committed
     883bc2cView commit details

     Copy full SHA for 883bc2c

     View code at this pointBrowse repository at this point
9. Commits on Jun 21, 2024

   • Update changelog with details about the fix

     benbrandt

     committed
     7c3cbbdView commit details

     Copy full SHA for 7c3cbbd

     View code at this pointBrowse repository at this point

   • New attempt at finding best effort binary search window

     benbrandt

     committed
     b8b2184View commit details

     Copy full SHA for b8b2184

     View code at this pointBrowse repository at this point
10. Commits on Jun 8, 2024

    • Update benchmark output

      benbrandt

      committed
      ac9f17aView commit details

      Copy full SHA for ac9f17a

      View code at this pointBrowse repository at this point
11. Commits on Jun 2, 2024

    • Prep CodeSplitter Python bindings and release

      Show description for 9c589f5
      benbrandt

      committed
      9c589f5View commit details

      Copy full SHA for 9c589f5

      View code at this pointBrowse repository at this point
12. Commits on May 10, 2024

    • Add benchmark code for code splitter

      benbrandt

      committed
      90aeff5View commit details

      Copy full SHA for 90aeff5

      View code at this pointBrowse repository at this point
13. Commits on Apr 27, 2024

    • Update benchmark output

      benbrandt

      committed
      7c5d994View commit details

      Copy full SHA for 7c5d994

      View code at this pointBrowse repository at this point
14. Commits on Apr 26, 2024

    • Remove test_utils because it will block publish

      benbrandt

      committed
      4d37942View commit details

      Copy full SHA for 4d37942

      View code at this pointBrowse repository at this point

    • Move vocab files to cached downloads and reuse more encoding logic

      benbrandt

      committed
      da3c069View commit details

      Copy full SHA for da3c069

      View code at this pointBrowse repository at this point

    • Add rust_tokenizers (#145)

      Show description for e73a18e
      

      dkuku

      and

      benbrandt

      committed
      e73a18eView commit details

      Copy full SHA for e73a18e

      View code at this pointBrowse repository at this point
15. Commits on Apr 20, 2024

    • feat!: Change ChunkCapacity from a Trait to a Struct

      Show description for 86e7287
      benbrandt

      committed
      86e7287View commit details

      Copy full SHA for 86e7287

      View code at this pointBrowse repository at this point

    • feat!: Update Splitters to require a ChunkConfig, and trim by default

      Show description for 42bfcbb
      benbrandt

      committed
      42bfcbbView commit details

      Copy full SHA for 42bfcbb

      View code at this pointBrowse repository at this point
16. Commits on Apr 3, 2024

    • bump bench output

      benbrandt

      committed
      47d86fbView commit details

      Copy full SHA for 47d86fb

      View code at this pointBrowse repository at this point
17. Commits on Mar 25, 2024

    • Remove Text level from markdown levels

      Show description for 35f30dc
      benbrandt

      committed
      35f30dcView commit details

      Copy full SHA for 35f30dc

      View code at this pointBrowse repository at this point

    • Fix bug with ranges after offset

      Show description for ac678ab
      benbrandt

      committed
      ac678abView commit details

      Copy full SHA for ac678ab

      View code at this pointBrowse repository at this point

    • Remove extra allocation for ranges

      benbrandt

      committed
      1b282d0View commit details

      Copy full SHA for 1b282d0

      View code at this pointBrowse repository at this point

    • Simplify the level selection logic

      benbrandt

      committed
      3f13f97View commit details

      Copy full SHA for 3f13f97

      View code at this pointBrowse repository at this point

    • Reduce allocations in chunk generation

      Show description for e442344
      benbrandt

      committed
      e442344View commit details

      Copy full SHA for e442344

      View code at this pointBrowse repository at this point
18. Commits on Mar 22, 2024

    • Switch from Criterion to Divan for benchmarks (#113)

      Show description for 8556955
      benbrandt

      authored
      8556955View commit details

      Copy full SHA for 8556955

      View code at this pointBrowse repository at this point
19. Commits on Mar 9, 2024

    • Update documentation and Python library (#110)

      Show description for 24e1d3f
      benbrandt

      authored
      24e1d3fView commit details

      Copy full SHA for 24e1d3f

      View code at this pointBrowse repository at this point
20. Commits on Mar 2, 2024

    • Add markdown benchmark code (#107)

      benbrandt

      authored
      e1bf7acView commit details

      Copy full SHA for e1bf7ac

      View code at this pointBrowse repository at this point
21. Commits on Feb 3, 2024

    • Move clippy config to cargo.toml (#93)

      benbrandt

      authored
      1b38bc8View commit details

      Copy full SHA for 1b38bc8

      View code at this pointBrowse repository at this point
22. Commits on Dec 23, 2023

    • Benchmarks (#70)

      Show description for 6637a46
      benbrandt

      authored
      6637a46View commit details

      Copy full SHA for 6637a46

      View code at this pointBrowse repository at this point
23. Loading