I plan on this to be fully backwards compatible, except now instead of automatically detecting the image protocol to use (for terminal), to use imageprotocol option instead (yet to be implemented), similar to keyprotocol.

A suggestion for improvement:

The SIXEL protocol is the most inferior protocol supported by Vim right now since it's limited to 255 colors. For good results any non trivial image must be properly quantized to 255 colors. While it would be nice to have quantization in Vim, I understand that quantization is a non-trivial problem and not in the scope of the project. After all Vim isn't an image viewer.

I've experimented with quantization through pngquant (properly the best quantization implementation out there) and was able to yield good results.

This image is just 'resize by ImageMagick -> convert to RGBA -> display by Vim'. You can see the artifacts caused by missing quantization:

This one was rendered by 'resize by ImageMagick -> quantize with pngquant -o $OUTPUT --posterize 2 255 -> convert to RGBA -> display by Vim'. The quality is drastically better:

Source code of my highly experimental plugin implementing this can be found here: https://codeberg.org/yamagi/vim-imagepopup

Long story short: It would be very appreciated if Vim provided a way to detect the color depth supported by the active image backend. Or at least what image backend is active. That way a plugin could determine automatically how the image should be processed before it's displayed by Vim.

I plan on this to be fully backwards compatible, except now instead of automatically detecting the image protocol to use (for terminal), to use imageprotocol option instead (yet to be implemented), similar to keyprotocol.

A suggestion for improvement:

The SIXEL protocol is the most inferior protocol supported by Vim right now since it's limited to 255 colors. For good results any non trivial image must be properly quantized to 255 colors. While it would be nice to have quantization in Vim, I understand that quantization is a non-trivial problem and not in the scope of the project. After all Vim isn't an image viewer.

I've experimented with quantization through pngquant (properly the best quantization implementation out there) and was able to yield good results.

This image is just 'resize by ImageMagick -> convert to RGBA -> display by Vim'. You can see the artifacts caused by missing quantization:

This one was rendered by 'resize by ImageMagick -> quantize with `pngquant -o $OUTPUT --posterize 2 255` -> convert to RGBA -> display by Vim'. The quality is drastically better: Source code of my highly experimental plugin implementing this can be found here: https://codeberg.org/yamagi/vim-imagepopup

Long story short: It would be very appreciated if Vim provided a way to detect the color depth supported by the active image backend. Or at least what image backend is active. That way a plugin could determine automatically how the image should be processed before it's displayed by Vim.

I suppose there can be a v: variable for this. However I'm not sure how you would detect the color depth from the image backend? Shouldn't this be possible by just knowing what image backend is being used?

I suppose there can be a v: variable for this. However I'm not sure how you would detect the color depth from the image backend? Shouldn't this be possible by just knowing what image backend is being used?

Good point. Thinking about it the only real advantage of a colordepth variable would be the addition of new backends existing plugins don't know about. But I would suspect that any new backend, be it in GUI or some new terminal protocol, would likely support 24 bit colors. So knowing what backend is used would be enough for plugin authors to special case SIXELs.

I like both of your ideas of a more generic image_add and embedding images inside documents.

I'd suggest keep the idea/impl open to the idea of animated images and fixing to lines with the tallest image setting the lineheight.

Animated images/shaders inside a markdown document templeOS style;

I have conceal on all my markdown formatting and animated images unconcealing to file-refs on cursor over would look so cool, especially if blog writing from within vim.

I've decided to switch to using the libsixel library for the sixel support. Sixel is honestly really complex and has a bunch of edge cases that does not seem right for a text editor to have to implement. Additionally libsixel seems to be very well known and well distributed, and supported on all relevant platforms. This makes maintainance in the future easier

I'm going to put off creating the tests in this PR. I've thought of several solutions, but I think the most suitable one would to literally implement sixel in the builtin terminal (libvterm). Then there could be a term_dumpwrite() like function that can dump a pixel buffer view of the terminal, which can be compared. This should be maybe feasible? (with the help of the Cairo library and maybe Pango to render text? (probably unecessary for testing)). This is a nice feature in its own right as well.

The initial sixel support can be very rudimentary, possibly without scrollback support or sixel scrolling (DECSDM).

This all sounds a bit overkill for just a test suite, but it looks like a fun challenge for me 😄

I have done a big refactor, now pixman is a dependency for the image feature. This is because I need to handle having some parts of the image being under some text (e.g. border). For sixel this is simply a matter of ordering how you draw everything to the screen. However for the kitty graphics protocol and GTK4, images are actual "objects", not just pixels blit to the screen, meaning there has to be a way to figure out what rectangles in an image are visible. Pixman does that, its very fast (SIMD optimized) and well tested, and has no dependencies. Additionally pixman has some image manipulation features that can we possibly expose to the user (e.g. cropping, scaling).

Before:

After:

Not sure why this test_smoothscroll_zero_width() test is failing...