Add terminalprops().rgb and TermResponseAll 'rgb' - #21191
AstroSnail wants to merge 2 commits into
Conversation
|
I disagree that v:termrgbresp should hold the raw response. The scope of this PR is strictly about a positive RGB response (the same behaviour #16490 used to have). Within this scope, if and when a positive RGB response arrives, the response will always have the format Ncurses is irrelevant to this PR, this feature deals strictly with terminal response. The specification I used for the XTGETTCAP response is this one https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h3-Device-Control-functions, which in my understanding can't send a boolean value and will translate a numerical value into a string. Therefore, the space of responses is always a string, either containing a single number (e.g. 8, for bits per channel across 3 channels) or three numbers (e.g. 5/6/5, for bits per each channel). I decided that parsing this string is out of scope for this PR, so v:termrgbresp contains the "raw" capability string. The example in the PR description mirrors what #16490 used to do, and is not meant to be an all-encompassing solution (the same way #16490 evidently wasn't). I expect that users can edit the example to fit their needs. It is true that this PR doesn't provide a mechanism to detect a negative RGB response. This is based closely on #16490 which did nothing when a negative response arrived or no response arrived. I consider it sensible to start Vim in a state that assumes the terminal doesn't support RGB, and configure it to take advantage of RGB when the positive response arrives, because some terminals will never send a response and should be assumed to not support it. I didn't consider terminalprops() because I aimed to stay consistent with other uses of TermResponseAll. I don't have strong feelings either way. If terminalprops() is preferable to v:termrgbresp I can change the PR to use that. I didn't consider using "rgb" in lowercase because the capability name is "RGB" in uppercase. I don't have strong feelings either way. If all-lowercase is preferable I can change the PR to use that. I didn't touch the disabled code too much, under the impression that it was kept because it might still be valuable. I don't have strong feelings either way. If it's preferable to remove it once and for all I can change the PR to do that. I changed version.c because I use my own patches and like to keep track of which ones I'm using, and expect that the maintainer will remove it when it comes time to merge the PR. (This has happened to my past PRs without issue.) I agree that da1 and osc fixes should go in a separate patch, but I'm doubtful as to how to go about that. I've previously encountered trouble when I sumitted two patches that worked together, and during some time only one was merged which led to bugs. The fixes also don't address a specific bug, they're simply omissions that I happened to notice while working on this PR, so it's not clear to me how I should present such a patch or what tests (if any) I should add to go along with it. I agree that I'd like to hear back on some of my points. |
|
Thanks for the reply. Answers to your points, in your order. The capability valueYou are right - the XTGETTCAP reply is always a string, and ncurses' own terminalprops()Yes, please change the PR to use it. It reports 'y', 'n' or 'u' per property, Your reasoning for the negative side is sound: start out assuming no direct The final call on the interface belongs to @chrisbra, so it is worth waiting LowercaseThis one is not a preference. I put two autocommands on the existing da1 event, The disabled code9.1.1114 commented it out with "We may need another way to enable this in the version.cSomeone has to strip that hunk by hand before the change is committed. Please Splitting the da1 and osc fixPut it in this same PR as a separate commit, first in the series. The commits On tests: there is no TermResponseAll test in testdir today, and the delayed The testThe group is testRespRGB, so Your reason for feeding "100" is fair, and once the |
|
Thanks, your critique is helpful. I've given your ideas some thought, and come up with with a plan that sounds sensible to me. I will drop v:termrgbresp in favour of an entry in terminalprops() called 'rgb'. At the time when Vim receives the RGB response, it will check whether the response was positive, and whether the value decodes to '8' or '8/8/8' (the two known formats for "8 bits per component"), and only in that case set the 'rgb' prop to 'y', otherwise 'n'. The TermResponseAll event remains (renamed to "rgb" in lowercase) so that the vimrc can react to the response, should the user want it to. I'm not yet sure how values besides 8 are reflected in the terminal behaviour. I suspect that they always parse component values from 0 to 255 and either chop less-significant bits off or find the closest representable colour if the display isn't actually 8 bpc. But I haven't found documentation that says this for certain. Either way, 8 bpc is common these days and the direct-color format doesn't afford a greater range than 0 to 255, so I think requiring exactly 8 bpc is reasonable. What do you think? |
|
The plan is the right shape, but two parts of it will not hold. 'n' cannot be reachedctlseqs gives the failure reply as "DCS 0 + r ST for invalid requests" - no So 'rgb' will be 'y' or 'u', never 'n'. That still matches what you described: The value should not decideSet 'y' on any positive reply and do not read the value at all. The RGB capability is the terminal reporting that it understands direct color. This also drops the parsing: no "8" against "8/8/8", and nothing to revisit The rest of the plan - dropping v:termrgbresp, keeping TermResponseAll with |
|
That makes sense. It raises another question though. If a terminal doesn't respond to XTGETTCAP RGB, but does support direct colour and exposes this through its terminal description (e.g. TERM=xterm-direct), what should count, the number of colours (e.g. Co#16777216), the presence of RGB, or either? The original patch #16490 doesn't seem to do anything useful with RGB from the terminal description, and relies on number of colours for this purpose. It's not obvious to me whether the RGB capability is even exposed through the termcap library. |
RGB does not reach Vim through the terminal descriptionVim reads capabilities with the termcap-compatible calls - tgetent(), The terminfo calls do find it, as a boolean - hence the -2 from tigetnum - but Either source should countCo is what the description leaves, and it already lands in t_colors. 16777216 The two sources are independent. In #16568 the terminal was xterm under So set 'rgb' to 'y' when t_colors reaches 16777216, and on a positive A note on the valueRGB in the xterm-direct entry is a boolean and carries no number. The "8" and |
Problem: When "da1" and "osc" events were added to TermResponseAll,
they were left out of block_autocmds and unblock_autocmds.
Solution: Add them in the same way as the other TermResponseAll events.
Signed-off-by: AstroSnail <astrosnail@protonmail.com>
edafc60 to
528297f
Compare
|
I've begun implementing the changes we agreed on, and I have some questions. I'm encountering some confusion while implementing the number of colours in the terminal description as a source of direct colour support for terminalprops().rgb. The information is available right away as Vim initializes, before it even needs to wait for a response from the terminal. But when Vim eventually receives a terminal version response it reinitializes all term_props with the "tpr_set_by_termresponse" attribute set to TRUE. If I set this to FALSE for the 'rgb' termprop, it will never get reset (unless the term_prop test override is enabled), even when the 'term' setting changes which I think isn't ideal. If I set this to TRUE, then it gets reset in two ways: setting 'term' (in set_termname) and receiving a terminal version response (in handle_version_response). The obvious way forward here is to set the 'rgb' termprop when there are 16777216 colours in both places, so that it keeps up with changes to 'term', but this sounds kludgy to me, and I want to know what you think. EDIT: to be clear, the RGB response is downstream of termresponse, so if the terminal responds positively to RGB then the 'rgb' termprop will always be set no matter what happens to the 'term' setting. When I removed v:termrgbresp, the block/unblock_autocmds functions in autocmd.c lose the variable that helped track whether the 'rgb' TermResponseAll event should fire. The obvious substitute is term_props[TPR_RGB].tpr_status, but term_props is local to term.c. Should I make a new global variable, un-static term_props, ignore block/unblock_autocmds, or is there another more preferable solution? |
528297f to
47973a3
Compare
The color count does not need a copy in term_propst_colors is a value that can be read at any time, not an answer to remember, so keeping a copy of it in term_props is what creates the reset problem you ran into. Report it where the dictionary is built instead, in f_terminalprops(), and both assignments go away: value[0] = term_props[i].tpr_status;
if (i == TPR_RGB && t_colors == 0x1000000)
value[0] = TPR_YES;term_props then holds the XTGETTCAP answer alone, tpr_set_by_termresponse stays TRUE for it, and the property follows 'term' and t_Co on its own. An accessor keeps term_props to itselfFor the bookkeeping in block_autocmds() and unblock_autocmds(), an accessor in term.c that returns term_props[TPR_RGB].tpr_status is enough. The property only ever goes from unknown to yes, so autocmd.c can hold an int and compare it. Making term_props global is more than this needs. That also replaces what is still commented out there: the old_termrgbresp declaration and four lines in unblock_autocmds(). Vim uses // rather than /* */. The property does not depend on FEAT_TERMGUICOLORSThe XTGETTCAP branch in got_code_from_term() sits under #ifdef FEAT_TERMGUICOLORS, while the color count checks in ttest() and handle_version_response() do not. A build without the feature then reports 'y' from the color count but stays at 'u' however the terminal answers. What a terminal supports does not depend on how Vim was built. The branch no longer touches 'termguicolors' now that p_tgc_set is gone, so the #ifdef can go with it. Two-character names in key_namesIn get_term_entries(), da1 and oscAdding them to block_autocmds() and unblock_autocmds() is right, but it is unrelated to RGB and would be easier to see as a commit of its own. DocumentationIn term.txt, the note that |
47973a3 to
571fd74
Compare
The property may go from yes to unknown during the brief time after the 'term' setting is changed or a terminal version response is received and before the XTGETTCAP RGB response is received. EDIT: I've changed the unblock autocommand to only trigger when the 'rgb' termprop changes from 'u' to 'y', as it normally would if it wasn't blocked.
Are you referring to commit 605d740? I don't understand what you want me to change.
I would really like to solve the redraw problem before considering this PR complete. I did some testing and found that Vim can draw the intro screen as many as 3 times on startup:
As far as I can tell the redraws are triggered by calls to redraw_asap(UPD_CLEAR), but adding a redraw_asap(UPD_CLEAR) call at the end of did_set_termguicolors doesn't seem to force it to redraw the screen (and it adds another bug: Vim draws the intro before the screen has been put in termcap mode, leaving visible trash after Vim exits). EDIT: Solved below. |
571fd74 to
4de37a8
Compare
|
Nevermind that last point, I think I've solved it. |
e147975 to
1624e06
Compare
chrisbra
left a comment
There was a problem hiding this comment.
Thanks, I don't think it makes sense in f_termprops().rgb to return the RGB key only when the color count has been set to 16777216, instead let's just return the detected value from the XTGETTCAP query or the TERM value. It shouldn't be an issue to keep/remember the value during terminalprops() runtime. This also creates the inconsistency that setting t_Co manually can set the RGB value without a corresponding TermResponseAll autocommand.
I am fine with the suggested terminalprops() interface and it matches the existing terminal properties feature, so yeah, good idea to extend it.
For the term_is_rgb() function, just returning true/false should be fine. I don't think it makes sense to return the difference between undefined/yes/no.
| int | ||
| term_is_rgb(void) | ||
| { | ||
| return term_props[TPR_RGB].tpr_status; |
There was a problem hiding this comment.
should this return true/false instead of the raw status value?
return term_props[TPR_RGB].tpr_status == TPR_YES;
| mouse mouse type supported | ||
| kitty whether Kitty terminal was detected | ||
| decrqm whether sending DECRQM sequences work | ||
| rgb whether direct color is supported |
There was a problem hiding this comment.
I think this should keep ** marker
| set t_RV=x | ||
| call test_override('term_props', 1) | ||
|
|
||
| call feedkeys("\<Esc>[>0;0;0c", 'Lx!') |
There was a problem hiding this comment.
Not sure what this is for, but I think if this is supposed to trigger the terminalprops().rgb value, it should be asserted
| call assert_equal('y', terminalprops().rgb) | ||
|
|
||
| let &t_Co = '1' | ||
| call assert_equal('u', terminalprops().rgb) |
There was a problem hiding this comment.
Is this actually useful? First I think setting t_Co manually is quite unusual and after the RGB property has been detected from querying the terminal should not reset the flag back to undefined.
| if (t_colors == 0x1000000 && !p_tgc_set) | ||
| set_option_value((char_u *)"termguicolors", 1L, NULL, 0); | ||
| # endif | ||
| #endif |
There was a problem hiding this comment.
shouldn't this be something like this?
#ifdef FEAT_TERMRESPONSE
if (t_colors == 0x1000000)
term_props[TPR_RGB].tpr_status = TPR_YES;
#endif| // Use number of colors to indicate direct color support in case the | ||
| // terminal doesn't respond to XTGETTCAP. | ||
| if (i == TPR_RGB && t_colors == 0x1000000) | ||
| value[0] = TPR_YES; |
There was a problem hiding this comment.
hm, this is strange. Shouldn't we drop the whole block so that this stays like this?
value[0] = term_props[i].tpr_status;I am not sure I understand the comment, if the terminal does not respond to XTGETTCAP, then valud[0] shouldn't be TPR_YES, so this shouldn't be an issue
There was a problem hiding this comment.
I may need to rephrase the comment. What I meant was that, if the terminal does not respond to XTGETTCAP, then use number of colours as a fallback to see whether it supports RGB. The end result is that if the number of colours is 16 million, then value[0] is set to TPR_YES, if the terminal reports RGB support through XTGETTCAP, then term_props[TPR_RGB].tpr_status is set to TPR_YES, in other cases value[0] is TPR_UNKNOWN.
To make it clear to me: you are saying that colour count should not affect the value of terminalprops().rgb? The current implementation returns 'y' in either case that colour count is 16777216 or XTGETTCAP response is detected. |
|
Chris is right about t_Co, and it was my suggestion that put that check in |
1624e06 to
9f7a736
Compare
Problem: Vim does not know whether the terminal supports true color.
When a Vim user wishes to use true color and their terminal
description doesn't set t_Co to 16777216, there is no other
information and they must set termguicolors unconditionally in
their vimrc.
Solution: Reintroduce the RGB code request, add a new terminalprop 'rgb'
and a new TermResponseAll event 'rgb'. Don't change
termguicolors automatically.
Signed-off-by: AstroSnail <astrosnail@protonmail.com>
9f7a736 to
53d8c06
Compare
|
I tested it out with a sample vimrc file: Running this under xterm sometimes enables termguicolors but mostly doesn't. It looks like a race, the response is sent too early, before that part of the vimrc file was parsed. Not sure how easy it would be to fix this. Hm, found it. It seems autocmd_busy is set when calling But even when I set force=TRUE, it doesn't work because |
I'm having trouble reproducing this. When I use your sample vimrc like
When I add a sleep or a busy loop to the VimEnter autocommand, Vim seems to get as far as sending out the version and foreground/background requests before waiting, to which the terminal replies quickly, but Vim only starts sending xterm-codes after it ends. Same thing seems to happen when I open a file. I want to understand how you got the behaviour you described. I've found |
|
See here, on startup, Vim is not completely re-drawn, until I hit a key, then the statusline and command line is drawn. Screen.Recording.2026-09-07.233407.mp4 |
|
I'm extremely confused. None of my testing ever saw the cursor stop at the last word of the intro screen. Now and even back when this PR still caused a late redraw for me, the statusline was always drawn and the cursor was always at the beginning of the buffer. I would really appreciate if you sent the results of running Vim inside |
|
Will do later, once I am back at my developer machine. I think it may be related to having vim-airline +fugitive installed. I think this vim may be busy handling a VimEnter autocommand when it receives TermResponse result. |
|
here we go: |

Suggested fix for #16327.
This patch tries to implement a compromise both for users who want automatic termguicolors and users who don't. For users who don't want termguicolors, nothing needs to change. For users who want termguicolors and already added
set termguicolorsto their vimrc, nothing needs to change. For users who want their vimrc to adapt to different kinds of terminals which can and can't handle termguicolors, it's now possible to add this to their vimrc:(this is vim9script, I didn't check whether the syntax is valid for old vim script)
This script exactly matches the behaviour of #16490 (the original fix for #16327), and additionally runs
redraw!to fix the bug that the colours only change after the first user action. (I added the same fix to the #if-0-ed out code, for consistency)It is possible to add this to defaults.vim if this behaviour becomes desirable in the future.