SCS (Select Character Set) doesn't work with k95 terminal type

[j4james]

This may well be working as intended, but I thought it was worth raising because I found it very confusing. Steps to reproduce:

1. Start up k95g
2. Set the terminal type to k95
3. Connect to a linux host
4. echo -e '\e(0jklm\e(B'

I expected that to show line drawing characters, but it does not. I first noticed this when testing my games, and was wondering why none of the soft fonts were working, but realised later it was SCS in general that didn't work.

If I start up with the terminal type as vt220 then it works. But once I switch to k95 it stops working, and switching back to vt220 doesn't fix it. I eventually figured out that echo -e '\e%@' gets it working, which is what makes me think it might be intentional (I'm guessing you might be deliberately disabling SCS sequences when UTF-8 is enabled).

However, AFAIK most modern terminals do not do that. When UTF-8 is enabled, they won't let you map anything into GR, because bytes in the range 0x80 to 0xFF are reserved for UTF-8, but you can still map other character sets into GL. I believe they do this because there are still commonly used Linux apps that rely on SCS mapping to get line drawing characters, but in general a modern terminal will be expected to handle UTF-8.

And even if you want to keep things as they are, I think it's at least worth considering whether the vt terminal types should automatically switch UTF-8 back off again (i.e. re-enabling SCS), because that was the most confusing aspect for me. It seemed random whether SCS was working or not.

[davidrg]

Yeah, this is a bit of an annoying one - I run into it from time to time myself (and quite regularly when playing around with DRCS).

Currently SCS is processed in utf-8 mode, but all received bytes are assumed to be in utf-8 so all character set translation is skipped and received bytes are just converted to ucs-2 (a separate issue I'm not looking forward to - #315).

I found something from the original author a while back (usenet post perhaps? can't find it now) that very much implied that this is the intended behaviour, and based on my limited understanding of ISO-2022 its probably correct. I think as far as its concerned UTF-8 is a different and separate coding system and once you're in it (ESC % G or ESC % / G) ISO-2022 rules don't apply except that if you used ESC % G to escape ISO-2022 you can use ESC % @ to get back.

But I'm not sure if there are actually any terminals out there except perhaps xterm that are as strict with any of this - some don't even seem to implement ISO-2022 at all now. And even xterm has the vt100Graphics resource which is set by default - I guess this is what makes common line drawing stuff work there, though the documentation implies this doesn't allow just any character set to be accessed from UTF-8 land so perhaps you might still need to switch out of UTF-8 mode to access others. I assume PuTTY does something similar as IIRC it also supports the escape sequences for switching in and out of UTF-8.

I do have buried somewhere in my to-do list implementing something like xterms workaround - think that wouldn't be difficult, and as long as xterms behaviour is matched its probably not going to surprise anyone.

Switching character sets to something sensible when a VT emulation is chosen probably should be done too (and I guess VT320+ ought to default to the UPSS when that's implemented). I don't think that's too likely to cause problems for anyone out there who might be using K95s terminal emulator.

[j4james]

I found something from the original author a while back (usenet post perhaps? can't find it now) that very much implied that this is the intended behaviour, and based on my limited understanding of ISO-2022 its probably correct.

I can believe that it's technically correct, but I'm just saying that it makes for a less useful terminal in modern usage. I know full ISO 2022 is not widely supported anymore, but the one thing that all modern terminals do seem to support is DEC Special Graphics being mapped into G0 (i.e. my echo -e '\e(0jklm\e(B' test case). I've just tried a bunch of Linux terminals (while in UTF-8 mode) and they all supported that. And I think the reason they do, is because there are still quite a few apps that rely on it for line drawing (see the dialog utility as one example).

That said, I don't mean to try and push you to change this if you want to stick with the current behaviour. I just wanted to make you aware of the issue. I can always add an ESC % @ sequence to the start of all of my games so they'll work in K95 out of the box.

[davidrg]

Thinking about it a bit, you're probably right. Given no other terminals seem to enforce this requirement there isn't much benefit in K95 being incompatible with everything else here - not when its already going to be up against stuff like this for the foreseeable future. Plus I already accepted rule-breaking for SGR-38/48 for the same reason - not accepting the semicolon format just means apps that work everywhere else fail in K95.