Posted on
1. The keyboard emulation failure shows Wayland is failing to let a user connect multiple physical keyboards into a computer, each with distinct language/mapping. 2. The multi-user collaboration scenario further amplify this problem: each user may be using their own keyboard different in their language/mapping. 3. Traditional IME put their pre-edit text into their own floating box independent of the text input field of the application down below. I remember when I typed with Cangjie/Quick in old Windows, that floating box still remained on the screen post-commit. Auto-completion was completing text inside that floating box, not content in the text field. There were no confusion "what should the auto-completion be based on" and such mechanism wouldn't step into each other's toes even for collaborative editing, as only the "commits" were sent to the application while everything in between weren't. 4. Imagine 2 people typing Chinese into the same text field, each using their own keyboard/seat. Alice types "ni" while Bob types "hao" at the same time. A fair expectation of them is the field either ends up showing "你好" or "好你", not mixing their pre-edit together and produce gibberish. In collaborative editing each user is typing with their own cursor positions, so it will be crazy if the IME mix up their input just because their cursor positions happens to overlap. 5. The raw button events are out of business of most end-user applications. They only need to know the mapped keys and/or the IME commit. For those that care those raw events, combining events from multiple devices (+ virtual keyboards) into one mixed stream is unfavourable anyway. Imagine using traditional keyboard as gamepad. One may want to distinguish key events from each keyboard as different input (under user consent). 6. For modern auto-complete that takes text before the cursor in the existing text field, some sort of caching can be done, such that collaborated edit from Alice won't affect the auto-complete suggestion shown to Bob until after Bob committed his text.
Posted on
2. Multi-user collaboration is solved by seats which have independent layouts. 3. I didn't mention that, but a floating window was part of my protocol. But no, pre-edit is often still inside the text field. 4. The problem is not with overlapping cursors. The problem is that *nearby* cursors will modify state of one another because IME should be aware of the context. Or maybe it shouldn't. Anyway if 2 people write English with overlapping cursors, their text does get mixed up. I don't see why Chinese should be special in that regard :P 5. Sadly, we have to deal with how Wayland is designed today. 6. This doesn't help if Alice presses "commit" before Bob's event reaches the application but Alice's event arrives afterwards.
Posted on
3. Many new IMEs do put the pre-edit inside the text field, but the classic Cangjie/Quick originated in Win9x days didn't. The ibus-cangjie engine I use now also doesn't. 4. The difference of Chinese and English is, what users really want to commit are not a string of characters composed of the pre-edit. They are just hash keys of the Kanjis. There is near zero chance that the characters typed in pre-edit are really what the user want to commit. I always assume putting pre-edit inside the text field is just cosmetic behaviour. Of course I don't want cosmetic behaviour mess up typing. I guess the balance is a bit different for Japanese, so Japanese users may enjoy pre-edit filling up the text field prematurely. 5. I thought Wayland don't fear breaking backward compatibility for the sake of "security", as they proudly refuses to implement get/set window absolute position. The IME and pre-edit input sequence of how one get to type Chinese character(s) is also some form of privacy and may be utilized for fingerprinting, especially now that many Chinese IMEs are adaptive to user habit (same for many modern English touch screen keyboard). Masking that off from applications underneath is a privacy improvement. 5+6. I imagine for backward compatibility sake, an interesting take will be emulate an en-us keyboard for legacy applications, and pretend/remap what one type in other keyboard mapping or IMEs as if one was typing on that or somewhat extension of that. Not perfect but that should allow some reordering and prevention of mix-up of pre-edits.
Posted on
4. Pre-edit doesn't matter. The input method must know (or maybe currently knows but possibly shouldn't) about non-preedit as well, so all text modifications cause the same kind of trouble, even outside of preedit. 5. Wayland developers don't seem to be eager to change how keyboard input works now, but let me know if you want to convince them. I can kind of understand: this is not a security issue, and Wayland can't remain a moving target for the basics forever. 6. A state-based (as opposed to stream-of-events-based) architecture will always have this problem. I'd look for a solution in CRDTs. --dcz