Onscreen keyboard

[X512]

For work with Acer W500 we need onscreen keyboard. It can be based on keymap preflet. Also if arm port will be done onscreen keyboard will be needed on tablets and phones.

It can be implemented as input device or filter input_sever add on.

[korli]

Do you mean an input method maybe?

[X512]

Replying to korli:

Do you mean an input method maybe?

No, onscreen keyboard must work as real keyboard. Input methods as Canna will not work in another case.

[richienyhus]

Maybe something for the next Google Code-In?

[axeld]

This might indeed be a good job for the GCI; it's not much work, since most of it can be used directly from the Keymap code.

[X512]

Some days ago I implemented onscreen keyboard with touchscreen support (and I write this message using it). It implemented as input_server device and it don't based on any Haiku code and use only BeAPI and control look for keys drawing. Currently key positions are hardcoded, later I plan to add keymaps with GUI editor.

[X512]

screenshot of my onscreen keyboard

[dsjonny]

It looks fine for me :)

[leavengood]

That looks pretty good. Just be sure to check the Keymap preflet code before you try to implement keymap support, as I suspect that will have most of what you would need (as Axel suggested.)

Also I personally think this would be useful in general as sometimes I've had my keyboard stop working in Haiku, and this would be a good fallback for that case, at least to do a few things before trying to shut down.

[axeld]

Looks nice, patches welcome. However, I think it must share the same code (KeyboardView) that Keymap is using -- it does not make much sense to maintain code that pretty much does the same thing in two different places.

[X512]

Replying to leavengood:

That looks pretty good. Just be sure to check the Keymap preflet code before you try to implement keymap support, as I suspect that will have most of what you would need (as Axel suggested.)

BeAPI already has function get_key_map for handling keymaps. This function is enough for keyboard. I wrote special class "KeyboardHandler" that gets scancodes and produces messages. Also it handles states of caps, num and scroll lock keys, keystroke repeating by separate thread. For now handling deadkeys is missing. Keymap handling is a little part of this class. Most part is message generation that duplicates functionality of hardware keyboard input_server device.

Replying to axeld:

Looks nice, patches welcome. However, I think it must share the same code (KeyboardView) that Keymap is using -- it does not make much sense to maintain code that pretty much does the same thing in two different places.

At first KeyboardView in Keymap preflet is hardcoded to be keymap editor and not keyboard. It need to be changed a lot to be shared. Changing will be more complicated than writing independent implementation. Current Keymap preflet is also waste a lot of CPU time that is a problem for mobile devices.

At second I plan to use different keyboard layout format that layout keys like layouting of BViews using BLayout and had ability to use special keys such as keys without scancode (smiley, special characters), and keys that change layout or keymap. Also I plan to make GUI editor for layout to take benefit of modifiable nature of onscreen keyboard unlike hardware keyboard.

Also is it OK to use MacOS like icons for command and option keys? As I know BeOS keyboard layout has origin of MacOS one.

[axeld]

Replying to X512:

Replying to axeld:

Looks nice, patches welcome. However, I think it must share the same code (KeyboardView) that Keymap is using -- it does not make much sense to maintain code that pretty much does the same thing in two different places.

At first KeyboardView in Keymap preflet is hardcoded to be keymap editor and not keyboard.

So what? Same goes for BTextView, and it can still be used (and is used in a number of places) in non editable mode.

It need to be changed a lot to be shared. Changing will be more complicated than writing independent implementation.

Not really. Duplicating code is not a good idea, in any case.

Current Keymap preflet is also waste a lot of CPU time that is a problem for mobile devices.

I don't see this. Are you referring to the rounded borders, by any chance?

At second I plan to use different keyboard layout format that layout keys like layouting of BViews using BLayout and had ability to use special keys such as keys without scancode (smiley, special characters), and keys that change layout or keymap.

Sure, KeyboardView does not have these features. I just can't understand how rewriting it completely is any easier than adding them.

Also I plan to make GUI editor for layout to take benefit of modifiable nature of onscreen keyboard unlike hardware keyboard.

KeyboardView already supports arbitrary layouts.

Also is it OK to use MacOS like icons for command and option keys? As I know BeOS keyboard layout has origin of MacOS one.

This is probably nothing we would ship Haiku with, but other than that, it doesn't really matter if the keyboard is changeable.

Was there any particular reason to implement this as a keyboard device within the input_server rather than as a stand-alone application? How does the keyboard know when it should be visible?

[X512]

Replying to axeld:

Not really. Duplicating code is not a good idea, in any case.

I think that that duplicating code is OK when it makes development clearer and faster. I prefer to write as simple as possible and use as few as possible. Trying to generalize everything will produce unmanageable code full of bugs. Sometimes it's better to throw old code away and write it again than maintain it and fix bugs in it.

One thing that can be really utilized is KeyboardHandler. It incapsulates all keyboard logic. KeyboardHandler produce messages from key state changes, holds Caps Num and Scroll Lock key states and handles keymap. For now in Haiku message generation is duplicated in keyboard input_server device and Keymap preflet.

I don't see this. Are you referring to the rounded borders, by any chance?

It waste a lot of CPU time without round borders too. Just try to drag some window on top of Keymap preflet or resize it. It redraws very slow. I use drawing buttons in MouseDown, MouseUp directy instead of Draw for lower CPU time consumption when writing text fast (but for now it is disabled because some keys such as Shift update whole keyboard, KeyboardHandler have no API for it). Drawing one key is faster than all keys.

Sure, KeyboardView does not have these features. I just can't understand how rewriting it completely is any easier than adding them.

It requires understanding foregin code that is complicated task. Current KeyboardView has a lot of overengineering. Also new code is clear, easy understandable an free of legacy. First I have idea to use keyboard from that preflet, but don't understand how to make it work separate; it always crash. Actually writing keyboard view is not hard task, first version was done in few hours. In summary I worked on this keyboard for 3 days.

KeyboardView already supports arbitrary layouts.

It don't support editing layouts. Currentlly layouts can be edited using special delarative language only which is user-unfriendly.

Was there any particular reason to implement this as a keyboard device within the input_server rather than as a stand-alone application? How does the keyboard know when it should be visible?

Keyboard requires to send messages to system, so it can be implemented only as input_server driver or kernel driver to be fully-functional keyboard (but anyway it can not work in KDL unfortuneately). See ​http://www.haiku-os.org/legacy-docs/bebook/TheInputServer_Introduction.html.

For now keyboard is always visible. To hide keyboard just move add-on outside "add-ons" folder. Later I plan to add replicant to deskbar for showing/hiding keyboard.

About automatic detection when keyboard is needed: it could be shown when user focus BView that accept input. View that accept input can be recognized by B_INPUT_METHOD_AWARE flag or some new flag because B_INPUT_METHOD_AWARE actually means that view can show diffrerent text blocks. Any BView that supports input method is also support text input, but not all BView that don't support input methods don't support text editing (but this is bad and should be fixed if possible because it pervents convient japanese etc text editing). Obvoiusly user should have ability to disable automatic showing/hiding keyboard.

Also keyboard is used not only for text input. It is also used for different kinds of editing (selecting items in Tracker, copy-paste etc) and games. So user should have ability to showing keyboard manually.

[axeld]

Replying to X512:

Replying to axeld:

Not really. Duplicating code is not a good idea, in any case.

I think that that duplicating code is OK when it makes development clearer and faster.

That's a bit shortsighted. You don't only have to write code, you also have to maintain it. It may be more initial work to get acquainted with a foreign code base - but that doesn't mean it's not worth the effort, and actually saves time in the long run. The way I see it, you now have to search for bugs in two places instead of one.

Sure, there is a point where it makes sense to go different ways, but how different can a view that displays a keyboard can be? There is enough common ground to rectify a single solution.

I prefer to write as simple as possible and use as few as possible. Trying to generalize everything will produce unmanageable code full of bugs.

Sure, at a point - that is certainly far away in this situation (although I haven't seen your code yet either), but your approach is calling for trouble, too. You should always try to create code that is as maintainable as possible. That will naturally take longer to write.

Sometimes it's better to throw old code away and write it again than maintain it and fix bugs in it.

Nobody questions that, but that's either a direct result of poor coding in the first place, or changed requirements that the original code did not anticipate.

One thing that can be really utilized is KeyboardHandler. It incapsulates all keyboard logic. KeyboardHandler produce messages from key state changes, holds Caps Num and Scroll Lock key states and handles keymap. For now in Haiku message generation is duplicated in keyboard input_server device and Keymap preflet.

Feel free to fix that :-)

I don't see this. Are you referring to the rounded borders, by any chance?

It waste a lot of CPU time without round borders too. Just try to drag some window on top of Keymap preflet or resize it. It redraws very slow. I use drawing buttons in MouseDown, MouseUp directy instead of Draw for lower CPU time consumption when writing text fast (but for now it is disabled because some keys such as Shift update whole keyboard, KeyboardHandler have no API for it). Drawing one key is faster than all keys.

Drawing outside of Draw() is something that might not be supported in future versions anymore, and is generally not a good idea. It's not faster either as it generates the same amount of work in the app_server. The trick is to only invalidate the area that changed, and make your code in Draw() smart enough not to update everything everytime. Which the KeyboardLayoutView currently doesn't do, though.

Sure, KeyboardView does not have these features. I just can't understand how rewriting it completely is any easier than adding them.

It requires understanding foregin code that is complicated task. Current KeyboardView has a lot of overengineering. Also new code is clear, easy understandable an free of legacy.

Hey, I wrote that, so thanks for the fish :-) Anyway, old code does not equal legacy. And chances are that code that you write is clear only to you, too.

KeyboardView already supports arbitrary layouts.

It don't support editing layouts. Currentlly layouts can be edited using special delarative language only which is user-unfriendly.

Yes, but neither does your code. That's outside of its scope. A GUI tool to create these layouts could be added either way; it's not an argument in favor or against a common source.

Was there any particular reason to implement this as a keyboard device within the input_server rather than as a stand-alone application? How does the keyboard know when it should be visible?

Keyboard requires to send messages to system, so it can be implemented only as input_server driver or kernel driver to be fully-functional keyboard (but anyway it can not work in KDL unfortuneately). See ​http://www.haiku-os.org/legacy-docs/bebook/TheInputServer_Introduction.html.

Thanks for the link, but I actually (re)wrote many parts of the input_server. It's been a while, granted, but I can't think of any reason to implement it as an add-on.

For now keyboard is always visible. To hide keyboard just move add-on outside "add-ons" folder. Later I plan to add replicant to deskbar for showing/hiding keyboard.

About automatic detection when keyboard is needed: it could be shown when user focus BView that accept input. View that accept input can be recognized by B_INPUT_METHOD_AWARE flag or some new flag because B_INPUT_METHOD_AWARE actually means that view can show diffrerent text blocks. Any BView that supports input method is also support text input, but not all BView that don't support input methods don't support text editing (but this is bad and should be fixed if possible because it pervents convient japanese etc text editing). Obvoiusly user should have ability to disable automatic showing/hiding keyboard.

Also keyboard is used not only for text input. It is also used for different kinds of editing (selecting items in Tracker, copy-paste etc) and games. So user should have ability to showing keyboard manually.

Yes, that makes sense, although I think automatic should be the default where the user can change that when needed. Besides, I think additional flags would be helpful, so that the view can also add additional information, like "this is for entering numbers", or "this is for entering passwords, don't show the text or give hints".

[richienyhus]

What about an on-screen thumb keyboard mode?

The main buttons are located to the sides in this mode, sometimes with with extra buttons (like the arrows &c) in the centre.

​On-Screen thumb keyboard

[richienyhus]

Any news on the Onscreen keyboard?

Oh in my last comment I meant a keyboard along the lines the one in this article: ​http://www.macobserver.com/tmo/article/how-to-undock-and-split-your-ipad-keyboard

[X512]

Replying to richienyhus:

Any news on the Onscreen keyboard?

It was changed a litte from that time. For now keyboard layout is still hardcoded. I am too lazy because it work for me.

Oh in my last comment I meant a keyboard along the lines the one in this article: ​http://www.macobserver.com/tmo/article/how-to-undock-and-split-your-ipad-keyboard

About splitted keyboard: it seems not difficult to implement it. It require additional keyboard windows. I never tried this feature and I usually hold tablet by one hand and use by another one. Also I often use this tablet on a table like PC.

Do you have x86 tablet, compatible with Haiku to test keyboard?

[richienyhus]

No sorry, I don't have a x86 tablet, but rather only an arm tablet. I'm being hopeful for the future. :D

Do you have the on-screen keyboard app's code hosted anywhere like bitbucket or github? I ask because we could get these extra advancements added by students in the Google Code-In initiative.

On a side note, is there any other apps or small features that would make Haiku more useable on a tablet for you?

Of course major interface changes for touch would be out of the scope of GCI, but some of the little things might be able to be fixed as part of this initiative.

[Freeman]

X512, how did you to create an input_server device for something virtual?

[X512]

Replying to Freeman:

X512, how did you to create an input_server device for something virtual?

It doesn't matter does input_server device add-on represent physical device or not. It only generates input messages and nothing more. You can ever write device add-on that press or input something automatically. Device add-on was used because generated messages will be processed with all filters and methods. This makes keyboard layout switchers and input methods such as Japanese language IM avaliable.

[Freeman]

What did you specify for the cookie? And was your keyboard window directly used in the InputDevice child class?

[X512]

Replying to Freeman:

What did you specify for the cookie? And was your keyboard window directly used in the InputDevice child class?

I din't use cookie at all if remember. I will post latest sources if someone interests.

[X512]

source code

[pulkomandy]

Not really an R1 feature. Work on integrating this is still welcome :)

[modeenf]

as of 2014-11-13 we do some thing like that, don't know how its working and it's not in the build. (root/src/add-ons/input_server/devices/virtualkeyboard) think its based or have X512's work as inspiration :)

[Freeman]

Yes i remember during GCI, i thought i had the input server code setup correctly, but the onscreen keyboard wasn't actually inputting into a field. X512's code helped to clarify things. Currently the virtual keyboard looks bloated, since it uses the "locale changer" interface and has extras that we dont need, but i'm not sure how i can cleanly remove them. Iirc You can type with it though.

[bitigchi]

IMO, the implementation should have two modes:

• Tablet mode for touch screen devices, toggling automatically when an input field is tapped
• A smaller on-screen mode for accessibility purposes, intended for mouse use