Shortcuts: Improve interface

[Janus]

Actual window

Mockups

[waddlesplash]

I'd prefer to see us mimic the +/- from Repositories, instead of using a "toolbar" at the bottom of the view.

[KevinAdams]

Looks nice! I like the icons for the +/- buttons, as shown in screenshot5.

Are you also going to update the application path dialog box?

[Janus]

Replying to KevinAdams:

Looks nice! I like the icons for the +/- buttons, as shown in screenshot5.

Are you also going to update the application path dialog box?

The idea was to fix all the problems of the application. My first attempt to fix #7505 is stuck in review limbo...

[leavengood]

Shortcuts definitely needs work. Janus let me know if there are patches you want some help with merging. Even after a year I don't think anyone else has worked on this.

Either way I may at least take a cursory look at some of these Shortcuts bugs.

[KevinAdams]

Janus has ​Gerrit #339 open still, which looks fine to me other than the 80 character formatting that was mentioned.

[leavengood]

I would like to fix Shortcuts in a much more fundamental way which removes the need for the translation hack in Gerrit 339. Frankly the text based interface is terrible and I would like to replace it with a proper window for configuring the shortcut action. That would have tabs or similar to choose between things like running an app, sending messages, moving the mouse, etc.

After that I would like to adjust how the actual keyboard shortcut is chosen, but the above is more important.

[humdinger]

I have a proposal on how an improved Shortcuts prefs could look like.
How about a simple main window with a list, first column "Keycombo", second "Action". Under the list a "+" and "-" button to add/remove a shortcut. Very similar to the Repositories prefs, just with Revert/OK buttons instead of of Enable/Disable, because I don't think we'd need those.

Adding a new shortcut or double-clicking an existing one opens a window. Something like this:

Key combination: ___________________ [Clear]

Choose action:

* Launch application: ___________________________ [Browse...]

* Send message: _____________ to ________________ [Browse...]

* Mouse action: {popup:}   ____________
		MouseMove,
		MouseMoveTo,
		MouseUp,
		MouseDown
		MouseButton,
		etc.
				 
				        [Cancel] [OK]

When the text field of "Key combination" has the focus, the pressed keys are inserted, e.g. "ALT+SHIFT+V". The "Clear" button blanks the field.

The "Browse..." button opens a file dialog to find the app to launch. The path is inserted in the text field, the user may optional add parameters there.

When sending a message, the first text field takes the message string, the second the app's signature that gets inserted when choosing the app from the "Browse..." file dialog.

I'm not really sure the typical user needs all these "Mouse actions", but if it's decided to keep them, they could be chosen from a popup menu with the parameters to be entered in a text field. A tooltip could come in handy here...

[humdinger]

After reading the IRC log, maybe there should be another radio button above the general "Send message" with something like "Standard actions" that has a popup menu with "Volume up, Volume down, Tracker find, Hide all, ..." and other standard messages.

WRT special keys, I'd expect those to be mapped in the Keymap prefs. Shortcuts, I'd expect to be for customized key combonations. Probably these to panels are candidates for becoming joined. Not easy, considering Keymap's ultra wide window to accomodate the keyboard representation...

[pulkomandy]

There is a new Input preferences in the work that will eventually include all of this.

Some thoughts on the matters for shortcuts:

I don't like the idea of "standard actions". This is special-casing some actions and the underlying reason, is that no one is going to manually fiddle with "send message" if it is just a textfiled to fill in manually. So it is essentially already building a workaround for an otherwise bad user interface.

We can be more ambitious here. That is the whole point of the scripting interface and the "getsuites" message used to discover what an application can do. It should be possible to ask an application "what can you do?" and have the results exposed in the UI. With this in mind, the media_server should reply with something like:

Available actions:

• Set the volume (to some value)
• Mute
• Raise the volume
• Lower the volume

The Shortcuts prefs should parse the getsuites reply and expose these possible actions in its UI. Not just a "send message" textfield.

So, how this would work from the user side:

• Send message to [Popup menu]...

• In the popup, common/recently used applications and a "browse" entry for extra apps
• After selecting the app, Shortcuts asks said application what it knows to do, and builds an user interface offering the possible actions to select from, and fields to fill in the parameters. It allows to explore the available targets in the applications (windows, views, etc) in case these expose specific interfaces, and a shortcut can be set to one specific target there.

---

I don't like the idea of "mouse actions". Let's use scripting, we can invoke buttons directly and this can be done just fine through the "Send Message" system described above.

---

And one UI complain: I think the textfield/browse UI is a bad idea. No one is going to edit application paths manually. This UI pattern comes from lazy programmers using a textfield for a path, then users complaining that the UI is not good, then the programmer going "gran, okay, I'll add a browse button".

Please use a popup menu as done in the Background preferences, which allows quick access to known values and opening a filepanel if needed.

[nephele]

The standard actions would just be a list of "common actions" or something, certainly not to replace a textfield, I would like to have this because figuring out that "lower volume" is an action on the media_server makes sense but isn't that easily discoverable. (I.e a list of common actions above the list of servers to talk to that would list something like "Media server -> Increase Volume" which would allow people easy acces to this option and a good clue on where to find more).

[pulkomandy]

Just having "sane defaults" (such as mapping the "volume down" key to "media server -> decrease volume" in the default settings) could be enough to make this discoverable, I think?

[leavengood]

I am going to make changes incrementally to avoid getting overwhelmed, and to just fix some of the immediately broken aspects of Shortcuts. The first step will be to replace the text interface with a proper edit window, probably much like Humdinger's text mock-up. This will likely still include some text boxes for application path or signatures for messages.

Once we have that we can see about improving it with drop downs or QuickLaunch style search boxes, then proper support for GetSuites.

I haven't thought about it deeply yet, but I do like much about how the macOS shortcuts look:

But these only include "common actions" as nephele calls them, and we want to be more flexible than that. But I think it would be nice to have readable list of messages we know built-in Haiku servers or applications support so user's can quickly choose them, even if they just "expand out" to a message call with a signature and weird message code. But even that could be fixed by just having some mappings to nicer views in the UI: "Open find panel in Tracker".

[pulkomandy]

This UI is quite nice indeed. I think we should be able to do something like htis based on "getsuites". Maybe we should define a special suite for keyboard shortcuts or some other way to tag useful actions, so that the right part of the window doesn't get filled with thousands of useless commands? This way applications can tell if they want to expose some actions as keyboard shortcuts.

[humdinger]

And one UI complain: I think the textfield/browse UI is a bad idea. No one is going to edit application paths manually.

The textfield is handy if you let the user specify parameters for the launched app. It also makes it easy to copy to clipboard, in case the user wants to share the info or plans to use it in a script etc., or to quickly adjust a path (maybe you renamed a volume or want to redirect to another version). Of course, most people will use the Browse... button, so it's not an afterthought, but the main means of setting the app. The text field serves to inform e.g. which version of an app it's pointing to (for example the app from the HPKG or the debug-build you did).

Please use a popup menu as done in the Background preferences, which allows quick access to known values and opening a filepanel if needed.

This is a good approach if you'll be using the same couple of common locations (like home, desktop, downloads...), but it's less useful for a one-off selection of a specific app.

I think we should be able to do something like htis based on "getsuites".

The issues I can see with "getsuites": Most apps don't have those implemented and you just cannot fix them all. Also, the app has to be running to query it.

[nephele]

I still would not want a textfield, if an ipc takes paramaters i would expose them as such: "Media Server -> set volume to [int field]" "Medis Server -> Increase volume by [int field]" where some of those get default values one needen't change If you would like copy to clipboard i would just have a dedicated button for that, although it makes little sense if you can't paste it back.

As for the Browse button, I would suggest the UI to have the right side be the getsuites/application view by default, so one can select a shortcut on the left and then select from the right view what it is one really wants (and if you are so inclined you can make a third for experts tab, if really needed).

[pulkomandy]

Replying to humdinger:

And one UI complain: I think the textfield/browse UI is a bad idea. No one is going to edit application paths manually.

The textfield is handy if you let the user specify parameters for the launched app. It also makes it easy to copy to clipboard, in case the user wants to share the info or plans to use it in a script etc., or to quickly adjust a path (maybe you renamed a volume or want to redirect to another version).

I would still use the browse button and not allow the user to set an invalid path by careless hand editing. The text field just makes things more confusing.

Of course, most people will use the Browse... button, so it's not an afterthought, but the main means of setting the app. The text field serves to inform e.g. which version of an app it's pointing to (for example the app from the HPKG or the debug-build you did).

The browse button can open the directory with the file selected as a filepanel. This is much more handy than a path that you can only paste to Terminal with the default Tracker settings (otherwise you could enable the location bar in Tracker and copypaste the location here, carefully removing the app name itself from it).

We can have the app name and icon shown, and double clicking that opens the relevant Tracker window. Why would you bother with showing and allowing to manually edit the path for this?

Again, this is what Backgrounds does, it allows you to reach the recently used files, or to browse for more. Just what we need here too, and it's easier to locate apps (they are all in /boot/apps) than to locate backgrounds (they are pretty much anywhere in your home dir)

Please use a popup menu as done in the Background preferences, which allows quick access to known values and opening a filepanel if needed.

This is a good approach if you'll be using the same couple of common locations (like home, desktop, downloads...), but it's less useful for a one-off selection of a specific app.

Looking at the MacOS screenshot above, it doesn't even have a browse button. It has a preset list.

I think the browse button for looking for apps is not great. The user should not need to know where the apps are stored. Ideally we would just use a list like in "open with" when we need to list applications, except here we are probably more interested in scripts and command line tools.

But if the point is essentially running custom scripts, maybe Shortcuts should manage these completely. You drop them in a "shortcuts" directory somewhere in your settings dir and maybe the app offers to edit them from the UI. And this also makes more sense for sharing your nice shortcuts than painfully copypasting them, one by one, from a small textfield.

The textfield is clumsy for picking an app, the browse button will not know what to do if the previous text field content had arguments to the app, should these be preserved? And the text field will be too small for editing the full path and no one wants to be using full paths here - they will just want to do the same thing they do in Terminal.

I think we should be able to do something like htis based on "getsuites".

The issues I can see with "getsuites": Most apps don't have those implemented and you just cannot fix them all. Also, the app has to be running to query it.

The app has to be running for the shortcut itself to do anything as well.

Are we going to use this for anything else than servers anyway? Are there use cases for automating things in regular apps from Shortcuts? (I can imagine something like autohotkey, but I'm not sure Shortcuts is the right place for it?)

As for apps not implementing this yet, well, let's fix that. If they don't support this, how are users supposed to discover what messages they can send? Check out the app sourcecode?

[leavengood]

Let's see what I can come up with, then we can iterate. One major limitation of the macOS shortcuts is they just can perform those predefined actions. I do not know if other apps can add to that list, but I don't think so. That makes their UI much simpler. I think we definitely want the ability to launch apps, and probably add command-line parameters, even if hidden a bit under an "Advanced" view. Just that addition complicates things quite a bit (let alone all those mouse things which I think most of us just want to hide...)

I also want to add the checkbox to allow shortcuts to be enabled and disabled.

On a related note I also would like to expose some sort of API to allow apps like QuickLaunch or Clipdinger to add to this list, rather than having to make their own input_server filters or ask the user to add the shortcut.

[humdinger]

We can have the app name and icon shown, and double clicking that opens the relevant Tracker window. Why would you bother with showing and allowing to manually edit the path for this?

As I said, to optionally add parameters (if that is supported).

Again, this is what Backgrounds does, it allows you to reach the recently used files, or to browse for more. Just what we need here too, and it's easier to locate apps (they are all in /boot/apps) than to locate backgrounds (they are pretty much anywhere in your home dir)

I still don't see it. Choosing the app to open with a shortcut is a one-off occasion. Having the last 5 or 20 recently 'shortcutted' apps in a popup menu doesn't help. Only offering the last paths (/boot/apps/) and opening the file panel there to choose an app isn't very useful either. Same can be done by setting the file dialogs start location to it. The "Browse..." button even saves one mouse click compared to the popup menu. :)

The textfield is clumsy for picking an app, the browse button will not know what to do if the previous text field content had arguments to the app, should these be preserved?

I'd say, no. The chances that another app uses the same parameters are small. Frankly, I suspect that normally the target app won't be changed often. Once you 'shortcut' an app you may change the keycombo or remove the whole shortcut to that app, but only seldom decide to re-use the shortcut to one ap for another.

The issues I can see with "getsuites": Most apps don't have those implemented and you just cannot fix them all. Also, the app has to be running to query it.

The app has to be running for the shortcut itself to do anything as well.

That's a given. Users expect that the thing they want to control has to be running. They probably won't expect that simply creating a message shortcut does, too.

Are we going to use this for anything else than servers anyway? Are there use cases for automating things in regular apps from Shortcuts? (I can imagine something like autohotkey, but I'm not sure Shortcuts is the right place for it?)

Who knows? I'm not at all against providing 'standard actions' for known apps/servers, however we go about providing these messages. I'd just like to offer the possibility to send an arbitrary message to something.

As for apps not implementing this yet, well, let's fix that. If they don't support this, how are users supposed to discover what messages they can send? Check out the app sourcecode?

Some people like to tinker. If 'we' don't get around to fixing every application that could benefit from being messaged, someone might take a look at its sourcecode (and maybe share with others that don't like doing that).

[pulkomandy]

Replying to humdinger:

We can have the app name and icon shown, and double clicking that opens the relevant Tracker window. Why would you bother with showing and allowing to manually edit the path for this?

As I said, to optionally add parameters (if that is supported).

Why should the parameters and the app path be in the same small textbox? I can see the need for a textbox for the parameters, but not for the app path, really.

Again, this is what Backgrounds does, it allows you to reach the recently used files, or to browse for more. Just what we need here too, and it's easier to locate apps (they are all in /boot/apps) than to locate backgrounds (they are pretty much anywhere in your home dir)

I still don't see it. Choosing the app to open with a shortcut is a one-off occasion. Having the last 5 or 20 recently 'shortcutted' apps in a popup menu doesn't help. Only offering the last paths (/boot/apps/) and opening the file panel there to choose an app isn't very useful either. Same can be done by setting the file dialogs start location to it. The "Browse..." button even saves one mouse click compared to the popup menu. :)

Well, a browse button with no popup menu and no textfield either, then?

The textfield is clumsy for picking an app, the browse button will not know what to do if the previous text field content had arguments to the app, should these be preserved?

I'd say, no. The chances that another app uses the same parameters are small. Frankly, I suspect that normally the target app won't be changed often. Once you 'shortcut' an app you may change the keycombo or remove the whole shortcut to that app, but only seldom decide to re-use the shortcut to one ap for another.

You are contradicting yourself, as you previously brought the example of changing the path to an application that has been moved or to point to a custom build of it, in which case I think it makes sense the arguments should be preserved.

But, besides this special case (which I find somewhat unlikely), I don't think directly editing the path is desirable.

The issues I can see with "getsuites": Most apps don't have those implemented and you just cannot fix them all. Also, the app has to be running to query it.

The app has to be running for the shortcut itself to do anything as well.

That's a given. Users expect that the thing they want to control has to be running. They probably won't expect that simply creating a message shortcut does, too.

Once you have picked the app, Shortcuts could run it on its own, ask it its "getsuites", and then quit it, maybe? Or maybe we need to improve the "getsuites" system to work offline?

Are we going to use this for anything else than servers anyway? Are there use cases for automating things in regular apps from Shortcuts? (I can imagine something like autohotkey, but I'm not sure Shortcuts is the right place for it?)

Who knows? I'm not at all against providing 'standard actions' for known apps/servers, however we go about providing these messages. I'd just like to offer the possibility to send an arbitrary message to something.

I don't like this at all. As an application developer, it means all the internal messages I use in my app suddenly become API, and if I change them, some user will complain that I broke their setup. Or they could manage to corrupt the application state in ways I didn't anticipate. This is too brittle.

On the other hand, the "getsuites" system is actually designed for this, and allow the developers to make sure they expose sane things through the interface. This is much less dangerous, even if initially less flexible.

And I'm not sold on "who knows?" without some sample use cases.

There is a need to think more about these application specific shortcuts. For example, I expect the next/previous media buttons to do the right thing to whatever media application is running currently, be it APlayer, MediaPlayer, or VLC, for example. I don't want a system-wide shortcut entry that attempts to send a message to all 3 (and does not work everytime I add a new media playing app to my system).

Probably each app needs to hook for these by itself while it is running, that seems to be the best thing to do after all.

I think we should try to have something much simpler as part of the default system. I can imagine the need for a more advanced tool, but I think it's going to be a complex tool, and I'm not sure it's a great idea to do everything at once in a single application that we ship by default with the OS.

Anyway, I guess anything is better than the current situation?

[KevinAdams]

Replying to pulkomandy:

Once you have picked the app, Shortcuts could run it on its own, ask it its "getsuites", and then quit it, maybe? Or maybe we need to improve the "getsuites" system to work offline?

Replying to pulkomandy:

On the other hand, the "getsuites" system is actually designed for this, and allow the developers to make sure they expose sane things through the interface. This is much less dangerous, even if initially less flexible.

+1 this sounds logical and less likely to break.

[humdinger]

Anyway, I guess anything is better than the current situation?

I agree. We could continue this back-and-forth for quite a while, I'm sure... Let's see what Ryan comes up with and concentrate on the specifics if anyone takes it further into the discussed direction.