Hello,
There are two permissions that are required for any serious “VOiceOver”
scripting.
The first is in the general pane of VoiceOver utility.
“Allow VOiceOver to be controlled with AppleScript.”
This would allow you to do things lie:
tell application "VoiceOver"
tell commander
perform command "open notification center"
end tell
end tell
This is AppleScript code that I am showing here.
One could do essentially the same thing with a compiled Objective C or Swift
programming language using Apple Events It is telling the commander module of
the application “VOiceOver” to perform a command.
This includes most (all?) of the commands you can find when reviewing the
commander area of VOiceOver or found in the VO-H-H help command. Don’t try
interactive commands here like “item chooser”, they will hang and you can’t
actually proceed with the script.
The second function you already partially mentioned was the accessibility
permissions in the security pane of System (Settings/Preferneces0 this
permission allows one application to control the UI Elements of another
application.
Dropbox used to use this to put a flag next to a dropbox folder in the finder,
and there are a few other applications that use it. Keyboard Maestro also makes
extensive use of the UI Elements interface in controlling Mac applications.
There are about 40 specific types of ui elements but one can always do
get all UI Elements of WIndow 1 of system application “Mail” or some close
equivalent.
Also of course, most long standing Apple applications have their own apple
events defined that can allow direct manipulation of the application.
For ex…ample the below is code sample from a script I was using to close
notifications. I gave up on it after a while.
set NotificationUI to getUI("NotificationCenter")
using terms from application "System Events"
try
set aWin to window 1 ofNotificationUI
tell application "NotificationCenter" to activate
on error
OpenNotifications()
set aWin to window 1 of NotificationUI
end try
set aWin to window 1 of NotificationUI
set notContainer to UI element 1 of scroll area 1 of aWin
set LotificationList to UI elements of notContainer
set i to 1
repeat until i > (count of LotificationList)
set anElement to item i of LotificationList
set ActionInfo to the name of every action of anElement
perform action closeAction of anElement
end repeat
.
The above lines would probably fail, since not all notifications have a close
action. But if it did work it would close all notifications in the notification
center.
I haven’t run this code in about 2 years, but I believe it still works with
Ventura.
Jonathan Cohn
> On Jul 13, 2023, at 9:41 AM, Doug Lee <d...@dlee.org> wrote:
>
> I received an inquiry this morning asking about writing assistive technology
> scripts and/or more accessible
> apps for MacOS. I wrote the following rather involved response, as more of an
> expert in Windows than in Mac
> spaces; so I welcome corrections or comments here, and told my contact I
> would do so.
>
> =====
>
> I actually consider your question to amount to two questions:
> 1. Is it harder to write AT scripts under MacOS than on Windows? and
> 2. Is it harder to write an accessible app on MacOS than on Windows?
>
> I do consider it much harder to do assistive technology (AT) scripting for
> Mac than for Windows, but the
> reasons are complicated and subject to inaccuracy because that's not my area
> of best expertise. That said,
> I'll tell you what I can. Same for the second question, which I'll tackle
> after the first.
>
> 1. AT scripting on MacOS vs Windows:
>
> On Windows, AT scripting is accomplished within each screen reader, where it
> is possible anyway:
> * For JAWS, it's JAWS scripting.
> * For NVDA, it's add-ons in Python.
> * For Narrator, I don't know of anything possible yet - though I was told
> years ago this would happen.
> The ease of learning how to do AT scripting, then, depends on the screen
> reader. NVDA and JAWS vary in how
> and how much they document how to do this, but both do provide significant
> documentation.
>
> There is sort of an alternative to all that, which is making individual apps
> "self-voicing." RS Games and
> QC, an alternative gaming platform, are examples of this. In general this is
> often frowned upon though,
> because it leaves all the major interface choices to the AT developer more
> than to the individual user.
>
> On MacOS, we mostly just have one screen reader - VoiceOver. For specific
> spaces, there are alternatives;
> example, YASR or TDSR for terminal windows. I'll gloss past those for
> simplicity now though, because they
> are only meant to cover the unique challenges of terminal interaction.
>
> Apple does appear to provide some tools/interfaces that could be used for AT
> scripting. There's the
> Accessible Events system, which has a switch that you must turn on if you
> want apps to have such permission
> if I recall correctly. There are APIs in the Apple libs for accessibility,
> though I've never known much
> about them. I once saw a fascinating project proving this much though: A
> "file system" made out of
> accessibility information, where you could cd to a control in a running app
> and write to a file to press
> buttons etc. (This used the Fuse layer, if you've heard of that.) But unless
> I missed something,
> documentation of such interfaces has never been particularly easy to find.
>
> So the biggest reason I say it seems harder to do AT scripting on MacOS is
> that it seems much harder to
> figure out what can be done and how to do it. My second reason is that I
> suspect, but can't yet prove, that
> less is actually possible at least without doing things that Apple does not
> want non-Apple folks to do.
>
> But I have a third reason: I suspect, but again can't currently prove, that
> the availability of such options
> is, on MacOS as it is on Windows, going to depend on the specific app of
> interest. I further suspect that,
> on MacOS, there will be less available in the best of circumstances for an AT
> developer than on Windows. It
> is hard to know what can be done if nobody knows how to try, so I expect
> there is far less testing of the
> possibilities on MacOS than on Windows at this time. Take Qt apps for
> example: TeamTalk, TeamSpeak, and
> probably other apps' developers have sometimes worked with the developers of
> the Qt framework to improve the
> accessibility of apps developed using that framework on Windows; but I hear
> zero chatter out there about
> MacOS app developers working with Qt framework devs to improve
> framework-level accessibility for Macs. Again
> this is mostly speculation, but reasoned I think.
>
> 2. Accessible app writing on MacOS vs Windows:
>
> There is one actual advantage of the merkiness of AT scripting on MacOS I
> suppose: End users are, for the
> most part, required to work with app developers, not AT folk, to get problems
> fixed. Between this and the
> uniformity of the MacOS environment, owing of course to its "walled garden"
> nature, any results tend to be
> universal and not limited to specific circumstances, such as which screen
> reader you use. This would of
> course lead everyone to ask the second of your questions. :)
>
> I would argue that it is both easier and harder to write an accessible Mac
> app than an accessible Windows
> app - at least if we let end users define "accessible" for a moment.
>
> Reasons that MacOS is easier:
> * Less variation - only one screen reader and a generally uniform environment.
> * Clearly articulated documentation on how to make controls accessible (I
> believe this is there but have not sought it myself, to be fair).
> * Easier in-house testing for devs: VoiceOver is just as available to them as
> to us (though of course they must learn how to use it).
>
> Reasons MacOS is harder:
> * If you use a framework, you may have less or no control over
> end-user-visible fixes to your app's accessibility problems.
> * Fewer AT users than for Windows that you can ask. (There may be some areas
> of focus, like music creation, where this is not true.)
> * Fewer available AT professionals that can help you, such as via contracting.
>
> The second of those detractive reasons is actually my reason to suspect one
> more difficulty on MacOS: I
> think there is less effort toward making accessible Mac apps because there is
> less of a user base to write
> for there than on Windows. This means that it might be hard to get the
> developers of an existing Mac app to
> work on accessibility problems because they may perceive the effort to
> provide far less of a return on
> investment.
>
> In conclusion, my now overly complex answer to your question, or questions,
> boil down to this:
>
> I think it's harder to write AT scripts under Mac, easier and more likely to
> get accessible apps written for
> MacOS, but then harder to convince people to make the effort toward
> accessibility on Mac even if it ends up
> being easier to do than on Windows.
>
> These are my thoughts, again subject to correction because my expertise is
> far more centered on Windows.
>
> =====
>
> --
> Doug Lee d...@dlee.org http://www.dlee.org
> "If you refuse to be made straight when you are green,
> you will not be made straight when you are dry." {African}
>
> --
> The following information is important for all members of the Mac Visionaries
> list.
>
> If you have any questions or concerns about the running of this list, or if
> you feel that a member's post is inappropriate, please contact the owners or
> moderators directly rather than posting on the list itself.
>
> Your Mac Visionaries list moderator is Mark Taylor. You can reach mark at:
> mk...@ucla.edu and your owner is Cara Quinn - you can reach Cara at
> caraqu...@caraquinn.com
>
> The archives for this list can be searched at:
> http://www.mail-archive.com/macvisionaries@googlegroups.com/
> ---
> You received this message because you are subscribed to the Google Groups
> "MacVisionaries" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to macvisionaries+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/macvisionaries/20230713134127.7i4tc35k6qxtemmn%40dlee.org.
--
The following information is important for all members of the Mac Visionaries
list.
If you have any questions or concerns about the running of this list, or if you
feel that a member's post is inappropriate, please contact the owners or
moderators directly rather than posting on the list itself.
Your Mac Visionaries list moderator is Mark Taylor. You can reach mark at:
mk...@ucla.edu and your owner is Cara Quinn - you can reach Cara at
caraqu...@caraquinn.com
The archives for this list can be searched at:
http://www.mail-archive.com/macvisionaries@googlegroups.com/
---
You received this message because you are subscribed to the Google Groups
"MacVisionaries" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to macvisionaries+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/macvisionaries/DA039119-A94D-42F9-81F1-288CBEE6E7D2%40gmail.com.
