draft: Accesskit integration #3111
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add accesskit and accesskit_winit dependencies with winit fork patch - Create AccessibilityNode wrapper type in iced_core - Add accessibility() method to Widget trait with default None implementation - Implement accessibility() for Button widget - All changes compile successfully
- Create build_accessibility_tree() function to traverse widget tree - Implement stable ID generation based on tree path (hash-based) - Convert AccessibilityNode to AccessKit Node with proper fields - Handle bounds, label, value, role, enabled state - Build AccessKit TreeUpdate with node collection - Note: Children traversal is incomplete - needs widget state access
- Decision: Use iced's Operation pattern for tree traversal - Rationale: Non-breaking, future-proof, gets layout info for free - Phase 1: Full tree rebuild (MVP) - Phase 2: Diff-based incremental updates (future optimization) - Update both plan.md and research_status.md with architectural decisions
- Replace manual traversal with widget::Operation trait implementation - Implement build_tree() operation that collects accessibility info - Handle container, focusable, text_input, text, and scrollable widgets - Generate stable NodeIds using path-based hashing - Build AccessKit TreeUpdate in finish() method - Uses existing iced infrastructure (non-breaking, future-proof) - Compiles successfully
- Simplify TreeBuilder to implement Operation trait directly - Add UserInterface::accessibility() method that builds tree - Use simple counter for NodeIds (not path-based hash) - TreeBuilder collects nodes during operation traversal - Returns TreeUpdate directly from build() method - Compiles successfully - ready for testing
- Add accessibility: Option<accesskit_winit::Adapter> field to Window struct - Initialize as None (TODO: needs proper adapter setup with handlers) - Update plan.md with complete session progress and next steps - Update research_status.md with implementation details - Add architecture decision docs from earlier analysis - Ready for next session: adapter initialization and event loop wiring
- Add Accessibility variant to runtime Action enum - Create accessibility action types (ActionRequested, Deactivated) - Implement AccessKit handler traits (ActivationHandler, ActionHandler, DeactivationHandler) - Initialize adapter during window creation using with_direct_handlers - Process window events through adapter.process_event() - Update accessibility tree after UI rebuilds via interface.accessibility() - Handlers communicate with main loop via iced Proxy - Add Message: Send bound for thread-safe handler callbacks
- Add children Vec to TreeBuilder to track node hierarchy - Make all widget nodes children of the root window node - This fixes the panic: 'TreeUpdate includes nodes which are neither in the current tree nor a child of another node' - Counter example now runs without crashing
- Write first tree snapshot to /tmp/iced_accessibility_tree.txt - Shows tree structure, node roles, labels, and bounds - Helps verify accessibility integration without screen reader - Confirmed: tree building works, nodes have correct data
This commit wires up the Widget::accessibility() method to the actual tree building process, making widgets properly expose their accessibility information to screen readers. Changes: - Add Operation::accessibility() method to collect widget accessibility info - Implement Operation::accessibility() in TreeBuilder to convert iced nodes to AccessKit - Update Button::operate() to call operation.accessibility() - Add Text::accessibility() implementation with Role::Label - Update Text::operate() to provide accessibility info With these changes, both Button and Text widgets now properly appear in the accessibility tree with correct roles, labels, bounds, and actions. Testing: - Buttons expose Role::Button with press action - Text exposes Role::Label with text content - Accessibility tree correctly reflects UI structure - Verified via HTTP accessibility inspector
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Starting AccessKit Integration
This PR is to mainly judge the feasibility of integrating accesskit. It is now at a state where the counter
example works on MacOs with VoiceOver.
This is intentionally not finished. I'd like to get some feedback if I should continue down this path. Either way,
I've learned a lot putting this together.
Current state
Only Button and Text have accessibility support. I wanted to keep
the initial scope a little limited. I've tried to reduce the number of breaking changes. IDs are optional, and there
was a best effort to make things work when they aren't provided.
General structure
Changes in
coreadd the data types, and trait changes. Changes inruntimeCurrent problems
API changes
There are a couple of spots where trait bounds have been added to
Message, and I'd like to work towards reducingthat as much as possible. AFAIK,
Messages don't actually need to beSyncor'static, but those are currentbounds in some places.
Winit dependency
Since Iced depends on a vendored version of winit and accesskit-winit depends on winit, we'd probably need to vendor
accesskit-winit, or somehow depend on the normal winit. I've currently switched to winit on crates.io via
[patch.crates-io], but that obviously isn't a great long-term solution.OperationI haven't been happy with the current implementation of keeping
Operationobjectsafe, while still making the
AccessibilityNodegeneric over the type ofMessagethat it can emit. I'll definitely be spending more time trying to figure out a good compromise here.
Cargo Feature
I intentionally didn't add a cargo feature to toggle accessibility support on or off. This seems easier to do once
the hard bits have been figured out.