How do you use automated WCAG and accessibility audit tools effectively and efficiently?

If you're looking for a list of tools, here's a place to start: http://www.w3.org/WAI/ER/tools/ Note that the entries are not all up to date.

Choosing the right tool for the job requires understanding of the capabilities of their coverage over WCAG. It is widely considered that Deque's axe-core is the best one in many respects, including the least number of false positives and having extensive documentation. It's also open source. Axe-core is also being used in many popular accessibility defect scanners out there, including Microsoft's Accessibility Insights, Google's Lighthouse and Accessibility Cloud. Another popular engine is Siteimprove's Alfa. These tools are helpful in finding accessibility problems in many pages at once but it's always worthwhile remembering that their coverage over WCAG is low and you always need to test manually to be able to talk about WCAG compliance.

One of the biggest things to do when picking a tool is finding one that is consistent in a ruleset. For example, if you chose to use axe-devtools extension, axe-core is the underlying ruleset. Now, whenever your teams choose to use a library for UI testing, or even linting you can match the rulesets to use axe across the board. This allows for results to be consistent and developers and stakeholders to be familiar with the results

To benefit from this, you need to have a tool that can scan many pages and has support for recurring scans (AKA monitoring). Some powerful tools can be used for monitoring include Accessibility Cloud, Axe Monitor, Pope Tech, and Siteimprove. Accessibility defects can come from many sources, including design or code changes, but also addition of new content. A team member can publish a new article that has accessibility problems for example. In cases like this, having a tool that monitors your site's accessibility in short & recurring intervals and the ability to send notifications when new problems are found could be very beneficial. An important aspect to note here is that most monitoring tools have limited support for on-demand testing.

A low-hanging fruit can be the integration of automated accessibility testing into already existing automated frontend tests that you run continuously to detect potential regressions. For example, the testing framework Cypress supports easy integration with accessibility testing tools like Axe. Since Cypress provides a straightforward way to add custom commands and plugins, incorporating accessibility checks is relatively simple. Additionally, Cypress' ability to simulate user interactions and DOM manipulation aligns well with accessibility testing requirements. Therefore, while some initial setup may be required, the overall process is generally straightforward, making it an accessible enhancement to existing frontend tests.

It is important to understand what requirements can be reliably tested, and which parts of these requirements can and cannot be evaluated, and to supplement the automated test results with inputs from other testing methods such as heuristic evaluations, usability testing etc.

These are very good points and can't be stressed enough. Automated checkers will also only find a small number of genuine accessibility issues.

Results of some tools can be overwhelming since these results would include thousands of potential findings across thousands of pages. The differentiation of the tools comes to play here. Are the similar occurrences of the same problems being detected, grouped? Are the findings prioritized? Can they be marked as false positive to train the system? Does the system offer in-context learning material and the ability to generate solution suggestions? Does it allow issue tracking with external systems? Considerations like these could be very helpful in dealing with a larger number of problems. If the tool doesn't deal with this, you will have to do finding the needle in a haystack. Potential problems found by tools also require human review.

Prioritization requires understanding of how each problem is impacting which disability types and how severely. Prioritization should also take the following points into consideration: how widespread each problem is and how difficult it is to solve them. Prioritization could be made by using these values in a scoring system that is similar to RICE scoring system (reach, impact, confidence and effort). If this is done correctly, substantial improvements can be observed in a short amount of time in fixing a website's accessibility problems.

Prioritizing fixes from automated tools comes down to effectively choosing how your development team or organization wants to handle the results AND sticking to that plan. You can choose to go by the criticality of defects (Critical, Serious etc). OR you can prioritize by the overall impact of the issues (number of instances, etc).

After the fixes are applied / deployed, you need to test your site again. If it is a fix that applies to a single page, testing that particular page could be enough but if the fix applies to large number of pages, you need to test your entire site to ensure the fix works everywhere as intended and didn't cause other problems.

In addition to automated audits, follow up with manual testing to verify your results and identify any accessibility issues the tools might have missed.