Status: Applied addendum to the assessment methodology for all kinds of digital resources (user testing layer, section 4.3) and to the development methodology. Real user sessions under this protocol have not been carried out yet.
1. Purpose and grounds
The protocol helps to establish whether a person understands the purpose of a service, their own position in the process, the next step, the error, and the consequences of a decision. It also allows identifying requirements on memory, attention, reading, and pace of work that interfere with completing a task.
Reference documents cited in the preparation of this protocol: W3C COGA, Making Content Usable, 29.04.2021, W3C guidance on involving users in evaluation, general guidance of GOST R ISO 21801-1-2022, official scope of application, Uutilsynet practice of user testing, the Spanish national standards for easy-read text UNE 153101:2018 EX and UNE 153102:2018 EX.
All procedures, the designations K01-K12 and the decision rules below are an authorial draft. They do not constitute a separate level of WCAG conformance. A violation of a specific technical criterion is additionally recorded in the technical register, with its own applicability and evidence. The complexity of a task for a person does not by itself prove a violation of WCAG or of a national standard.
1.1. Scope of application and terms
The protocol is applied by researchers and assessors when preparing user checks of comprehensibility and cognitive accessibility, as an appendix to the assessment methodology for any kind of digital resource (user testing layer, section 4.3; for websites also section A.7, items 32-34) and to the development methodology. It does not replace technical testing against WCAG and does not create a separate level of conformance.
- K01-K12 check - an authorial scenario, a task for a user session on a specific aspect of comprehensibility (section 4).
- Project stop factor - a reproducible barrier in the main scenario that requires analysis before acceptance (section 6).
- Guiding assistance - a moderator hint that names an element or a path, or explains the content of the resource, as opposed to a neutral repetition of the goal (section 3).
- WCAG (Web Content Accessibility Guidelines) - accessibility guidelines for web content, developed and maintained by the W3C consortium.
- W3C (World Wide Web Consortium) - an international consortium that develops standards and recommendations for the internet, including WCAG.
- WAI (Web Accessibility Initiative) - an initiative (programme) of the W3C consortium that leads the development of WCAG and the accompanying accessibility documents.
- COGA (Cognitive and Learning Disabilities Accessibility Task Force) - a working group of the W3C consortium on cognitive accessibility; it issues advisory, non-mandatory addenda to WCAG for people with cognitive and learning impairments.
- Levels A, AA, AAA - the three steps of WCAG conformance with increasing strictness of requirements.
- Assistive technologies (AT) - programs and devices that a person with a disability uses to work with a digital resource: a screen reader, a screen magnifier, an alternative input device, and similar means.
- ISO - the International Organization for Standardization, which publishes international standards, including in the field of accessibility. An international standard is not, by itself, a normative legal act in any jurisdiction until it is adopted nationally or referenced by a contract or a technical specification; in the Russian jurisdiction (a national example) this means adoption in the form of a GOST R standard, or a reference to it in a contract or a technical specification.
- GOST R ISO - the designation of a Russian national standard adopted on the basis of an international ISO (or ISO/IEC) standard, with the degree of correspondence stated: identical (IDT), modified (MOD), or non-equivalent (NEQ); an identical standard (IDT) means that the text of the GOST R standard fully matches the international one.
1.2. Application to kinds of resources
The checks K01-K12 describe the tasks of a person, not the structure of an interface, so they apply to any kind of resource from Part II of the assessment methodology. The kind of resource determines the starting point, the device and the assistive technologies of the participant, and also which tasks are applicable:
- websites and web applications (section A) - all tasks where the function is present; the result of the K checks is used for items 32-34 of section A.7;
- electronic documents (section B) - primarily K01, K02, K03, K12, and for fillable forms also K04, K07 and K08;
- mobile applications (section V) and desktop software (section G) - all tasks where the function is present, on the device and with the settings the participant uses in real life;
- digital services and processes with authorisation (section D) - the tasks are carried out over the whole process, including transitions between kinds of resources; K09 is mandatory where a login exists;
- self-service terminals (section E) - the tasks are carried out at the installation site or on a demonstration terminal, with particular attention to K05, K06, K08 and K10; participation without the participant's own device and settings is recorded as a condition of the session.
The result of the K checks for any kind of resource is stored separately from the conformance decision (section 6) and does not replace the check protocols of the corresponding section.
2. Preparation of the study
First, the decision for which the study is carried out is defined: for example, whether a new application submission scenario can be released, or whether the payment confirmation has to be reworked. Real, significant tasks are chosen, and it is stated explicitly which understanding a person needs for a safe independent action.
The passport includes:
| Field | What to record before the session |
|---|---|
| Needs | Difficulties in reading, remembering, sustaining attention, understanding terms, processing several stimuli, or switching context |
| Context | Familiarity with the service, the language, the device and assistive means; customary support |
| Configuration | Product version, device and OS, assistive technology and its version, language, scale, input method, and material system settings |
| Task | The neutral goal of the person, without a hint about the name of the needed button |
| Key understanding | What the person must understand before acting: the amount, the deadline, the recipient, the commitment, the way to correct or cancel |
| Materials | Safe fictitious data and a state that can be restored for the next participant |
| Support | The way the instruction is delivered, the person's usual means, permitted assistance, and the stopping conditions |
| Coverage plan | Which needs are represented, which are absent, and why this composition of participants was chosen |
| Responsible parties | Moderator, observer, decision owner, responsible author or editor, support service, and the specialist who helps adapt communication |
The set is built by needs, not on the assumption that a diagnosis sets identical behaviour. Old age does not mean a cognitive impairment; the presence of autism or ADHD does not automatically describe reading and decision-making abilities. A single participant cannot be declared a representative of the whole group.
This draft does not establish a universal sufficient number of participants. For a qualitative pilot, what matters is the diversity of significant needs and repeated testing of changes. The headcount and the completion criterion of each iteration are agreed in advance. Success rates in a small target sample describe only that sample, and not the prevalence of barriers in the population.
3. Consent, assistance and stopping
Before the start, the participant receives a clear explanation of the goal, the actions, the duration, the recording, and the use of data. They are told that the resource is being evaluated and not the person, and that participation can be stopped without giving a reason. Consent to participate and consent to audio or video recording are recorded separately. If recording is not agreed, de-identified notes are used.
The consent information can be read aloud or presented in a convenient format. Understanding is checked by asking the person to explain in their own words what will happen and how to stop. This is not an examination. If a person cannot express a clear and voluntary decision, the session is not started; the further procedure is determined by the person responsible for the study, taking into account the applicable rules and the necessary representation. The fact of a disability by itself is not a basis for considering an adult person incapable of giving consent.
Compensation, if provided, does not depend on the successful completion of the task. A participant may use their usual device, reading software, magnification, alternative communication, and ordinary support. These conditions are described in the result.
For each action, the following are distinguished:
- independently under the agreed conditions: the person uses ordinary means and the help available on the site;
- with a neutral repetition: the moderator repeated the original goal without indicating a way to solve it;
- with guiding assistance: the moderator named an element or a path, or explained the content of the site;
- not completed: the participant stopped, refused, chose a different outcome, or a stopping condition was met.
Guiding assistance is not retroactively turned into independent success. Customary support is recorded separately from the researcher's prompts. A constant requirement to reason aloud is not mandatory: it can itself increase the load. Pauses and short questions after an action are allowed.
The session is stopped at the participant's request, on noticeable fatigue or discomfort, on a risk of disclosing personal data, of actually submitting an application, a purchase, or a payment. After stopping, a break or completion is offered, without pressure to continue. The study uses a test contour and fictitious data.
4. Checks K01-K12
Each task is applied only where the corresponding function is present. For non-applicability, the specific grounds are recorded. Before the user session, an expert goes through the same scenario, removes known technical blockers, and notes the remaining limitations. The expert forecast and the observation of a person are stored separately.
| ID | What to prepare and offer the person | What to observe and record |
|---|---|---|
| K02. Meaning of labels | Choose significant buttons and terms. Before a safe action, ask the person to predict what will happen, then carry out the action | Whether the expectation matches the actual state after the action; the salience of the result, the focus position, ambiguous labels; the need to guess the meaning of an icon or metaphor |
| K03. Orientation | Give a multi-step task, then offer to return to the previous decision and continue | Understanding of the current step, the remaining actions, and the saved result; loss of place after returning |
| K04. Memory load | Offer to fill in a form using test data and later to change one value | What has to be remembered, rewritten, or looked up again; the accessibility of viewing and correcting what was entered; the possibility to use notes and pasting |
| K05. Attention and stimuli | Choose a screen with optional movement, a pop-up message, or several tasks | Whether the stimulus diverts from the goal; whether it can be stopped or hidden; whether a clear path is preserved after closing |
| K06. Pace and interruption | At an agreed moment, make a pause and return to the task. If there is a timer, check its warning separately on the test contour | Understanding of the time limit, of extension, and of resumption; the place where information is lost; the possibility to continue without repeated complex input |
| K07. Error and recovery | In a safe form, use a predefined erroneous value, then offer to fix the problem and continue to a confirmed result | Whether the person understands the cause and the needed change; whether they find the field; whether the correct data is preserved; whether the previous error disappears visually and programmatically after the correction; whether a new error cycle arises; whether the successful outcome is clear |
| K08. Consequences of a decision | Before a conditional submission or payment, offer to check the outcome and explain what will be accepted | Correct understanding of the amount, the recipient, the deadline, and the commitment; the presence of a real way to correct and to cancel; a mismatch between the appearance and the consequences |
| K09. Login and access recovery | On a test account, go through the login and the forgotten-credential scenario, preserving the person's usual means | Requirements on remembering, rewriting codes, or solving a task; the clarity of switching channels; support for pasting and for a password manager |
| K10. Help | Offer to find an answer to a question during the process, to get help, and to continue | Whether help is found in the right place; whether the communication channel is suitable; whether the task context is preserved after the request |
| K11. Settings | Offer to apply the text, motion, or presentation setting the person needs, if it exists | Whether the setting is found, whether the effect is clear, whether it is preserved; whether choosing settings becomes an additional barrier |
| K12. Comprehensibility of text | Give an important text and offer to show the next step, choose the suitable option, or explain the meaning in a customary way; then solve the task on the basis of that text | Meaning errors, unclear terms and conditions; missed consequences; the difference between reproducing the text and applying it correctly |
For K08, the list of essential details is checked in advance by the service owner, and where necessary by a specialist of the profile. Simplifying the text must not hide the cost, the deadline, the limitations, and the consequences. Reproducing the wording verbatim is not required: what is assessed is the meaning needed for the decision.
For K12, an automatically calculated "readability" may serve as an auxiliary signal. It is not proof of understanding. The tasks use text that the person has not yet learned during joint editing. In a repeated iteration, it is useful to combine returning participants with people who have not seen the previous version, clearly distinguishing their results.
Project proposal. An oral retelling is not a mandatory way to answer in K03, K08 and K12. A participant may express understanding through an action, a choice of an image, or a customary means of communication. Before the session, the essential meaning and the observable sign of its understanding are recorded; the answer options must not give away the correct solution. The technique is based on a reader check from the manual FRC DS "Accessible and comprehensible in communication", 2025, pp. 113-115. Separate editorial techniques and training pairs are given in the appendix "Examples of clear instructions"; its examples are not the results of user testing.
In K05 and K11, the calmness of the composition is checked separately from contrast. The distinguishability of labels and of the focus must not be reduced solely to lower the visual load. The participant chooses the available settings themselves; the effect, the preservation, and the return are recorded, including combined needs. The need to reduce distractions is not attributed to all people with autism, and the need for a bright screen not to all people with visual impairments. The approach to differences is described in section 3.1 of the development methodology and section 4.4 of the assessment methodology.
5. Observation card
Study / protocol version:
Participant: de-identified code
Agreed conditions and customary support:
Object / version / role / state:
Device, OS, assistive technology, version and settings:
Need and context material for the task:
Comprehensibility check under section 4:
Neutral wording of the task:
Predetermined key details and expected outcome:
Test data:
Actions and the point of difficulty:
State before the action and state after the action:
Accessible name, role, state and actions of the element, if AT was used:
Explanation by the participant in their own words:
Assistance: ordinary / repetition of the instruction / guiding
Outcome: independently / with repetition / with assistance / not completed
Reason for stopping or refusing, if any:
Evidence: de-identified note, agreed recording, state
Interpretation and possible alternative explanations:
Related technical criterion, if a violation is separately confirmed:
Change proposal:
Owner / deadline / retest:
The execution time is recorded only together with pauses, assistance, and conditions. Slow execution is not by itself a defect. The participant's explanation should not be replaced by the observer's guess. If the cause may lie in an unfamiliar service, in the wording of the task, or in an unfamiliar device, this is recorded as an alternative explanation, and a check is planned.
If a participant uses an assistive technology, the full route from the starting point to the result is recorded, and not just a single utterance of the synthesiser. The expected behaviour is described through the accessible name, role, state, and action, the detectability of the change, the focus position, and the possibility to continue the task. A verbatim transcript may confirm the observation in that version and those settings, but does not set a universal phrase for other programs.
6. Decision rules
For each significant task, a matrix of participants and outcomes is compiled. Irreversible or dangerous misunderstandings, task abandonment, guiding assistance, and independent completion are shown separately. People with different needs are not combined into a single average "cognitive accessibility percentage".
Project stop factor: a reproducible barrier in the main scenario, because of which a participant cannot complete the task or wrongly understands a significant consequence, requires analysis before acceptance. A single case may be a sufficient signal of a serious risk, but not an assessment of its prevalence. The release decision is made by the service owner on the basis of the described risk and the verified options for correction. The presence of an alternative channel does not by itself confirm WCAG conformance of the site.
For the remaining findings, a specific change and an expected effect are set. The retest covers the original scenario, the new explanation, and the consequences for adjacent steps. If a correction changes the text, the sequence, or the interaction model, a simple confirmation of task closure by a developer is not enough.
Project proposal. A version labelled "simplified text" or "easy-read text" is not recognised as ready without a K12 check (section 4) by at least one person with difficulties in reading or comprehension, who has not previously participated in editing that text; the order of consent, assistance, and stopping for such a check is determined by section 3 and is not repeated here. Passing only an internal editorial check without such a participant gives no basis to claim that the version matches its purpose. If the direct participation of such a person is unavailable, a check by a representative is allowed, a teacher or an accompanying person familiar with the needs of the target group, with an explicit note in the result that the strength of such evidence is lower than that of direct participation. The requirement is formulated by analogy with the Spanish standards for easy-read text UNE 153101:2018 EX and UNE 153102:2018 EX, which require such validation as a condition of conformance.
An acceptable conclusion: "In version X, four participants with the stated needs independently completed task T; two needed an explanation of the term Y. The observation requires a change of the text and a retest". The numbers in this sentence are an example of the wording, not the result of a study carried out.
Unacceptable conclusions: "the resource is cognitively accessible to everyone", "WCAG conformance is confirmed by users", "80% accessibility" without a definition of the object and the denominator. The absence of comments in a small sample means only the absence of identified problems under the given conditions.
7. Storage and transfer of results
In the work plan, the data owner, the permitted circle of access, the place of storage, and a specific deletion date or a review of the term are set in advance. The public report contains de-identified results. Diagnoses, contacts, credentials, the record of consent, and the researcher's observations are not combined into a publicly available defect register.
The result is handed over as a list of the checked tasks, the composition of the needs, the conditions, the observations, the limitations, the decisions, and the retest plan. Corrected technical defects and changes in comprehensibility have related, but separate, acceptance results.
The expert forecast, the manual technical route, and the observation of the participant are stored separately. Only a user session that has actually been carried out constitutes user evidence; a conclusion by the author, by the support service, or by an accessibility specialist is not marked as the voice of a participant.
