3.0 Comparison of Research Methodologies
There were many differences in the research studies, including the documentation provided, samples recruited, methods used and data reporting formats. Table 1 provides a summary of the main differences. The sub-sample studied for any sub study could differ greatly from the total sample size and was not always documented. (Refer to Appendix 1 for more detailed descriptions of the methods used in each study.)
Table 1. Overview of Research Studies
Study |
Sample |
Methods |
Reliability |
Scope |
Bails, 1983, AUS
|
Total unknown, manual and power chairs, from institutions
|
2-D, manual
|
No reliability or validity study |
Body and device size, reaching, maneuvering, door use |
Seeger, Costi and Hartridge, 1994, AUS |
240, all devices. 75% from institutions |
2-D, manual measurements |
No reliability or validity study.
|
Body and device size |
Department of the Environment, Transport and the Regions (DETR): Stait, Stone and Savill, 2000, U.K.
|
745, all devices, attendees at Mobility Roadshow |
2-D, photography with digital measurements |
Reliability study |
Body and device size |
BS8300: 2001 Appendix (research commissioned by the DETR), U.K. |
164, all devices, but only 91 for space allowances, source unknown
|
Not reported. |
Unknown |
Body and device size, knee and toe clearances, reaching, maneuvering, door use |
Universal Design Institute (UDI): Ringaert, Rapson, Qui, Cooper and Shwedyk, 2001, CA
|
50, power chair and scooter users, diverse sources |
2-D, manual measurements, detailed interview |
No reliability or validity study |
Body and device size, reaching, maneuvering |
IDEA Center: Steinfeld, Paquet and Feathers, 2005, U.S.
|
275, all devices, diverse sources |
3-D, digital probe, video, detailed interview |
Reliability and validity study |
Body and device size, reaching, maneuvering, door use |
Bails (1983) recruited participants from attendees at disability support centers and institutions. Eligible participants were between 18 and 60 years of age and used a manual or powered wheelchair. Scooter users were not included in the study. The total sample size is not known. The research focused primarily on testing of full-size simulations of elements found in the built environment, such as doorways, environmental controls, furniture and fixtures that were configured to meet the Australian standards at the time. Many of the findings, therefore, could not be used to make generalizations or to determine the ideal spaces needed for access.
Seeger et al. (1994) studied only device size. About seventy-three percent 73% of the 240 individuals in the sample lived in nursing homes and other institutions. Forty-five percent were over 65. Eleven percent used power chairs and two percent used scooters. Both unoccupied and occupied dimensions of device width and length were measured as well as a set of other basic dimensions. Measurements were taken manually using conventional measuring tools including a tape measure, steel square and spirit level.
The DETR study (Stait et al., 2000) was limited to measurement of device size and weight. Participants in the DETR study were recruited at an exposition of equipment for people who use wheeled mobility devices for traveling around the community. Of the 745 participants whose data was acceptable, 59% used self-propelled manual chairs, 9% used attendant powered chairs, 25% used power chairs and 9% used scooters. Nine percent of the sample were judged to be 16 years of age or younger. Two cameras were used to take photographs of the participants from top and side after they wheeled into position on a scale. A checkerboard pattern on the floor and wall provided references to take measurements off the photographs. Parallax was corrected during the calculation of the dimensions. Prior to data collection the reliability of the method was verified by comparing dimensions taken directly from individuals with those calculated from photographs. Device dimensions were defined clearly. Although a wide variety of accessories were observed on the devices, they were not measured as part of the width calculation.
The BS8300:2001 research covered knee and toe clearances, reaching abilities of wheeled mobility users, clear floor area space requirements and maneuvering clearances. A total of 164 individuals were included in the sample but only 90 participated in the research on space allowances. Due to the lack of a full research report, it is not clear how the measurements were collected and, in many cases, the landmarks used to define them. From the information available, it appears that some scooters and attendant propelled chairs were included in the sample but it is not clear whether these individuals were included in the device, body or reach measurements.
The UDI study (Ringaert et al., 2001) was recruited from disability and senior organizations in Winnipeg by written invitation. Of the 50 participants, 35 used power chairs and 15 used scooters. The cause of disability for individuals in the sample included a wide range of conditions. Device size, maneuvering space needed and reaching ability were measured. All dimensions were taken to the extremes of the equipment including any object attached to the device like a ventilator. However, the actual landmarks on the devices are not well documented. Measurements were made with rulers and tape measures but no information is given on the accuracy and reliability of these techniques. Maneuvering trials were recorded using overhead video cameras while participants completed standardized movements in simulated environments built with plywood floors and wood framed dividers. Measurements were later taken off the videotapes although the method used to do this and the reliability of the technique are not described. An observer rating was used to determine successful trials.
The IDEA study (Steinfeld, Paquet, Feathers, 2004; 2005; Feathers, Paquet, Steinfeld, 2004; Paquet and Feathers, 2004) included static anthropometry of occupied wheeled mobility devices, reach measurements and measurements of maneuvering clearances. At the date of the analyses conducted for this report, 275 participants with a wide range of chronic conditions had been recruited through outreach efforts with several organizations in Western New York and mass media. Fifty-three percent of the sample was male and 47% female. The mean age of the sample was 51.5 with a range of 18-94 years of age. Fifty-five percent used manual wheelchairs, 36% used power wheelchairs and 9% used scooters. Three-dimensional locations of body and wheelchair landmarks were collected with an electromechanical probe (Feathers, Paquet and Steinfeld, 2004). A reliability study was completed before data collection (Feathers, Paquet and Drury, 2004). The reach protocol utilized a standardized procedure in which participants reached to a target (free reach) and also while lifting .46, 1.4 and 2.3 kg (0, 1, 3 and 5 lb.) weights. Reach limits were digitized with the mechanical probe, and, from the data, a reach envelope for each individual was constructed. Maneuvering clearances were measured while participants conducted standardized maneuvers inside a set of lightweight movable walls. The walls were gradually moved further apart until the maneuver could be completed without the participants moving the walls. Clearances were pre-measured on the floor of the test site using tape and marker and the locations of walls were recorded after each trial.
The studies utilized widely different methods to recruit participants. The DETR study has a large sample size but the participants were all self-selected in their interest and ability to attend the Mobility Roadshow. Thus the sample is likely to be more mobile than the wheeled mobility population as a whole and their choice of chairs may reflect that. A high rate of unusable data due to problems with photography may have introduced some sampling bias as well. The Seeger and Bails samples were drawn primarily from institutional settings, which would definitely introduce bias in the types of devices used and maneuvering abilities. Both the IDEA and UDI studies recruited a diverse sample and provided transportation to the research site ensuring that low mobility would not be a barrier to participation. However, in both cases, the samples could not be considered representative of the entire population of wheeled mobility users in their respective countries. Users of powered chairs, are definitely overrepresented in the IDEA sample. The UDI study did not include manual wheelchair users at all. It should be noted that overrepresentation of one group or another is not necessarily bad because it can help to address lack of attention to specific populations.
The UDI and Bails studies had very small samples. Bails, in particular, had sub samples for some of his protocols that included as few as five individuals. In the U.S. we calculate that the bare minimum size of a sample that would represent the U.S. population of wheeled mobility users would be 500. Although the current size of the IDEA Center sample (275) is the largest assembled in the U.S. and is almost five times larger than the study on which ANSI A117.1 (1980) was based, it still cannot be considered a representative sample on size alone. It is important to note that, in this type of research, total sample size is not as important as inclusion of a diverse group of users, enough people in each category to cover the range of abilities and sizes and spending enough time with individuals to obtain a comprehensive data set. Powered device users were over-sampled on purpose in the IDEA sample because the technology has changed so much since 1979 and the users of powered devices are more seriously impaired.
A limitation of both the IDEA sample and the UDI sample is that they are drawn entirely from cold weather cities. This may introduce some bias toward larger and more durable equipment. Data was collected all year round in the IDEA study so that season should not have introduced a bias in recruitment, however, the UDI study was conducted only in winter which may have influenced participation. The IDEA sample is growing more representative as time goes on as we target new recruitment to underrepresented groups. We are planning on collecting data in two additional cities, one in a hot climate and one in a city with hilly terrain. We have made a concerted effort to promote the use of our methods so that other researchers can contribute data to merge into our existing database.
The methods Bails and the BS8300 used are not reported in detail. Without the details, it is difficult to evaluate them. The reliability of measurements taken in the UDI and Seeger research is not known. It appears that UDI did not address issues of distortion caused by camera lenses in their analysis of some maneuvering trials. Both UDI and Seeger used rulers and tape measures rather than more accurate anthropometric instruments designed for such a purpose. In the UDI research, the lack of a reliability study to determine the level of agreement among researchers to rate the level of “accessibility” observed is a key limitation because those ratings were used to determine success in managing specific space clearances. The IDEA and DETR studies are very well documented and reliability studies were completed in the preparation phase.
UDI’s research was focused on validating the B651-95, precursor to the B651-04. All the maneuvering trials started with the clearances in the standard but the results of fitting trials can be influenced by the starting position. In contrast, the IDEA Center research started with the smallest possible clearance. We discovered that many of our participants could manage maneuvers in much smaller spaces than the minimum clearances of the ADAAG.
The reach protocol used by the IDEA Center did not collect data on the lowest reach possible because of safety concerns. Lowest reach was obtained for free lateral reach without bending but this cannot be used to represent the lowest forward or extended lateral reach. It is not known how the UDI researchers addressed the safety concerns related to low reach but it is clear that differences in human subject approval procedures certainly can influence the methodologies used.
DETR, UDI, the BS8300 researchers and IDEA provided results in percentile form. This is very valuable for making decisions about criterion values. Ideally, standard deviations and median values should be provided at a minimum so that all percentiles can be calculated. BS8300 failed to include median values so percentiles other than what they reported cannot be computed. UDI, Bails and the BS8300 researchers did not provide any information on whether outliers were identified and eliminated from the data reported.
User Comments/Questions
Add Comment/Question