RESEARCH into Allen Cognitive Levels - Taken from Cognition Disabilities Model:  How to Make Clinical Judgments by Claudia K. Allen, MA, OTR/L, FAOTA, and Tina Blue, OTR/L, (1998).  In the publication ‘Cognition and Occupation in Rehabilitation: Cognitive Models for Intervention in Occupational Therapy’ by Noomi Katz, PhD, OTR (Editor)

The credibility studies of the components of the Allen Battery were started in the early 1970’s, with the many studies (published and un-published) undertaken on several versions of the screening tools (ACLS). Most of the quantitative studies have used correlations to establish the strength of the association between measures. The strongest correlation is reported as r = + or -1.0.  For reliability, the r number should be higher than + or -.80.  The r for validity is usually lower, between .30 and .70.  A small p represents the statistical significance, with the customary minimum at .05, meaning that the result may be incorrect 5 out of 100 times. Lower p numbers are better reducing the number of incorrect results.  N is the number in the sample; larger Ns increase the confidence in the study.

Inter-rater reliability has always been high.  The original six cognitive levels were used in the first studies that found nearly perfect inter-rater reliability (r - .99, N = 32, range of levels 2-6).  At that time the ACLS was used to place clients in a groups that matched their score; the predictive validity was r =.76, N = 23 (Moore, 1978).  Newman (1989 examined the next version of the ACLS and found a percentage of agreement between two raters of 95.2% (N = 21). Test-retest reliability for New­man's sample of chronic schizophrenic patients was r = .75 (N = 22, p. 01). Partida (1992) looked at the inter-rater reliability of the large and small ACL with a small sample (N = 4) and two raters, finding perfect reliability. Howell (1993) found similar inter-rater reliability with the most recent version of the small ACLS (r = .91, p<.0001, N = 20). Eight raters trained by Penny, Musser, and North (1995) achieved an impressive r = .98. The first step in establishing a credible use of the Allen Battery is to be sure that the rating of the screening tool is accurate. These studies suggest that thera­pists and their students should be able to establish accurate inter-rater reliability.

Kehrberg (1992) reported significant correla­tions between the large leather lacing and the small one, with higher associates with the senile dementia population (r = .95, p<.001) than the control sub­jects (r = .58, p<.001). The client population was more impaired when gender and test order were controlled. With both sizes, both the dementia population and the controls over the age of 75 did not do as well as those under 75.

Validity is the extent to which a test measures what it says it measures, in this case cognitive processes, global ability to function₎ and severity of a disability. The ACLS has been correlated with well-known instruments commonly used with a variety of diagnostic categories to check the validity of the scale. The most widely used cognitive mea­sure is the Wechsler Adult Intelligence Scale (WAIS), which is often reported as verbal, perfor­mance, and full-scale intelligence (IQ). Katz (1979) found a moderate correlation between the Block Design and the ACI.S ( r = .45, p<.00I ). Mayer (1988) used all of file subtests of the WAIS to clarify the type of information processing that is related to adaptation. The strongest correlations were between the ACLS and the Block Design and Object Assembly (r .⁷/29, p<.0001). Performance IQ also showed a high correlation with the ACLS (r - .55, p<.0003). The most enlightening use of the WAIS was to divide the test into crystallized and fluid abilities. Fluid abilities "subsume information processing functions such as attention, perception, flexibility, and problem-solving. Crystallized abili­ties are dependent on previous training, education, and acculturation (e.g., vocabulary)" (Mayer, 1988, p. 176). The correlations with fluid abilities gave credence to the notion that a pattern of perfor­mance in adapting to a changing environment was being tested. No significant correlations were found with vocabulary, arithmetic, or picture com­pletion, which are part of crystallized intelligence that is influenced by cultural background. This has been further supported by a lack of significant correlations with age, sex, education, occupation, work history, or socioeconomic status in several disabled populations. While these demographic factors do influence intelligence in normal popula­tions, the benefits of cultural experiences do not overcome difficulties in processing the information necessary for planning and problem-solving with new information (Alsberg, 1987; Averbuch & Katz, 1988; Breeding, 1993; Camp & Person, 1987; David & Riley, 1990; Gokey, 1986; Heimann, Allen, & Yerxa, 1989; Herzig, 1978; Heying, 1983; Howell, 1993; Josman & Katz, 1991; Kaeser, 1992; Katz, 1979; Katz, Josman, & Steinmetz, 1988; Mayer, 1988; Moore, 1978; Newman, 1987; Partida, 1992; Penny, Musser, & North, 1995; Richard,^,;, 1983; Shapiro, 1992; Skinner, Denton, & Levy, 1989; Wil­liams, 1981; Wilson, Allen, M cCormack, & Bur­ton, 1989).

Shapiro's (1992) finding that leather lacing did not correlate with the perceptual Memory "Task (PMT), but did correlate with Visual Motor Inte­gration (VMI), was surprising. A deeper look at underlying mental processes was required. The in­put of PMT is similar to the Block Design of the WAIS, using colored blocks and increasingly com­plex designs printed on cards. With the PMT, sub­jects are given cards for 10 seconds, and then the designs are removed. To match or recognize a pattern, subjects must store an image of the design. The formation and storage of images are not re­quired with leather lacing. With leather lacing peo­ple are given examples of stitches to copy or match. With the VMI, subjects arc also given geometric patterns to copy. Shapiro's study suggests a need to look beyond the input of visual and auditory cues and consider the underlying mental processes that must be applied to the cues.

The relationship between leather lacing and hand dexterity has been investigated, using the Purdue Pegboard, without a significant correlation as expected. With a sample of depressed patients the time to complete the ACL was used to success­fully remove the ceiling effect on the ACLS, which has been a problem with other studies of depres­sion (Carmel, Katz, & Modal, 1996). When the major problem is slowness, or loss of energy, tim­ing the ACLS may provide objective data.

The relationship between leather lacing and ver­bal abilities has also been studied. David and Riley (1990) found modest correlations between the ACLS and the Shipley Institute of Living Scale, which is a paper-and-pencil test for choosing syn­onyms and writing responses to a sequence of terms. With N = 57, modest correlations were found with vocabulary, r = .25, p<.02; abstraction, r =.35, p<.001; and IQ, r =- .31, p<.005. A stronger correlation was found with the Symbol Digit Mo­dalities Test, which times the translation of novel geometric shapes into written responses (r = .52, p<.001). As suspected, leather lacing is more re­lated to novel learning of perceptual motor tasks than to abstract reasoning and verbal abilities. A similar modest correlation was found between leather lacing and the Social Interaction Test (SIT) (Penny, Musser, & North₎ 1995). The total SIT correlation was r =--.32, p<.01; nonverbal r = -.27, p<.03; conversation r = ---.32, p<.01; voice quality was not significant, r -.16. A low ACLS score was associated with reduced social competen­cies. The scales go the opposite directions, which explains the expected negative correlations. While there is a relationship between what people say and what people do, the relationship is modest.

In an effort to examine the cognitive abilities that emerge during Piaget's period of concrete op­erations, the Riska Object Classification (ROC) was developed with interrater reliability at r =.83, p<.01. The structured part of this test required too great a leap in cognitive abilities and has not produced significant correlations. The spontane­ous portion has been more productive, with a sig­nificant association with leather lacing, education, occupation, and social position in a non-disabled population (Williams, 1981). Wilson (1985) also found a significant correlation between the sponta­neous portion of the ROC and leather lacing (r = .66, p<.001) and the Mini-Mental Status Exam in her population with senile dementia (r = .90, p<.001). In a study of depressed people, Katz (1979) found significant correlations with leather lacing (r = .42, p<.001) and the Block Design Por­tion of the WAIS (r = .60, p<.0001). The results of the early studies with the ROC were uneven, suggesting that the developmental continuum needed to be replaced with a different way of organ­izing our understanding of cognitive abilities. Katz began to work on this by developing a series of class inclusion questions that were found to differentiate significantly between psychiatric patients and nor­mal controls (Katz, Josman, & Steinmetz, 1988; Tolchinsky-Landsmann & Katz, 1988). The ROC is also included as part of the Loewenstein Occupa­tional Therapy Cognitive Assessment (Katz, Itz­kovich, Averbuch, & Elazar, 1989; (Cermak, Katz, McGuire, Greenbaum, Peralta, & Maser-Flanagan, 1995). The convenient explanation of regression to lower levels of development, in a sequence taken from normal children, was not working. The differ­ences between the way disabled people think, as distinguished from normal children and adults, needed clearer explanations.

Some differences between a psychological view of intelligence and an occupational therapy view of function are emerging from these studies. Psy­chological tests of intelligence tend to include words, concepts, paper-and-pencil tasks, and even when visual cues are used they must be processed and stored in a sequential manner within a timed period. As a whole, psychological tests have a strong tradition in favouring the left-hemisphere abilities. Leather lacing, crafts, and ADL is probably favour the right hemisphere's contribution to visual spatial abilities. Visual spatial cues are demon­strated and copied while doing an activity. Learn­ing may be instantaneous or acquired through practice in doing the activity. Right-hemisphere processes prepare people for immediate action, containing spontaneity not seen in left-hemisphere processes. Shapiro's (1992) discussion of different responses to the tests she used provides insight into these differences. Because both hemispheres are important, an effort has been made to include both types of ability in the description of the modes to provide a global view of ability to function (Allen, Earhart, & Blue, 1992).

The idea that the scale measured the severity of a disability was first investigated with the Brief Psychiatric Rating Scale; the concurrent validity with the ACLS was r == .53 at admission and r = .43 at discharge (Moore, 1978).  In another sample with a schizophrenic population, Newman (1987) found a correlation with the Global Assessment Scale (r = .46, p<.01, N - 34). Differences between schizophrenic, depressed, and control populations were found in California, and replicated in Israel (Katz, 1979; Katz & Heimann₎ 1990; Williams, 1981).  Heying (1983) expanded the investigation of the ACLS to senile dementia, finding a correla­tion with the Mini-Mental Status Exam (r = .66, p<.001, N = 33). The results were replicated by Wilson (1985) with N = 20 and r = .59. The Cogni­tive Performance Test has been used to predict institutionalisation in low-functioning patients and trace the functional decline and death in years for people with senile dementia (Burns, 1992). At the other end of the age range, emotionally dis­turbed boys showed decreased ability to do leather lacing in a school in New York (N = 24). A moder­ate correlation with Developmental Test for Visuomotor Impairment (r = .44, p<.04) was found on a test designed to screen children for neurological impairments and learning disabilities (Shapiro, 1992). Breeding (1993) found a difference between age groups in a study of normal six- to nine-year­ olds from a school with parents in high socio-economic status (N - 84). The ACLS has been used to differentiate between adolescents, 12- to 18­vears-old, with psychiatric disorders (N -= 49) and controls (N = 29) matched for age, sex, education, and place of residence (Katz, Josman, & Steinmetz, 1988; Josman & Katz, 1991). In non-disabled adults 18 to 65, Williams (1981) found a positive correlation between ACLS and social, position based on education, social and occupational status as determined by the Hollinghead's Two-Factor Index of Social Position.  Cultural factors influence cognitive ability in normal populations but have not had an impact on the disabled scores. The cognitive levels do correlate with well-known measures of the se­verity of a variety of mental disorders.

The relationship between activities that therapists typically use in practice and leather lacing has also been examined. Newman ( 1987) found a correlation (r = .63, p<.01) with the Task Oriented Assessment of the Bay Area Assessment of Func­tional Performance. The study of activities of daily living was started by Heying's (1983) finding of a high correlation (r = 82, p<.001 ) with the Physical Self Maintenance Scale and Instrumental Activities of Daily Living Scale presented by Lawton and Brody (1969). Lawton and Brody activities of daily living were modified and turned into a quantitative measure by Heimann (1985), in the form of the Routine Task Inventory (RTI). Heimann studied psychiatric outpatients, finding an ACLS correla­tion with the total RTI of r =.64, p<.001 (Heimann, Allen, & Yerxa, 1989). The test-retest reliability of the RTI was established at r = .99, p<.0001 and inter-rater reliability was established at r = .99, p<.001 (Heimann, 1985). Wilson (1985) found similar test re-test reliability after a 2-week interval (r = .99, p<.000 1). Wilson described a community ­based sample of people with senile dementia, find­ing an ACLS correlation with the RTI similar to Heimann's at r = .56, p<.01 (Wilson, Allen, Mc­Cormack, & Burton, 1989). Gokey (1987) found a modest correlation between leather lacing and the RTI (r = .44, p<.006). The strength of the correlation between leather lacing and activities of daily living was given an in depth investigation by Gokey (1987), Heimann (1985), and Wilson (1985), leading to the suggestion that eating is one level lower than all other activities. Gokey (1987) found that the RTI had a stronger correlation with working than leather lacing did in a sample of schizophrenic patients. In a comparison of the R TI correlations with leather lacing and classification abilities, a distinction between motor and verbal abilities began to emerge (Gokey, 1987; Heimann, 1985; Wilson, 1985). The safety concerns implicit in the RTI received their first explicit investigation by Alsberg (1987) finding an association between leather lacing and the errors made while making macaroni and cheese in a sample functioning at ACL. 4 and 5. Burns (1992) translated six tasks from the RTI into a Cognitive Performance Task, finding a correlation with the Mini-Mental Status Exam (r = .67, N 77). At four weeks with the ('1¹T, inter-rater reliability was r = .91 (N - 18) and test-retest was r = .89 (N = 36). The strength of the validity correlations was not great, suggesting that prior experience, motivation, social situations and underlying mental processes can have impor­tant influences on ADL.

The investigation of craft activities was initiated by Kaeser (1992), showing improved performance when activities are matched to the cognitive level.  The sample was elderly persons with dementia doing tiling tasks. Subjects at cognitive level 3 per­formed better on level 3 tiling activities than on a level 4 tiling activities (F(1,14) = 125, p<.001). As expected there was significance difference in the way the level 4 subjects performed on the level 3 and 4 activities.

The first controlled research investigation of treatment effectiveness has been completed in Israel with two groups of schizophrenics in post acute care. The research group (N = 11 )was given activities from the ADM that matched their ability to function and probed for higher abilities. The control group (N = 8) was in a sheltered workshop with tasks given according to the work to be done. Both groups showed a significant improvement on the RTI (z = 2.80, p<.005 for research; z = 2.52, p<.01 for control), but the research group showed a higher gain. The research group showed a sig­nificant improvement on the ACLS (z = 2.52. p<.01) but there was no significant improvement in the control group (z = .13). Although the sample size is very small, the study supports the idea that greater gains may he made when the therapist's treatment methods match the capacities of the cli­ent (Raweh, 1996).

Since 1998, there have been further studies undertaken. 

These studies include works by:

·               Dr I. Lazzarini and C. Darby, OTR/L., Saint Antonio Hospital, Texas, (2002)

·              Project I. AOTF-funded collaborative study by Alexis D. Henry, ScD, OTR/L, FAOTA, Principle Investigator, and Deane B. McCraith, MS, OTR/L, Co-investigator; Project II, completed in partial fulfillment of requirements for the Doctor of Education by Linda W. Duncombe, Ed.D, OTR/L, FAOTA --New Evidence Validates and Supports Use of the Allen Model (2002)

·               Wilhelmina J. Lima, Julie A. Dopheide, Barry A. Kramer, Catherine A. Earhart and Michael Z. Wincor, (1999).  A naturalistic comparison of adverse effects between slow titration and loading of divalproex sodium in psychiatric patients.

·                Sevier S and Gorek B. Cognitive evaluation in care planning for people with Alzheimer disease and related dementias. Geriatric Nursing: Vol. 21, 2000.

·                Trafton E. The Adapted Work Program and its application in VA hospitals and other facilities; An interview with Theressa Burns: Activities Directors’ Quarterly: American Journal of Alzheimer’s Disease and Other Dementias: Vol. 3, 2002.

Allen Cognitive Advisors Symposium - Research in Cognition: Advances in Dynamic Systems Theories, Effects on Brain Science and these influences on the Allen Cognitive Levels

The following provides the details of the research abstracts that were presented at the Allen Cognitive Advisors Symposium on Cognition 2003 held at the University of Southern California, Tampa, Florida

·           Henry, A.D. and McCraith, D. B. (2003). The Usefulness of the Cognitive Disabilities Model in Predicting Community Functioning among Persons  with Mental Illness.

·           Lazzarini, I. (2003).  Neuro-occupation:  The Non-linear Dynamics of Intention, Perception and Meaning.  American Journal of Occupational Therapy. (In process of publication).

·          Lazzarini, I., Daby, C. and Mu, K. (2003).  Current Validity of the Allen Diagnostic Module in a Psychiatric Population.

·          Wotton, G., Champagne, T., Schweitzer-Hendriks, A., Evans, P., Young, G. (2003). Comparison of the Allen 4.6 level and The Developmental Test of Visual Perception-Adolescent and Adult (DTVP-A) in a Psychiatric Population.