Assessment

ferent points and then adding them up to give a score, which may then convert into a grade. It is characteristic of this approach that the categories are weighted, i.e. the categories do not each account for an equal number of points.

Tables 2 and 3 in Chapter 3 provide self-assessment and examiner assessment examples respectively of analytic scales of criteria (i.e. grids) used with a holistic rating strategy (i.e. match what you can deduce from the performance to the definitions, and make a judgement).

9.3.12 Series assessment/category assessment

Category assessment involves a single assessment task (which may well have different phases to generate different discourse as discussed in section 9.2.1.) in which performance is judged in relation to the categories in an assessment grid: the analytic approach outlined in 9.3.11.

Series assessment involves a series of isolated assessment tasks (often roleplays with other learners or the teacher), which are rated with a simple holistic grade on a labelled scale of e.g. 0–3 or 1–4.

A series assessment is one way of coping with the tendency in category assessments for results on one category to affect those on another. At lower levels the emphasis tends to be on task achievement, the aim is to fill out a checklist of what the learner can do on the basis of teacher/learner assessment of actual performances rather than simple impression. At higher levels, tasks may be designed to show particular aspects of proficiency in the performance. Results are reported as a profile.

The scales for different categories of language competence juxtaposed with the text in Chapter 5 offer a source for the development of the criteria for a category assessment. Since assessors can only cope with a small number of categories, compromises have to made in the process. The elaboration of relevant types of communicative activities in section 4.4. and the list of different types of functional competence outlined in section 5.2.3.2 may inform the identification of suitable tasks for a series assessment.

9.3.13 Assessment by others/self-assessment

Assessment by others: judgements by the teacher or examiner. Self-assessment: judgements about your own proficiency.

Learners can be involved in many of the assessment techniques outlined above. Research suggests that provided ‘high stakes’ (e.g. whether or not you will be accepted for a course) are not involved, self-assessment can be an effective complement to tests and teacher assessment. Accuracy in self-assessment is increased (a) when assessment is in relation to clear descriptors defining standards of proficiency and/or (b) when assessment is related to a specific experience. This experience may itself even be a test activity. It is also probably made more accurate when learners receive some training. Such structured self-assessment can achieve correlations to teachers’ assessments and tests equal to the correlation (level of concurrent validation) commonly reported between teachers themselves, between tests and between teacher assessment and tests.

191

Common European Framework of Reference for Languages: learning, teaching, assessment

The main potential for self-assessment, however, is in its use as a tool for motivation and awareness raising: helping learners to appreciate their strengths, recognise their weaknesses and orient their learning more effectively.

Users of the Framework may wish to consider and where appropriate state:

•which of the types of assessment listed above are:

•• more relevant to the needs of the learner in their system

•• more appropriate and feasible in the pedagogic culture of their system

•• more rewarding in terms of teacher development through ‘washback’ effect

•the way in which the assessment of achievement (school-oriented; learning-oriented) and the assessment of proficiency (real world-oriented; outcome-oriented) are balanced and complemented in their system, and the extent to which communicative performance is assessed as well as linguistic knowledge.

•the extent to which the results of learning are assessed in relation to defined standards and criteria (criterion-referencing) and the extent to which grades and evaluations are assigned on the basis of the class a learner is in (norm-referencing).

•the extent to which teachers are:

•• informed about standards (e.g. common descriptors, samples of performance)

•• encouraged to become aware of a range of assessment techniques

•• trained in techniques and interpretation

•the extent to which it is desirable and feasible to develop an integrated approach to continuous assessment of coursework and fixed point assessment in relation to related standards and criteria definitions

•the extent to which it is desirable and feasible to involve learners in self-assessment in relation to defined descriptors of tasks and aspects of proficiency at different levels, and operationalisation of those descriptors in – for example – series assessment

•the relevance of the specifications and scales provided in the Framework to their context, and the way in which they might be complemented or elaborated.

192

Assessment

with a large number of categories and in addition, the full range of levels presented may not be appropriate in the context concerned. Rather, the set of scales is intended as a reference tool.

Whatever approach is being adopted, any practical assessment system needs to reduce the number of possible categories to a feasible number. Received wisdom is that more than 4 or 5 categories starts to cause cognitive overload and that 7 categories is psychologically an upper limit. Thus choices have to be made. In relation to oral assessment, if interaction strategies are considered a qualitative aspect of communication relevant in oral assessment, then the illustrative scales contain 12 qualitative categories relevant to oral assessment:

Turntaking strategies

Co-operating strategies

Asking for clarification

Fluency

Flexibility

Coherence

Thematic development

Precision

Sociolinguistic competence

General range

Vocabulary range

Grammatical accuracy

Vocabulary control

Phonological control

It is obvious that, whilst descriptors on many of these features could possibly be included in a general checklist, 12 categories are far too many for an assessment of any performance. In any practical approach, therefore, such a list of categories would be approached selectively. Features need to be combined, renamed and reduced into a smaller set of assessment criteria appropriate to the needs of the learners concerned, to the requirements of the assessment task concerned and to the style of the pedagogic culture concerned. The resultant criteria might be equally weighted, or alternatively certain factors considered more crucial to the task at hand might be more heavily weighted.

The following four examples show ways in which this can be done. The first three examples are brief notes on the way categories are used as test criteria in existing assessment approaches. The fourth example shows how descriptors in scales in the Framework were merged and reformulated in order to provide an assessment grid for a particular purpose on a particular occasion.

193

Common European Framework of Reference for Languages: learning, teaching, assessment

Example 1:

Cambridge Certificate in Advanced English (CAE), Paper 5: Criteria for Assessment (1991)

Test criteria	Illustrative scales	Other categories
Fluency	Fluency
Accuracy and range	General range
	Vocabulary range
	Grammatical accuracy
	Vocabulary control
Pronunciation	Phonological control
Task achievement	Coherence	Task success
	Sociolinguistic appropriacy	Need for interlocutor support
Interactive communication	Turntaking strategies	Extent and ease of maintaining
	Co-operative strategies	contribution
	Thematic development

Note on other categories: In the illustrative scales, statements about task success are found in relation to the kind of activity concerned under Communicative Activities. Extent and ease of contribution is included under Fluency in those scales. An attempt to write and calibrate descriptors on Need for Interlocutor Support to include in the illustrative set of scales was unsuccessful.

Example 2:

International Certificate Conference (ICC): Certificate in English for Business Purposes, Test 2: Business Conversation (1987)

Test criteria		Illustrative scales	Other categories
Scale 1	(not named)	Sociolinguistic appropriacy	Task success
		Grammatical accuracy
		Vocabulary control
Scale 2	(Use of discourse	Turntaking strategies
features to initiate and		Co-operative strategies
maintain flow of		Sociolinguistic appropriacy
conversation)

194

Assessment

Example 3:

Eurocentres – Small Group Interaction Assessment (RADIO) (1987)

Test criteria	Illustrative scales	Other categories
Range	General range
	Vocabulary range
Accuracy	Grammatical accuracy
	Vocabulary control
	Socio-linguistic appropriacy
Delivery	Fluency
	Phonological control
Interaction	Turntaking strategies
	Co-operating strategies

Example 4:

Swiss National Research Council: Assessment of Video Performances

Context: The illustrative descriptors were scaled in a research project in Switzerland as explained in Appendix A. At the conclusion of the research project, teachers who had participated were invited to a conference to present the results and to launch experimentation in Switzerland with the European Language Portfolio. At the conference, two of the subjects of discussion were (a) the need to relate continuous assessment and selfassessment checklists to an overall framework, and (b) the ways in which the descriptors scaled in the project could be exploited in different ways in assessment. As part of this process of discussion, videos of some of the learners in the survey were rated onto the assessment grid presented as Table 3 in Chapter 3. It presents a selection from the illustrative descriptors in a merged, edited form.

Test criteria	Illustrative scales	Other categories
Range	General range
	Vocabulary range
Accuracy	Grammatical accuracy
	Vocabulary control
Fluency	Fluency
Interaction	Global interaction
	Turntaking
	Co-operating
Coherence	Coherence

195

Common European Framework of Reference for Languages: learning, teaching, assessment

Different systems with different learners in different contexts simplify, select and combine features in different ways for different kinds of assessment. Indeed rather than being too long, the list of 12 categories is probably unable to accommodate all the variants people choose, and would need to be expanded to be fully comprehensive.

Users of the Framework may wish to consider and where appropriate state:

•the way in which theoretical categories are simplified into operational approaches in their system;

•the extent to which the main factors used as assessment criteria in their system can be situated in the set of categories introduced in Chapter 5 for which sample scales are provided in the Appendix, given further local elaboration to take account of specific domains of use.

196