Abstract
Study Design
Descriptive.
Introduction
The Hamilton Inventory for Complex Regional Pain Syndrome (HI-CRPS) is a multidisciplinary assessment tool under development.
Purpose of the Study
This study examined the assessment practices, beliefs and preferences of health care professionals working with CRPS to inform the content and structure of the clinician-based portion of the HI-CRPS (CB-HI-CRPS), as well as refine the user manual.
Methods
Semi-structured cognitive interviews were conducted with health care professionals from a spectrum of disciplines working with CRPS. Assessment practices and scaling preferences for 15 assessment concepts were collected, relating directly to items on the CB-HI-CRPS. Interviews were transcribed and coded with emergent themes.
Results
Participants reported using the concepts from the CB-HI-CRPS 85.2% of the time. Physicians and nurses preferred present/absent judgements, while therapists used none/mild/moderate/severe scaling. Emerging themes highlighted assessment values, beliefs about CRPS, professional roles, and knowledge translation.
Conclusions
Lack of uniformity in terminology and assessment behaviours underscores the need for clear scoring frameworks and standardized assessment instructions to improve reliability across the proposed users of the HI-CRPS.
Level of Evidence
Level 4.
Complex regional pain syndrome (CRPS) is a perplexing neurological condition that commonly arises after a traumatic injury, and can be associated with a peripheral nerve injury.
1
, 2
, 3
De Mos et al.4
calculated the incidence of CRPS at 26.2 per 100,000 person years in a retrospective cohort in the Netherlands; CRPS is thought to affect from 5% to 40% of patients after upper extremity injuries or surgeries, and becomes a chronic condition in just under 2% of these patients.5
, 6
The presentation of the syndrome is variable; usually affecting a single limb, but occasionally seen to spread bilaterally or hemispherically.
1
The symptoms are variable: while most patients report some form of burning pain, they may also have swelling, circulatory changes, skin changes, sensory complaints, stiffness, and altered movement patterns.7
, 8
CRPS is often divided into two subtypes (CRPSI and CRPSII), distinguished by the presence of a known nerve injury in CRPSII.2
Although consensus-based diagnostic criteria
1
and assessment recommendations exist,9
there is as yet no gold standard for diagnosis.2
, 3
, 10
Despite this, there have been attempts to quantify some of the symptoms associated with CRPS.11
, 12
A recent systematic review of outcome measures for CRPS13
highlights that most tools have focused on specific symptoms and have limited validation; no comprehensive CRPS scale has been accepted into practice or research. Preliminary work has been undertaken to develop a condition-specific outcome measure, the Hamilton Inventory for Complex Regional Pain Syndrome (HI-CRPS).Previous Development of the HI-CRPS
Development of the HI-CRPS was undertaken with the goal of providing clinicians and patients with a condition-specific tool for outcome assessment of both CRPSI and CRPSII affecting any limb. A foundational literature review found 96 different descriptors or evaluation points: these concepts were formulated into signs for clinician measurement (20 items) and symptoms for patient self-report (SR) (45 questions) and reviewed by patients and experts for clarity, face, and content validity. This was combined with item-total correlation data from a small pilot study to remove redundant items (T. Packham, unpublished data, 2007), resulting in the current prototype (16 multidisciplinary items for clinician-based [CB] assessment, and a 35-item patient SR). This study focuses on the CB portion of the assessment tool (the CB-HI-CRPS). While still under development, the patient SR portion of the assessment tool addresses symptoms, functional concerns, and the social and emotional impact of the syndrome, and is intended to compliment the objective assessment components of the CB portion within a biopsychosocial paradigm. It is perhaps important to note that the work described herein represents a single facet of psychometric evaluation in the evolutionary tool development process.
Cognitive Interviewing
Cognitive interviewing is a qualitative method that can be used to examine how participants interpret and respond to survey questions.
14
, 15
The debriefing data can help survey developers to not only discover errors made by respondents from the target population, but also to identify where those errors arise in the response process, thus facilitating item revision and the development of new items for SR assessments.16
, Willis G, Caspar R, Lessler J. Cognitive Interviewing: a how-to guide. 1999. Available at: http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf. Last accessed October 7, 2008.
17
Traditionally, qualitative studies seeking to refine CB assessment tools have used a Delphi or other consensus method for validation of the content coverage,18
, 19
but these do not generally allow for an understanding of the sources of difference. This mixed-methods study used a novel application of cognitive debriefing, where those interviewed were the potential future users of a CB assessment tool (the HI-CRPS), rather than the patients completing a SR questionnaire.Improving Observational and Direct Measures: Do You See What I See?
Reliability can be described as a measure of the reproducibility, agreement, or degree of error inherent in any measurement or measurement tool.
19
Inherent in classical measurement theory is the principle that every clinical measure is composed of the true score and a component of measurement error. Error can arise during CB assessment because of a lack of clarity or consistency in how a measure is conceptualized/interpreted or applied;20
others may not see what we see simply because they are looking elsewhere. For this reason, understanding the cognitive underpinnings of how clinicians interpret measures is a potential basis for enhancing reliability. Others have focused on how reliability of an assessment tool can be improved by reducing the amount of errors through standardization, training of assessors, and using precise rating scales.19
, 21
Few studies have used qualitative approaches to understanding measurement error.Validity focuses on the extent that a tool can measure what is purported to measure and thus encompasses multifaceted analyses,
22
including face validity (the overt ways that the tool reflects concepts that the reader or user feels are important) and content validity (the degree to which the instrument includes the spectrum of assessment concepts). Measurement tools can be used to describe, predict, and evaluate the concepts of interest.23
It is important for instruments to clearly define which of these purposes are being undertaken when a tool is developed; and to test whether it can perform these different measurement functions.Purpose of the Study
This study examined the assessment practices, beliefs, and preferences of the proposed users of the CB-HI-CRPS with the intent of improving the reliability, face validity, and content validity for future testing. The specific areas of exploration were intended to define:
- 1.User-generated definitions and descriptions for assessment techniques and scale anchors,
- 2.Similarities and differences in how health care professionals (HCPs) from different disciplines assess and formulate judgments on the clinical signs of CRPS, and
- 3.Implications for altering the measurement properties of a CRPS prototype tool with the aim of reducing possible sources of user error, and ultimately potentially improving reliability, content, and face validity.
Materials and Methods
Present Study Overview
This study consisted of a series of semistructured cognitive debriefing interviews conducted by a team of researchers from McMaster University. Interview content was intended to reflect the key assessment concepts included in the CB-HI-CRPS; see Table 1 for a listing of the concepts. Participants were asked to provide definitions for the concepts covered by the assessment (e.g., allodynia and guarding) and to provide detailed description for the end anchors of the scales, or for individual scale points. Opportunity was also given for participants to state their preferences for different formats of scales that could be used for ranking the individual constructs. Although a working draft of the CB-HI-CRPS tool was provided to participants for reference, the focus was content rather than the format. Although cognitive debriefing typically is used for the refinement of patient SR measures,
15
, 16
, Willis G, Caspar R, Lessler J. Cognitive Interviewing: a how-to guide. 1999. Available at: http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf. Last accessed October 7, 2008.
17
this study undertook identification of potential sources of consensus and difference in clinical assessment strategies across the professional groups, and to use this information for tool refinements that should strengthen statistical reliability and correspondingly, validity.Table 1Key Assessment Concepts of the CB-HI-CRPS
| Categories | Concepts |
|---|---|
| Pain | Allodynia |
| Cold intolerance | |
| Hyperpathia—pinprick | |
| Guarding | |
| Autonomic | Skin temperature |
| Mottling | |
| Hyperhydrosis | |
| Edema | |
| Trophic | Hair growth |
| Skin quality | |
| Nail quality | |
| Motor | Muscle tone |
| Movement given time since initial injury | |
| Incoordination | |
| Movement given severity of initial injury | |
As part of the cognitive debriefing interview, participants were asked to describe their current assessment practices. The descriptive methodology of contemporary content analysis
24
was then used to garner insight into how individual clinicians have addressed the clinical challenge of CRPS assessment.Participants and Research Team
Professionals with an interest in or experience with the treatment of CRPS were purposively selected to reflect the anticipated future users of the CB-HI-CRPS from six different professions commonly involved in the care of persons with CRPS: anesthesia, occupational therapy, orthopedic surgery, plastic surgery, physiatry, and physiotherapy. The goal of including multiple professions was to gain perspective on the scope of professional variations
25
and ensure that the language of the tool was clear and inclusive for all disciplines.The research team for this project consisted of an occupational therapist (OT) (TP: author of the HI-CRPS) and two occupational therapy students from McMaster University. One or two members of the team conducted each interview after a joint initial practice interview. The sessions were audio-recorded, then transcribed verbatim for analysis. Reflecting the mixed-methods design, the analyses had two forms: 1) quantitative content analysis of demographics, assessment practices, and scaling preferences using an item-by-item format, and 2) descriptive thematic analysis and coding of the transcripts. The research team met for coding and consensus sessions four times over a one-year period.
This study received approval from the joint Research Ethics Board of McMaster University and Hamilton Health Sciences in Hamilton, Ontario.
Procedures
Sampling
The intended sampling strategy was to include at least three representatives of each of the target professional groups. Invitations to participate in the study were sent to members of a multidisciplinary clinical and academic network (MacHANd: McMaster Hand, Arm and Nerve), and therapist interest groups (Hamilton Hand Interest Group and London Hand Interest Group). Interviews were conducted at a place of the interviewee’s choosing: ranging from offices to clinics to cafes. In the one-year study period, interviews were conducted with 20 clinicians; however, two interviews were lost to technology failures (degradation of audio-cassettes), leaving 18 transcripts for review. Unfortunately, despite multiple requests only a single orthopedic surgeon responded and provided consent to participate; subsequently this interview consent was withdrawn because the surgeon felt they could not contribute the amount of time required for the interview (45 minutes–one hour). Using a snowballing technique, those who participated were given the opportunity to recommend other professionals whom they knew to be interested in the area of CRPS; these individuals were then also invited to participate in an interview. Although registered nurses (RNs) had not originally…etc been included as a target group, other professionals recommended nursing colleagues as part of the “snowballing,” and these RNs were accordingly incorporated into the sampling frame. No volunteers who gave consent to participate in the full interview process were excluded.
We were unable to meet initial multidisciplinary recruitment targets because of the lost data and imbalanced recruitment: only two transcripts were available from each of the plastics, physiatry, and physiotherapy professions (see Table 2 for participant demographics). However, six OTs volunteered for the study and were interviewed. Five of the six OT participants each represented a distinct geographic or practice area, and were included by the research team because of the breadth they contributed to the overall study. The two OTs included from the same facility represented very different levels of experience and training: again, the team felt this reflected the spectrum of future users of the CB-HI-CRPS.
Table 2Description of Participants
| Mean | SD | Range | |
|---|---|---|---|
| Years of experience | 14.9 | 9.9 | 3–35 |
| Percentage of practice in CRPS | 10.6 | 7.0 | 2–20 |
| Self-rated expertise in CRPS (0–10) | 7.5 | 1.9 | 5–10 |
| Self-rated expertise in assessment skills (0–10) | 8.1 | 1.7 | 5–10 |
| Profession | Anesthesia=3, OT=6, PMR=3, PT=2, Plastics=3, RN=3 | ||
| Practice area | Pediatrics=2, Pain=4, Hands=8, General=6 | ||
CRPS=complex regional pain syndrome; OT=occupational therapist; PMR=physiatrist; PT=physical therapist; RN=registered nurse.
Several potential biases are acknowledged: 1) a high number of participants specialized in upper extremity practice, and 2) the interviews were limited geographically within the area of Southern Ontario for ease of conducting face-to-face interviews. It is also important to note that the study was closed after a one-year period when leads on new participants had been seemingly exhausted, rather than because data saturation or informational redundancy
9
had been achieved. New themes continued to emerge throughout the interview and review process, and this study cannot be interpreted as an exhaustive review of the rich scope of ideas generated from this examination.Data Collection
After giving informed consent, participants underwent a “verbal probing” format of cognitive interview. Using a combination of established questions and responsive probing, subjects were asked to describe their assessment practices, define concepts, and indicate scale preferences related to each item (refer to Table 3 for sample questions). The face-to-face interviews, typically about an hour long, were audio-recorded then transcribed for analysis. Order bias can arise when respondents modify their responses based on 1) how they have previously answered other questions, 2) fatigue, or 3) boredom,
25
, 26
so the 15 items for discussion were presented in a random order to each participant.Table 3Sample Questions from Cognitive Interview Structure
| Guarding behavior |
| What do you think this question (from CB-HI-CRPS) is asking you to assess? |
| Which scale would you prefer to categorize guarding behavior in your patients? |
| Why? |
| What cues would you use to pick a response category? |
| How confident are you in making a judgment about this characteristic? |
| Hypersensitivity—allodynia |
| How do you define allodynia? |
| How do you assess allodynia in your patients? |
| Which scale would you prefer to assess allodynia in your patients? Why? |
| How would you characterize the anchors of the scale (mild allodynia, severe allodynia, etc)? Can you describe a patient to me that you think would fit this category? |
| How hard is it to judge allodynia? |
CB-HI-CRPS=clinician-based portion of the Hamilton Inventory for Complex Regional Pain Syndrome.
After the first three interviews were completed and transcribed, the team reviewed the transcripts individually and jointly to generate themes and to modify the interview format to reflect emergent concerns. More themes were added as the interviews continued, with subsequent recoding of previously completed interview transcripts. Each of the joint sessions included time for reflection and discussion of how each individual member of the research team guided the interviews with their questions, and responded to the opinions and emotions of the participants.
Several issues that emerged from the reflection sections were team perceptions around experience and relationships. Student members of the research team felt their limited clinical experience constrained the interviews, and they tended to follow the question framework closely. However, as the study progressed and they became more familiar with the subject matter, they became more confident in pursuing alternate lines of questioning when participants’ responses indicated an opportunity to explore an alternate area. The interviews conducted by the students were generally richer in descriptive detail as participants seemed to assume a teaching role. Conversely, the clinical researcher (TP) was sometimes seen as an expert, and participants would ask questions and seek opinions on the basis of the perceived expertise. These interviews tended to be more broad ranging in scope, and raised global issues such as 1) timing and patterns of referrals within the health care system, 2) shifting theoretical frameworks for treatment, and 3) issues related to research, including development and use of outcome measures.
Analysis Plan
The coding system was developed after each reviewer listened to the first three interviews to generate some overall categories; more categories were added as they emerged. Working independently to reduce bias
27
each member of the research team identified categories within each anonymous transcript using a color coding system; this coding was then reviewed by the team and discussed until consensus was reached. Using a cross-case analytical approach,28
data from the interviews were then transcribed into Microsoft Excel in an item-by-item format, with respondents identified only by discipline. This second coding focused more on content-specific analysis that relates to the individual items and measurement concepts, whereas the initial thematic analysis highlighted the similarities and differences among the disciplines related to the process of making clinical judgments. Contemplation on each individual category generated the subcategories as the team attempted to reflect the scope of each meaning within each grouping. Subsequent reflections and discussions by the research team then contracted the categories into the four overarching themes. This level of analysis is more in-depth than the tradition of cognitive debriefing16
, Willis G, Caspar R, Lessler J. Cognitive Interviewing: a how-to guide. 1999. Available at: http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf. Last accessed October 7, 2008.
17
yet was in keeping with the overall study plan of not only refining the CB-HI-CRPS tool, but also understanding the collective wisdom of HCPs relating to outcome measurement for this condition.Results
Descriptive Content Analysis Findings
Although we do not presume to have exhausted the potential of themes arising from cognitive debriefing on this topic, four overarching themes emerged from these interviews:
- 1.Assessment beliefs and values;
- 2.Professional roles and multidisciplinary functions;
- 3.Beliefs about CRPS; and
- 4.Knowledge translation.
The four themes arose from ten content categories; these are presented with related qualifiers, and quotes reflecting their essence are listed in Table 4. The remainder of this section will define each theme and expand on each facet of the categories.
Table 4Qualitative Themes Identified as Arising from Interviews
| Themes | ||
|---|---|---|
| Categories | Qualifiers | Illustrative Quotes |
| Assessment beliefs and practices | ||
| The role of experience in assessment | Necessary for competence and competance | “I have seen beads of sweat on their hand and they are doing nothing – they are sitting! … It’s rare- but it does happen. But I’ve been in practice for 27 years.” |
| “I work with CRPS patients but not that many I think this [confidence] would come with more experience. I think this [guarding behavior] is sometimes hard to discriminate this between the increased muscle tone.” | ||
| The role of comparing to unaffected limb | Integral to the assessment process | “I’ll note it that there is something different, that it is a true difference within that individual compared to the other side” [in reference to nail quality] |
| “I look for it [hyperhydrosis] and then I feel. So if they’re like ‘this palm is always sweaty,’ then I feel the two palms.” | ||
| The need for observation, patient report, and direct measurement | Varies at different points in the assessment process | “…usually when you first start with somebody, it’s just going to be through questioning ‘do you have pain,’ ‘where is it and how much is it,’ ‘does it hurt when you touch things,’ before I even do any hands on assessment.” |
| Correlating patient reports and direct measurements | “When I am doing ROM you can almost feel the hand get cold and clammy a lot of the time or change in terms of temperature… then I will ask them as well, how does it feel? Does it feel cold, does it hot, to make sure that we correlate.” | |
| Finding a balance between subjective and objective measures | “…less subjective from the patient and more objective from us would be a huge benefit.” | |
| “…I think it is ok to use the descriptive factors, but I need some kind of objective measurement to back up my observations.” | ||
| The perceived value of direct measurements | “If you are going to do ROM, then you have got to measure it. If you are not going to measure it, then you have to have some standardized functional task….” | |
| Professional roles and functions | ||
| Diagnostic evaluation vs. outcome evaluation | Both perspectives are useful | “Unfortunately with CRPS, and sometimes it’s [hair growth] very subtle, it doesn’t mean that it’s mild or mild disease. It just means there are mild changes …usually by the time they’re having changes that are more severe, they’re also having a huge decrease in function. Whereas my approach to CRPS is more if I think it’s present, I treat it aggressively. And all I’m looking for is evidence of CRPS.” |
| “The present or absent is simpler, and maybe better for diagnosing, but in terms of looking at change over time…it depends what you want the tool for.” | ||
| Certain features are more useful for diagnosis than measuring outcome | “I think, that at that point, present and absent would be sufficient information in the clinical needs to really come to any conclusions about CRPS. [in regards to mottling]” | |
| “I tend to use the allodynia piece as information to help me figure out if they do have CRPS…it’s one of those real trigger things. But do I use it as a tool to see whether or not they’re getting better…? I don’t know, I don’t think I would re-evaluate that in any way.” | ||
| Utility for ruling in and ruling out | But if it is present, it is a diagnostic sign…. But if it isn’t [present], it does not rule out CRPS. (in reference to skin temp) | |
| Focus reflected in scaling preferences | “… you either have it [allodynia] or you don’t. If you have it – mild, moderate, severe, it doesn’t matter – it is the presence of it that matters.” | |
| “I would use the 4pt scale and I’ll tell you why. Because when that patient comes back or if the CRPS is resolving, I want to be able to look at my descriptors and say it was moderate, and now is maybe mild or none….” [in reference to hyperhydrosis] | ||
| Health care professional roles and scope of practice | Specialization supports skill for CRPS assessment | `` I think that question [movement given severity of injury] would be answered slightly differently, based on what their frame of reference was. A generalist may not know enough about what to expect for a particular injury.” |
| Assessment partnerships within the multidisciplinary team | “We [RNs] don’t do that. Our physiotherapists do it as part of their assessment and treatment plan….” | |
| “I would rely on the notes from the hand therapist to give me that information [related to functional assessment].” | ||
| “That [allodynia] would make me think, oh I really have to talk to the doctor about this person.” | ||
| Practice environment influences roles and scope of practice | “[Interviewer] How hard is it to judge allodynia?” | |
| “It depends on the age of the patient. So in pediatrics, it can be pretty tough. We have some kids with cognitive impairments, and other kids are very young and just not able to verbalize… some of them are just scared that they are even there, that someone is going to touch them, so it may be anxiety related rather than true hyperalgesia.” | ||
| “…I am dealing with patients in their more acute phase, and passing them along to pain management centers … so most of the patients I see will not be in the severe stage….” | ||
| Roles influence scope of practice, and assessment practices and opportunities | “… we [RNs] don’t clinically do that [assess Incoordination]. To be quite honest, I don’t know that I would have good cues to know how to assess that.” | |
| “But it is difficult to tease out the separate components when you are treating someone as a therapist… you are doing it all at once.” | ||
| Beliefs about CRPS | ||
| Physical vs. psychological symptoms | Reflective of attitudes toward signs vs. symptoms | “Well, you are always relying on the patients’ perceptions, so if the patient wants to fool you, they can fool you. It’s pain, so it is subjective….” |
| “…you think they may be malingering, but if you step back and think ‘wait a minute, they haven’t actually used this arm normally for a while, and it’s [coordination] not just going to come back like that [snapped fingers].’” | ||
| Relationship to impaired perceptions and/or altered body image | “Kids come in with slight swelling, and they come in saying ‘Look at how bad my swelling is’ and you are looking at it saying ‘Riiight!’ because you can barely see it…’ | |
| [Interviewer asked: Do you attribute that to the child just being really focused or perseverative on it, or do you attribute it to changes in the sensory map in the brain, and body perception in CRPS?] | ||
| …I think it is both: a perceptual component, and I believe there is a huge psychological component to chronic pain. Huge. There has got to be. I mean they are hypervigilant, it’s almost like an anorexic with their weight…. | ||
| “… they’ll have the sensation of swelling and I try and distinguish that and actual swelling…I’ll say ‘You might have the sensation of swelling, but is that swelling actually present? Look at the other hand and see if you can see any changes” | ||
| The influence of pain on categorical judgments | Avoiding assessments that might be painful | “I would use the FROM to the distal palmar crease, because I am not going to measure each joint one by one, it’s ridiculous with someone in pain.” |
| “I find it too traumatizing for the patients to pull out a pin (to assess hyperpathia) when they’re having allodynia.” | ||
| Rapport valued over complete assessment | “I don’t see the point of subjecting them to a stimulus that I know is noxious, and I know they are hypersensitive… I don’t think it benefits the theraputic relationship that I have with them…. It’s the same thing as if they have told you their bone is broken – why are you doing passive ROM when you know it is counter-productive?” | |
| Severity vs. chronicity | Some changes only present after a long time | “[Interviewer] Do you see decreased hair growth? |
| Yes – later stage.” | ||
| “[Interviewer] Do you look at nail quality or changes in the nail growth? | ||
| Yah again, to us, those are late signs.” | ||
| Signs found in every stage of syndrome, but may change with severity | “…Well, I have seen hyperhydrosis in every stage of CRPS” | |
| “… it depends on how long you’ve had it, because there would be more guarding behaviors and more contractures the longer you’ve had it.” | ||
| Different assessment practices for different phases | “[Interviewer] Do you see altered hair growth? | |
| Rarely. We try to get them in quick – we try to see potential new CRPS patients as soon as possible, so it is pretty rare that we would see changed hair growth.” | ||
| “I think probably 0–3 would be the best I could do in terms of assessment. You have got to remember those nail changes happen over a long period of time, and I am dealing with patients in their more acute phase….” | ||
| “… if they are in the later stages, and you are not really going to do anything for them, then what is the point of assessing them, other than to document” | ||
| Factoring in deconditioning to judgments about movement in the later phases | “The difference is, how long it has been since the injury, is in the conditioning…. So I don’t think that much on how much they can do – I think whether they can or cannot do it… if they cannot walk a certain distance, because the leg gets tired, it could just be part of de-conditioning….” | |
| Symptom variability as an assessment challenge | Hard to make judgments based on a single time point | “…they will say that ‘When I wake up in the morning it’s mottled, or on and off it’s mottled all day,’ but of course the hour that you see them, nothing is present….” |
| “And just assess it throughout treatment, see what they present with then they get there, see what they present with partially through, monitor it throughout treatment and see what it looks like at the end… and let me know what happens after they leave therapy.” | ||
| Knowledge translation | ||
| Practice evolves as new research is incorporated | “In the beginning I did used to do it [measure skin temperature] because I thought that it had to be cold, but the more I read about it, it really doesn’t matter, because the cold can be presented late, late in the problem – in the beginning it can be warm.” | |
| Clinicians unsure if their practices are up to date | “I don’t know why we don’t [assess pinprick hyperpathia], but we don’t. I wonder if it’s pediatrics, I wonder if they are already so anxious and so fearful, that if we started putting painful stimuli…. | |
| [Interviewer] In actual fact, we are likely going to drop this item, because most clinicians told us they would never do this. When we sent the tool out to experts, they thought it was important, but most of them were researchers, and doing it in that context. But clinicians told us they would not test this. | ||
| Thank God, I was thinking, shoot, are we supposed to be doing that?” | ||
| Clinicians expressing lack of certainty around knowledge and practices | “…lack of hair….so why is that? This is for my own education. So people can have lack of hair with the sympathetic response as well?” | |
| “[Interviewer] Do you see this [increased hair growth] in children? | ||
| No. Do you? | ||
| [Interviewer] We see it in adults frequently. | ||
| Really? Nope, not in kids.” | ||
| KT barriers and supports | “…maybe I should do the figure-8 method [to measure edema] … but it takes time to change your practice and so when you’re already in the routine of doing something a certain way, it’s hard to change.” | |
| “I don’t have a huge strong preference …. I know that the likert scale is the 7 point scale, and probably has the best research to support it.” | ||
| Using interviews to reflect on evidence-based practice | “And after I was reading it over again, I was thinking, oh maybe I should be assessing that….” | |
| “Now you are making me think about it! In my mind if you use the term hyperhydrosis, it is to the point that the person needs to seek treatment for it, to do something to the nerves or glands or whatever…. Yes I am comfortable [making a judgment about hyperhydrosis], but I hesitate now just thinking about how I have used the term a bit differently.” | ||
CRPS=Complex Regional Pain Syndrome; ROM=range of movement.
Assessment Beliefs and Values
The Role of Experience in Assessment
Many participants articulated that clinical experience was a general prerequisite for competent assessment of this population. Others saw experience as a critical factor to their confidence in their own assessment skills. The evolution of skill mirroring the evolution of practice with experience was also reflected on. The research team noted that experience also appeared to contribute to advanced description for scale end anchors.
The Importance of Comparison to the Unaffected Limb
This topic emerged so frequently throughout the interviews, it appears to be a foundational assessment value. In fact, it was articulated by every participant and referred to in relationship to every concept on the CB-HI-CRPS.
The Need for Observation, Patient Report, and Direct Measurement
Clinicians often raised the idea of the importance of using different sources of assessment information, including objective measurements, clinical observations, and listening to the subjective reports of the patient being assessed. Value was attributed to all of these forms of information; however, participants also expressed a need to find a balance between the different sources of information, suggesting that all forms were not valued equally. This was reflected in the process of assessment by seeking to correlate subjective data with objective findings. The taxonomy used by the participants also appeared to reflect a hierarchical value structure, as “subjective” was used far more frequently than “patient reported.”
The influence of time pressures on assessment choices was also noted, as it appeared that some participants made assessment choices based on time constraints rather than best practices. Others expressed strong opinions about the value of making actual measurements of range of movement (ROM) as opposed to the judgments about movement relative to anticipated recovery proposed by the current draft of the CB-HI-CRPS.
Professional Roles and Functions
HCP Roles and Scope of Practice
Participants were from a purposive multidisciplinary sample of HCPs, and encompassed physicians, nurses, and therapists from a variety of practice roles, including pain management, pediatrics, and hand therapy or hand surgery (refer to Table 2). However, most clinicians in the sample considered themselves specialists having a moderate to high level of expertise in CRPS, and this was reflected in their comments. The practice environment was also cited as a key influence on the scope of practice and assessment considerations.
Interviewees shared how their professional roles sometimes delineated what and how they would assess, and their comfort level with the assessment components. They also talked about the collaborative interactions between members of the multidisciplinary team as it pertained to the assessment of CRPS, and expressed that they valued the team approach.
Diagnostic Evaluation vs. Outcome Evaluation
Assessment of CRPS can have a diagnostic focus or can be used for measuring progress over time, and/or final outcomes. Participants reflected the value of all of these perspectives. However, certain features of CRPS were thought to be more useful as diagnostic evidence than for monitoring progress—refer to Table 5.
Table 5Numbers of Participants Who Labeled CB-HI-CRPS Items as Diagnostic or Outcome
| Category | Concept | Diagnostic Item | Outcome Item |
|---|---|---|---|
| Pain | Allodynia | 2 | 2 |
| Hyperpathia—pinprick | 4 | 0 | |
| Cold intolerance | 1 | 0 | |
| Guarding | 1 | 3 | |
| Autonomic | Hyperhydrosis | 3 | 1 |
| Skin temperature | 4 | 1 | |
| Mottling | 4 | 0 | |
| Edema | 2 | 7 | |
| Trophic | Hair growth | 0 | 5 |
| Nail quality | 4 | 0 | |
| Skin quality | 2 | 0 | |
| Motor | Muscle tone | 1 | 1 |
| Incoordination | 0 | 2 | |
| Movement given time since initial injury | 2 | 0 | |
| Movement given severity of initial injury | 1 | 2 | |
CB-HI-CRPS=clinician-based portion of the Hamilton Inventory for Complex Regional Pain Syndrome.
Scaling preferences also reflected the purpose of assessment, with dichotomous scaling (present/absent) being preferred for features that were perceived as diagnostic, and more descriptive scales (four point or seven point) being preferred for monitoring change over time. Practice roles appeared to drive preferences: both OTs and physiotherapists preferred to look at progress and outcome evaluation, while physicians’ preferences reflected a diagnostic focus.
Beliefs about CRPS
Physical vs. Psychological Symptoms
Clinicians’ descriptions of how patients recounted their symptoms sometimes reflected differing attitudes toward items seen as measurable clinical signs when compared with symptoms subject to the perceptions of the patient. Patient descriptions of pain intensity, guarding and neglect, incoordination, and fluctuating swelling were thought to be difficult to correlate or reconcile with behavioral observations and measurements, such as ROM and volumetry. Several participants also related psychological symptoms to impaired perception or altered body schema, reflecting the evolving understanding in the literature
29
, 30
, 31
of the relationships between sensation, perception, and altered cortical representations in CRPS.The Influence of Pain on Categorical Judgments
Another subject that emerged was how the presence of pain might influence both the assessment process and the ranking of other symptoms. Clinicians sometimes avoided testing situations that they thought would induce or increase pain in persons with CRPS. In fact, most of our participants said they would not test pinprick hyperpathia (see Table 6), citing concerns for rapport and the therapeutic alliance.
Table 6Assessment Practices and Scaling Preferences
| Scaling Preferences | |||||
|---|---|---|---|---|---|
| Assessment Concepts | Present/Absent (%) | Four Point (0–3) (%) | Seven Point (0–6 or −3 to 3 Likert) (%) | Written Description/Other Scale | Do You Currently Assess? (% Yes) |
| Allodynia | 20 | 45 | 15 | —/5% | 85 |
| Hyperpathia—pinprick | 15 | 5 | 5 | —/10% | 35 |
| Cold intolerance | 20 | 15 | 5 | 10%/5% | 55 |
| Guarding | 30 | 50 | 10 | — | 90 |
| Skin temperature | 20 | 15 | — | 5%/45% | 85 |
| Mottling | 50 | 40 | — | — | 90 |
| Hyperhydrosis | 45 | 30 | 10 | — | 85 |
| Edema | 10 | 75 | — | — | 85 |
| Hair growth | 30 | 25 | 5 | 10%/15% | 85 |
| Skin quality | 35 | 15 | — | 35%/— | 85 |
| Nail quality | 40 | 25 | 5 | 5%/10% | 85 |
| Muscle tone | 15 | 15 | 15 | —/15% | 60 |
| Incoordination | 15 | 20 | 10 | 20%/— | 65 |
| Movement given severity of initial injury | 10 | 10 | 40 | 5%/10% | 75 |
| Movement given time since initial injury | 15 | 15 | 35 | 5%/10% | 80 |
Severity vs. Chronicity
Interviewees discussed the challenge of separating the severity of this neuropathic pain syndrome from the duration of symptoms experienced by the person. They felt that some changes were only present in patients who had been affected by CRPS for a greater time interval. Other signs were seen in every phase of the syndrome, although the severity of the findings might increase over time. Differences in assessment findings pertaining to timing was also influenced by practice area, with some pediatric practitioners indicating unique patterns of signs in children, and other clinicians citing their acute practice focus as precluding seeing what they perceived as later signs of CRPS.
The influence of deconditioning with prolonged CRPS was also raised as a consideration when making comparative judgments on the severity of impairments, for example, coordination or ROM; given that these could be affected by the duration of time since the original injury.
Symptom Variability as an Assessment Challenge
The inherent variability of the signs and symptoms of CRPS was shared by many participants as a challenge and a barrier to relying solely on a single time point of assessment. Health professionals from all disciplines stressed that this variability underscored the need to listen to the report of the patient in addition to undertaking objective physical examination.
Knowledge Translation
Knowledge translation can be defined as group of methods to close the gap between new research findings and clinical practice.
32
In the health care arena, it includes aspects of awareness of evidence, willingness to change, and environmental/system supports in order for health professionals to alter assessment and treatment practices.33
During our interviews, the clinicians made reference to how their practice had evolved in step with new research and ideas. Some also expressed concern that they were not keeping up with contemporary evidence, occasionally asking the interviewer if what they were doing was correct. Many participants also used the cognitive debriefing interviews as an opportunity to muse about their own practices, sometimes questioning what they were doing and why they were doing it. They also reflected on the barriers to knowledge translation like lack of time, and established practice patterns and identified strategies that could support change such as working toward change as a team.Additional Considerations
Several themes are perhaps noteworthy by their absence. No mention was ever made of gender, and all participants used gender-neutral terms such as person, client, or patient when referring to this population. Additionally, the sole references which could be attributed to culture or ethnicity pertained to 1) the difficulty in assessing mottling in persons with darker skin pigment, and 2) the need to carefully compare to the unaffected side, factoring in skin and hair color when assessing changes in hair growth. The role of culture in the assessment of pain, and the influence of culture on the expression of pain was not overtly addressed.
The findings of these interviews must be viewed in the context of the research team. One of the researchers (TP) is the developer of the HI-CRPS, and was involved in the analysis of input on her tool from her colleagues and peers. Her interviews often took on a different flavor, as participants sometimes asked questions of the interviewer, seeking additional information or verification for some of their beliefs and practices. Issues raised in these interviews sometimes reflected a broader spectrum of issues, including referral patterns and health system concerns. The impact of this dynamic bears consideration when reviewing these findings.
The other members of the research team were students, and they sometimes perceived a power imbalance when asking questions of experienced clinicians. Conversely, the participants often gave richer answers to the students, appearing to undertake an educational role with the students by providing detailed descriptions and definitions. By contrast, the interviewer with clinical experience sometimes received briefer answers, as if participants assumed she already knew the information.
Responding to perceived time pressures was a challenge experienced by all of the interviewers. The research team would edit the interview structure to fit the available time period, and omit questions if repeated probes were failing to elicit the depth of information sought. This experience seemed to justify the a priori decision to randomize the order of the concepts covered by each interview, but it is unclear to what extent the veracity of the interviews was compromised by these omissions. Finally, the large proportion of participants from the occupational therapy profession must also be acknowledged. The potential exists that this may bias the evaluation toward the taxonomy and theoretical frameworks of client-centered practice and occupational performance that underpin the profession.
34
In summary, the cognitive debriefing interview format created an opportunity for our participants to reflect not only on the content of the CB-HI-CRPS, but also on a variety of practice challenges and concerns, beliefs, and attitudes related to the assessment of CRPS. Categories such as diagnostic evaluation vs. outcome evaluation, and the need for observation, patient report, and direct measurement reflected our own struggles with developing a clinical assessment tool with a transdisciplinary focus (T. Packham, Canadian Society of Hand Therapists, 2009: Toronto, Ontario). Issues surrounding knowledge translation, although not an anticipated theme, also resonated with the research team. The themes, categories, and content we have shared here are not an exhaustive exploration of these issues, but are intended to give a broader context and depth of meaning when entwined with the quantitative analysis that follows.
Quantitative Analysis
This section is intended to share the numerically descriptive data collected from the interview process. It is important to note that while 20 HCPs participated in this study, not all participants provided answers for every question, so the number of respondents varies from question to question. Additionally, two audio-recordings were lost due to technical malfunctions before transcription, but their demographic and preference data (recorded on paper) are included in the overall summaries.
Scaling Preferences
Participants were asked to identify which scale they would prefer to use for scoring each individual concept on the CB-HI-CRPS (see Table 1 for the list of concepts). Grouped responses are summarized in Table 6; the data illustrate how preferences changed according to concept. For example, the dichotomous present/absent scale was preferred by 50% of respondents for rating mottling, but 75% preferred a four-point scale (none, mild, moderate, and severe) to rate edema, and 35% indicated that they would rather just write a description of skin quality instead of making a scalar judgment. Seven-point scales were rarely selected (receiving no endorsements for four of the concepts) with two exceptions: the movement scales scored with a seven-point likert agreement scale (from strongly disagree to strongly agree) were preferred by 35% (duration) and 40% (severity) of participants. However, when participants were grouped according to their self-reported roles with 1) physicians and RNs (diagnostic group) and 2) therapists (both occupational and physiotherapists) (outcome group), a general trend did emerge. The diagnostic group tended to endorse the present/absent scales, in keeping with the type of judgments required to use the International Association for the Study of Pain (IASP) diagnostic criteria for CRPS, and with their consultative practice patterns. Conversely, therapists in the outcome grouping chose four-point scales that reflected their current behaviors for assessment and monitoring change over time, and were in keeping with seeing persons with CRPS for repeated visits over a longer period of time.
Assessment Practices
One of the purposes of the study was to examine HCPs’ current assessment practices for CRPS, and to compare and contrast those with the assessment template provided by the CB-HI-CRPS. The most striking departure from CB-HI-CRPS related to the concept of pinprick hyperpathia. Only 35% of participants currently assess this routinely: furthermore, among those who did assess it, it was used selectively on a case-by-case basis rather than as a standard part of their initial test battery. Hyperpathia for cold (used by 55%) and muscle tone (used by 60%) were also less consistently assessed concepts. However, there was some confusion with respect to the terminology: muscle tone was chosen as a neutral term encompassing both elevated or lowered levels, but some participants associated the phrase “muscle tone” with spasticity (12.5%), whereas others interpreted it as muscle strength (12.5%) as opposed to the intended classic definition of the amount of background contraction of a muscle at rest (43.8%). This is reflected in the 15% of participants who endorsed “other scales” as the way they would assess this concept: standardized grading systems for manual muscle testing and for the assessment of spasticity were among the additional scales cited by the participants as their current practices.
Overall, the participants reported using the concepts represented by the CB-HI-CRPS 85.2% of the time (SD=11.1). When analyzed by professional groups, physicians followed the concepts a little more closely (mean 89.5%), therapists were consistent with the overall scores at 85%, and RNs tended to leave off a few more items (they used the concepts only 75.6% of the time). This represented a statistically significant correlation between professional group and use of the CB-HI-CRPS concepts (p=0.002).
User Manual Definitions and Recommendations for Standardization
Within the interviews, participants were asked to define or describe each concept covered by the CB-HI-CRPS, first as a check to ensure they understood what the question was asking them to assess, and secondly as a way of generating definitions for the user manual that reflected a multidisciplinary lexicon. Many participants were quick to define allodynia using formal terms that reflected the IASP definition, which clearly states the pain is generated from “a stimulus that does not normally provoke pain” (p. 18),
35
but several participants substituted the word hypersensitivity and made reference to an exaggerated reaction to a noxious stimulus. The interview questions did not overtly ask participants to define hyperpathia, but instead asked them how they would assess pinprick hyperpathia. Ninety-five percent of participants did not answer the question as they did not include it in their assessment; 10% directly referred to testing protective sensation as sharp/dull, whereas another 15% did refer to their testing procedure as testing hyperpathia, but whose methods in doing so did not reflect the concept accurately (i.e., using Semmes–Weinstein monofilaments, which are intended to measure threshold).36
Only 30% described an appropriate testing procedure (it is perhaps noteworthy that all of the anesthetists fell into this group). Clinicians were much more comfortable with the concept of cold hyperpathia: they equated it with cold intolerance. However, those who included it in their assessment were evenly split on whether to objectively assess or simply ask the patient to describe their perceptions.The differing interpretations of the term muscle tone have already been alluded to a precise definition in the user manual may help to clarify the issue, but that presumes that all users of the instrument will familiarize themselves with the operational definitions before using it in clinical practice. Brunner et al.
9
proposed the term “motor changes” to encompass dystonia, weakness, bradykinesia, and tremor. However, such broad umbrella terms present a challenge to reliability as different raters may introduce variability into the scoring by perceiving all of those symptoms need to be present for severity vs. any single attribute being scored for severity.19
Skin temperature was another category demonstrating great variation within the assessment process. Seventy-seven percent of those who currently measured temperature were doing so simply by touching the patient and comparing to the other side, whereas the remaining 23% are using some form of thermometer; however, an additional 23% stated they would like to measure temperature formally, but did not have the equipment available. No clinicians relied on patient SR for this information. The lack of uniformity in terminology and assessment behaviors underscores the need for a user manual with a clear scoring framework and standardized assessment instructions to improve reliability across a multidisciplinary user group.
Discussion
Reflections
This mixed-methods study used cognitive debriefing interviews to investigate the presence and patterns of how HCPs from different disciplines define, assess, and formulate judgments on the clinical signs of CRPS. The study also explored the potential to use the content of the interviews to modify the scoring system of a condition-specific outcome measure currently under development (the CB-HI-CRPS) to eliminate potential sources of difference between user groups, with the goal of improving reliability, content, and face validity in future psychometric testing. Using semistructured interviews, we asked a small purposive sample of health professionals from six different disciplines to define the concepts of the CB-HI-CRPS and describe their current assessment practices and potential scaling preferences. The interview transcripts were reviewed using a descriptive content analysis paradigm to examine the scope, themes, and relationships of the information
37
and quantitative data reflecting practices and preferences was also collected and analyzed. Four interrelated themes emerged: 1) assessment beliefs and values, 2) professional roles and functions, 3) beliefs about CRPS, and 4) knowledge translation. The HCPs in our sample raised fundamental issues related to the assessment of signs and symptoms including balancing objective measurements with clinical observations and patient report, and the essential differences between assessment for the purpose of diagnosis and assessment for the purpose of measuring changes over time.We also found that assessment practices appeared to differ in specific ways across professional groups; however, it is difficult to make generalizations given our small sample. Although there was good overall agreement for all of the occupational groups, the current practices of physicians reflected most closely the literature-based concepts of the CB-HI-CRPS, followed by therapists, then nurses. Preferences for dichotomous scales were higher among the professionals who saw their primary role as diagnosis or screening (physicians and nurses), whereas therapists generally preferred four-point scales, seeing them as more responsive and thus reflective of their need to evaluate the same patient over time.
Limitations
One of the challenges of this study was the level of self-reflection or reflexivity required of the research team. Interviewing and reviewing data from colleagues was particularly challenging for the experienced clinician (TP) who was aware of the ethical tensions with both recruitment and participation and of the inherent bias of the direct involvement of the tool developer in validation of an assessment tool. Although a strong rapport often existed which may have fostered authenticity in the interviews, the semistructured format of the interviews created by the research team nevertheless represented an intrinsic bias toward specific content areas. This could also be considered a weakness of the cognitive debriefing interview format as a vehicle for qualitative exploration, as the interviews are necessarily built around the questions from tool under examination.
38
This study also has several limitations related to sampling. It was conducted over a one-year period, but failed to meet the initial recruitment targets for the sample. All of the interviews were conducted within the Southern Ontario region (ranging from London to Toronto to St. Catherines); however, several of the clinicians interviewed had international training and clinical practice experience in diverse areas such as India, Saudi Arabia, Mexico, Brazil, and the UK. Although the range of time in clinical practice spanned from three to 35 years, the average experience of participants was 14.9 years, representing a relatively experienced group. It would have been valuable to include clinicians with less experience to provide a greater breadth; however, the snowballing sampling method may have unwittingly eliminated this population, as clinicians tended to nominate other more experienced colleagues, and those with less experience may have been hesitant to volunteer if they did not perceive themselves to have a requisite level of expertise. The large component of OTs included in the sample may also add a bias toward the foundational theories and values of that profession, including occupational performance and client-centered practice.
39
It should also be acknowledged that no psychologists or behavioral pain management specialists were included in this group, and this oversight may also bias the findings.Implications
This mixed-methods study is intended to supplement and complement the traditional quantitative approach initially used to develop the CB-HI-CRPS. The information gathered from the clinicians will help to inform the scoring system and user manual, adding reliability and validity by strengthening the clinical utility for target users from a variety of disciplines. Moving forward, the intent is to test a revised version of the CB-HI-CRPS that omits the pinprick hyperpathia item; other categories such as muscle tone may be better served with clearer definitions or different terminology. The opportunity exists to test alternate forms of the assessment, using either four-point or seven-point scales with expanded descriptors/anchors on the individual assessment items in a larger study of reliability and validity, and comparing the results to recommend the most reliable scaling format.
The second contribution of this study is to add insight into how individual clinicians have grappled with the challenges of assessing persons with CRPS. The rich descriptions provided by the participants highlight their struggles to balance observations with patient report and direct measurement: this illustrates the challenge of “objectifying” the variety of signs and symptoms found within this syndrome.
40
A holistic perspective appears to underpin their collective awareness of the multifactorial influences on impairments; and they harnessed their experience to identify symptom patterns within the inherent variability of CRPS. Although our sample was very experienced with a mean 14.9 years spent in clinical practice, this ability to be adaptive and address variability could be considered true expertise.41
This study underscores the realities of knowledge translation: HCPs need clear reasons to inform behavior change, and support to overcome barriers to practice change such as time constraints in the clinical environment.
31
, 32
, 42
In our study, the assessment practices of health professionals tended to mirror their current conduct; in essence, they saw their own behaviors as the standard. This may be explained in part by their self-rankings of expertise, with an average of 7.5 of 10. However, some practitioners did acknowledge the need to evaluate their practices against ever-evolving theories and research, and were actively seeking input to do so: this reflects behavior that supports the development and maintenance of expertise in HCPs.41
, 43
Additionally, the insight into the process differences in making clinical judgments between different health professions may provide the foundation for a future grounded theory study on models of clinical judgment formation, and correlation studies on the differences in assessment skills between professional groups, and between experienced and novice clinicians, both in the arena of CRPS assessment and beyond.Conclusion
Do You See What I See?
In order for us to see our patients in the same way we must start to come to some consensus on what we look for (assess) and how we interpret those findings. Standardized scales can serve a role in helping to create more uniform assessments and provide structure to how they are interpreted. This article describes a novel application of the cognitive debriefing technique, where potential users of the CB-HI-CRPS (a tool designed to guide the standardized assessment of clinical signs in persons with CRPS by HCPs) were interviewed to collect information and insight on assessment beliefs, practices, and preferences. This appeared to be a useful method for formulating user-generated definitions and descriptions for assessment techniques and scale anchors. Developers of outcome measurement tools may wish to explore the use of this method in addition to traditional Delphi rounds or other consensus methods.
The results generated from this small sample suggest there may be systematic differences in how health professionals from different disciplines assess and formulate judgments on the clinical signs of CRPS, and that these may be driven by the underlying focus on diagnosis or outcome assessment as directed by the role of the professional within the care team. The information collected also highlighted a number of potential areas of user error, such as variations in operational definitions and assessment practices that can be addressed by refining the user manual of the CB-HI-CRPS before further psychometric examination of the reliability and validity of the tool. As the components of the HI-CRPS continue to evolve, so can our understanding of this challenging syndrome. As individual caregivers and members of multidisciplinary teams, our shared goal is to pursue comprehensive assessment of CRPS leading to informed treatment decisions that will continue to advance evidence-based care.
Acknowledgments
The authors thank Erin Gaiger, MScOT, and Alysha Friedman, MScOT, for their dedication, insights, and assistance in conducting many of the interviews included in this article. They also thank Pam Ball, OTR, for her comments on an early version of this article.
Supplementary Data
- User Manual
References
- An updated interdisciplinary clinical pathway for CRPS: report of an expert panel.Pain Pract. 2002; 2: 1-16
- External validation of IASP diagnostic criteria for Complex Regional Pain Syndrome and proposed research diagnostic criteria.Pain. 1999; 81: 147-154
- Complex Regional Pain Syndrome—mechanisms, diagnosis and management.Curr Anaesth Crit Care. 2009; 20: 209-214
- The incidence of complex regional pain syndrome: a population-based study.Pain. 2007; 129: 12-20
- Complex regional pain syndrome: a review.Ann Vasc Surg. 2008; 22: 297-306
- Outcomes of the Complex Regional Pain Syndrome.Clin J Pain. 2009; 25: 590-597
- Onset and progression of dystonia in complex regional pain syndrome.Pain. 2007; 130: 287-293
- The natural history of Complex Regional Pain Syndrome.Clin J Pain. 2009; 25: 275-280
- Diagnostic criteria and follow-up parameters in complex regional pain syndrome type I—a Dephi survey.Eur J Pain. 2008; 12: 48-52
- Development of a Severity Score for CRPS.Pain. 2010; 151: 870-876
- Impairment Level Sum Score in reflex sympathetic dystrophy of one upper extremity.Arch Phys Med Rehabil. 1998; 79: 979-989
- Development of a symptoms questionnaire for complex regional pain syndrome and potentially related illnesses: the Trauma Related Neuronal Dysfunction Symptoms Inventory.Arch Phys Med Rehabil. 2008; 89: 1114-1120
Packham T, MacDermid J, Henry J, Bain J. A systematic review of the psychometric evaluations of outcome assessments for complex regional pain syndrome. Disabil Rehabil. Accepted manuscript in press.
- Using cognitive interviews to develop surveys in diverse populations.Med Care. 2006; 44: S21-S30
- What the resident meant to say: use of cognitive interviewing techniques to develop questionnaires for nursing home residents.Gerontologist. 2008; 48: 158-169
Willis G, Caspar R, Lessler J. Cognitive Interviewing: a how-to guide. 1999. Available at: http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf. Last accessed October 7, 2008.
- The analysis and interpretation of cognitive interviews for instrument development.Res Nurs Health. 2007; 30: 224-234
- Consensus methods: characteristics and guidelines for use.Am J Public Health. 1984; 74: 979-983
- Health Measurement Scales: A Practical Guide to Their Development and Use.3rd ed. Oxford University Press, New York2003
- Measuring what matters: does ‘objectivity’ mean good science?.Disabil Rehabil. 2010; 32: 1011-1019
- Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences.Acta Psychol. 2000; 104: 1-15
- Applying evidence on outcome measures to hand therapy practice.J Hand Ther. 2004; 17: 165-173
- Communicating evidence to clients, managers and funders.in: Law M. Evidence Based Rehabilitation: A Guide to Practice. Slack Inc, Thorofare, NJ2002: 221-254
- Three approaches to qualitative content analysis.Qual Health Res. 2005; 15: 1277-1288
- Self Reports: how the questions shape the answers.Am Psychol. 1999; 54: 93-105
- Problems completing questionnaires on health status in medical rehabilitation patients.J Rehabil Med. 2007; 39: 633-639
- Qualitative research in health care: analyzing qualitative data.BMJ. 2000; 320: 114-116
- Qualitative Evaluation and Research Methods.2nd ed. Sage Publications, Thousand Oaks, CA1990
- Body perception disturbance: a contribution to pain in Complex Regional Pain Syndrome.Pain. 2007; 133: 111-119
- Wherever is my arm? Impaired upper limb position accuracy in Complex Regional Pain Syndrome.Pain. 2010; 149: 463-469
- Cortical changes in complex regional pain syndrome (CRPS).Eur J Pain. 2009; 13: 902-907
- Defining knowledge translation.Can Med Assoc J. 2009; 181: 165-168
- From best evidence to best practice: effective implementation of change in patients’ care.Lancet. 2003; 362: 1225-1230
Law M, Baum C. Measurement in occupational therapy. In: Law M, Baum C, Dunn W (eds). Measuring Occupational Performance: Supporting Best Practice in Occupational Therapy. (2nd ed.) Thorofare, NJ: Slack Incorporated, 2005.
- Pain terms and taxonomies of pain.in: Loeser J.D. Bonica’s Management of Pain. 3rd ed. Lippincott Williams and Wilkins, Philadelphia, PA2001
- Handbook for Somatosensory Rehabilitation.English ed. Sauramps Medical, Paris, France2006
- Qualitative Research Methods: A Health Focus.Oxford University Press, Melbourne1999
- A briefing on cognitive debriefing.Good Clin Pract J. 2006; 13: 25-29
- A review of evidence on the conceptual elements informing client-centred practice.Can J Occup Ther. 2006; 73: 153-162
- Objectification of the diagnostic criteria for CRPS.Pain Med. 2010; 11: 1212-1215
- Developing expertise in surgery.Med Teach. 2010; 32: 830-836
- Using an evidence-based approach to measure outcomes in clinical practice.Hand Clin. 2009; 25: 97-111
- The first year of practice: an investigation of the professional learning and development of promising novice physical therapists.Phys Ther. 2010; 90: 1758-1773
Article info
Identification
Copyright
© 2012 Hanley & Belfus. Published by Elsevier Inc. All rights reserved.
