In Search of Prosthetic Outcome Measures

This chronicle outlines a voyage of mystery research, and deception as a lone adventurer traveled into the depths of` Canada to find the ultimate .yet of prosthetic and orthotic outcome measures.

The voyage started by my asking the local natives for direction, however they greeted my question about the ultimate outcome rneasure with the response, "What's an Outcome measure?" At this point, I realized my journey may be a difficult one.

Luckily, l had my trusty notebook computer in my pack, so I, hooked onto the Webster Dictionary site (http://www.m-w.com). Outcome was defined as something that follows as a result or consequence and measure as the dimensions, capacity, or amount of something ascertained by measuring. My search was starting to take shape - I must find a way to measure the result or consequences of the prosthetic and orthotic process. Sounded easy enough ...

The word on the street...

I figured the best way to start was to hit the street and see what the boys in the 'hood' (i.e., fume hood) would say. The best thing about being on the street again was that both the clients and the practitioners have their say. Not surprisingly, their views are not always the same.

Two big points of contention are: how to define a good prosthesis/orthosis and how to define a talented practitioner. These two items are related, since the definition of a talented prosthetist is usually a prosthetist who makes a great device. Unfortunately, most people have trouble deciding what makes a good prosthesis and what makes one prosthesis better than another. Finding the ultimate outcome measure is difficult when practitioners and clients cannot agree on what makes a great device.

One think is certain: outcome measures are directly linked to quality. However, agreement on what constitutes a quality prosthesis or orthosis must occur before appropriate outcome measures can be selected. Clients who are vocal on the OANDP-L e-mail list focus on the socket as the main factor of a good prosthesis. In the words of Wayne Renardson, (a client from OANDP-L), "a talented prosthetist is one who can make a socket that permits me to walk without pain or discomfort". The socket should feel comfortable throughout the day. A talented prosthetist should also be able to deal with the psychological aspect of prosthetic fitting (i.e., expectations, dealing with limb loss, etc.). Comfort is so important that we must include it in any outcome measure. Unfortunately comfort is an abstract concept that is hard to measure accurately.

Much of how a person grades comfort is based on his or her personal experience, physiological makeup, and expectations. This makes between-client comparisons tricky. The best measure of comfort may be

Table One: A simple outcome measure.

Unfortunately, this measure is so broad that we are unable to confirm if one prosthetic design is more comfortable than another. As well, we should address the interplay between comfort and function in the ultimate outcome measure. For many consumers, the comfort question is easy to answer - if a prosthesis is not comfortable they will not wear the device, no matter how functional it is.

Cosmesis is another abstract component of prosthetic quality. The ultimate outcome measure would grade the overall device cosmesis and provide information on areas that require improvement not an easy task since beauty is in the eye of the beholder.

Practitioners have similar responses as their clients, however prosthetists do not place as much emphasis on comfort. This could he because prosthetists assume that all devices must he comfortable before the fitting process is completed. !1 quote from Reed Coleman, CP (from OANDP-L) stated shat, "if the prosthesis lets the wearer do all their activities of daily living (ADL) and extracurriculars in a comfortable and proficient manner, then you have a quality prosthesis". Another factor is that clients do not necessarily understand the effects of alignment and/or components on comfort (i.e., most clients focus on where it hurts and rely on the prosthetist to find out why it hurts).

Mr. Coleman's definition leads to the next level of outcome measurement:

Table Two: Another simple outcome measure.

This approach is easy to implement, easy to interpret, and provides a good checklist for any prosthetic or orthotic device that is dispensed. However, these outcome measures are not useful when comparing devices, clinical procedures, or client-relation approaches.

P&O is a difficult field to assess. since most outcomes are client-specific and many clinical decisions are based on the clinician's experience (and not on scientific principles). If 10 experienced prosthetists all fit the same 10 clients, you would likely have 100 prostheses with differences in sockets, materials, alignment, and components. Client responses to the questions in Table Two would be positive for all prostheses. Many prosthetists would also tell you that their design is the best (perhaps convincing the client of this as well).

This tells us a few things: all people with amputations must accommodate to the device secured to their residual limb, prosthetists can work within a tolerance, and interpersonal skills can influence client satisfaction. Fitting tolerance is the range of differences in socket shape and alignment within which there is no effect on comfort and function. Knowing this tolerance is important, since we would not need an outcome measure to compare devices that are similar enough to tall into the accommodation range. The effect of charisma on medical outcomes is a problem when trying to define a quality measure. The ultimate outcome measure would provide information that is independent of a prosthetist's ability to promote their work to a client.

Also, clients who have only been exposed to one or two prosthetists may believe that their prosthesis is great just because they have not experienced a better product. Many outcome measures rely on the client to know how a truly excellent product feels; for example. rate the comfort on a scale of one to true where one is poor and five is excellent. What you may he getting is a client's rating based on their most comfortable device or a rating based on their idea of an excellent device - and you cannot tell which basis they use.

Into the depths of rehab...

It seemed that I was finding more questions than answers, so I headed in the direction of a local P&O tribe. As I rounded a corner, I came across an older fellow sitting cross-legged at the side of the road. "Look no further," he said to me, "the ultimate outcome measures were written long ago". Since I had not mentioned my quest to anyone before, I decided to sit and listen to this strange man.

After paying a small service fee (it appears even learned gurus arc union members) the man handed me a stack of wrinkled paper. "Read and learn," was all he said before he lowered his head and paid me no more attention. A quick scan of the papers revealed summaries of many measures that could Undoubtedly he put to use in the prosthetic and orthotic field. Here is a summary of two popular measures:

• COVS (Clinical Outcome Variable Scale) is a method for measuring physiotherapy clinical outcomes. Eleven items are used to assess mobility, such as rolling, lying-to-sitting balance, transfers, and arm function. Each item is rated on a seven-point scale and entered on a data sheet.
• FIM (Functional Independence Measure) was designed to provide a "basic indication of severity of disability" for rehabilitation clients. Eighteen items, grouped into six categories, are rated on a seven-point scale. The scale measures what the person with a disability does, not what they should be able to do. FIM is good at predicting the burden of care (one point change relates to 3.38 minutes of assistance per day).

Sounded good to me. COVS and FIM appeared to be tile ultimate outcome measures. I was feeling quite pleased with myself as I got up to return with the answers to the ultimate outcome question. I had not gone further than a few kilometres when a small woman caught me froth behind. "I hear that you are looking for P&O outcome measures." she said in a sweet voice. Having no need for secrecy, I told her that I had already found the answer and was heading back to pass on tile good news. "That man on the street does not understand," she said, ..he gives everyone the same tools that you now have: however, the measures are so broad that you will never be able to really understand what is happening with a prosthesis or orthosis." "So, what do you look for in the ultimate outcome measure'?" I responded smugly.

Well, she had obviously thought things through because her answer made tile reconsider whether my stack of sheets was really the right tool for P&O. The guidelines she listed are shown in Table Three.

Re-examination of tile COVS and FIM rehabilitation measures showed that these tools were easy to apply, valid, accurate, reliable, cost-effective, and accepted by many rehabilitation professionals. However, the tests failed oil a few counts. They did not deal with the technical issues related to prosthetic and orthotic devices, and some were not precise enough to measure the range of changes we see in P&O (i.e., most people with a prosthesis or orthosis start at the higher range of scales that cover all disabilities).

As I sat thinking about these guidelines, another idea sprang to mind: the ultimate outcome measure should eventually be used to define tile ultimate prosthesis/orthosis.

On to R&D...

Rather than feeling annoyed by an apparent setback in my journey, I used this last encounter to help map out my next step. If general rehabilitation measurement tools would not work, I might find outcome measures in the research and development world that were specific to P&O. fly sources in cyberspace came up with two options for evaluation: Prosthetic Profile of the Amputee (PPA) and the Prosthesis Evaluation Questionnaire (PEQ).

The PPA questionnaire (Griseé, et al., 1993) was developed in Montreal to evaluate prosthetic use by people with lower extremity amputations after discharge from rehabilitation. This extensively validated questionnaire contains -1-1 questions. takes approximately 25 minutes to complete, and is available in French and English These questions are divided into six groups: physical condition, the prosthesis, prosthetic use, the environment, leisure activities, and general information. While this questionnaire would be effective for a research study, it is not appropriate for use in a clinical setting due to the relatively long time requirement for its completion.

The Prosthesis Evaluation Questionnaire (PEQ) was developed by the Prosthetics Research Study team in Seattle, Washington (Legro, et al., 1998). This questionnaire is also long (43 questions) but was developed to compare lower extremity prostheses or methods of care. The PEQ was also designed to measure small differences in prosthesis function. Subdivisions within the questionnaire cover areas such as prosthesis function, mobility, perceived responses, frustration, social burden, and well-being. While the PEQ should be more precisc than tile FIM at measuring prosthetic function, this tool does not seem to he better than other prosthetic questionnaires at measuring small differences (however, I am willing to be proved wrong on this).

The PEQ also uses visual analog scales (VAS) to record client responses. The client makes a mark on a 10 cm line to indicate his or her rating (i.e., an "average" rating is denoted by a mark at the five cm point along the line). While VAS is frequently used in psycho-social research, the results are often misinterpreted. For example, if a client rated his comfort as a two on a first visit and as a four during the next visit, it does not mean that the person is twice as comfortable although you could reasonably say that the prosthesis is more comfortable.

Table Three: Outcome measure guidelines.

Figure One: A technical/clinical checklist.

Some researchers even try to report differences in millimetres. Besides the fact that moat pen nibs are approximately one millimetre chick, the average person is not able to draw, freehand, a nook 67 mm along a line and then perform the same feat a week later on a new piece of paper. When trying to make sense of prosthetic VAS test results, the data can he converted to the common five-point scale (i.e., poor = 0-2 cm, below average = 2-4 cm, average = 4-6 cm, above average = 6-8 cm. excellent = 8-10 cm).

Although this may paint a bleak picture about the PPA and PEQ questionnaires, these are good tools for evaluating prosthetic cervices. Any facility planning to implement a quality improvement questionnaire should consider using these outcome measures.

Another positive point about these questionnaires is that they cover many aspects of prosthetic use. If we want to use outcome measures to compare ICEROSS, PTB-SC, PTB, or Alpha Liner-type sockets, we need to discover how these designs rate in terms of comfort, function, cosmesis, ease of use/maintenance, etc. These measures would also have to be evaluated based on the clients ADL and ambulatory goals.

But, are these questionnaires the ultimate outcome measure?... not quite.

What's missing...

As I looked at the growing pile of paper in front of me, I realized that a big part of the outcome measure picture was missing: the hardware. In such a technology -based field, we need to have outcome measures to rate the machine-factors as well as the human-factors. Luckily 1 already had some information on this subject tucked away in a shoebox at the back of my closet.

Technical tests

The International Organization for Standardization (ISO) technical committee 168 has defined testing criteria for many lower extremity prosthetic components. While some may not consider these to be outcome measures, components tested to this standard can be considered appropriate for use by most prosthetic clients. In this case, the task of collecting and reporting the outcome measures falls on the manufacturer and nut the practitioner. Canadian organizations have not officially adopted this standard as a requirement for prosthetic components sold in our country. Countries within the European Union will likely be the first to enforce this standard.

Other technological outcome measures are the 1S0 9000 series. These standards help to identify companies that have control systems in place to ensure quality manufacturing. Again, the burden of proof falls on the manufacturer and not the practitioner.

Health Canada has recently revised its medical devices regulations to require quality management procedures for Class II, III, IV device manufacturers (ISO 9002 & ISO 13488 standards for Class 11). Since P&O devices have a Class I risk rating, quality management procedures are not required; however, they are recommended.

Even though the global community is working toward an outcome-based system for testing prosthetic products, many outcome treasures can be used oil a clinical level (especially since the final assembly work is usually done on-site). During manufacturing, a log sheet can he used to verify that the appropriate steps have been used to fabricate and assemble a particular device. Completing this sheet can also serve as a reminder for technical staff. (See Figure One)

Simple outcome measures such as number of adjustments, number of repairs, length of time between repairs, and type of repairs can be collected at the clerical level. If the numbers from these measures surpass a reasonable amount, a more in-depth analysis can be conducted to discover and solve the problem.

In cases where a central fabrication facility is being used to assemble the prosthesis, evidence of outcome-haled, product evaluation procedures should be available to ensure that the practitioner is receiving a consistently -good product. A good facility will not just collect information and use the resultant stack of paper to prop up wobbly tables; manufacturing outcome information should be used to continuously improve service. The practitioner who can rely on the quality of a centrally-fabricated product is then free to focus on adjusting the product to make the best device for the client.

Service tests

A client-focused field like prosthetics and orthotics should also be concerned with quality of service. An ideal service would have no waiting list, offer short delivery times, and provide prompt follow-up. For inpatient facilities, the clients should have a minimal hospital stay, have all their issues resolved, and not have to be readmitted because they\ were discharged too soon. Many of these data call be obtained at the clerical level by cataloguing client visits and the timing of critical events throughout tile fitting process (i.e., date, reason for visit. device manufacturing status.)

Specialized tests

Much of my search for the ultimate outcome measure has focused on measures that call easily be implemented in a clinical situation. Some measures require more effort to capture, but they provide specialized information that can improve prosthetic outcomes.

Quantitative motion analysis is art important outcome measure that is becoming more accessible to health care practitioners with every improvement in computing power. Clinical gait analysis systems are now available that can help you analyze and compare interventions. For example, motion analysis tools (http://www.rohcg.on.ca/cag/mat/) can be downloaded from the Internet and videometrics (http://www.ptexsys.com/prod03.htm) can be purchased to complete simple gait evaluations. Clinical gait evaluation systems typically allow you to step through the motion in 2D, measure joint angles, tuning, stride length, etc. Full gait evaluation facilities call also measure muscle activation (EMG), ground reaction forces, net joint forces and moments. and conduct complete 3D kinematic analyses.

Examples of how these gait outcome measures are useful include: showing a client (and their physician/therapist) how much better they walk with a certain prosthetic approach or discovering if a new foot design has a perceptible effect on motion.

Diagnostic imaging systems (MRI, CT, ultrasound) have been used to collect outcome data on prosthetic function; however, these systems have mainly been used in research efforts. These outcome measures are typically used to analyze the relationship between the prosthetic hardware and the client's anatomy. While diagnostic imaging systems are not currently practical for use in day-to-day practice, these technologies will eventually he integrated into the field as anatomical outcome measures. For example, a client of the future could walk through a real-time MRI machine so that the practitioner could verify that a device properly conforms to the client's anatomy.

Pressure measurement is often cited as an important outcome measure for prosthetics and orthotics. Unfortunately, technical difficulties in obtaining accurate and reliable 3D pressure measures have limited the use of this tool. In-shoe and seating pressure measurements are the only outcome measures that have emerged in P&O clinical practice. While pressure magnitudes are useful for identifying potential problem areas, timing of events in the gait cycle is the best use of these pressure sensors. As clinicians get better at interpreting magnitude and timing data, pressure measurement will continue to evolve as a clinical decision-making tool. In-socket pressure measurement will likely not appear as a regular clinical tool until someone develops art array of appropriate pressure sensors that call be pulled on as easily as a sock.

Another interesting outcome measure is the integration of diagnostic imaging, pressure measurement, tissue property measurements, and finite element analysis to predict the load on a client's physical structures as a result of using a prosthesis. Today's clinicians will not see this technology appear as a clinical tool during their careers; however, future generations will use these models to obtain outcome measures at the tissue level. The usefulness of these measures is starting to appear ill the research area. For example, a recent study by Houston, et al. (1998) used finite element analysis to demonstrate that current trams-tibia (prosthetic designs call produce such high internal loads that they can impact tissue circulation and metabolism. In Dr. Houston's presentation at the ISPO 1998 congress, he stated that some tissue atrophy seen in residual limbs could be directly attributed to circulatory/metabolic restrictions due to the prosthesis design. Better outcome measures could allow a prosthetist to design a device that optimizes both internal pressure distribution and surface loads.

Computer modelling will also be beneficial for predicting how a device functions. For example, if' a leaf-spring type AFO is designed oil a CAD/CAM system and you know the material properties, you can create a computer model of the device. As you apply loads to the model, the computer program provides outcome data on how the AFO deforms. An orthotist could adjust the trim lines and repeat the test until the best compromise between motion and stability has been attained. This use of quantitative measures in the design process could help improve AFO function by reducing the number of devices that err on the safety side (i.e., if you trim too much by hand you must remake the AFO).

Knowledge management

Great ... now 1 have found so many outcome measures that clinical staff could never collect all the data on every client in their practice. "What good is the ultimate outcome measure if I do not have time to collect the data?," I asked myself. "As well, what good is a pile of data if you cannot use it to improve client care?," I responded. Realizing that I was talking to myself, I thought it would be a great time to lie down for a snooze. Almost immediately, I started to dream...

...I was standing in a large room with file cabinets looming all around me. Not a door was in sight. As I moved from cabinet to cabinet, I found each to be locked. I was thoroughly frustrated when I bumped into a spectacled man in a polka-dot bow tie. "I am the knowledge manager," he said in a condescending tone, "stay away from my information - you might mess it up." What sort of setup is this?" I countered, "this is more like knowledge storage than knowledge management." Looking at me over his glasses, the man said, "I have catalogued and indexed all the world's data, I cannot have people ruining my hard work."

Typical, I thought as I eyed the room full of cabinets. We collect all this great information and then let it sit in some database. The main problem is that data is often collected without having a plan in place for its use. This was definitely the time to bring out Bo's Quality Loop (Klasson, 1997). This loop provides a framework for integrating quality improvement and outcome measures into clinical practice:

1. Define the client's goals.
2. Quantitatively measure and document your success rate at satisfying these goals.
3. Define the client's values as a product performance specification (i.e., what constitutes a quality device).
4. Review the methods/technologies used to satisfy the product performance specification.
5. Evaluate the available methods/technologies with reference to 1, 3, and 4.
6. Select the most appropriate method/technology and improve if necessary.
7. If no appropriate methods/technologies exist, develop new methods/technologies (include field tests, official approvals, etc.).
8. Implement the method/technology by preparing facilities, supplies, instructions, training, and environmental conditions. Use outcome measures to monitor these procedures.
9. Develop and implement a quality follow-up protocol that includes a time-schedule and management review intervals (i.e., review the outcome data and make decisions based on the results).
10. Return to stage 1 and repeat the process.

Back to the drawing board...

... I woke up with a start and slowly got to my feet. As I stood, a smile crept up on my face. It seems I had been sent nn a fool's errand. The ultimate outcome measure does not yet exist for prosthetic and orthotic practitioners. Too much work needs to be done on deciding what makes a quality device before people throughout the field can develop common measures. However, my voyage did uncover some useful information.

The PEQ and PPA questionnaires can be used in man situations to get a general view of how well a prosthetic device is functioning. Outcome measures from the ISO standard should be used to evaluate many prosthetic and orthotic components. Gait analysis and pressure measurement tools and starting to appear in clinical settings and many technical am service factors can be monitored by simple outcome measures.

I felt as if my job was finished - for today. Somewhere out there, the ultimate outcome measure waits to be discovered. Until then, the adventure continues...

References

1. Program Evaluation in Physical Rehabilitation &r Mental Health. Institute for Rehabilitation Research and Development, 1996.
2. Grisé ML. Gauthier-Gagnon C, Martineau GG (1993) Prosthetic Profile of People with Lower Extremity Amputation: Conception and Design of a Follow-up Questionnaire. Archives of Physical Medicine and Rehabilitation, 74. 862-870.
3. Houston Vt., Luo G, Mason CP. Beanie AC, LaBlanc KP, Garbarini M, Arena L, Cruise CM (1998) Tissue Biomechanical Modeling and Finite Element Analysis for Prosthetic Socket Design. International Society for Prosthetics and Orthotics World Congress, 597-600.
4. Klasson B. (1997) Appreciation of Prosthetic Socket Fitting from Basic Engineering Principles. National Centre for Training and Education in Prosthetics and Orthotics. Glasgow.
5. Legro MW, Reiber GD, Smith DG, del Aguila D, Larsen J, Boone D (1998) Prosthesis Evaluation Questionnaire for Persons with Lower Limb Amputations: Assessing Prosthesis-Related Quality of Life. Archives of Physical :Medicine and Rehabilitation, 79, 931-938.

Reader Comments: Read all user comments Osseointegrated Finger Prosthesis Hi everyone... I'm looking for any information about " Osseointegrated Finger Prosthesis" and Doctor Manurangsee. If anyone has any information, pictures, or news paper adds... it wouldbe greatly appreciated!! You can e-mail it to: maubicch... read more - Mauro   12/6/2002 John's Corner Hi John, John's Corner looks great!! I'm sure one of the most visited sections on the site. Keep up the fantastic job...as always. I miss ya! Karen Dinsmore - Karen Dinsmore   10/8/2000 HELP!!!!! Hi everyone... I have to do a project on what Geometry has to do with prostehtics. If anyone has any nformation, pictures, or news paper adds... it wouldbe greatly appreciated!! You can e-mail it to me @ Jamizhott10@aol.com Jami - Jami Guido   9/16/2000 Re: HELP!!!!! Take a close look at these prosthetic knees: the Total Knee is a six bar mechanism, the 3R46 is a four bar mech, and the Mauch is a single axis joint. There is a LOT of geometry involved in the design of such joints, for them to swing and... read more - Juan Jose Ochoa   10/12/2004 More details on prepreg carbon fibre orthoses Andreas Wuersching is a friend and very good orthotist who has quite a bit of experience with custom made carbon fibre orthoses using prepreg materials, as highlighted in last month's Corner. He is open to sharing his experiences or answer... read more - John Michael CPO   9/1/2000