카카오톡채널
search for




 

The Impact of Manual Therapy on Pain Catastrophizing in Chronic Pain Conditions: A Systematic Review and Meta-analysis
Phys Ther Rehabil Sci 2023;12:177-84
Published online June 30, 2023
© 2023 Korean Academy of Physical Therapy Rehabilitation Science.

Hyunjoong Kima and Seungwon Leeb*

aNeuromusculoskeletal Science Laboratory
bDepartment of Physical Therapy, Sahmyook University, Republic of Korea
Correspondence to: Seungwon Lee (ORCID https://orcid.org/0000-0002-0413-0510)
Department of Physical Therapy Sahmyook University, 815, Hwarang-ro, Seoul, Republic of Korea
Tel: +82-2-3399-1630 Fax: +82-3399-1639 E-mail: swlee@syu.ac.kr
Received June 16, 2023; Revised June 26, 2023; Accepted June 26, 2023.
cc This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Objective: Manual therapy is a commonly utilized approach in managing chronic pain, but its specific impact on pain catastrophizing remains uncertain. The objective of this systematic review and meta-analysis was to examine the effects of manual therapy on pain catastrophizing in individuals with chronic pain.
Design: A systematic review and meta-analysis
Methods: A comprehensive search was conducted in electronic databases to identify relevant studies published from 2014 onwards. Studies that evaluated the impact of manual therapy on pain catastrophizing in individuals with chronic pain were incorporated. The risk of bias in the selected studies was evaluated using the Cochrane tool for risk of bias in qualitative analysis. For the quantitative analysis, RevMan 5.4 software was utilized, employing a random-effects model as the analysis model. The effect measure used in the analysis was the standardized mean difference (SMD).
Results: In total, 26 studies were collected, and following the screening process, three of them were incorporated into the final analysis. The included studies involved a total of 153 patients with chronic pain. The interventions comprised various manual therapy techniques targeting different areas of the body. Pain catastrophizing and pain intensity were the primary outcomes of interest. The meta-analysis revealed a significant reduction in pain catastrophizing scores following manual therapy intervention compared to control conditions (SMD = -0.91, 95% CI: -1.25 to -0.58). However, heterogeneity between the studies was observed.
Conclusions: Despite the limited quantity and heterogeneity of studies, it has been demonstrated that manual therapy intervention is effective in reducing pain catastrophizing in individuals with chronic pain.
Keywords : Pain, Chroinc pain, Manual therapy, Pain Catastrophizing
Introduction

Chronic pain is a significant global health concern, affecting a substantial portion of the population [1]. It leads to physical, psychological, and social impairments, significantly impacting the quality of life for individuals afflicted by this condition [2, 3]. Pain catastrophizing, characterized by an exaggerated negative assessment of pain and its consequences, has been recognized as a crucial psychological factor contributing to the experience of chronic pain [4-6].

Manual therapy, a widely used approach in the management of chronic pain, encompasses a range of hands-on techniques applied to soft tissues and joints [7-9]. This therapy aims to reduce pain, improve function, and enhance overall well-being [10, 11]. According to clinical practice guidelines for chronic low back pain, manual therapy is recommended as strong evidence[12]. Although manual therapy is frequently utilized in clinical practice, its effect on pain catastrophizing in individuals with chronic pain remains uncertain [13].

To bridge this knowledge gap, a systematic review and meta-analysis were conducted to investigate the influence of manual therapy on pain catastrophizing in individuals with chronic pain. Through the synthesis of available evidence, this study aims to provide a comprehensive understanding of the effects of manual therapy on pain catastrophizing. These findings have the potential to inform clinical decision-making and optimize treatment strategies for patients suffering from chronic pain.

Methods

Study design

The present study represents a systematic review and meta-analysis that aims to explore the influence of manual therapy on pain catastrophizing among individuals suffering from chronic pain. Our review strictly adheres to the established guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) to ensure methodological rigor and transparency in reporting the findings.

Search strategy and selection of studies

Inclusion criteria

Inclusion criteria were classified using PICOSD (P =participants, I=intervention, C=comparisons, O=outcomes, and S=study design), which are key methological questions of systematic review and meta-analysis.

Participants included chronic pain, persistent pain, and recurrent pain conditions. Interventions included manual therapy, manipulative medicine, joint mobilization, soft tissue mobilization, manipulation, thrust manipulation, and nonthrust manipulation. Comparison applies to the group that does not include manual therapy, but if not, a comparison before and after intervention was added. Outcomes included pain catastrophizing and pain intensity, an evaluation tool for pain catastrophism. The study design is a randomized controlled trial (RCT).

Exclusion criteria

Studies after 2014 were included, and studies not written in English or for which data were not provided were excluded.

Literature-search strategy

For this review, a comprehensive search was conducted in May 2023 to identify relevant studies published from 2014 onwards. The search process was carried out independently by the researchers involved, all of whom possessed experience in conducting meta-analyses. The search strategy utilized the following terms: (chronic pain OR persistent pain OR recurrent pain) AND (manual therapy OR manipulative medicine OR joint mobilization OR soft tissue mobilization OR manipulation OR non-thrust manipulation OR thrust manipulation) AND (pain catastrophizing) AND (randomized controlled trial). International electronic databases searched by researchers used Medical literature analysis and retrieval system online (MEDLINE), Web of Science, and PubMed.

Study selection and data extraction

The studies obtained from the database search were imported into reference management software (EndNote 20, Thomson Reuters, USA), and duplicate records were removed. After the deduplication process, the researchers independently evaluated the titles and abstracts of the remaining articles according to the PRISMA guidelines. Subsequently, the full texts of the selected studies were thoroughly reviewed, and any discrepancies were resolved through consultations among the researchers.

Risk of bias assessment

The risk of bias (RoB) in the RCTs included in the analysis was evaluated using a specific tool for assessing risk of bias [14]. The researchers assigned ratings of low (+), uncertain (?), or high (-) to indicate the level of RoB for each item. In cases where discrepancies arose in the results across different items, such as random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other biases, a re-evaluation was conducted after reaching a consensus among the researchers.

Strategy for Data Synthesis

The selected studies were synthesized using RevMan 5.4 software for quantitative analysis. Given the observed heterogeneity among the studies, a random effects model was utilized as the analysis model. The effect measure employed was the standardized mean difference (SMD). To evaluate the heterogeneity between the studies, both the chi-square test and I2 test were utilized. A high degree of heterogeneity was indicated if the I2 value exceeded 75%, while a low degree of heterogeneity was indicated if it was below 40% [15].

Results

Literature search and characteristics of the included studies

By conducting a search across three international electronic databases, a total of 26 studies were identified. After removing four duplicate studies using EndNote 20, the researchers reviewed the titles and abstracts of the remaining 22 studies. Through this initial screening, 16 studies were excluded. Subsequently, a comprehensive evaluation of the full texts of the remaining studies led to the exclusion of three studies that did not correspond to the outcome items specified in the PICOSD framework. As a result, three studies that met the search criteria were included in the analysis [16-18] (Figure 1).

Risk of bias assessment of the manual therapy applied to chronic pain conditions

After conducting individual RoB assessments for each study, the researchers arrived at a consensus on the results through a consensus process. The RoB assessment results for three studies were as follows: random sequence generation (low: 3), allocation concealment (low: 2, uncertain: 1), blinding of participants and personnel (low: 3), blinding outcome assessment (low: 3), incomplete outcome data (low: 2, uncertain: 1), selective reporting (low: 3), and other bias (low: 2, uncertain: 1) (Figure 2). The evaluation of other bias was determined by referring to a systematic survey [19].

Effectiveness of manual therapy on chronic pain

The review included three studies involving a total of 153 patients with chronic pain. The focus of the analysis was to assess the effectiveness of manual therapy in the management of chronic pain conditions. The participants in these studies had chronic neck and back pain, and the intervention provided was manual therapy. However, it is important to note that the specific techniques and areas of application varied among the included studies. The outcomes of interest were synthesized based on measures of catastrophizing and pain intensity. The duration of the intervention was consistent across all included studies, spanning a period of four weeks (or one month) (Table 1).

Effects of manual therapy on pain catastrophizing of chronic pain

Figure 3 displays a forest plot illustrating the changes in pain catastrophizing before and after manual therapy (Figure 3; 1.1.1) and the comparison with the control group (Figure 3; 1.1.2). The analysis, conducted using a random-effects model, revealed a significant improvement in pain catastrophizing following manual therapy (SMD=-0.91, 95% confidence interval [CI]: -1.25 to -0.58, heterogeneity (x2=11.49, df=6, I2=48%), and overall effect (Z=5.34, p<0.0001)). Subgroup analysis indicated a significant improvement before and after manual treatment (SMD=-0.97, 95% CI: -1.25 to -0.58, heterogeneity (x2=6.28, df=4, I2=36%), and overall effect (Z=5.07, p<0.0001)), but no significant improvement was observed when comparing with the control group (SMD=-0.77, 95% CI: -1.67 to 0.12, heterogeneity (x2=4.79, df=1, I2 =79%), and overall effect (Z=1.70, p=0.09)).

Effects of manual therapy on pain intensity of chronic pain

Figure 4 displays a forest plot illustrating the changes in pain intensity before and after manual therapy (Figure 4; 1.2.1) and the comparison with the control group (Figure 4; 1.2.2). The analysis, conducted using a random-effects model, revealed a significant improvement in pain catastrophizing following manual therapy (SMD=-1.02, 95% CI: -1.87 to -0.17, heterogeneity (x2=67.59, df=6, I2=91%), and overall effect (Z=2.35, p=0.02)). Subgroup analysis showed no significant improvement before or after manual treatment (SMD=-1.19, 95% CI: -2.39 to 0.01, heterogeneity (x2=54.93, df=4, I2=93%), and overall effect (Z=1.95, p=0.05)), and no significant improvement was observed when compared with the control group (SMD=-0.63, 95% CI: -1.96 to 0.71, heterogeneity (x2=10.68, df=1, I2=91%), and overall effect (Z=0.92, p=0.36)).

Discussion

Chronic pain is a common and debilitating condition that has a significant impact on the lives of individuals affected by it [20]. Pain catastrophizing, characterized by an exaggerated negative perception of pain and its consequences, is recognized as a crucial psychological factor contributing to the experience of chronic pain [21, 22]. The objective of this systematic review and meta-analysis was to investigate the effects of manual therapy on pain catastrophizing in individuals with chronic pain.

The findings of this study suggest that manual therapy has a notable effect in reducing pain catastrophizing in chronic pain conditions. The forest plot analysis demonstrates a clear improvement in pain catastrophizing before and after manual therapy (SMD=-0.97, 95% CI: -1.25 to -0.58) (Figure 3; 1.1.1). This indicates that manual therapy interventions effectively decrease the negative appraisal of pain and its consequences, leading to a more adaptive cognitive response to pain among individuals with chronic pain.

However, when comparing the manual therapy group with the control group, no significant improvement in pain catastrophizing was observed (SMD=-0.77, 95% CI: -1.67 to 0.12) (Figure 3; 1.1.2). Several factors could potentially contribute to this result. First, the heterogeneity among the included studies, including variations in the techniques and areas of manual therapy application, may have influenced the overall effect. Second, the control groups in the studies might have received other interventions or treatments that could have affected their pain catastrophizing scores. Additionally, the limited number of studies included in the analysis might have reduced the statistical power to detect significant differences between the manual therapy and control groups.

The results of this review indicate that manual therapy interventions have a positive impact on reducing pain catastrophizing in individuals. This finding aligns with the understanding that manual therapy can provide physical benefits, such as pain reduction and improved function, which may contribute to a more positive perception of pain and its consequences [13, 23, 24]. Furthermore, the hands-on nature of manual therapy may also have a therapeutic effect on psychological aspects, including pain catastrophizing, by promoting relaxation, reducing stress, and enhancing the therapeutic relationship between the therapist and the patient [25-27].

It is important to note that the effectiveness of manual therapy in reducing pain catastrophizing, as observed in this review, does not imply that manual therapy should be used as a standalone treatment for chronic pain. Instead, it should be considered as part of a multimodal approach that incorporates comprehensive pain management strategies, such as exercise, education, and psychological interventions [28-30].

This review has several limitations. Firstly, the number of studies included was relatively small, which limits the generalizability of the findings. Further research with larger sample sizes is necessary to provide more robust evidence. Secondly, there was heterogeneity in the interventions and outcome measures employed across the included studies, which may have influenced the results. Future studies should strive for greater consistency in the application of manual therapy techniques and utilize standardized outcome measures to allow for more accurate comparisons and meta-analyses.

Conclusion

In this systematic review and meta-analysis, compelling evidence is presented to support the efficacy of manual therapy in alleviating pain catastrophizing among individuals suffering from chronic pain conditions. The results demonstrate a noteworthy reduction in pain catastrophizing following manual therapy interventions. However, the considerable diversity observed among the studies included and the limited quantity of research conducted emphasize the necessity for further investigation to validate these findings and determine the most effective techniques and treatment parameters. Manual therapy emerges as a valuable component within comprehensive pain management strategies, effectively addressing the physical and psychological dimensions of chronic pain.

Conflicts of interest

The authors declare no conflict of interest.

Figures
Fig. 1. PRISMA flow diagram
Fig. 2. Risk of bias summary
Fig. 3. Forest plot on pain catastrophizing. Lóez-de-Uralde-Villanueva, et al., 2020(a): manual therapy; Lóez-de-Uralde-Villanueva, et al., 2020(b): manual therapy plus therapeutic patient education based on a biobehavioral approach; Lóez-de-Uralde-Villanueva, et al., 2020(c): manual therapy, therapeutic patient education based on a biobehavioral approach, therapeutic exercise; MartíSalvador, et al., 2018(a): before and after comparison of manual therapy; MartíSalvador, et al., 2018(b): comparison of manual therapy and control group; Zafereo, et al. 2018(a): before and after comparison of manual therapy; Zafereo, et al. 2018(b): comparison of manual therapy and control group.
Fig. 4. Forest plot on pain intensity. Lóez-de-Uralde-Villanueva, et al., 2020(a): manual therapy; Lóez-de-Uralde-Villanueva, et al., 2020(b): manual therapy plus therapeutic patient education based on a biobehavioral approach; Lóez-de-Uralde-Villanueva, et al., 2020(c): manual therapy, therapeutic patient education based on a biobehavioral approach, therapeutic exercise; MartíSalvador, et al., 2018(a): before and after comparison of manual therapy; MartíSalvador, et al., 2018(b): comparison of manual therapy and control group; Zafereo, et al. 2018(a): before and after comparison of manual therapy; Zafereo, et al. 2018(b): comparison of manual therapy and control group.
Tables

Table 1

Characteristics of included studies

Study Sample size Therapeutic intensity Outcomes
Lóez-de-Uralde-Villanueva, et al., 2020 [16] CNCP
EG1=15
EG2=16
EG3=16
A total of 8 individual treatment sessions were received twice a week for 1 month. EG1=manual therapy; EG2=manual therapy plus therapeutic patient education based on a biobehavioral approach; EG3=manual therapy, therapeutic patient education based on a biobehavioral approach, therapeutic exercise. Catastrophizing=PCS
Intensity=VAS
MartíSalvador, et al., 2018 [17] CNLBP
EG=33
CG=33
In the study, both groups received a total of 5 manual therapy sessions. The first 4 sessions were administered twice a week, and the fifth session was scheduled to take place 1 month after the first session. On average, each session had a duration of 45 minutes. The control group received the sham treatment. Catastrophizing=PCS
Intensity=VAS
Zafereo, et al. 2018 [18] CLBP
EG=20
CG=20
It consists of 30 minutes per session twice a week for 4 weeks, 10 minutes of manual therapy and 20 minutes of motor control exercise program.
EG=local lumbar spine manual therapy plus regional thoracic, pelvic, and hip manual therapy; CG=local lumbar spine manual therapy
Catastrophizing=PCS
Intensity=NPRS

CG, control group; CNCP, chronic nonspecific neck pain; CLBP, chronic low back pain; CNLBP, chronic nonspecific low back pain; EG, experimental group; NPRS, numeric pain rating scale; PCS, pain catastrophizing scale; VAS, visual analog scale.


References
  1. Cohen SP, Vase L, Hooten WM. Chronic pain: an update on burden, best practices, and new advances. Lancet. 2021;397:2082-97.
    Pubmed CrossRef
  2. Clauw DJ, Essex MN, Pitman V, Jones KD. Reframing chronic pain as a disease, not a symptom: rationale and implications for pain management. Postgrad Med. 2019;131:185-98.
    Pubmed CrossRef
  3. Capano A, Weaver R, Burkman E. Evaluation of the effects of CBD hemp extract on opioid use and quality of life indicators in chronic pain patients: a prospective cohort study. Postgrad Med. 2020;132:56-61.
    Pubmed CrossRef
  4. Miller MM, Meints SM, Hirsh AT. Catastrophizing, pain, and functional outcomes for children with chronic pain: a meta-analytic review. Pain. 2018; 159:2442-60.
    Pubmed KoreaMed CrossRef
  5. Darnall BD, Sturgeon JA, Cook KF, Taub CJ, Roy A, Burns JW, et al. Development and Validation of a Daily Pain Catastrophizing Scale. J Pain. 2017; 18:1139-49.
    Pubmed KoreaMed CrossRef
  6. Elphinston RA, Sullivan MJL, Sterling M, Connor JP, Baranoff JA, Tan D, et al. Pain Medication Beliefs Mediate the Relationship Between Pain Catastrophizing and Opioid Prescription Use in Patients With Chronic Non-Cancer Pain. J Pain. 2022;23:379-89.
    Pubmed CrossRef
  7. Bernal-Utrera C, Gonzalez-Gerez JJ, Anarte-Lazo E, Rodriguez-Blanco C. Manual therapy versus therapeutic exercise in non-specific chronic neck pain: a randomized controlled trial. Trials. 2020;21:682.
    Pubmed KoreaMed CrossRef
  8. O'Keeffe M, O'Sullivan P, Purtill H, Bargary N, O'Sullivan K. Cognitive functional therapy compared with a group-based exercise and education intervention for chronic low back pain: a multicentre randomised controlled trial (RCT). Br J Sports Med. 2020;54:782-9.
    Pubmed KoreaMed CrossRef
  9. Llamas-Ramos R, Pecos-Martín D, Gallego-Izquierdo T, Llamas-Ramos I, Plaza-Manzano G, Ortega-Santiago R, et al. Comparison of the short-term outcomes between trigger point dry needling and trigger point manual therapy for the management of chronic mechanical neck pain: a randomized clinical trial. J Orthop Sports Phys Ther. 2014;44:852-61.
    Pubmed CrossRef
  10. Jacobson E. Structural integration, an alternative method of manual therapy and sensorimotor education. J Altern Complement Med. 2011;17:891-9.
    Pubmed KoreaMed CrossRef
  11. Cumplido-Trasmonte C, Fernández-González P, Alguacil-Diego I, Molina-Rueda F. Manual therapy in adults with tension-type headache: a systematic review. Neurologia. 2021;36:537-47.
    Pubmed CrossRef
  12. George SZ, Fritz JM, Silfies SP, Schneider MJ, Beneciuk JM, Lentz TA, et al. Interventions for the Management of Acute and Chronic Low Back Pain: Revision 2021. J Orthop Sports Phys Ther. 2021; 51:Cpg1-cpg60.
    Pubmed CrossRef
  13. Kamonseki DH, Christenson P, Rezvanifar SC, Calixtre LB. Effects of manual therapy on fear avoidance, kinesiophobia and pain catastrophizing in individuals with chronic musculoskeletal pain: Systematic review and meta-analysis. Musculoskelet Sci Pract. 2021;51:102311.
    Pubmed CrossRef
  14. Kim H, Jung J, Park S, Joo Y, Lee S, Lee S. Effects of Repetitive Transcranial Magnetic Stimulation on the Primary Motor Cortex of Individuals with Fibromyalgia: A Systematic Review and Meta-Analysis. Brain Sci. 2022;12:570.
    Pubmed KoreaMed CrossRef
  15. Deeks JJ, Higgins JP, Altman DG, Group CSM. Analysing data and undertaking meta‐nalyses. Cochrane handbook for systematic reviews of interventions. 2019:241-84.
    CrossRef
  16. López-de-Uralde-Villanueva I, Beltran-Alacreu H, Fernández-Carnero J, La Touche R. Pain management using a multimodal physiotherapy program including a biobehavioral approach for chronic nonspecific neck pain: a randomized controlled trial. Physiother Theory Pract. 2020;36:45-62.
    Pubmed CrossRef
  17. Martí-Salvador M, Hidalgo-Moreno L, Doménech-Fernández J, Lisón JF, Arguisuelas MD. Osteopathic Manipulative Treatment Including Specific Diaphragm Techniques Improves Pain and Disability in Chronic Nonspecific Low Back Pain: A Randomized Trial. Arch Phys Med Rehabil. 2018;99:1720-9.
    Pubmed CrossRef
  18. Zafereo J, Wang-Price S, Roddey T, Brizzolara K. Regional manual therapy and motor control exercise for chronic low back pain: a randomized clinical trial. J Man Manip Ther. 2018;26:193-202.
    KoreaMed CrossRef
  19. Babic A, Pijuk A, Brázdilová L, Georgieva Y, Raposo Pereira MA, Poklepovic Pericic T, et al. The judgement of biases included in the category “other bias” in Cochrane systematic reviews of interventions: a systematic survey. BMC Med Res Methodol. 2019;19:1-10.
    Pubmed KoreaMed CrossRef
  20. Fillingim RB, Loeser JD, Baron R, Edwards RR. Assessment of Chronic Pain: Domains, Methods, and Mechanisms. J Pain. 2016;17:T10-20.
    Pubmed KoreaMed CrossRef
  21. Meints SM, Edwards RR. Evaluating psychosocial contributions to chronic pain outcomes. Prog Neuropsychopharmacol Biol Psychiatry. 2018;87: 168-82.
    Pubmed KoreaMed CrossRef
  22. Malfliet A, Coppieters I, Van Wilgen P, Kregel J, De Pauw R, Dolphens M, et al. Brain changes associated with cognitive and emotional factors in chronic pain: A systematic review. Eur J Pain. 2017;21:769-86.
    Pubmed CrossRef
  23. Kim H, Lee S. The Efficacy of Pain Neuroscience Education on Active Rehabilitation Following Arthroscopic Rotator Cuff Repair: A CONSORTCompliant Prospective Randomized Single-Blind Controlled Trial. Brain Sci. 2022;12.
    Pubmed KoreaMed CrossRef
  24. Martinez-Calderon J, Flores-Cortes M, Morales-Asencio JM, Luque-Suarez A. Pain Catastrophizing, Opioid Misuse, Opioid Use, and Opioid Dose in People With Chronic Musculoskeletal Pain: A Systematic Review. J Pain. 2021;22:879-91.
    Pubmed CrossRef
  25. Nadal-Nicolás Y, Rubio-Arias J, Martínez-Olcina M, Reche-García C, Hernández-García M, Martínez-Rodríguez A. Effects of Manual Therapy on Fatigue, Pain, and Psychological Aspects in Women with Fibromyalgia. Int J Environ Res Public Health. 2020;17.
    Pubmed KoreaMed CrossRef
  26. Alansari SM, Youssef EF, Shanb AA. Efficacy of manual therapy on psychological status and pain in patients with neck pain. A randomized clinical trial. Saudi Med J. 2021;42:82-90.
    Pubmed KoreaMed CrossRef
  27. Weerapong P, Hume PA, Kolt GS. The mechanisms of massage and effects on performance, muscle recovery and injury prevention. Sports Med. 2005;35: 235-56.
    Pubmed CrossRef
  28. Song J, Kim H, Jung J, Lee S. Soft-Tissue Mobilization and Pain Neuroscience Education for Chronic Nonspecific Low Back Pain with Central Sensitization: A Prospective Randomized Single-Blind Controlled Trial. Biomedicines. 2023;11.
    Pubmed KoreaMed CrossRef
  29. Watson JA, Ryan CG, Cooper L, Ellington D, Whittle R, Lavender M, et al. Pain Neuroscience Education for Adults With Chronic Musculoskeletal Pain: A Mixed-Methods Systematic Review and Meta-Analysis. J Pain. 2019;20:1140.e1-.e22.
    Pubmed CrossRef
  30. Kim H, Lee S. Effects of pain neuroscience education on kinesiophobia in patients with chronic pain: a systematic review and meta-analysis. Phys Ther Rehabil Sci. 2020;9:309-17.
    CrossRef

 

Full Text(PDF) Free

Cited By Articles
  • CrossRef (0)
  • Authorship and ethical issues