Value of apparent diffusion coefficient (ADC) in evaluating response of carcinoma of the cervix treated with chemoradiotherapy
Department of Diagnostic Radiology, Faculty of Health Sciences, University of the Free State, Bloemfontein
A le Roux, MB ChB
F Gebremariam, MD, FC Rad Diag (SA)
W Harmse, MB ChB, MMed (Rad), FC Rad Diag (SA)
Objective. To determine whether the apparent diffusion coefficient (ADC) value obtained by diffusion-weighted magnetic resonance imaging (DW-MRI) can be used as a reliable detector of response of carcinoma of the cervix treated with chemoradiotherapy, compared with conventional T2-weighted MRI.
Design. A prospective cohort study was performed.
Setting. Department of Oncology, Universitas-National Hospital Complex, Bloemfontein.
Subjects. Seventeen women with advanced cervical cancer, FIGO staging IIB - IVB, were selected for chemoradiation.
Outcome measures. Patients underwent pelvic MRI before therapy, 14 days after onset of therapy, and in the last week of treatment (5th/6th week). Axial and sagittal conventional T2 was followed by DW-MRI in the axial plane from which a tumour region of interest (ROI) was manually drawn to calculate ADC values using b-values of 500 and 1 000 s/mm2.
Results. ADC values for cervical carcinoma increased after treatment with chemoradiation. The most significant observation was seen 14 days after treatment was started. The mean ADC value increased with 20% (b=500 s/mm2) and 24% (b=1 000 s/mm2) (statistically significant, p<0.05) compared with a decrease in tumour size of only 8%, which was not statistically significant (p=0.075). Responders showed a larger change in ADC values than non-responders.
Conclusion. The study showed considerable promise in the ability of ADC to identify early tumour response to therapy. DW-MRI is a non-invasive functional imaging technique that may in future change management in oncology by early identification of non-responders, hence avoiding unnecessary treatment.
S Afr J Rad 2013;17(1):11-15. DOI:10.7196/SAJR.752
Cervical carcinoma is the third most common gynaecological malignancy in women. It typically occurs in women who are on average 45 years old at onset of the disease.1 The Federation of Gynaecology and Obstetrics (FIGO) system for staging of cervical carcinoma (classification IIB - IVB) is widely used for treatment planning, and also for standardisation of epidemiological and treatment results. The system is based on clinical examination to assess tumour extent and parametrium status, and histological samples obtained by colposcopic biopsy or diagnostic conisation. There are, however, significant inaccuracies in the FIGO system, with a 24 - 39% error rate in gynaecological examinations. Furthermore, two important prognostic factors, i.e. lesion volume and nodal metastases, are not assessed.2
Owing to its high-contrast resolution and direct multiplanar imaging capability, conventional T2-weighted magnetic resonance imaging (MRI) can clearly demarcate cervical carcinomas and is the method of choice for evaluation of the lymph nodes. Additionally, MRI provides a non-invasive means of evaluating the pelvis, making it a widely accepted method for assessment of the main prognostic factors and selection of therapeutic strategies for cervical carcinoma.3 , 4 T2-weighted MRI can also be used to evaluate the response of cervical carcinoma to treatment. This application, however, only relies on evaluating the size of the lesion, and a delayed response of tumour shrinkage after therapy could potentially be missed. Initial response to treatment, such as cell death, causes increased diffusion of water molecules within the tumour which could be assessed by diffusion-weighted MRI.
Aim
The aim of our study was to determine whether the apparent diffusion coefficient (ADC) values could be used as a reliable detector of response in carcinoma of the cervix treated with chemoradiotherapy as compared with conventional morphological MRI response.5-7
Methods
A prospective cohort study was conducted at the Faculty of Health Sciences of the University of the Free State (UFS), in the Universitas and National Hospital Complex, in Bloemfontein. Women of all ages and race with advanced cervical cancer, FIGO staging (IIB - IVB), were selected between June 2010 and August 2011 by the Department of Oncology to participate in the study. The clinical criteria included biopsy-proven untreated cancer of the cervix that was suitable for chemoradiotherapy, using the FIGO staging classification of carcinoma of the cervix. Approval to perform the investigation was obtained from the Ethics Committee of the Faculty of Health Sciences (UFS). All subjects gave written informed consent.
MRI protocol
All examinations were done on a 1.5 Tesla MRI machine (GE Healthcare, Buckinghamshire, UK) using an 8-channel body coil. Axial and sagittal T2-weighted images and axial diffusion-weighted images were acquired. Imaging parameters for T2-weighted fast recovery fast spin echo (FRFSE) were the following: repetition time (TR) 4 225 - 6 200 ms, echo time (TE) 15 - 100 ms, echo train length 15 kHz, bandwidth 20.83 kHz, field of view (FOV) 22 cm, slice thickness 4 mm, gap 1 mm, number of excitations 4, matrix 256 x 256, and acquisition time 4 minutes. Once the tumour had been visualised on conventional T2-weighted imaging, diffusion-weighted images were obtained in the axial plane that contained the largest tumour cross-section, using b-values of 500 and 1 000 s/mm2. Imaging parameters for these were TR 5 200 ms, TE 69.3 ms, bandwidth 62.5 kHz, FOV 32 cm, slice thickness 6 mm, gap 0 mm, number of excitations 16, matrix 128 x 128, and acquisition time 5 minutes. All patients were imaged before commencement of chemoradiation therapy, after 2 weeks of therapy, and during the final week of therapy (either the 5th or 6th week), depending on the exact treatment protocol instituted by the Department of Oncology.
T2-weighted imaging in the sagittal and axial planes was used for localisation of the tumour, measurement of its dimensions and assessment of morphological response. The tumour size was calculated as maximum craniocaudal x transverse diameter (cm2) and was expressed as a percentage decrease in tumour size according to the time protocol, as well as the mean from start to completion of treatment. ADC maps were created on a GE Advantage 4.1 workstation. A tumour region of interest (ROI) was manually drawn on the diffusion-weighted images by using the highest signal intensity areas within the tumour, avoiding reduced signal intensity areas which could indicate necrotic areas and potentially bias the results. The ROIs were then superimposed on the ADC maps and a mean ADC value (x 10–3 mm2/s) was determined by the workstation. To assist in follow-up measurements, an anatomic pelvic bony point was located, such as the acetabular notch or symphysis pubis, at the level of the ROI and used as reference point on follow-up scans.
ADC values were determined with b-values of 500 and 1 000 s/mm2, respectively. All measurements were expressed quantitatively and as a percentage change from the previous measurements. We evaluated results collectively, as well as in 2 groups comparing responders and non-responders. Responders were categorised as those patients with a decrease in tumour size of more than 30% at the end of treatment. Non-responders were regarded as those patients in whom the tumour size either increased or decreased by less than 30% at the end of treatment.
Results
Twenty patients were initially included in the
study. Patients’ ages ranged from 30 to 80 years, with a mean of
60 years. All had advanced cervical cancer (FIGO staging IIB -
IVB) and clinical indications for chemoradiotherapy. Data of 17
patients were included in the analysis, with 3 patients (numbers
6, 17 and 19) being excluded. One patient died before therapy
could be completed. Two patients’ MRI studies were unsuitable
for evaluation owing to motion artefacts. Fig. 1 shows an
example of the pelvic axial MRI of one of the patients.
Fig. 1. Pelvic axial MRI of a patient
diagnosed with carcinoma of the cervix. (a) Pre-treatment
study. (b) Images obtained 14 days after the onset of
treatment. Sequences from left to right are T2, DWI and ADC.
On average, the ADC values increased during
the study and tumour sizes decreased. The ADC values (calculated
with b-values of 500 and 1 000 s/mm2)
and tumour sizes collected over the time of the study are shown
in Table 1.
Table 1. Comparison of ADC values in axial plane with b-values of 500 and 1 000 s/mm 2 and tumour size in women with cervical cancer, FIGO staging IIB - IVB |
|||||||||||
Patient no. |
ADC x 10–3 mm2/s; b-values 500 and 1 000 s/mm2 |
|
Tumour size (cm2) |
||||||||
ADC 500 |
|
ADC 1 000 |
|
||||||||
Initial value |
After 14 days |
Final week |
|
Initial value |
After 14 days |
Final week |
|
Initial size |
After 14 days |
Final week |
|
1 |
1.26 |
1.45 |
2.10 |
|
1.02 |
1.23 |
1.46 |
|
29.24 |
35.09 |
9.56 |
2 |
1.21 |
1.39 |
1.66 |
|
0.99 |
1.14 |
1.63 |
|
33.80 |
17.10 |
10.17 |
3 |
1.35 |
1.43 |
1.73 |
|
0.99 |
1.23 |
1.43 |
|
14.31 |
9.3 |
7.54 |
4 |
0.86 |
1.31 |
1.43 |
|
0.64 |
1.13 |
1.25 |
|
5.24 |
2.27 |
3.29 |
5 |
1.11 |
1.49 |
1.90 |
|
0.93 |
1.30 |
1.53 |
|
13.81 |
15.35 |
5.61 |
7 |
1.14 |
1.30 |
1.51 |
|
0.95 |
1.17 |
1.29 |
|
25.38 |
26.69 |
9.12 |
8 |
1.71 |
1.81 |
2.00 |
|
1.38 |
1.49 |
1.58 |
|
6.7 |
4.96 |
5.84 |
9 |
1.20 |
1.52 |
2.29 |
|
0.94 |
1.15 |
1.51 |
|
19.50 |
17.57 |
4.72 |
10 |
1.32 |
1.75 |
2.02 |
|
1.01 |
1.42 |
1.41 |
|
13.8 |
12.09 |
5.77 |
11 |
1.03 |
1.32 |
0.15 |
|
1.00 |
1.09 |
1.29 |
|
22.15 |
20.00 |
18.18 |
12 |
1.71 |
1.59 |
2.10 |
|
1.41 |
1.32 |
1.83 |
|
13.33 |
16.00 |
13.45 |
13 |
1.61 |
1.35 |
1.51 |
|
1.38 |
1.19 |
1.18 |
|
12.09 |
12.46 |
12.71 |
14 |
1.25 |
1.59 |
1.71 |
|
0.78 |
1.27 |
1.54 |
|
20.68 |
17.53 |
13.76 |
15 |
0.93 |
1.94 |
2.15 |
|
0.87 |
1.46 |
1.71 |
|
10.61 |
13.71 |
7.67 |
16 |
1.28 |
1.37 |
1.47 |
|
0.98 |
1.10 |
1.25 |
|
58.36 |
49.37 |
21.96 |
18 |
1.41 |
1.32 |
1.56 |
|
1.13 |
1.24 |
1.23 |
|
30.25 |
30.46 |
19.07 |
20 |
1.73 |
1.86 |
2.08 |
|
1.46 |
1.54 |
1.52 |
|
21.60 |
22.15 |
12.58 |
Mean |
1.30 |
1.52 |
1.73 |
|
1.05 |
1.26 |
1.45 |
|
20.64 |
18.95 |
10.65 |
During the first 14 days, the ADC values
increased by a mean of 20% and 24% for b=500 s/mm2 and b=1 000 s/mm2, respectively, while tumour sizes
decreased by a mean of only 8%. These results are summarised in
Table 2.
Table 2. Mean ADC values and tumour sizes with percentage change during the study |
||||
|
Initial value |
At 14 days |
Day 14 - last day |
Initial - last day |
ADC 500 value |
|
|
|
|
Mean (x 10–3 mm2/s) |
1.30 |
1.52 |
1.73 |
– |
Percentage change |
– |
20% |
13% |
35% |
ADC 1 000 value |
|
|
|
|
Mean (x 10–3 mm2/s) |
1.05 |
1.26 |
1.45 |
– |
Percentage change |
– |
24% |
15% |
43% |
Tumour size |
|
|
|
|
Mean size (cm2) |
20.64 |
18.95 |
10.05 |
– |
Percentage change |
– |
8% |
32% |
42% |
The relationship between the mean ADC values and tumour sizes is
reflected graphically in Fig. 2, demonstrating the early
detection of tumour response using ADC values compared with
tumour sizes, where response was seen at a later stage in the
treatment regime.
Fig. 2. Percentage change in ADC 500 and ADC 1 000 values and
tumour size during different periods of the study.
In only 2 (11.8%) patients did tumour size not
decrease by the final week of treatment, and a further 3 (17.6%)
patients’ tumour size decreased by less than 30%. These 5
patients were considered to be non-responders in the analysis
and were compared with the 12 (70.6%) responders in whom tumour
sizes decreased by more than 30%. In general, the changes were
larger in responders than in non-responders. Table 3 compares
the ADC values and sizes of the responders and non-responders.
Table 3. Comparison of ADC values and tumour size in responders and non-responders to chemoradiotherapy |
|||||||
|
Responders ( n =12) |
|
Non-responders ( n =5) |
||||
ADC 500 (x 10–3 mm2/s) |
ADC 1 000 (x 10–3 mm2/s) |
Tumour size (cm2) |
|
ADC 500 (x 10–3 mm2/s) |
ADC 1000 (x 10–3 mm2/s) |
Tumour size (cm2) |
|
Mean value and size |
|
|
|
|
|
|
|
Day 1 |
1.26 |
0.99 |
23.83 |
|
1.40 |
1.21 |
12.98 |
Day 14 |
1.48 |
1.24 |
21.25 |
|
1.60 |
1.31 |
13.43 |
Last day |
1.79 |
1.42 |
10.26 |
|
1.58 |
1.52 |
11.57 |
Percentage increase |
|
|
|
|
|
|
|
Day 1 - 14 |
19.30 |
29.30 |
-12.87 |
|
23.90 |
12.90 |
3.33 |
Day 14 - last |
20.80 |
15.00 |
-41.63 |
|
-4.70 |
15.90 |
-9.87 |
Day 1 - last |
44.00 |
48.80 |
-54.48 |
|
15.90 |
31.10 |
-10.49 |
Discussion
In 2000, Therasse et al. published a set of rules to evaluate tumour response (morphological) during treatment.8 The most recent version (RECIST 1.19 ) suggests that the functional methods for tumour response assessment include positron emission tomography (PET) and PET-computed tomography (PET-CT) using 18-fluorodeoxyglucose (18FDG) as a tracer, and dynamic contrast-enhanced MRI and MR spectroscopy. DW-MRI has so far not been considered in RECIST 1.1. This MR technique as a cancer biomarker has been discussed at a meeting of the International Society for MR in Medicine (ISMRM) in 2008 and, since then, many reports have been published highlighting the potential of this promising new technique in cancer patients. In 1990, Chenevert et al.10 established the initial use of this principle of DWI-MRI in the treatment of brain tumours and found that malignant tumours, owing to their high cellularity, have a restriction of water movement (diffusion) reflected by a low ADC value; and, when the tumours respond to treatment, the ADC value will increase (less restriction of water movement) because of a decrease in tumour cellularity owing to apoptotic cell death. This indication will appear well before changes in tumour volume.11 In 2008, McVeigh et al. found that the mean ADC value of cervical carcinoma (1.09 x 10–3 mm2/s) was significantly lower than for normal cervix tissue (2.09 x 10–3 mm2/s).12 At present, its use in female pelvic tumours, especially cervical cancers, is limited.9-13
The ADC value is calculated after DW-MRI that derives its contrast from the diffusion of water molecules (Brownian motion) within tumour tissue. Cell lysis is often the first effect of many different types of therapy, and theoretically this would lead to an increase in water diffusion, and therefore an increased ADC value. DW-MRI essentially involves T2-weighted MRI sequences during which a dephasing gradient is applied, followed a short while later by a rephasing gradient. Water molecules without any significant motion would not lose any signal apart from normal T2 decay. However, if significant motion of a molecule occurred, it would lose signal owing to the dephasing gradient, and not regain this signal with the rephasing gradient, as it would have moved from its original position. A b-value is assigned to DW-MRI, which effectively is a measurement of the size and duration of the dephasing gradient. A DW-MRI image with increased signal indicates restricted diffusion or possible T2 shine-through. Post-processing of this image yields an ADC map where low signal indicates restricted diffusion, whereas areas with T2 shine-through would remain high.
Our results indicate that an increase in ADC values for cervical carcinoma can be observed after treatment with chemoradiation. During the course of the study, we saw a mean decrease of 42% in tumour size, which correlated with a mean increase in ADC values of 35% (b=500 s/mm2) and 43% (b=1 000 s/mm2), respectively. This finding suggests that ADC measurement can be used in the evaluation of cervical carcinoma tumour response. The mean ADC value for all patients increased from 1.05 x 10–3 mm2/s prior to treatment, to 1.45 x 10–3 mm2/s after treatment (b=1 000 s/mm2). These values compared well with those reported previously, namely that ADC values increased from 1.35 - 1.69 x 10–3 mm2/s with a b-value of 1 000 s/mm.2 , 14 We are of the opinion, however, that insufficient data are available to determine exact cut-off ADC values at the end of treatment for clinical practice, and further studies are necessary.
The most significant observation was seen when evaluating the ADC and size values 14 days after treatment was commenced. From day 1 to day 14, the mean ADC value increased by 20% and 24% with b-values of 500 and 1 000 s/mm2, respectively. During this time, the tumour size decreased by only 8% on conventional T2-weighted MRI. These findings were reversed for the period from 14 days to the last week of treatment, with the mean ADC value increasing by 13% (b=500 s/mm2) and 15% (b=1 000 s/mm2), respectively, and a relatively larger decrease in tumour size of 32%. These results indicate that ADC values will give an earlier indication of tumour response compared with tumour size, consistent with the pathophysiological principle of cell apoptosis during tumour response to treatment.
When considering those patients with >30% decrease in size after treatment as responders, we found that the responders had an initial mean ADC value of 1.26 x 10–3 mm2/s (b=500 s/mm2) and 0.99 x 10–3 mm2/s (b=1 000 s/mm2). After 2 weeks, the mean ADC value in this group increased to 1.48 x 10–3 mm2/s (b=500 s/mm2) and 1.24 x 10–3 mm2/s (b=1 000 s/mm2). At both these b-values, the difference was statistically significant (p<0.05). In the same patients, the mean tumour size decreased from 23.83 cm2 to 21.25 cm2 over a period of 2 weeks. This decrease in tumour size was not statistically significant (p=0.075). At the end of treatment, however, the tumour size in the group of responders decreased to a mean of 10.26 cm2, which was statistically significant (p<0.05). The change in ADC value was statistically significant for all measurements in these patients. In the group of non-responders, the change in both ADC value and tumour size was not statistically significant at any stage.
Our findings, in addition to results reported in the literature, support the theory that ADC values show earlier change in tumours responding to therapy compared with size measurements alone. No statistical difference in pre-treatment ADC values between the responders and non-responders was observed.
Regarding the ideal b-value to use, there is no universal consensus. In our study, the mean ADC values obtained during the time protocol of the study did not show a marked difference between the two b-values of 500 and 1 000 s/mm2. Thoeny and Ross, however, suggested using a moderate b-value (100 - 600 s/mm2) due to its effect to minimise the vascular signal of perfusion.15 Other authors16-18 recommended the use of high b-values (1 000 s/mm²), as it provides a more accurate measurement of water diffusion free of vascular contamination, as well as a low b-value (100 - 600 s/mm2) for better tumour visualisation to distinguish the lesions from normal anatomy.16-18 We agree with the latter recommendation of using a minimum of two b-values to quantify ADC.
Conclusion
The study shows considerable promise in the use of ADC values to evaluate treatment response of cervical carcinoma, especially in detecting early response after 14 days of treatment. This non-invasive, non-contrast and non-ionising radiation, functional imaging technique might in future contribute to more advances in oncological therapy.
Acknowledgements. The authors thank the radiographers at Universitas Academic Hospital and personnel of the Department of Oncology for assistance; Dr Daleen Struwig, medical writer, Faculty of Health Sciences, UFS, for technical and editorial preparation of the manuscript; and Professor Gina Joubert, Department of Biostatistics, Faculty of Health Sciences, UFS, for statistical analysis of data.
1. Togashi K, Nishimura K, Sagoh T, et al. Carcinoma of the cervix: Staging with MR imaging. Radiology 1989;171:245-251.
2. Nicolet V, Carignan M, Bourdon F, Prosmanne O. MR imaging of cervical carcinoma: A practical staging approach. Radiographics 2000;20:1539-1549.
3. Nakai G, Matsuki M, Inada Y, et al. Detection and evaluation of pelvic lymph nodes in patients with gynecologic malignancies using body diffusio-weighted magnetic resonance imaging. J Comput Assist Tomogr 2008;32:764-768. [http://dx.doi.org/10.1097/RCT.0b013e318153fd43]
4. Whittaker CS, Coady A, Culver L, Rustin G, Padwick M, Padhani AR.Diffusion-weighted MR imaging of female pelvic tumours: Apictorial review. Radiographics 2009;29:759-774. [http://dx.doi.org/10.1148/rg.293085130]
5. Van de Bunt L, Van der Heide UA, Ketelaars M, De Kort GA, Jürgenliemk-Schultz IM. Conventional, conformal, and intensity-modulated radiation therapytreatment planning of external beam radiotherapy for cervical cancer: The impact of tumor regression. Int J Radiat Oncol Biol Phys 2006;64:189-196. [http://dx.doi.org/10.1016/j.ijrobp.2005.04.025]
6. Van de Bunt L, Jürgenliemk-Schultz IM, De Kort GA, Roesink TM, Tersteeg RJ, Van der Heide UA. Motion and deformation of the target volumes during IMRT for cervical cancer: What margins do we need? Radiother Oncol 2008;88:233-240. [http://dx.doi.org/10.1016/j.radonc.2007.12.017]
7. Kerkhof EM, Van der Put RW, Raaymakers BW, Van der Heide UA, Jürgenliemk-Schultz IM, Lagendijk JJ. Infrafraction motion in patients with cervical cancer: The benefit of soft tissue registration using MRI. Radiother Oncol 2009; 93:115-121. [http://dx.doi.org/10.1016/j.radonc.2009.07.010]
8. Therasse P, Arbuck SG, Eisenhauer EA, et al. New guidelines to evaluate the response to treatment in solid tumors (RECIST Guidelines). J Natl Cancer Inst 2000;92:205-216. [http://dx.doi.org/10.1093/jnci/92.3.205]
9. Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumors: revised RECIST guideline (version 1.1). Eur J Cancer 2009;45:228-247. [http://dx.doi.org/10.1016/j.ejca.2008.10.026]
10. Chenevert TL, Brunberg JA, Pipe JG. Anisotropic diffusion in human white matter: Demonstration with MR techniques in vivo. Radiology 1990;177:401-405.
11. Moffat BA, Chenevert TL, Lawrence TS, et al. Functional diffusion map: A noninvasive MRI biomarker for early stratification of clinical brain tumor response. Proc Natl Acad Sci USA 2005;102:5524-5529. [http://dx.doi.org/10.1073/pnas.0501532102]
12. McVeigh PZ, Syed AM, Milosevic M, Fyles A, Haider MA. Diffusion-weighted MRI in cervical cancer. Eur Radiol 2008;18:1058-1064. [http://dx.doi.org/10.1007/s00330-007-0843-3]
13. Xue HD, Li S, Sun F, et al. Clinical applicationof body diffusion wieghted magnetic resonance imaging in the diagnosis and preoperative N staging of cervical cancer. Chin Med Sci J 2008;23:133-137. [http://dx.doi.org/10.1016/S1001-9294(09)60027-4]
14. Harry VN, Semple SI, Gilbert FJ, Parkin DE. Diffusion-weighted magnetic resonance imaging in the early detection of response to chemoradiation in cervical cancer. Gynecol Oncol 2008;111:213-220. [http://dx.doi.org/10.1016/j.ygyno.2008.07.048]
15. Thoeny HC, Ross BD. Predicting and monitoring cancer treatment response with diffusion-weighted MRI. J Magn Reson Imaging 2010;32:2-16. [http://dx.doi.org/10.1002/jmri.22167]
16. Padhani AR, Liu G, Koh DM, et al. Diffusion-weighted magnetic resonance imaging as a cancer biomarker: consensus and recommendations. Neoplasia 2009;11:102-125. [http://dx.doi.org/10.1593/neo.81328]
17. Koh DM. DW-MRI for disease detection. In: Koh DM, Theony HC, eds. Diffusion Weighted MR Imaging: Applications in the Body. Berlin: Springer, 2009:97-115.
18. Sbano H, Padhani AR. Diffusion-weighted MRI of female pelvic tumors. In: Taouli B, ed. Extra-Cranial Application of Diffusion-Weighted MRI. Cambridge: Cambridge University Press, 2010:119-140.
Crossref Citations
1. Role of apparent diffusion coefficient as a biomarker in the evaluation of cervical cancer
Sunita Dashottar, T Preeth Pany, Nishant Lohia
Indian Journal of Radiology and Imaging vol: 29 issue: 01 first page: 25 year: 2019
doi: 10.4103/ijri.IJRI_441_18