Virtual and augmented reality systems and three-dimensional printing of the renal model—novel trends to guide preoperative planning for renal cancer
Claudia-Gabriela Moldovanuab*()
aDepartment of Radiology, Municipal Clinical Hospital, Cluj-Napoca, Romania bDepartment of Radiology, Emergency Heart Institute “N. Stancioiu”, Cluj-Napoca, Romania
Objective: This study aimed to explore the applications of three-dimensional (3D) technology, including virtual reality, augmented reality (AR), and 3D printing system, in the field of medicine, particularly in renal interventions for cancer treatment.
Methods: A specialized software transforms 2D medical images into precise 3D digital models, facilitating improved anatomical understanding and surgical planning. Patient-specific 3D printed anatomical models are utilized for preoperative planning, intraoperative guidance, and surgical education. AR technology enables the overlay of digital perceptions onto real-world surgical environments.
Results: Patient-specific 3D printed anatomical models have multiple applications, such as preoperative planning, intraoperative guidance, trainee education, and patient counseling. Virtual reality involves substituting the real world with a computer-generated 3D environment, while AR overlays digitally created perceptions onto the existing reality. The advances in 3D modeling technology have sparked considerable interest in their application to partial nephrectomy in the realm of renal cancer. 3D printing, also known as additive manufacturing, constructs 3D objects based on computer-aided design or digital 3D models. Utilizing 3D-printed preoperative renal models provides benefits for surgical planning, offering a more reliable assessment of the tumor's relationship with vital anatomical structures and enabling better preparation for procedures. AR technology allows surgeons to visualize patient-specific renal anatomical structures and their spatial relationships with surrounding organs by projecting CT/MRI images onto a live laparoscopic video. Incorporating patient-specific 3D digital models into healthcare enhances best practice, resulting in improved patient care, increased patient satisfaction, and cost saving for the healthcare system.
. [J]. Asian Journal of Urology, 2024, 11(4): 521-529.
Claudia-Gabriela Moldovanu. Virtual and augmented reality systems and three-dimensional printing of the renal model—novel trends to guide preoperative planning for renal cancer. Asian Journal of Urology, 2024, 11(4): 521-529.
Hamad A, Jia B. How virtual reality technology has changed our lives: an overview of the current and potential applications and limitations. Int J Environ Res Publ Health 2022; 8:11278. https://doi.org/10.3390/ijerph191811278.
[2]
Wohlgenannt I, Simons A, Stieglitz S. Virtual reality. BISE 2020; 62:455-61.
[3]
Anthes C, Garcia-Hernandez RJ, Wiedemann M, Kranzlmuller D. State of the art of virtual reality technology. In: IEEE Aerospace Conference. MT, USA: Big Sky; 2016. https://doi.org/10.1109/AERO.2016.7500674.
[4]
Flores-Arredondo JH, Assad-Kottner C. Virtual reality: a look into the past to fuel the future. Bull Roy Coll Surg Engl 2015; 97:3. https://doi.org/10.1308/rcsbull.2015.42.
[5]
Wake N, Rosenkrantz AB, Huang R, Park KU, Wysock JS, Taneja SS, et al. Patient-specific 3D printed and augmented reality kidney and prostate cancer models: impact on patient education. 3D Print Med 2019; 5:4. https://doi.org/10.1186/s41205-019-0041-3.
doi: 10.1186/s41205-019-0041-3
pmid: 30783869
Pavan Kalyan BG, Kumar L. 3D printing: applications in tissue engineering, medical devices, and drug delivery. AAPS PharmSciTech 2022; 17:23-92.
[8]
Ghazi AE, Teplitz BA. Role of 3D printing in surgical education for robotic urology procedures. Transl Androl Urol 2020; 9:931-41.
doi: 10.21037/tau.2020.01.03
pmid: 32420209
[9]
Segaran N, Saini G, Mayer JL, Naidu S, Patel I, Alzubaidi S, et al. Application of 3D printing in preoperative planning. J Clin Med 2021; 26:917. https://doi.org/10.3390/jcm10050917.
[10]
Mudgal KS, Evolving trends in kidney cancer. IntechOpen; 2018. p. 100-54.
[11]
Bastawrous S, Wu L, Liacouras PC, Levin DB, Ahmed MT, Strzelecki B, et al. Establishing 3D printing at the point of care: basic principles and tools for success. Radiographics 2022; 42:451-68.
[12]
Doebbeling BN, Burton MM, Wiebke EA, Miller S, Baxter L, Miller D, et al. Optimizing perioperative decision making: improved information for clinical workflow planning. AMIA Annu Symp Proc 2012; 2012:154-63.
pmid: 23304284
[13]
Krupinski EA. Current perspectives in medical image perception. Atten Percept Psychophys 2010; 72:1205-17.
doi: 10.3758/APP.72.5.1205
pmid: 20601701
[14]
Zheng B, Wang X, Zheng Y, Feng J. 3D-printed model improves clinical assessment of surgeons on anatomy. J Robot Surg 2019; 13:61-7.
doi: 10.1007/s11701-018-0809-2
pmid: 29693206
[15]
Bücking TM, Hill ER, Robertson JL, Maneas E, Plumb AA, Nikitichev DI. From medical imaging data to 3D printed anatomical models. PLoS One 2017; 31:e0178540. https://doi.org/10.1371/journal.pone.0178540.
[16]
Flaxman TE, Cooke CM, Miguel OX, Sheikh AM, Singh SS. A review and guide to creating patient specific 3D printed anatomical models from MRI for benign gynecologic surgery. 3D Print Med 2021; 5:17. https://doi.org/10.1186/s41205-021-00107-7.
[17]
Archip N, Rohling R, Dessenne V, Erard PJ, Nolte LP. Anatomical structure modeling from medical images. Comput Methods Progr Biomed 2006; 82:203-15.
[18]
Popescu D, Marinescu R, Laptoiu D, Deac GC, Cotet CE. DICOM 3D viewers, virtual reality or 3D printingda pilot usability study for assessing the preference of orthopedic surgeons. Proc Inst Mech Eng H 2021; 235:1014-24.
[19]
Liu S, Liao W, Yu Q, Cheng X, Dai N, Zhang X. [The development of a system for 3D reconstruction from DICOM data and collaborative visualization]. Sheng Wu Yi Xue Gong Cheng Xue Za Zhi 2007; 24:1152-6. [Article in Chinese].
[20]
Wake N, Rosenkrantz AB, Huang WC, Wysock JS, Taneja SS, Sodickson DK, et al. A workflow to generate patient-specific three-dimensional augmented reality models from medical imaging data and example applications in urologic oncology. 3D Print Med 2021; 28:34. https://doi.org/10.1186/s41205-021-00125-5.
[21]
Bernhard JC, Isotani S, Matsugasumi T, Duddalwar V, Hung AJ, Suer E, et al. Personalized 3D printedmodel of kidney and tumor anatomy: a useful tool for patient education. World J Urol 2016; 34:337-45.
[22]
Smith M, Faraci A, Bello F. Segmentation and generation of patient-specific 3D models of anatomy for surgical simulation. Stud Health Technol Inf 2004; 98:360-2.
[23]
Wake N, Alexander AE, Christensen AM, Liacouras PC, Schickel M, Pietila T, et al. Creating patient-specific anatomical models for 3D printing and AR/VR: a supplement for the 2018 Radiological Society of North America (RSNA) hands-on course. 3D Print Med 2019; 30:5-17.
[24]
Abdelrahman A, Viriri S. Kidney tumor semantic segmentation using deep learning: a survey of state-of-the-art. J Imaging 2022; 25:8-55.
[25]
Tingelhoff K, Moral AI, Kunkel ME, Rilk M, Wagner I, Eichhorn KG, et al. Comparison between manual and semiautomatic segmentation of nasal cavity and paranasal sinuses from CT images. Annu Int Conf IEEE Eng Med Biol Soc 2007; 2007:5505-8.
pmid: 18003258
[26]
Daniel AJ, Buchanan CE, Allcock T, Scerri D, Cox EF, Prestwich BL, et al. Automated renal segmentation in healthy and chronic kidney disease subjects using a convolutional neural network. Magn Reson Med 2021; 86:1125-36.
doi: 10.1002/mrm.28768
pmid: 33755256
[27]
Huang W, Li H, Wang R, Zhang X, Wang X, Zhang J, et al. A selfsupervised strategy for fully automatic segmentation of renal dynamic contrast-enhanced magnetic resonance images. Med Phys 2019; 46:4417-30.
[28]
Rahim MSM, Norouzi A, Rehman A, Saba T. 3D bones segmentation based on CT images visualization. Biomed Res 2017; 28:3641-4.
[29]
Kang Y, Engelke K, Kalender WA. A new accurate and precise 3-D segmentation method for skeletal structures in volumetric CT data. IEEE Trans Med Imag 2003; 22:586-98.
[30]
Tsuneki M. Deep learning models in medical image analysis. J Oral Biosci 2022; 64:312-20.
doi: 10.1016/j.job.2022.03.003
pmid: 35306172
[31]
Türk F, Lüy M, Bar???? N. Kidney and renal tumor segmentation using a hybrid V-Net-based model. Mathematics 2020; 8:1772. https://doi.org/10.3390/math8101772.
[32]
Milletari F, Nassir N, Ahmadi SA. V-Net:fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings of the 2016 fourth international conference on 3D vision (3DV), stanford, CA, USA; 2016. p. 565-71. https://doi.org/10.48550/arXiv.1606.04797.
[33]
Xin Y, Minh HL, Cheng K, Sung KH, Liu W. Renal compartment segmentation in DCE-MRI images. Med Image Anal 2016; 32:269-80.
doi: 10.1016/j.media.2016.05.006
pmid: 27236222
[34]
Xiang D, Ulas B, Jin C, Shi F, Zhu W, Yao J, et al. CorteXpert: “A model-based method for automatic renal cortex segmentation”. Med Image Anal 2017; 42:257-73.
[35]
Tuncer SA, Alkan A. A decision support system for detection of the renal cell cancer in the kidney. Measurument 2018; 298:303. https://doi.org/10.1016/j.measurement.2018.04.002.
[36]
Yang G, Li G, Pan T, Kong Y, Wu J, Shu H, et al. Automatic segmentation of kidney and renal tumor in CT images based on 3D fully convolutional neural network with pyramid pooling module. In: 24th international conference on pattern recognition. Beijing, China: ICPR; 2018, Aug 2018. p. 3790-5.
[37]
Marie F, Corbat L, Chaussy Y, Delavelle T, Henriet J, Lapayre JC. Segmentation of deformed kidneys and nephroblastoma using case-based reasoning and convolutional neural network. Expert Syst Appl 2019; 127:282-94.
[38]
Couteaux V, Si-Mohamed S, Renard-Penna R, Nempont O, Lefevre T, Popoff A, et al. Kidney cortex segmentation in 2D CT with U-Nets ensemble aggregation. Diagn Interv Imaging 2019; 100:211-7.
[39]
Mihaylova AD, Georgieva VM, Petrov PP, Aleksandar T. Novel algorithm for segmentation of renal cyst from CT image sequences. In: 14th international conference on advanced technologies, systems and services in telecommunications (TELSIKS). Nis, Serbia; 2019. p. 380-3. https://doi.org/10.1109/TELSIKS46999.2019.9002209.
[40]
Rundo L, Han C, Nagano Y, Zhang J, Hataya R, Militello C, et al. USE-Net: incorporating squeeze-and-excitation blocks into U-Net for prostate zonal segmentation of multiinstitutional MRI datasets. Neurocomputing 2019; 365:31-43.
[41]
Fuzhe M, Sun L, Liu H, Jing H. Detection and diagnosis of chronic kidney disease using deep learning-based heterogeneous modified artificial neural network. Future Generat Comput Syst 2020; 111:17-26.
[42]
CruzaJose′ LB, Lima D, Jonnison A, Ferreira L, Ota′vio J, Silva AC, et al. Kidney segmentation from computed tomography images using deep neural network. Comput Biol Med 2020; 123:103906. https://doi.org/10.1016/j.compbiomed.2020.103906.
[43]
Li C, Tan Y, Chen W, Luo X, He Y, Gao Y, et al. ANU-Net: attention-based nested U-Net to exploit full resolution features for medical image segmentation. Comput Graph 2020; 90:11-20.
[44]
Nithya A, Appathurai A, Venkatadric N, Ramjia DR, Anna Palagan C. Kidney disease detection and segmentation using artificial neural network and multi-Kernel K-means clustering for ultrasound images. Measurement 2020; 149:106952. https://doi.org/10.1016/j.measurement.2019.106952.
[45]
Zhao W, Jiang D, Queralta JP, Westerlund T. MSS U-Net: 3D segmentation of kidneys and tumors from CT images with a multi-scale supervised U-Net. Inf Med 2020; 19:100357. https://doi.org/10.1016/j.imu.2020.100357.
[46]
Fogarasi M, Coburn JC, Ripley B. Algorithms used in medical image segmentation for 3D printing and how to understand and quantify their performance. 3D Print Med. 2022;24:18. https://doi.org/10.1186/s41205-022-00145-9.
[47]
Paul GM, Rezaienia A, Wen P, Condoor S, Parkar N, King W, et al. Medical applications for 3D printing: recent developments. Mo Med 2018; 115:75-81.
pmid: 30228688
[48]
Cornejo J, Cornejo-Aguilar JA, Vargas M, Helguero CG, Milanezi de Andrade R, Torres-Montoya S, et al. Anatomical engineering and 3D printing for surgery and medical devices: international review and future exponential innovations. BioMed Res Int 2022; 24:6797745. https://doi.org/10.1155/2022/6797745.
[49]
Sun Z, Wong YH, Yeong CH. Patient-specific 3D-printed low-cost models in medical education and clinical practice. Micromachines 2023; 16:464. https://doi.org/10.3390/mi14020464.
[50]
Meyer-Szary J, Luis MS, Mikulski S, Patel A, Schulz F, Tretiakow D, et al. The role of 3D printing in planning complex medical procedures and training of medical professionals-cross-sectional multispecialty review. Int J Environ Res Publ Health 2022; 11:3331. https://doi.org/10.3390/ijerph19063331.
[51]
Keller M, Guebeli A, Thieringer F, Honigmann P. Overview of in-hospital 3D printing and practical applications in hand surgery. BioMed Res Int 2021; 26:4650245. https://doi.org/10.1155/2021/4650245.
[52]
Daoud GE, Pezzutti DL, Dolatowski CJ, Carrau RL, Pancake M, Herderick E, et al. Establishing a point-of-care additive manufacturing workflow for clinical use. J Mater Res 2021; 36:3761-80.
[53]
Marro A, Bandukwala T, Mak W. Three-dimensional printing and medical imaging: a review of the methods and applications. Curr Probl Diagn Radiol 2016; 45:2-9.
doi: 10.1067/j.cpradiol.2015.07.009
pmid: 26298798
[54]
Garcia J, Yang Z, Mongrain R, Leask RL, Lachapelle K. 3D printing materials and their use in medical education: a review of current technology and trends for the future. BMJ Simul Technol Enhanc Learn 2018; 4:27-40.
[55]
Ventola CL. Medical applications for 3D printing: current and projected uses. P T 2014; 39:704-11.
[56]
Moldovanu CG, Lebovici A, Buruian MM. A systematic review of the clinical value and applications of three-dimensional virtual reconstructions in renal tumors. Med Pharm Rep 2022; 95:11-23.
[57]
Porpiglia F, Manfredi M, Checcucci E, Mele F, Bertolo R, De Luca S, et al. 3D prostate MRI reconstruction for cognitive robot assisted radical prostatectomy: is it able to reduce the positive surgical margin rate? Eur Urol Suppl 2017;16:e110. https://doi.org/10.1016/S1569-9056(17)30133-1.
[58]
Porpiglia F, Bertolo R, Checcucci E, Amparore D, Autorino R, Dasgupta P, et al; ESUT Research Group. Development and validation of 3D printed virtual models for robot-assisted radical prostatectomy and partial nephrectomy: urologists’ and patients’ perception. World J Urol 2018; 36:201-7.
[59]
Amparore D, Pecoraro A, Checcucci E, de Cillis S, Piramide F, Volpi G, et al. 3D imaging technologies in minimally invasive kidney and prostate cancer surgery: which is the urologists’ perception? Minerva Urol Nephrol 2022; 74:178-85.
[60]
Porpiglia F, Amparore D, Checcucci E, Manfredi M, Stura I, Migliaretti G, et al. Three-dimensional virtual imaging of renal tumours: a new tool to improve the accuracy of nephrometry scores. BJU Int 2019; 124:945-54.
doi: 10.1111/bju.14894
pmid: 31390140
[61]
Sofia C, Magno C, Silipigni S, Cantisani V, Mucciardi G, Sottile F, et al. Value of three-dimensional volume rendering images in the assessment of the centrality index for preoperative planning in patients with renal masses. Clin Radiol 2017; 72:33-40.
doi: S0009-9260(16)30370-1
pmid: 27729105
[62]
Amparore D, Piramide F, Checcucci E, Verri P, De Cillis S, Piana A, et al. Three-dimensional virtual models of the kidney with colored perfusion regions: a new algorithm-based tool for optimizing the clamping strategy during robot-assisted partial nephrectomy. Eur Urol 2023; 84:418-25.
[63]
Ukimura O, Nakamoto M, Gill IS. Three-dimensional reconstruction of renovascular-tumor anatomy to facilitate zeroischemia partial nephrectomy. Eur Urol 2012; 61:211-7.
doi: 10.1016/j.eururo.2011.07.068
pmid: 21937162
[64]
Von Rundstedt FC, Scovell JM, Agrawal S, Zaneveld J, Link RE. Utility of patient-specifc silicone renal models for planning and rehearsal of complex tumour resections prior to robotassisted laparoscopic partial nephrectomy. BJU Int 2017; 119:598-604.
[65]
Fujisaki A, Takayama T, Yamazaki M, Kamimura T, Katano S, Komatsubara M, et al. Utilization of a three-dimensional printed kidney model for favorable TRIFECTA achievement in early experience of robot-assisted partial nephrectomy. Transl Androl Urol 2020; 9:2697-704.
doi: 10.21037/tau-20-927
pmid: 33457241
[66]
Zargar H, Allaf ME, Bhayani S, Stifelman M, Rogers C, Ball MW, et al. Trifecta and optimal perioperative outcomes of robotic and laparoscopic partial nephrectomy in surgical treatment of small renal masses: a multi-institutional study. BJU Int 2015; 116:407-14.
doi: 10.1111/bju.12933
pmid: 25220543
[67]
Checcucci E, Piramide F, De Cillis S, Volpi G, Piana A, Verri P, et al. Icon study group. Health Information Technology Usability Evaluation Scale (Health-ITUES) and User- Experience Questionnaire (UEQ) for 3D Intraoperative Cognitive Navigation (ICON3DTM) system for urological procedures. Medicina 2023; 21:624. https://doi.org/10.3390/medicina59030624.
[68]
Chepelev L, Wake N, Ryan J, Althobaity W, Gupta A, Arribas E, et al. RSNA special interest group for 3D printing. Radiological Society of North America (RSNA) 3D Printing Special Interest Group (SIG): guidelines for medical 3D printing and appropriateness for clinical scenarios. 3D Print Med 2018; 21:11. https://doi.org/10.1186/s41205-018-0030-y.
[69]
Lee H, Nguyen NH, Hwang SI, Lee HJ, Hong SK, Byun SS. Personalized 3D kidney model produced by rapid prototyping method and its usefulness in clinical applications. Int Braz J Urol 2018; 44:952-7.
doi: 10.1590/S1677-5538.IBJU.2018.0162
pmid: 30044595
[70]
Knoedler M, Feibus AH, Lange A, Maddox MM, Ledet E, Thomas R, et al. Individualized physical 3-dimensional kidney tumor models constructed from 3-dimensional printers result in improved trainee anatomic understanding. Urology 2015; 85:1257-61.
doi: 10.1016/j.urology.2015.02.053
pmid: 26099870
[71]
Campi R, Pecoraro A, Vignolini G, Spatafora P, Sebastianelli A, Sessa F, et al. The first entirely 3D-printed training model for robot-assisted kidney transplantation: the RAKT box. Eur Urol Open Sci 2023; 53:98-105.
[72]
Cacciamani GE, Okhunov Z, Meneses AD, Rodriguez-Socarras ME, Rivas JG, Porpiglia F, et al. Impact of three-dimensional printing in urology: state of the art and future perspectives. A systematic review by ESUT-YAUWP group. Eur Urol 2019; 76:209-21.
doi: S0302-2838(19)30361-6
pmid: 31109814
[73]
Checcucci E, Amparore D, Volpi G, De Cillis S, Piramide F, Verri P, et al. Metaverse surgical planning with three-dimensional virtual models for minimally invasive partial nephrectomy. Eur Urol 2024; 85:320-5. https://doi.org/10.1016/j.eururo.2023.07.015.
[74]
Yamazaki M, Takayama T, Fujita A, Kikuchi T, Kamimura T, Myoga H, et al. 3D printed kidney model could be an important educational tool for residents. Asian J Endosc Surg 2023; 16:197-202.
[75]
Detmer FJ, Hettig J, Schindele D, Schostak M, Hansen C. Virtual and augmented reality systems for renal interventions: a systematic review. IEEE Rev Biomed Eng 2017; 10:78-94.
doi: 10.1109/RBME.2017.2749527
pmid: 28885161
[76]
Hyde ER, Berger LU, Ramachandran N, Hughes-Hallett A, Pavithran NP, Tran MGB, et al. Interactive virtual 3D models of renal cancer patient anatomies alter partial nephrectomy surgical planning decisions and increase surgeon confidence compared to volume-rendered images. Int J CARS 2019; 14:723-32.
[77]
Hameed BMZ, Somani S, Keller EX, Balamanigandan R, Mahapatra S, Pietropaolo A, et al. Application of virtual reality, augmented reality, and mixed reality in endourology and urolithiasis: an update by YAU Endourology and Urolithiasis Working Group. Front Surg 2022; 1:866946. https://doi.org/10.3389/fsurg.2022.866946.
[78]
Zhao Y, Liu Y, Dai Y, Yang L, Chen G. Application of 3D bioprinting in urology. Micromachines 2022; 13:1073. https://doi.org/10.3390/mi13071073.
[79]
Soliman Y, Feibus AH, Baum N. 3D printing and its urologic applications. Rev Urol 2015; 17:20-4.
pmid: 26028997
[80]
Chen MY, Skewes J, Desselle M, Wong C, Woodruff MA, Dasgupta P, et al. Current applications of three-dimensional printing in urology. BJU Int 2020; 125:17-24.
doi: 10.1111/bju.14928
pmid: 31622020
Luis G. Medina, Randall A. Lee, Valeria Celis, Veronica Rodriguez, Jaime Poncel, Aref S. Sayegh, Rene Sotelo. Robotic management of urinary fistula[J]. Asian Journal of Urology, 2024, 11(3): 357
-365
.
