Automatic 3D Modeling of Liver Segments Including Segmental Branches of Portal Triad and Hepatic Vein Based on the Sectioned-Images/Modelado 3D Automatico de Segmentos Hepaticos, con las Ramas Segmentarias de la Triada Portal y la Vena Hepatica Basadas en las Imagenes Seccionadas.
The liver segments including the portal triad (portal vein, hepatic artery, and bile duct) and hepatic vein are very important clinically. Therefore, anatomical dissection of liver is being performed for medical education of not only medical students but also clinician. However, it is not easy to dissect a liver because its segments, portal triad, and hepatic vein are complicated. To overcome the difficulty of a liver dissection and improve the accuracy of a surgical operation, three dimensional (3D) models of a liver from computed tomography (CT) or magnetic resonance imaging (MRI) are used in anatomy and surgery education (Shin et al., 2009; Goryawala et al, 2014; Le et al., 2015; Li et al, 2015). On the other hand, it is difficult to produce 3D models of the segments with segmental branches of its artery, duct, and vein from CT, MRI, and sectioned-images. In case of CT and MRI, 3D models can be made automatically, but the 3D models are not sophisticated (Liu et al., 2013; Goryawala et al.; Dong et al., 2015; Le et al.; Li et al.; Gotra et al., 2017).
Unlike CT and MRI, in sectioned-images of high resolution (0.1 mm X 0.1 mm X 0.1 mm sized-voxel) and real color (48 bit color), the terminal branches of the portal triad and hepatic vein can be seen (Park et al., 2005, 2015). Despite this, 3D models of the segments and segmental branches of the liver cannot be made automatically (Shin et al., 2009) for the following reasons. First, there are no boundaries of the liver segments at not only outer surface, but also at the inner parenchyma. Second, the portal triad and hepatic vein cannot be distinguished in the two dimensional (2D) images. Third, each branch of the portal triad and hepatic vein in liver is quite complicates. Therefore, we tried to outline the sectioned-images manually in previous work (Shin et al., 2009), but it was quite tedious and time consuming.
The aim of this study was to develop automatic and accurate methods for producing liver 3D models from high resolution sectioned images. Another purpose was to present 3D models of the liver segments and terminal branches of the portal triad and hepatic vein in a PDF file that can assist in learning and training with anatomy and clinical surgery. To achieve of this 3D models of liver segments with segmental branches of the portal triad and hepatic vein were produced from the sectioned-images.
MATERIAL AND METHOD
In a previous study, 4,935 sectioned-images of a whole female body and 1,642 color-filled-images of 27 structures including the liver surface were made (Park et al., 2015). For this study, the 651 sectioned-images of the abdominal region (intervals, 0.2 mm; pixel size, 0.1 X 0.1 mml; bit depth, 48 bit color) and the color-filled-images, including the liver and inferior vena cava, were used.
1. First step: Producing a Ref-3D of the portal triad and hepatic vein together. Using the 'Magic wand' tool on Photoshop CC 2015 (Adobe Systems, Inc., San Jose, CA, USA), by clicking the inferior vena cava in the sectionedimages, similar-colored structures both on the inside and outside of the liver were selected simultaneously. The most selected-structures were the portal triad and hepatic vein in inside the liver and some selected-structures were several structures in outside the liver. Using the sections of liver surface in the previous study (Park et al., 2015), the outside selected-structures were deleted automatically by 'SelectInverse' and 'Action-Batch' tools. Consequently the portal triad and hepatic vein in the liver selected. The selected structures were filled automatically with a color and saved as A-color-filled-images in BMP format (Fig. 1A; Table I).
The A-color-filled-images were imported in Mimics version 10.01 (Materialize, Leuven, Belgium). On Mimics, the A-color-filled-images were reconstructed using the 'Calculate 3D' tool to produce a referential 3D model (Ref30) of STL format (Fig. 1B; Table I).
In the Ref-3D including both the portal triad and hepatic vein, the hepatic vein was removed manually to produce a Portal-triad-3D using Maya version 2016 (Autodesk, Inc., San Rafael, CA, USA) because the hepatic vein was not as complicated as the portal triad in either Ref-3D or the real liver. In the Ref-3D, the Portal-triad-3D was subtracted automatically using the 'Boolean' tool to leave the Hepaticvein-3D (Fig. 1C, 1D; Table I).
2. Second step: Making eight segments of the liver. After each segmental branch of the portal triad was identified in Ref-3D using the cross-sectioning view of the 'Toggle Cross Section' tool on Adobe Reader version 9 (Adobe Systems, Inc., San Jose, CA, USA) (Fig. 1E), the identified branches were marked on the printed paper of the sectioned images (Fig. 1F). In the identified branches, the tertiary branches were set as the criterion of the liver segments according to the anatomy textbook (Moore et al, 2014). According to the criterion, the boundaries of eight segments of the liver are marked on the papers (Fig. 1F). Referring to the papers, eight segments were outlined on A-color-filled-images using the 'Lasso' tool and filled automatically with a specific color to make the Color-filled-images on Photoshop (Fig. 1G). The Color-filled-images were reconstructed by surface modeling and saved as Segment 3D-I to Segment-3D-VIII in STL format on Mimics (Fig. 1H).
3. Third step: Producing 3D models of the segmental branches of the portal triad and hepatic vein. Portal-triad3D in the first step and segment-3D-I in the second step were loaded in Maya together. The intersection of the portal triad and segment I were extracted automatically using the 'Boolean' tool and saved as Portal-triad-branch-3D-I. In the same way, segmental branches of the portal triad in segments II to VIII were divided and saved as Portal-triad-branch-3D-II to -VIII in STL format. Also, in the same manner, branches of the hepatic vein were divided by segment-3D-I to -VIII and saved as Hepatic-vein-branch-3D-I to -VIII of STL format (Fig. 1I, 1J; Table I).
4. Final step: Combining all 3D models. All STL files of eight segments and segmental branches of the portal triad and hepatic vein were loaded on the Deep Exploration Standard (Right Hemisphere, San Ramon, CA, USA), which was used to produce a list tree of the 3D structures. The combined STL files were saved as Liver-3D of PDF format (Fig. 1K; Table I).
RESULT AND DISCUSSION
The 3D models of the liver including the eight segments and segmental branches of the portal triad and hepatic vein were produced and stored as STL file, which has high applicability (Fig. 2; Table I). For the common user, the STL files were put into a PDF file and the PDF file can be downloaded freely at neuroanatomy.kr.
A human liver is classified into two viewpoints: Anatomical lobes and functional subdivision. By functional subdivision, segments II, III, and IV of the left liver and segments V, VI, VII, and VIII of the right liver in the Liver3D could be shown in bookmark window of Acrobat Reader. In addition, in Liver-3D, the primary to tertiary segmental branches of the portal triad could be shown in different colors. Although the tertiary or more branches of the portal triad and hepatic vein could be shown in Liver-3D, the color of the branches was identical (Fig. 2). In the case of the hepatic vein, each segmental branches of the vein was shown instead of the right, intermediate, and left hepatic vein in the textbook (Moore et al.).
The volume of the each structure could be measured by the STL file size. In contrast, the right liver was larger than the left, the largest segment was segment IV in the left lobe (55.8 MB). Segment II in the left lobe was the smallest (27.6 MB) to almost half that of segment IV. Segmental branch IV of the portal triad was the largest like segment IV. Segmental branch VI was the smallest. In the segmental branches of the hepatic vein, the largest was VII, which was similar to that segment VII (Table II).
In the sectioned-images, the main to terminal branches of the portal triad and hepatic vein could be identified by the naked eye due to the high resolution and real color (Park et al., 2009, 2010; Shin et al., 2012) unlike MRI and CT. In the sectioned-images, however, it was difficult to separate automatically or manually the whole branches in the liver for 3D modeling because of the considerable color information by the high resolution and real color. In particular, the high resolution and true color of the images are both a strength and weakness. Therefore, easier and more accurate separation methods of the liver structures are needed for 3D modeling.
In liver 3D modeling, there are two difficulties. First was selection of the segmental branches of the portal triad and hepatic vein in sectioned-images. Second was that the selected branches were divided into segmental branches. To solve the difficulties, 2D (sectioned-images) selection and 3D division were used as follows.
The criterion of auto-selection for the portal triad and hepatic vein from main to terminal segmental branches was the similar color in the sectioned-images. We choose the inferior vena cava with a similar color to the portal triad and hepatic vein. The inferior vena cava existed in every sectioned-image, was large enough, and did not change its position significantly. Therefore, anyone would find it easy to locate and select it automatically. In this study, by clicking the inferior vena cava in the sectioned-images using the 'Magic wand' tool in Photoshop, most of the portal triad and hepatic vein were selected automatically.
The criteria of auto-division for the segmental branches of a portal triad and hepatic vein were the intersection regions in the 3D models. There is 'Boolean' tool in Maya in that the intersection region at two objects can be chosen. After making Segment-3D of liver, the intersection regions of each segment in the Segment-3D and each segmental branch in Portal-triad-3D or Hepatic-vein3D occurred. The intersection segmental branches could be divided automatically using 'Boolean' tool in Maya. Automatic process would be easier and time saving.
The liver 3D models of this study will assist in the anatomical and clinical education of medical students and medical doctors. In addition, the methods for making liver 3D models will assist in 2D selection and 3D modeling of other structures. We distributed the Liver-3D of PDF file free of charge at neuroanatomy.kr.
ACKNOWLEDGMENTS. This research was financially supported by the Ministry of Trade, Industry and Energy (MOTIE) and Korea Institute for Advancement of Technology (KIAT) through the International Cooperative R&D program. (Grant number: N0002249)
Dong, C.; Chen, Y. W.; Foruzan, A. H.; Lin, L.; Han, X. H.; Tateyama, T.; Wu, X. ; Xu, G. & Jiang, H. Segmentation of liver and spleen based on computational anatomy models. Comput. Biol. Med., 67:146-60, 2015.
Goryawala, M.; Gulec, S.; Bhatt, R.; McGoron, A. J. & Adjouadi, M. A lowinteraction automatic 3D liver segmentation method using computed tomography for selective internal radiation therapy. BioMed Res. Int., 2014:198015, 2014.
Gotra, A.; Chartrand, G.; Vu, K. N.; Vandenbroucke-Menu, F.; Massicotte-Tisluck, K.; de Guise, J. A. & Tang, A. Comparison of MRI- and CT-based semiautomated liver segmentation: a validation study. Abdom. Radiol. (NY), 42(2) :478-89, 2017.
Le, T. N.; Bao, P. T. & Huynh, H. T. Fully automatic scheme for measuring liver volume in 3D MR images. Biomed. Mater. Eng., 26Suppl. 1:S1361-9, 2015.
Li, G.; Chen, X.; Shi, F.; Zhu, W.; Tian, J. & Xiang, D. Automatic liver segmentation based on shape constraints and deformable graph cut in CT images. I. E. E. E. Trans. Image Process, 24(12):5315-29, 2015.
Liu, X. J.; Zhang, J. F.; Sui, H. J.; Yu, S. B.; Gong, J.; Liu, J.; Wu, L. B.; Liu, C.; Bai, J. & Shi, B. Y. A comparison of hepatic segmental anatomy as revealed by cross-sections and MPR CT imaging. Clin. Anat., 26(4):486-92, 2013.
Moore, K. L.; Dalley, A. F. & Agur, A. M. R. Clinically Oriented Anatomy. 7th ed. Philadelphia, Wolters Kluwer Health/Lippincott Williams & Wilkins, 2014.
Park, H. S.; Choi, D. H. & Park, J. S. Improved sectioned images and surface models of the whole female body. Int. J. Morphol., 33(4) :1323-32, 2015.
Park, J. S.; Chung, M. S.; Chi, J. G.; Park, H. S. & Shin, D. S. Segmentation of cerebral gyri in the sectioned images by referring to volume model. J. Korean Med. Sci., 25(12):1710-5, 2010.
Park, J. S.; Chung, M. S.; Hwang, S. B.; Lee, Y. S.; Har, D. H. & Park, H. S. Visible Korean human: improved serially sectioned images of the entire body. I. E. E. E. Trans. Med. Imaging, 24(3):352-60, 2005.
Park, J. S.; Chung, M. S.; Shin, D. S.; Har, D. H.; Cho, Z. H.; Kim, Y. B.; Han, J. Y. ; Chi, J. G. Sectioned images of the cadaver head including the brain and correspondences with ultrahigh field 7.0 T MRIs. Proc. I. E. E. E., 97(12):1988-96, 2009.
Shin, D. S.; Chung, M. S.; Lee, J. W.; Park, J. S.; Chung, J.; Lee, S. B. & Lee, S. H. Advanced surface reconstruction technique to build detailed surface models of the liverand neighboring structures from the Visible Korean Human. J. Korean Med. Sci., 24(3):375-83, 2009.
Shin, D. S.; Park, J. S.; Park, H. S.; Hwang, S. B. & Chung, M. S. Outlining of the detailed structures in sectioned images from Visible Korean. Surg. Radiol. Anat., 34(3):235-47, 2012.
Jin Seo Park
Department of Anatomy
Dongguk University School of Medicine, 87
REPUBLIC OF KOREA
Sang Eun Lee & Jim Seo Park
Department of Anatomy, Dongguk University School of Medicine, 87 Dongdae-ro, Gyeongju, Republic of Korea.
Grant sponsor: This research was financially supported by the Ministry of Trade, Industry and Energy (MOTIE) and Korea Institute for Advancement of Technology (KIAT) through the International Cooperative R&D program. (Grant number: N0002249).
Caption: Fig. 1. Procedures for making automatically Liver-3D.pdf. In the first step, A-color-filled-images of whole branches of both the portal triad and hepatic vein are automatically made of the sectioned images (A). Ref-3D is automatically made of a pink color in A-colorfilled-images (B). From Ref-3D, Portal-triad-3D (C) and Hepatic-vein-3D (D) are separated manually and automatically respectively. In the second step, the segmental branches of the portal triad are identified using the cross-sectioning view the Portal- triad-3D (E). The identified branches are drawn and their names are written on printed papers (F). Referring to the identified branches in the papers, 8 segments of the liver are outlined on the papers (F). Referring to the papers (F), Color-filled-images of the liver's 8 segments are manually made of A-color-filled-images (G). Segment-3D was made automatically from the Color-filled-images by surface modeling (H). From Portal-triad-3D (C) and Hepatic-vein-3D (D) using Segment-3D (H), each segmental branch of are separated automatically (I) to make a Portal-triad-branch-3D and Hepatic-vein-branch-3D (J). All 3D results are combined into the Liver-3D (K).
Caption: Fig. 2. Liver-3D in Adobe Reader. In the Liver-3D of the 3D window, whole branches of portal triad and hepatic vein can be selected and manipulated freely by mouse dragging and wheel rotating (A). By selecting a view in the bookmark window, each segmental branches of the portal triad (B) or hepatic vein (C) can be shown either individually or together. The portal triad and hepatic vein in liver-3D can be magnified to the tertiary or more branches.
Table I. Sequential procedures for segmentation and 3D modeling of liver. Step Procedures (software) Raw data First Automatic selecting (Photoshop) Sectioned-images Automatic 3D modeling (Mimics) A-color-filledManual separating (Maya) Ref-3D Automatic separating (Maya) Second Manual segmenting (Photoshop) A-color-filledimages Automatic 3D modeling (Mimics) Color-filled-images Third Semi-automatic dividing (Maya) Portal- triad-3D Hepatic-vein-3D Final Automatic combining all 3D All 3D files of STL Step File name Resultants Structures Fig. (file format) First A-color-filled- Whole branches of both PT A Ref-3D (stl) and HV B Portal-triad-3D Portal triad C (stl) Hepatic-vein-3D Hepatic vein D (stl) Second Color-filled- Eight segments of liver G images (bmp) Segment-3D H Third Portal-triad- Segmental branches of portal J branch-3D (stl) triad Hepatic-vein- Segmental branch of hepatic J branch-3D (stl) vein Final Liver-3D (pdf) 8 segments of liver including K PT, portal triad; HV, hepatic vein. Table II. STL file sizes of eight segments and segmental branches of the portal triad and hepatic vein. Part Segment (MB) Segmental branch Portal triad (KB) hepatic vein (KB) Left I 33.9 1,046 459 ** II 27.6 ** 1,227 1,561 III 32.6 1,302 829 IV 55.9 * 6,493 * 3,232 Right V 33.7 2,885 1,557 VI 29.2 572 ** 2,408 VII 47.7 729 9,029 * VIII 43.0 816 2,878 * Biggest structure in the column; **Smallest structure in the column; The primary segmental branch of the portal triad were 11.1 MB.