1. Introduction

Innovations in laser surveying technology and promotion of CIM and i-Construction informatization by the Ministry of Land, Infrastructure, Transport and Tourism (MLIT) have allowed point cloud data for public structures to be measured and accumulated. Among them, many cases have been proposed in which a laser scanner is mounted on construction machinery to perform 3D measurement (Kato, Y., et al. 2019, Kubota, S., et al. 2020). They have led to the reduction of the cost of measuring point cloud data for public structures during construction, creating momentum for the accumulation and utilization of point cloud data.

Thus, point cloud data, which is expected to become more and more promising in the future, is a huge aggregate of points each of which holds x, y, and z coordinates and RGB values (color data) in urban space. In addition, as the data does not hold information about the features (maintenance objects) represented by each point or the relationship between the point and other points, it is difficult to use point cloud data wisely in accordance with its intended purpose. In an interview survey with managers of public structures, they mentioned their wishes to utilize point cloud data for daily patrols, regular inspections, and response to complaints and disasters. They also recognized the following issues regarding the use of point cloud data.

• There is no software environment for checking and editing point cloud data easily.
• Because of the large data size of point cloud data, it is not easy to search, retrieve, and view the point cloud data for the target location.
• It is not easy to grasp the deformation and time-series changes of respective features, such as slope swelling and road surface deformation.
• As there is no way to grasp the local situation when they receive a complaint or inquiry, it is necessary for them to actually visit the site.

As a course of action for solving these problems, it is essential to develop an environment for utilizing point cloud data, define a data structure for efficiently utilizing “point cloud data” and “machine-readable data”, and construct a digital twin for public structures to standardize the environment for utilization.

Against this background, a variety of system has been proposed. For example, there are software such as InfiPoints (ELYSIUM. 2022), ReCap Pro (AutoCAD. 2022), TREND-POINT (FUKUI COMPUTER. 2022), etc., but these are expensive and have not been widely used by local governments. On the other hand, when free software is checked, CloudCompare (cloudcompare.org. 2022) and MeshLab (MeshLab. 2022) are available, but they do not have functions that comply with national specifications or meet the needs of the field, and they have not solved the above issues. This paper shows a case study of developing 3D Point Studio, a platform for utilizing and sharing “point cloud data” and “machine-readable data” and actually constructing a digital twin environment for public structures.

2. Overview of 3D Point Studio

2.1. Outline
3D Point Studio was designed based on the following plans for “common platform”, “management and linkage”, and “sharing and reuse” respectively.

• [Common platform] To realize the mechanism capable of checking via a Web browser, with reference to 3D Tiles, a standard approved by OGC (Open Geospatial Consortium).
• [Common platform] To realize not only utilization of point cloud data in units of road features but also efficient linkage between the point cloud data and the machine-readable data by using the data conforming to the “Attribute Management Specifications for Point Cloud Data [for Road] (Draft) (JACIC. 2018)” published by the National Institute for Land and Infrastructure Management (hereinafter called as “Semantic Point Cloud Data” (Nakamura, K., et al. 2021)).
• [Management and linkage] To make a hybrid system (Fig. 1) consisting of an offline version (Local) that can process point cloud data stored in the user’s local environment and an online version (Web) that can check point cloud data without downloading it, enabling mutual data linkage.
• [Sharing and reuse] To provide architecture capable of extending a wide variety of functions with plug-ins in order to improve the reusability of the program code and the extensibility of functions to meet the users’ requests.

As mentioned above, by dividing the system into the offline version that handles data processing and analysis and the online version that handles data viewing and information sharing, a variety of choices can be made to meet users’ needs. If point cloud data is centrally managed on a server equipped with this system, it will always allow construction of an environment in which point cloud data can be easily viewed at any time, thereby reducing the cost of sharing large-scale point cloud data.

By designing a system based on the above plans, we realize a utilization and sharing platform that enables efficient use of “point cloud data” and “machine-readable data” in relation to each other.

2.2. Attribute management specifications for point cloud data
This system uses Semantic Point Cloud Data that conforms to the Attribute Management Specifications for Point Cloud Data [for Road] (Draft) (JACIC. 2018) as the basic data to realize efficient utilization of point cloud data by associating it with “machine-readable data”. Specifically, we have developed a technology for automatically extracting point cloud data for a road feature as a unit by assigning attributes that indicate the location and area of road features as “area data” to the point cloud data. Since this technology is implemented in conformity with the specifications (JACIC. 2018) shown in Fig. 2, it can be edited into point cloud data that can be handled in units of road features for both.

The capability of searching and analyzing point cloud data for the road space in units of road features not only improves work efficiency, but also leads to applications of multiple purposes, such as automation through program processing.

2.3. List of main functions
3D Point Studio has functions shown in Table 1. Standard functions such as RGB display of point cloud data, grayscale or color scale display of arbitrary values, point selection, distance measurement between two points, and cross-section view display are provided with both the online and offline versions.

The online version is characterized by capability of sharing the result of visualization of point cloud data with other users on the system (Fig. 3). This makes it possible to promote smooth communication and collaboration between different departments within the same company (planning section, survey section, engineering section, management section, etc.), as well as between the client and the contractor. The offline version is characterized by its functions to extract single features from approximately 20,000,000 points in a 150m square in less than 10 seconds, with an average F value of 0.78 or higher, using existing materials and machine learning (Nakamura, K., et al. 2017, Nakamura, K., et al. 2020, Umehara, Y., et al. 2021), to extract the difference between two different periods of time (Fig. 4), and to manage point cloud data in units of features in conformity with the Attribute Management Specifications.

Table 1. Main functions of 3D Point Studio

The offline version has a function of uploading the analysis results of point cloud data to the online version. This enables the online version to perform spatial searches for features using the area data as an index in conformity with the Attribute Management Specifications for Point Cloud Data [for Road] (draft) (JACIC. 2018). In addition, it allows the user to make direct access to the target feature within a few minutes of starting the system. It also has a function of downloading point cloud data by selecting the required area. This makes it possible to easily download point cloud data for the same place in multiple time periods or with different measurement devices, allowing utilization of such functions on the offline version as detection of difference between the point cloud data and the design data, as well as that between the point cloud data obtained during two different time periods.

2.4. Digital Twin for Public Structures using 3D Point Studio
Using 3D Point Studio, we have constructed an environment that allows management of point cloud data in units of features (a digital twin for public structures), as shown in Fig. 5. In this environment, the Semantic Point Cloud Data, with which the point cloud data can be handled in units of road features, is used as the basic data. Consequently, it becomes easy to associate it with ledgers or make time-series assessment of deformation.

In this study, we also constructed and released a browsing environment for Semantic Point Cloud Data (Fig. 6), a prototype of the digital twin, using 3D Point Studio. We also conducted its operations test at a civil engineering office in Shizuoka Prefecture. Its results won the i-Construction Grand Prize (MLIT. 2019) in fiscal 2019. As described above, by realizing the digital twin for public structures (cyberspace of roads) capable of treating point cloud data in units of road features, it becomes possible to generate cross-sectional and longitudinal views of any location immediately, as well as handle road ledgers or disaster prevention and periodic inspection reports and so forth in relation to the road features. This enables easy acquisition of information about the features for inspection during daily patrols and regular inspections, thereby avoiding omissions of inspection and significantly reducing data access cost at the time of inspections. Moreover, the digital twin of public structures also enables the delivery and acceptance of order drawings and as-built drawings for repair works, as well as grasping the situation of damage at the time of disaster (including assessment of disaster).

3. Future Plans

3D Point Studio has versatile functions as a platform for utilizing point cloud data, such as efficient management and visualization of point cloud data and estimation of features using machine learning. Therefore, the system can be used throughout the construction lifecycle, and especially in the maintenance phase, it is considered capable of contributing to the near-real-time sharing of information, both at normal time and the time of disaster. Fig. 7 shows the situations in which 3D Point Studio is utilized in the construction lifecycle.

This is the result of summarizing the possible applications in respective phases of the construction lifecycle. In the future, as the figure indicates, we aim to develop technologies specialized for each of these phases, such as automatic construction of 3D models in the design phase, management of piecework in the execution of works phase, and inspection of deformation in the maintenance phase, thereby advancing the digital twin for public structures using 3D Point Studio.

4. Conclusion

In this study, we developed 3D Point Studio, a platform for utilizing point cloud data that contributes to the deepening of i-Construction. Using 3D Point Studio, we constructed a digital twin environment for public structures that allows management of point cloud data in units of features and easily enables association with ledgers, evaluation of deformation over time and so on.

In the future, based on the technology cultivated in the studies on 3D model generation, work progress control and UAV measurement, development of a 3D CAD engine, UAV-mounted laser scanners, and the utilization of point cloud data, we are planning to extend the functions suitable for each construction lifecycle, promoting and disseminating the results of the studies. This paper was translated and written based on the technical report published by the Image Lab (Nakamura, K., et al. 2022a) and a part of the article published by Intelligence, Informatics and Infrastructure of Japan Society of Civil Engineers (Nakamura, K., et al. 2022b).

References
AutoCAD. (2022). ReCap Pro. https://www.autodesk.co.jp/products/recap/overview.

cloudcompare.org. (2022). CloudCompare. https://www.danielgm.net/cc/.

ELYSIUM. (2022). InfiPoints. https://www.elysium-global.com/ja/product/infipoints/.

FUKUI COMPUTER. (2022). TREND-POINT. https://const.fukuicompu.co.jp/products/trendpoint/.

JACIC Social Infrastructure Information Standardization Committee. (2018). Subcommittee on Attribute Management Specifications for Point Cloud Data in the Road Field. Attribute Management Specifications for Point Cloud Data [for Road] (Draft) – Version 1.0 -. https://www.jacic.or.jp/hyojun/2017shouiinnkai-02.html.

Kato, Y., et al.(2019). Efforts to develop a construction machine-mounted planning device for work progress control of civil engineering and pavement construction, Proceedings of 33rd Japan Road Congress, 33.

Kubota, S., et al.(2020). A Study on Visualization System for Crane Operation Support by 3D Map Generation and Feature Recognition, Intelligence and Information, 32(1), 604-615. https://doi.org/10.3156/jsoft.32.1_604.

MLIT. (2019). 25 organizations for innovative initiatives in construction sites announced! -We are pleased to announce the winners of the i-Construction Grand Prize for fiscal 2019. https://www.mlit.go.jp/report/press/kanbo08_hh_000653.html.

MeshLab. (2022). MeshLab. https://www.meshlab.net/.

Nakamura, K., et al.(2017). A Study on Feature Extraction Techniques for Point Cloud Data Based on As-built Drawings, Journal of JSCE F3 (Civil Engineering Informatics), 73(2), I_424-I_432. https://doi.org/10.2208/jscejcei.73.I_424.

Nakamura, K., et al.(2020). A Study on Extraction of Point Cloud Data of Road Surface Feature Using As-built Drawings, Intelligence and Information, 32(1), 616-626. https://doi.org/10.3156/jsoft.32.1_616.

Nakamura, K., et al.(2021). Product Modeling of Point Cloud Data: Proposal of Semantic Point Cloud Data, Proceedings of the Symposium on Civil Engineering Informatics, 46, 117-180.

Nakamura, K., et al.(2022). 3D Point Studio; Utilization Platform for Point Cloud Data, Image Lab, 33(10), 10-16.

Nakamura, K., et al.(2022). Development on 3D Point Studio; Utilization Platform for Point Cloud Data, Intelligence, Informatics and Infrastructure, 3(J2), 854-869.

Umehara, Y., et al.(2021). Research on Identification of Road Features from Point Cloud Data Using Deep Learning, International Journal of Automation Technology, 15(3), 274-289. https://doi.org/10.20965/ijat.2021.p0274.