Keywords

1 Introduction

The packaging market has been showing constant growth rates around the world, according researchs conducted by [1], which is accentuated by the fact that food packaging can be developed with any input, as long as there is compatibility with the food product.

In this sense, 3D scanning and product simulation can enhance Packaging Design options, as is already happening in other applications: architecture and civil engineering [2], materials science [3], reverse engineering [4, 5], ergonomics or human factors [6], medicine [7], dentistry [8], assistive technology [9], preservation of historical and cultural heritage [10] and archeology [11].

However, as stated by [2], the 3D scanning technology carries with it the fact that the limitations of laser equipment is, on average, 15 times more expensive than a high quality photographic camera. This makes the photogrammetry process by capturing images a viable option in terms of costs.

On the other hand, smartphones have been improving in performance over the last decade and, when properly calibrated, can provide images with a good level of sharpness and quality. So, photo-based scanning presents itself as an efficient and affordable solution.

From there, a simpler process can combine capture actions and comparison between photos acquired from different angles, to generate a cloud of points. This cloud of points, by itself, is not very useful for design processes, but when combined with the processing of specific software, it can generate information about volume, perimeter, width and depth [12]. In this way, the 3D modeling process of fresh food generates a reduction in waste, reducing the need for storage and conservation of products, which are now viewed and analyzed in a virtual environment.

Within this perspective, this article covers a stage of the research conducted by [13], whose general objective was to develop a low-cost 3D scanning method for fresh food packaging design. In the proposed method, image capture is performed with smartphones, followed by processing in digital photogrammetry and analysis of results in a virtual environment. The relevance of this study is to provide cost reduction in the Packaging Design process, by making 3D scanning cheaper.

2 Methodological Procedures

This article discusses a research stage whose general objective was to determine how the use of three-dimensional models, obtained by photogrammetry carried out with the aid of smartphones, can reduce the costs associated with the structural design of packaging.

The global methodological procedures adopted can be consulted in [13], and their goals were: adapt a low-cost three-dimensional scanning method using images collected by smartphones; identify and select free or open source software for the workflow for obtaining three-dimensional models; compare the results of the low-cost photogrammetric scanning process with those using high-cost technology; illustrate possible contributions of low-cost digitization to structural packaging design phases, generated through the application of 3D mesh; and systematize a low-cost digitization flow for fresh fruit packaging design. In this article, specifically, the procedures for adapting a low-cost three-dimensional scanning method using images collected by smartphones is presented, and some results will illustrate the contributions of this proposal.

The objective was, therefore, evaluate whether the shooting mode, the camera resolution and the model would affect the final quality of the 3D objects generated.

In the experiments, only fresh and perishable foods were used, with textures, colors and structural complexities distinct from each other, as illustrated in Fig. 1, where the green avocado has a rough surface, the red apple is smooth and textured, the brown pear has a matte and textured surface, the green apple is smooth, the papaya has mixed and matte colors, the dark green avocado is rough and the yellow melon has few texture.

Fig. 1.
figure 1

Fruit samples selected for the study

Full size image

Intermediate and high-end smartphones were tested, with different prices, all with a minimum sensor size of 12 megapixels, regardless of the release date, as shown in Table 1.

Table 1. Main features of smartphones used
Full size table

The experiments also evaluated possible image capture scenarios, with fixed, spinning (turntable) and mini studio with turntable and specifically controlled lighting (Fig. 2). In this sense, characteristics of the workbench, lighting, platform with rotation angle markings, support to minimize object occlusion and tripod with tilt adjustment were considered.

Fig. 2.
figure 2

Scenarios for capturing images: fixed base, swivel base and mini studio

Full size image

In the scenarios with a rotating base, stabilizers were used for the fruits and remote shooting (headphone trigger) for the smartphones, minimizing the variation of unwanted movements.

The environment for the setup of the three types of scenarios and the execution of the experiment aimed to simulate a simple home office. This environment focused on ease of configuration, human factors and application of the three image capture techniques for low-cost photogrammetry. In addition, the same general lighting settings were used for all scenarios. Figure 3 shows a schematic view with the location of the fixed base scenario (Fig. 3-A), rotating base (Fig. 3-B) and mini studio (Fig. 3-C).

Fig. 3.
figure 3

Schematic views of the environment and scenario settings

Full size image

The samples received markers and small volumetric pieces to aid in their positioning. They were measured and weighed, as illustrated in Fig. 4, before being photographed. Altogether, for each of the seven fruits, three scenarios and four smartphone models were considered, totaling 84 image sequences (see Fig. 5). They underwent photogrammetric processing, carried out by the Meshroom software, and were edited and evaluated by the CloudCompare and Blender software. For more information about these software, see [14, 15] and [16].

The Hardware used to process the image files and analyze the three-dimensional models was an Avell® notebook, model 2019 with a ninth generation i7 9750H processor, Geforce® GTX 1660 Ti video card, 512 SSD disk with 32 GB of DDR 4 RAM on the Windows 10® platform.

In meshroom software it was adopted for all image sequences the high processing mode with the depthmap resource set to 2.

Fig. 4.
figure 4

Instruments, items and previous procedures performed with the samples

Full size image
Fig. 5.
figure 5

Images samples illustrating the data capture process

Full size image

The capture times of the image sequences by the operator, for both the different scenarios and types of fruit, provided a set of variables to be analyzed. Other interesting variables are related to the processing times to different image sizes and smartphones, using the same processing parameters in photogrammetry software.

Also, this study considered the size of the cache file and 3D file generated by the processing of the photogrammetry software, took into account that these elements influence the handling and management of objects and disk space. In this sense, some hypothesis tests were performed (parametric, whenever the data distributions could be considered Normal, with p > 0.05, and non-parametric approaches in other cases) to infer about possible significant differences in the results of the experiments, taking into account the comparaition of these variables.

3 Results and Discussion

The 3D files of the generated and treated samples, including the application of texture (see some examples in Fig. 6), were analyzed in order to identify the best parameters to be used in procedures based on the low-cost photogrammetry proposed in this research.

Fig. 6.
figure 6

Examples of 3D objects generated by photogrammetry

Full size image

The average processing time for each fruit, using the same hardware and photogrammetry software parameters, is ranged from 98 to 132 min, as illustrated by the box plots in Fig. 7. Analysis of Variance showed (p = 0.0352) that, regardless of morphology, size, shape, roughness, color or texture of the fruits used in the experiments, there was no significant difference in these processing times.

Fig. 7.
figure 7

Variables associated with fruit image processing times by photogrammetry

Full size image

The charts in Fig. 8 present, for the four smartphone camera resolutions (horizontal axis), the mean cache values (left), mesh file size (center) and processing time (right).

Fig. 8.
figure 8

Results associated to the resolutions of the tested cameras

Full size image

A significant difference was identified between the cache sizes when compared by camera resolution (p < 0.001), and the post hoc analysis showed that the camera with 16 MP is associated to a higher value than the others, and the one with 25 MP is related to a smaller value. Those with 12 MP and 13 MP generated similar measures.

A significant difference was also identified between the sizes of the mesh files when compared by camera resolution (p < 0.001). In this case, the post hoc analysis indicated that this difference is associated with the 25 MP resolution.

Finally, a difference (p < 0.001) was identified in the processing times, and the post hoc analysis showed the camera with 16 MP is associated with to a longer time than the others, and the one with 25 MP is related to a shorter time. Those with 12 MP and 13 MP generate similar processing times.

It is quite surprising that the highest resolution camera (25 MP) was associated to the smallest cache size, mesh file size and image processing time. One possible reason for that is the smaller aperture (f-stop 1.7) used in taking the images by this equipment. However, there is a possibility that these variables are correlated with each other and with other hardware and software configurations. To deepen this analysis, it would be interesting to perform a multivariate analysis.

Figure 9 illustrates the time variations for shooting in the three scenarios. The Kruskal-Wallis test indicated that there was a significant difference (p < 0.001), which was established, from the post hoc analysis, as smaller for the fixed base scenario in relation to the spinning base scenario and the mini studio. This result is interesting, because in addition to reducing the time to photograph, the configuration is simpler than the others.

Fig. 9.
figure 9

Results associated to time for taking photos in each scenario

Full size image

The longer time required to obtain the photos in the mini studio setting is justified by the care taken to no collide with such structures and to no harm the alignments at each image taking, due to the reduced movement space, amplified in the case of larger samples. In the scenario composed by the turntable, the increase in time may be due to the care taken to rotate the table without move the samples.

The downside of the fixed-base scenario is that it requires more space to use, and it generates greater discomfort for the photographer, who must work standing up.

Figure 10 shows the average time taken to obtain the images of each fruit, with no significant difference in values (p > 0.05). However, we remarked that higher values were observed for the bulkier samples and those with greater instability (avocado, papaya and melon).

Fig. 10.
figure 10

Means procedure times for taking photos in each fruit groups

Full size image

With the exception of the avocado samples, the other fruits, when using the elevation base, had a reduction in image capture time, and an increase in the generated structural quality (further details on the measurement of structural quality can be obtained in [13]).

To exemplify, we present, in Fig. 11, some results os structural projects and concepts of packaging, carried out by [13] and [17], from the images generated by the use of the proposed low-cost method.

According to [13], in general, the resulting models present an adequate reconstruction, with several well-structured areas in the center of interest. Although it does not have a significant participation in the structural improvement of the objects, the use of the scenario with a mini studio helps minimize reflections and shadows generated in samples. The same author observed that automatic modes with remote triggering (headphone button) were the most suitable, and that the sitting position, when using the swivel base, was the most practical, safe and comfortable. In addition, the use of a tripod was fundamental in the revolving base scenarios, as it kept the smartphone fixed, allowing greater control over angles and heights.

Fig. 11.
figure 11

Examples of packaging concepts generated with the 3D models produced

Full size image

4 Final Remarks

This article was focused on a specifical research stage whose general objective was to determine how to use three-dimensional models, obtained by photogrammetry carried out with the aid of smartphones, to reduce the costs associated with the structural design of packaging.

Experiments were described, carried out by a quantitative study of the variables involved in the proposed low-cost three-dimensional digitization method.

The analyze indicated to use a camera with 25 MP at least, if there is a need to reduce the processing times of the images, the cache or 3D mesh sizes of the objects that will be generated.

If these are not restrictions, 12 MP resolution is sufficient. To obtain the photographs, a fixed base scenario is the best option in terms of time and simplicity. The alternative scenarios compared require extra care to avoid collisions with structures, misalignment of objects and their displacement.

The use of an elevation base generates reduction in the time to edit the 3D models, and an increase in the structural quality of the generated objects, due to the fact that it minimized the occlusion of contact between the fruits and the base. In addition, the use of a tripod is considered essential in swivel base scenarios, to keep the smartphone fixed, allowing greater control over angles and heights.

Because there was no significant difference in time to obtain the images of each fruit, the time used for the procedure is independent of the different sizes, shapes, roughness, colors or textures of the product.

Some exemples of structural packaging projects were presented, showing the importance of images obtained by 3D models generated by the use of the proposed low-cost method.

This low-cost option proved to be viable for the application tested. Because of that, there is important to apply it to samples with higher levels of complexity, as well as to test different methods of artificial texturing in the models, to develop supports that reduce the occlusion and explore additive manufacturing analyzes to associate and evaluate physical models with their virtual counterparts.

In order to deepen and continue the research, it is suggested, for future work: to apply, in samples with higher levels of complexity, the technique proposed here, in order to verify more precisely the processing and generation times of the models; test different methods of artificial texturing in the models, with the objective of investigating the application times and the most suitable materials for this purpose; apply other types of markers or study ways to reduce them, in order to minimize the virtual texturing work in the covered areas; develop supports that reduce the occlusion, without damaging the samples and that maintain stability in the handling stage of the turntables; explore other possibilities of analysis, such as additive manufacturing, for example, for association and evaluation of physical models with their virtual counterparts; and develop more in-depth studies in interactive virtual environments, aiming to investigate the efficiency and accuracy of samples in low-cost systems.