Journal für Kulturpflanzen, 75 (09-10). S. 235–247, 2023 | DOI: 10.5073/JfK.2023.09-10.02 | Müller-Blenkle et al.
1*, Ulrich Simon2, Ralf Meyer3, Isabell Szallies4, Daniela Lorenz4, Sabine Prozell5, Matthias Schöller5, Cornel S. Adler1A method for acoustic storage pest detection and its challenges
Eine Methode zur akustischen Erkennung von Vorratsschädlingen und ihre Herausforderungen
 | (c) The author(s) 2023 This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/deed.en). |
Submitted/accepted for publication: 6 June 2023/16 August 2023 |
Insects in grain can cause serious problems, not only because they feed on the grains. Mass reproduction also causes additional moisture and heat due to the insects' metabolism. This leads to favourable conditions for moulds, which can cause major losses and the formation of mycotoxins. It is therefore important to detect and treat an infestation at an early stage. The "Beetle Sound Tube" system was developed as an acoustic early detection system for insects in grain, which makes it possible to detect even very low levels of infestation and inform the storekeeper by e-mail. The acoustic system remains in the grain during the storage period, and permanently records insect sounds. Challenges were encountered in the development of this acoustic monitoring system, such as analysing very quiet insect sounds in a noisy agricultural environment. In addition, the amount of data collected and the speed of analysis had to be optimised to achieve just-in-time detection of insects. The acoustic tube system was developed for silos, flat stores and big bags and is therefore widely applicable.
Storage protection, acoustics, early detection, insects, "Beetle Sound Tube"
Insekten im Getreide sind ein Problem, nicht nur, weil sie das Getreide fressen, sondern weil sie durch Massenvermehrung mit ihrem Stoffwechsel dazu führen, dass Wärme und Feuchte und damit ein guter Lebensraum für Schimmel entsteht, der das Getreide unbrauchbar macht. So droht die Kontamination mit Mykotoxinen. Daher ist es wichtig, Insektenbefall frühzeitig zu erkennen und zu bekämpfen. Mit dem "Beetle Sound Tube" wurde ein System zur akustischen Früherkennung von Insekten in Getreide entwickelt, mit dem es möglich ist auch sehr geringen Insektenbefall akustisch zu erfassen und den Lagerhalter per E-Mail zu benachrichtigen. Das System befindet sich während der Lagerperiode dauerhaft im Getreide und überwacht das Getreide auf Insektengeräusche. Während der Entwicklung dieses akustischen Monitoringsystems ergaben sich unterschiedliche Herausforderungen bei der Analyse der sehr leisen Insektengeräusche in einer Umgebung mit vielfältigen Hintergrundgeräuschen. Auch anfallendes Datenvolumen und Analysegeschwindigkeit mussten optimiert werden, um Befall „just-in-time“ erfassen zu können. Das Akustik-Röhrensystem wurde für Silos, Flachläger und big bags entwickelt und ist damit für viele Lagerformen geeignet.
Vorratsschutz, Akustik, Früherkennung, Insekten, "Beetle Sound Tube"
In agriculture, great efforts are made to control plant diseases and insect pests and to harvest a good crop. To ensure a continuous supply of food, the crop must then be stored until the next harvest or for tough periods. Therefore, the protection of stored products is important to keep them in good condition for a longer period of time with minimal losses.
However, storage protection should not only be considered as something at the end of the production chain – it also protects the seed for the next cultivation, ensuring that it is healthy and germinable, and therefore storage protection plays an important role in the crop production cycle.
Dry stored products are kept in different types of facilities such as silo bins, flat storages or big bags. In many cases large amounts of grain are stored, which makes quality monitoring elaborate and more important. Essential for good storage is to keep the grain cool and dry and to avoid storage conditions that allow condensation of water on walls or on the floor. In many cases, the state of storage facilities is not optimal and flawed sealing allow mice, rats and birds to feed on the stored products and to contaminate them. In addition, a number of beetles, moths, dust lice and mites are specialized on stored products. While living in the grain mass they produce moisture and heat by metabolism, which improves the conditions for more insects and mites. As temperature and humidity increase, conditions also improve for moulds, which can spoil whole batches of grain (Zain, 2011; Peter et al., 2013). Data on post-harvest losses are sparse and vary in different parts of the world. The FAO (2011) estimates post-harvest losses in cereal grains of up to 10% in industrialized Asia and about 8% in Africa while about 4% are lost in Europe and Latin America. For Germany, post-harvest losses of wheat in on-farm storage are estimated at 3.3% (Peter et al., 2013).
Climate change with rising temperatures and mild winters can make it more difficult to protect stored products. Higher temperatures provide better conditions for storage pests and may enable them to have more generations per year. Mild winters and dry summers might allow stored product pests to survive outside of storage facilities and infest crops, in the field as it is common in warm climates (Adler et al., 2021). In addition, thermophilic species from the south could invade new habitats.
Today, successful protection of stored product is based on three pillars: 1. prevention of losses through good storage conditions, 2. early detection of infestation, 3. early treatment of infestation.
There are various ways to detect insects in grain such as traps, sieving, visual or acoustic inspection. Traps have the advantage of collecting insects over a longer period of time from a larger area around the trap and are therefore more effective than visual inspection or sieving of a sample taken at a specific time. However, the traps have to be checked regularly, which can be very time-consuming depending on the inspection interval and the number of traps. Systems for detecting insects with cameras are currently developed (Schott, 2021; Adler et al., 2022b), but species identification is challenging for small storage pests.
When insects are detected at an early stage of infestation, the choice of measures against infestation is larger than when the infestation level is high, and losses can be kept low. This is of particular importance for organic farming, where chemical treatment of pests is restricted. For example, the use of beneficial insects is most effective when the number of insect pests is low and the infestation is controlled with a large number of natural enemies (inundation). In case of severe infestation, natural enemies may only slow down the reproduction of the pests.
Early detection also prevents the infestation from spreading to other parts of grains, which could cause high losses.
One way of early detection is to record the sounds insects make as they move and feed in the grain. This possibility was discovered almost a century ago when Brain (1924) detected the rice weevil Sitophilus oryzae with a microphone. Unlike other species that use acoustics for communication, this is not the case with stored product pests, where only movement and feeding sounds can be detected. Identifying these sounds in the grain mass would allow earlier detection of infestation. Therefore, a number of researchers have conducted studies in this field, only a few of which can be mentioned here (Fleurat-Lessard & Andrieu, 1986; Vick et al., 1988; Hagstrum et al., 1990; Hagstrum & Flinn, 1993; Shuman et al., 1993; Mankin et al., 1996; Fleurat-Lessard et al., 2006; Potamitis et al., 2010; Mankin et al., 2021).
Acoustic detection has mainly been used on a small scale with acoustic probes or acoustic containers that used sensors for structure-born sounds (Busnel & Andrieu, 1966; in Fleurat-Lessard, 1988; Hagstrum et al., 1994; Reichmuth et al., 1996). A disadvantage of acoustic probes is the disturbance of both the grain and the insects, so that the the acoustic sensor may either detect sounds caused by the movement of the grain, or the insects may feign death (thanatosis) and no signals may be detected at all. Experiments with acoustic sensors embedded in grain were carried out by Hagstrum and colleagues (1994; 1996). Infestation could be detected and the results provided a good estimation of the level of infestation. Using a large number of acoustic sensors the authors found that insects were detected mainly in the upper layers of the grain.
The “Beetle Sound Tube”-system uses perforated tubes embedded in the grain mass that act as a large insect trap. They contain one acoustic sensor at the bottom of the tube directly above the trap container and thus listen to anything caught inside the tube. The tubes remain in the grain throughout the storage period and permanently monitor the insect sounds without causing disturbances. Additional sampling with other traps is not necessary and continuous monitoring increases the probability of detection compared to short-term observations.
The name of the system is derived from the large, at least 1.5 m long, perforated metal tubes that serve as insect traps and contain the acoustic equipment. Due to the comparatively large size of the tubes, they are able to detect even low infestation levels. The storage keeper is automatically informed about the sounds produced by insects in the traps and can decide on the next steps against the infestation (Fig. 1).
Fig. 1. Procedure of the "Beetle Sound Tube"-system. Grain is stored, insects appear in the grain, they can be recorded, the storage keeper is informed by email and can decide on appropriate measures to control the infestation.
In a previous project the grain weevils Sitophilus granarius could be detected acoustically about nine weeks earlier in a volume of 1 or 8 m3 of wheat with a piezoelectric precision microphone PCB-378B02 (PCB Synotech, Hückelhoven, Germany) compared to temperature monitoring in the grain mass or inspection of the grain surface (Müller-Blenkle et al., 2018). Different tubes were used in the experiments with perforated tubes being the most suitable for detecting grain weevils first. However, beetles were also detected quite early in non-perforated tubes, showing that the insects could also be detected in the grain around the tube. Based on these findings the "Beetle Sound Tube"-system was developed to detect insects mainly but not exclusively in the collection container inside the tube.
The aim of the project was to develop an automatic detection system that uses sounds coming from insects to detect infestation. Since stored product insects species do not use sounds to communicate, identification can only be based on passive sounds caused by feeding or moving, which makes identification more challenging. Therefore, we have tried to answer the following questions
Is it possible to identify typical insect signals?
Can the number of signals be related to the level of infestation?
Can a permanent acoustic monitoring system be used under field conditions in storage facilities?
This publication describes the development and methods of the "Beetle Sound Tube"-system.
The "Beetle Sound Tube"-system consists of a varying number of perforated metal tubes embedded in the grain mass. The tubes are completely perforated except from smaller parts at the bottom and top. Each tube contains an insect trap at the bottom that can be pulled up on a string for clearing. Clearing was done at regular 14-day intervals, and the contents of the traps were determined in the laboratory.
Each tube is equipped with one acoustic sensor near the trap container at the bottom of the tube and 1 or 3 climate sensors depending on the size of the tube. The climate sensors were located directly above the trap at the bottom of the tube and with tubes of more than 1.5 m in the middle and near the top of the tube. Details of the system are described below. The "Beetle Sound Tube"-system was installed on the premises of four different agricultural companies with different types of storage facilities. In two cases the system was installed in organic farms where the harvest was stored mainly for animal husbandry. The other companies received grain from surrounding farms for feed production or as a grain trader.
In 2018, the first farm with a 70-t silo was equipped with the system, followed by three other systems installed in 2019 in a 300-t silo, flat storage and big bags, respectively. Trials were conducted over three to five storage periods.
A main component of the acoustic system were the "Beetle Sound Tubes" manufactured by the project partner WEDA Dammann & Westerkamp GmbH (Lutten, Germany). These tubes were made of stainless steel and placed in the grain mass in a vertical position.
The project partner AGRAR TECHNIK BARNIM (Bernau, Germany) had to develop a specific mounting system for the different local conditions in order to install the tube system in silos. The first system was installed on an organic farm in a 70-t indoor silo (Sketch see Fig. 2A). A scaffold structure was erected around the silo with a working platform above the silo to which the three meter long tubes with an outer diameter of 100 mm, 1 mm wall thickness were attached by chains.
Fig. 2: "Beetle Sound Tube"-system in different storage facilities. A: three tubes in a 70 t silo, B: three tubes in a 300 t silo, C: one tube in a 1 m3 big bag, D: nine tubes in flat storage.
For the second silo with a volume of 300 t it was not possible to attach the tubes to an existing structure in the silo or to build a structure strong enough to withstand the forces that occur when filling the silo or removing the grain mass. Therefore, the tube system consisting of three tubes with 80 mm outer diameter (2 mm wall thickness) was extended to the full height of the silo of 18 m and placed on the concrete floor inside the silo (Sketch see Fig. 2B). Metal crossbars provided additional stability every two meters between the three tubes. The upper part of the tubes, which was exposed to the weather above the roof, was embedded in a plastic pipe, covered with a plastic lid and insulated against temperature differences to avoid condensation. All tube tops were accessible from a walkway above the silo to allow maintenance and inspection of the insect traps. For this purpose, traps and microphones were suspended from above on plastic cords.
While the tube system inside the silos could remain there all year round, the tubes in flat storages or big bags would have been in the farmer’s way as soon as the grain was moved. In flat storage front loaders are usually used, in big bags the whole bag is moved. Thus, only a mobile solution that can be installed at the beginning of storage and removed again at the end of the storage period came into question. Therefore, for a flat storage facility and an organic farm that used big bags, individual tubes in two sizes were developed that can be drilled into the grain by hand. The 1 m3 big bags are the smallest storage unit in which the "Beetle Sound Tube"-system was tested (Sketch see Fig. 2C). The shortest sound tube is 1.5 m long (diameter 100 mm, 2 mm wall thickness). The pointed lower end and a screw blade make it possible to drill the tube into the grain at the beginning of the storage period. For three-meter sound tubes used in large flat storage facilities, an extension piece of 1.5 m can be added (Sketch see Fig. 2.D). In this case the lower part of the tube is drilled into the grain before the extension is screwed on.
To prevent anything like grain, mice, birds etc. from falling into the tube, the tubes were covered with metal lids.
While the tube system has been refined according to the conditions at the different storage facilities the changes also might influence the acoustic properties of the system. Different conditions may require some calibration of the software after a "Beetle Sound Tube" system has been set up.
Each "Beetle Sound Tube" was equipped with one to three climate sensors (EE 071, E + E Elektronik, Engerwitzdorf, Austria) to record temperature and relative humidity. Additional sensors were placed below and above the grain surface.
Each tube was equipped with an insect trap located at the bottom of the tube with a cord attached to pull it upwards. It also contained a microphone freely suspended in the "Beetle Sound Tubes" to record air-born sounds and one to three climate sensors (depending on the length of the tube). All sensor cables were attached to the cord of the trap container with cable ties. Therefore, the entire bundle of sensors, cables and trap was removed from the tube during trap clearance. For insulation, the bnc-microphone cables were wrapped in foam material where they came in contact with metal parts (e.g. the lid of the tube) or power cables.
Within the grain mass sounds are mainly transmitted through air passages (inter-grain spacing) between the grains (Hickling et al., 1997). Structure-borne sound is very strongly dampened by the grains (Hickling et al., 1997). The airborne sound, which reaches the inner volume of the tube, can propagate to the microphone only attenuated by the air. Since the sound pressure caused by the insects is very low and is further attenuated by the grains and the air, the acoustic system we used in the experiments consisted of a precision piezoelectric microphone PCB-378B02 with a linear intrinsic noise of 18.5 dB (PCB Synotech, Hückelhoven, Germany). This microphone had already proven suitable for recording beetle noises in a previous project (Müller-Blenkle et al., 2018). Special low-noise microphones were considered too expensive. During the experiments, the microphone recorded five minutes every hour to limit data collection. Permanent recording and evaluation is possible but was omitted for reasons of data reduction.
For recording of climate and acoustic data, the project partner Müller-BBM Acoustic Solutions GmbH (Planegg, Germany) developed the recording system shown in Figure 3. The measurement units supplied the IEPE constant current, performed the impedance conversion and converted the analogue signal to digital values with a sampling rate of 48 kHz and a resolution of 24 bits. It also collected the data from the climate sensors and forwarded all the data to the central microcomputer via a LAN interface. Each climate sensor was given an ID so that the measurement unit could address each sensor individually. All measurement units were supplied with power via Power-Over-Ethernet (POE) network switches.
Fig. 3: Technical setup of the "Beetle Sound Tube“-system. A microphone and climate sensors were located in each sound tube and connected to the measurement units (MU). A microcomputer (μC) collected and saved all data from the measurement units via Ethernet connections and could be accessed remotely.
A Raspberry-PI was chosen as the microcomputer. A Web-Server was installed on the Raspberry, which enabled access and configuration the recording system. Parameters for the configuration were for example, the time of the recording, the recording length or the storage location. It was also possible to listen to acoustic recordings and check climate measurements. Remote access via internet was possible by using a WLAN-router or an LTE-Modem and a daily status email sent by the system facilitates the regular monitoring.
The structure of the measurement system was easily expandable by adding further measurement units. So far, the system was operated with up to nine sound tubes simultaneously.
It was necessary to identify individual pulses, store them separately and discard the parts of the recording without signals to reduce the amount of data. Therefore, a pulse detection algorithm was developed, that cuts the pulse signals from the recording and stores them into individual sound files. Figure 4 shows the principle of the algorithm: The signal was divided into blocks of 128 samples, corresponding to a duration of 2.7 ms, and the energy per packet was determined by squaring the sample values and summing them up. The minimum energy in a 5-minute recording was taken as the background energy level. Now the recording was searched for energy blocks exceeding the background energy level by a certain dB level (blue block). If such a block was found, the adjacent block with the maximum level (green block) was taken to determine the beginning and end of the pulse, by taking the two left blocks the maximum block itself and the seven right blocks, resulting in a pulse length of 1280 samples.
Fig. 4: Principle of the algorithm for identifying individual pulses. The energy levels of 128 sample blocks were determined and related to the minimum background noise level in a 5-minute recording. A pulse contained a block that exceeded the background noise level by a certain value (blue block) and a block with highest energy (green block). The start and end of the pulse were defined as the two blocks before and seven blocks after the block with the highest energy.
In Addition a 1/3 octave analysis was performed, calculating the energy level for 16 1/3-octave-bands between 250 and 8000 Hz. Again, for the filtered signals, 128 samples were squared and summed up to calculate the energy level. Due to different environmental factors (e.g. storm, rain, heavy machinery, ventilation), for every recording an individual background noise was calculated as the 5% quantile of the energy levels for each 1/3 octave band. If the energy level for a 1/3-octave band exceeded a threshold of 25 dB above background noise in the frequency range between 250 and 800 Hz, 20 dB between 800 and 2500 Hz or 15 dB between 2500 and 8000 Hz this band “scored”. To ensure, that broadband noise was not counted as beetle signal, all signal with threshold exceedances below 800 Hz were discarded. The number of scores was calculated for each signal.
We present data from a flat storage with infestation of barley with the sawtoothed grain beetle Oryzaephilus surinamensis. The infestation could be heard as short cracking sounds (Fig. 5).
Fig. 5: Recording of insect sounds produced by Oryzaephilus surinamensis, displayed as spectrogram (top) and amplitude over time (bottom)., filtered (highpass 500 Hz, low pass 15,000 Hz, Butterworth, second order).
Figure 6 shows an example of the number of pulse signals from trap 1 over a period of about three-month of storage. Shown is the number of signals per five-minute recording per hour. The green squares indicate the time when the trap was cleared, and the numbers in the squares next to the markings indicate the number of beetles found in the trap at that time. The green 3 dB line contains all signals that could be identified, i.e. beetles and all background noise. A large part of the background noise is likely to come from the solar panels on the roof of the building. They mainly occur around midday and the number of pulses increases from spring into summer. With the help of the correlation analysis (see below), these disturbances could easily be filtered out. Despite the background noise, the beetle signals in the green 3 dB line are still distinguishable and increase between trap clearings.
Fig. 6: Number of detected events in 5-minute recordings over a period of about 3 months. Signal detection with varying minimum amplitude levels of 3 to 7 dB. The green squares indicate the time of trap clearance with the number of insects found in the squares next to the green mark.
The maximum level of the pulses in relation to the background noise was recorded to find a suitable sensitivity for pulse detection. 5 dB was the threshold for detecting the pulses, coming from the insects. Immediately after clearing the trap, hardly any pulses were detected, with the number of signals increasing over time until the next clearing. Higher sensitivity resulted in the detection of additional pulses caused by other sources and background noise and even when traps were empty, many signals were detected.
The number of insects in the traps corresponded to the number of signals based on the signal detection with minimum amplitude level of 5 dB before clearance. Falling temperatures in December/January lead to lower insect activity and lower number of detected signals. While the number of events increased with the number of insects, the database is not solid enough to come to a statistical correlation. The number of sound events depends not only on the number of insects but on the activity level, which is influenced by factors such as temperature, humidity and infestation level.
The number of pulse signals with an amplitude height of 5 dB or more in Figure 6 still contains outliers originating from other sources e.g. from other animals like mice or birds, or from technical sources such as the solar panels on the roof of the warehouse.
Therefore, all pulse signals of an arbitrary chosen 5-minute recording were cross-correlated with each other to create a correlation matrix containing the peak value of each correlation. The height of the peak is a measure of the similarity of the two signals. Using this matrix, the statistical method of principal component analysis (Smith, 2002) was performed. This method identifies the principal components in a dataset and gives an indication of how many different signal types are contained in the data set. In addition, for each impulse signal we obtain the value of how it cross-correlates with the principal components.
The result shows that the first components already represented the events very well. The second eigenvalue was only 20% of the first eigenvalue. Figure 7 shows the pulse signal that has the highest cross-correlation value of 0.933 with the most dominant component displayed as spectrogram and waveform. This pulse signal looks like a pulse that could come from a beetle falling into the trap.
Fig. 7: Pulse signal with the highest cross-correlation of 0.933 with the most dominant principal component displayed as spectrogram (top) and waveform (bottom).
A second dominant component displayed in Figure 8 that cross-correlates with a value of 0.55 was found that is displayed in Figure 8. The second pulse signal looks like regular pulses occurring at a frequency of 100 Hz. It is highly unlikely that this signal comes from an insect. There are very few species of stored product pests that have been shown to be capable of producing some active sounds (Reid, 1942; Bailey & Lemon, 1968; Arnett, 1968). In O. surinamensis and its close relative O. mercator, there is no evidence of active sound production.
Fig. 8: Pulse signal with the highest cross-correlation of 0.55 with the second most dominant principal component displayed as spectrogram (top) and waveform (bottom).
Figure 9 shows the number of signals that matched the template pulse by at least 20, 30 or 40%. Signals with a smaller correlation were removed from the dataset since they are most likely to originate from background noise. In this way, the disturbance most likely caused by the solar panel (Fig. 8) could be mostly discarded.
Fig. 9: Number of identified events with different levels of correlation (k) to a typical beetle signal (>20% grey, >30% brown, >40% green). The green squares indicate the time of trap clearance with the number of insects found in the squares next to the green mark.
It turned out, that the correlation analysis worked well for some recordings, but failed for others. Therefore, a more robust analysis was developed, that was also less time consuming and could be performed in time by the measurement system itself. An example for different signals is shown in Figure 10. The 1/3 octave analysis was implemented in the measurement unit and enabled an automatic evaluation, which was summarized in a daily email to the storage keeper that also contained climate data.
Fig. 10: Results of a 1/3 octave analysis. The background noise and thus the thresholds (25 dB above background noise in the frequency range between 250 and 800 Hz, 20 dB between 800 and 2500 Hz or 15 dB between 2500 and 8000 Hz) vary with every recording. A signal is counted as beetle signal if it exceeds the threshold in at least one 1/3 octave band of 800 Hz or above but not in lower frequency bands.
For the early acoustic detection of insect infestations, the identification of insect signals is crucial. The combination of quiet insect signals and a noisy environment in storage facilities makes detection a challenge. In addition, an infestation must be detected as soon as possible to avoid mass development of insects.
In order to inform storage keepers about an infestation at an early stage an acoustic early detection system for stored product pests was developed. The "Beetle Sound Tube“-system was adapted for different types of storage such as silos, flat storage and big bags using different tubes, sizes and materials as well as different numbers of tubes. There was a concern, that the different types of tubes would make insect detection more challenging and that data would not be comparable. However, it was clear from the results that the influence of the trap container was more important than the tube system and that a standardized trap container is important. Nevertheless, a more standardized type of tube should be used for future systems, differing in length but not in diameter and wall thickness.
Different methods have been used to detect insect signals in recordings, which have their advantages and disadvantages. To answer our initial questions:
Yes, typical insect signals can be identified and used for automatic infestation detection. Different detection methods were used to separate the signals from different types of background noise. The correlation analysis proved to be time-consuming and required considerable computer capacity. Therefore, it is not very suitable for timely infestation detection. The 1/3 octave analysis is fast and can be used “just in time” to count insect signals. However, the 1/3 octave analysis is not accurate enough for species identification. So the system would be able to detect the presence of an infestation but not the species causing it. Species identification would be needed when parasitoids are to be used for treatment. So a second step to identify the infestation would still be needed to identify the infestation and the trap contents would need to be checked manually. However, for further work the use of deep learning could enable species identification. Acoustics have been successfully used for environmental studies to identify species based on sounds (Aide et al., 2013, Fairbrass et al., 2019, Kahl et al., 2021, Sueur et al., 2012). Since stored product pest insects do not produce sound for communication, identification will be much more challenging. Deep learning algorithms and methods have been greatly improved in recent years, so that it may be possible to obtain spectrograms of insect movement or feeding sounds or even the sound of an insect hitting the bottom of a trap container to determine species identity. This method has already been used to monitor pollination, through the buzzing sounds of insects (Folliot et al., 2022). The authors created spectrographic images of sounds and used them to train a convolutional neural network (CNN) to automatically recognise the sounds. Initial attempts have also been made to identify flying insect pests in greenhouses using deep learning (Branding et al., 2022). Deep learning can be used for insect detection in agriculture in different ways and it has the potential to improve the efficiency and effectiveness of insect monitoring (Teixeira et al., 2023). But to use deep learning for acoustic detection of stored product insects, the environment needs to be standardized as much as possible so that small differences between species can be distinguished. The algorithms for automatic recognition are being developed further, but depend on the availability of reference libraries for sounds (Ross et al., 2023). Further research is needed here to generate data sets of different stored product pest species under controlled conditions in a standardized trap to train a model. This could allow species to be identified from the sounds of insects moving or feeding, or perhaps even from the sounds made when beetles of different sizes and shapes fall into the trap.
Using cross correlation, the number of signals can be related to the level of infestation. Figure 6 shows the number of detected events in 5-minute recordings and the correlating number of beetles found when checking the trap. However not all traps showed comparable results. This led to the conclusion that the detected signals were strongly dependent on the acoustic properties of the collection container. If the container was suspended freely in the sound tube, beetles produced signals when getting in contact with the container. But if the container hung askew and was in contact with the sound tube, the resonance was disturbed and hardly any signals were recorded. By adaption of the trap container, this problem could be solved. Five O. surinamensis in the trap produced enough signals for reliable detection, but the number of signals depends on the activity of the insects and can therefore vary. Since the “Beetle Sound Tubes” are effective insect traps, even a small infestation is detected. It depends for example on the insect species, its behaviour, the temperature and on the decision of the storage keeper what level of infestation is acceptable and when control is necessary. The detection system therefore allows the storage keeper to set different warning levels depending on the information needs. For future work, it would be useful to determine a correlation between the number of signals detected and the number of insects caught in the trap. The calculation would at least have to take into account the factors mentioned above that influence the activity of the insects.
It has been possible to develop a permanent acoustic monitoring system that can be used under field conditions in storage facilities. In close contact with storage keepers, it was possible to develop a system that detects insect signals and sends necessary summary information including temperature and humidity data to the storage keeper at specific time intervals.
Acoustics in storage protection is a rather small field of research. In view of climate change and rising temperatures, stored product protection will be an area of increasing importance due to better conditions for storage pests and decreasing harvests (Adler et al., 2022a).
The "Beetle Sound Tube“-system shows that permanent acoustic monitoring can also work with passive acoustic signals. The next step should therefore be to bring the "Beetle Sound Tube“-system to the market and to make it much easier and faster for storage keepers to detect storage pests.
Automation and the use of AI that is already on the rise in agriculture (De Baerdemaeker et al., 2023, Teixeira et al., 2023) and will also gain important in the protection of stored products (Ajisegiri et al., 2022). Automatic detection by camera (Adler et al., 2022b; Mendoza et al., 2023), microphone or artificial nose (Ali et al., 2023) is likely to play an important role in future storage facilities.
The species composition during the experiment showed mainly O. surinamensis, the sawtoothed grain beetle. Experiments conducted by Simon & Müller-Blenkle (2021, unpublished work) with O. surinamensis next to a sound tube embedded in grain indicate, that the small and flat beetles cannot be heard moving in grain as they move through the gaps between grains without moving the grain. They are also not expected to produce strong feeding sounds, as they feed on grain fragments and softer parts and only larger larvae can bore into the kernel. Weevils on the other hand push the grains aside as they move, which can be heard inside the tube (Müller-Blenkle et al., 2018). In addition, feeding sounds are stronger in weevils, as they drill holes into kernels to lay their eggs in them. During development the larvae scoop out the kernel, which is audible as scraping sound. To enable acoustic detection for a wide variety of stored pests, the focus of acoustic monitoring was shifted to the insect collector. The acoustic system has recently been optimized in both in terms of data interpretation and cost effectiveness. Preliminary results indicate that it was possible to reduce most interference and make the recording system more robust for harsh conditions. Detailed results will be published in following work.
We thank the funding agencies European Agricultural Fund for Rural Development (EAFRD) and the Ministry of Rural Development, Environment and Agriculture of the Federal State of Brandenburg (MRDEA) that made this work possible.
We also thank the whole project team that consists of twelve project partners from agriculture, science, associations and industry, led and coordinated by agrathaer GmbH for lots of work, inspiring discussions and great ideas. Additionally, colleagues of Müller-BBM and JKI contributed practical help and inspiring ideas.
Thanks a lot to Torsten Meiners for his helpful comments on the manuscript.
All authors contributed to the study conception and design and were deeply involved in the development of the "Beetle Sound Tube"-system. The “Beetle Sound Tubes" were built by Ralf Meyer, parts of the technical equipment was developed by Ulrich Simon. Data collection and analysis were performed by Ulrich Simon and Christina Müller-Blenkle. The first draft of the manuscript was written by Christina Müller-Blenkle and Ulrich Simon. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
“Beetle Sound Tube” is an eip-agri Agriculture & Innovation project funded by the European Agricultural Fund for Rural Development (EAFRD) on behalf of and co-funded by the Ministry of Rural Development, Environment and Agriculture of the Federal State of Brandenburg (MRDEA), Grant number 204017000004/80170084.
The author(s) declare that they do not have any conflicts of interest.
Adler, C., C. Athanassiou, M.O. Carvalho, M. Emekci, S. Gvozdenac, D. Hamel, J. Riudavets, V. Stejskal, S. Trdan, P. Trematerra, 2022a: Changes in the distribution and pest risk of stored product insects in Europe due to global warming: Need for pan-European pest monitoring and improved food-safety. Journal of Stored Products Research 97, 101977, DOI: 10.1016/j.jspr.2022.101977.
Adler, C., G. Böttcher, K. Große, C. Hentschel, 2022b: Insektenlaser, StarWars gegen Schädlinge. Der praktische Schädlingsbekämpfer, 3, 22-24.
Adler, C., C. Müller-Blenkle, B. Fürstenau, 2021: Vorratsschutz bei Getreide in Zeiten des Klimawandels. Mühle + Mischfutter 158 (9), 254-256.
Aide, T.M., C. Corrada-Bravo, M. Campos-Cerqueira, C. Milan, G. Vega, R. Alvarez, 2013: Real-time bioacoustics monitoring and automated species identification. PeerJ 1, e103, DOI: 10.7717/peerj.103.
Ajisegiri, E., A.A. Adediran, T. A Adekanye, A. Salami, J. Audu, 2022: Development of a smart grain storage silo using the internet of things (IoT) technology.
Ali, A., A.S. Mansol, A.A. Khan, K. Muthoosamy, Y. Siddiqui, 2023: Electronic nose as a tool for early detection of diseases and quality monitoring in fresh postharvest produce: A comprehensive review. Comprehensive Reviews in Food Science and Food Safety 22 (3), 2408-2432, DOI: 10.1111/1541-4337.13151.
Arnett, R.H., Jr., 1968: Measurement of adventitious sounds produced by some stored product insects. Annals of the Entomological Society of America 61 (5), 1170-1172.
Bailey, S.W., R.W. Lemon, 1968: Sound production by the arder beetle, Dermestes lardarius Linnaeus (Coleoptera Dermestidae). Journal of Stored Products Research 4 (3), 271-273.
Brain, C.K., 1924: Preliminary Note on the Adaptation of Certain Radio Principles to Insect Investigation Work. Annals of the University of Stellenbosch, Ser. A 2, 45-47.
Branding, J., D. von Hörsten, J.K. Wegener, 2022: Akustische Insektenerkennung–Deep Learning zur Klassifikation leisester Fluggeräusche.
Busnel, R.G., A.J. Andrieu, 1966: Appareil pour la detection d'insectes ou autres parasites dans des produits ou matériaux divers. INRA, France.
De Baerdemaeker, J., S. Hemming, G. Polder, A. Chauhan, A. Petropoulou, F. Rovira-Más, D. Moshou, G. Wyseure, T. Norton, B. Nicolai, F. Hennig-Possenti, I. Hostens, 2023: Artificial intelligence in the agri-food sector: Applications, risks and impacts. Technology, S.P.f.t.F.o.S.a., Brussels, European Union, 78 pp., DOI: 10.2861/516636.
Fairbrass, A.J., M. Firman, C. Williams, G.J. Brostow, H. Titheridge, K.E. Jones, 2019: CityNet—Deep learning tools for urban ecoacoustic assessment. Methods in Ecology and Evolution 10 (2), 186-197, DOI: 10.1111/2041-210X.13114.
FAO, 2011: Global food losses and food waste – Extent, causes and prevention., Rome.
Fleurat-Lessard, F., 1988: Determination of insect infestation. In: Multon, J.L., A.M. Reimbert, D. MarshA.J. Eydt (Eds.), Preservation and storage of grains, seeds and their products: cereals, oilseeds, pulses and animal feed. New York, Lavoisier Publishing, 495-515.
Fleurat-Lessard, F., A.J. Andrieu, 1986: Development of a rapid method to determine insect infestation in grain bins with electroacoustic devices. Proceedings of the 4th International Working Conference on Stored-Product Protection, Tel Aviv, Israel, Maor-Wallach Press.
Fleurat-Lessard, F., B. Tomasini, L. Kostine, B. Fuzeau, 2006: Acoustic detection and automatic identfication of insect stages activity in grain bulks by noise spectra processing through classification algorithms. 9th International Working Conference on Stored Product Protection, Campinas, São Paulo, Brazil.
Folliot, A., S. Haupert, M. Ducrettet, F. Sèbe, J. Sueur, 2022: Using acoustics and artificial intelligence to monitor pollination by insects and tree use by woodpeckers. Science of The Total Environment 838, 155883, DOI: 10.1016/j.scitotenv.2022.155883.
Hagstrum, D.W., P.W. Flinn, 1993: Comparison of Acoustical Detection of Several Species of Stored-Grain Beetles (Coleoptera: Curculionidae, Tenebrionidae, Bostrichidae, Cucujidae) Over a Range of Temperatures. Journal of economic entomology 86 (4), 1271-1278, DOI: 10.1093/jee/86.4.1271.
Hagstrum, D.W., P.W. Flinn, D. Shuman, 1994: Acoustical monitoring of stored-grain insects: An automated system. Proceedings of the 6th International Working Conference on Stored-Product Protection, Canberra, Australia.
Hagstrum, D.W., P.W. Flinn, D. Shuman, 1996: Automated monitoring using acoustical sensors for insects in farm-stored wheat. Journal of economic entomology 89, 211-217.
Hagstrum, D.W., K.W. Vick, J.C. Webb, 1990: Acoustical Monitoring of Rhyzopertha dominica (Coleoptera: Bostrichidae) Populations in Stored Wheat. Journal of economic entomology 83 (2), 625-628, DOI: 10.1093/jee/83.2.625.
Hickling, R., W. Wei, D.W. Hagstrum, 1997: Studies of Sound Transmission in Various Types of Stored Grain for Acoustic Detection of Insects. Applied Acoustics 50 (4), 263-278.
Kahl, S., C.M. Wood, M. Eibl, H. Klinck, 2021: BirdNET: A deep learning solution for avian diversity monitoring. Ecological Informatics 61, 101236, DOI: 10.1016/j.ecoinf.2021.101236.
Mankin, R., D. Hagstrum, M. Guo, P. Eliopoulos, A. Njoroge, 2021: Automated Applications of Acoustics for Stored Product Insect Detection, Monitoring, and Management. Insects 12 (3), 259, DOI: 10.3390/insects12030259.
Mankin, R.W., D. Shuman, J.A. Coffelt, 1996: Noise Shielding of Acoustic Devices for Insect Detection. Journal of economic entomology 89 (5), 1301-1308, DOI: 10.1093/jee/89.5.1301.
Mendoza, Q., L. Pordesimo, M. Neilsen, P. Armstrong, J. Campbell, P. Mendoza, 2023: Application of Machine Learning for Insect Monitoring in Grain Facilities. AI 2023, 4, 348–360.
Müller-Blenkle, C., S. Kirchner, I. Szallies, C. Adler, 2018: A new approach to acoustic insect detection in grain storage. Proceedings of the 12th International Working Conference on Stored Product Protection, October 7-11, Berlin, Germany, DOI: 10.5073/jka.2018.463.077.
Peter, G., H. Kuhnert, M. Haß, M. Banse, S. Roser, B. Trierweiler, C. Adler, 2013: Einschätzung der pflanzlichen Lebensmittelverluste im Bereich der landwirtschaftlichen Urproduktion. Bericht im Auftrag des Bundesministeriums für Ernährung, Landwirtschaft und Verbraucherschutz (BMELV). 44 pp.
Potamitis, I., T. Ganchev, D.C. Kontodimas, P.A. Eliopoulos, 2010: On automatic bioacoustic detection of stored product pests: the case of Sitophilus oryzae. Proceedings of the 10th International Working Conference on Stored Product Protection Estoril, Portugal, Julius Kühn-Institut Berlin, Germany.
Reichmuth, C., H. Welp, H. Bentert, B. C., A. Wudtke, 1996: A simple and cheap acoustic device to detect immature stages of the internal feeders Sitophilus granarius and Sitotroga cerealella in grain. Proceedings of the XX. International Congress of Entomology, August 25-31 1996 Firence, Italy.
Reid, J.A., 1942: The species of Laemopholeus (Coleoptera: Cucujidae) occurring in stored foods in the British Isles. Proceedings of the Royal Entomological Society of London. Series A 17, 27-33.
Ross, S.R.P.-J., D.P. O'Connell, J.L. Deichmann, C. Desjonquères, A. Gasc, J.N. Phillips, S.S. Sethi, C.M. Wood, Z. Burivalova, 2023: Passive acoustic monitoring provides a fresh perspective on fundamental ecological questions. Functional Ecology 37 (4), 959-975, DOI: 10.1111/1365-2435.14275.
Schott, J., 2021: Digitale Insektenfall "traptice". Neues Konzept zur digitalen Überwachung von Schadinsekten und -nagern. Mühle + Mischfutter 158 (19), 580.
Shuman, D., J.A. Coffelt, K.W. Vick, R.W. Mankin, 1993: Quantitative Acoustical Detection of Larvae Feeding Inside Kernels of Grain. Journal of economic entomology 86 (3), 933–938, DOI: 10.1093/jee/86.3.933.
Smith, L.I., 2002: A tutorial on principal components analysis. 26p.
Sueur, J., A. Gasc, P. Grandcolas, S. Pavoine, 2012: Global estimation of animal diversity using automatic acoustic sensors. Sensors for ecology. Paris: CNRS, 99-117.
Teixeira, A.C., J. Ribeiro, R. Morais, J.J. Sousa, A. Cunha, 2023: A Systematic Review on Automatic Insect Detection Using Deep Learning. Agriculture 13 (3), 713, DOI: 10.3390/agriculture13030713.
Vick, K.W., J.C. Webb, D.W. Hagstrum, B.A. Weaver, C.A. Litzkow, 1988: A Sound-Insulated Room Suitable for Use with an Acoustic Insect Detection System and Design Parameters for a Grain Sample Holding Container. Florida Entomologist 71 (4), 478-484.
Zain, M.E., 2011: Impact of mycotoxins on humans and animals. Journal of Saudi Chemical Society 15 (2), 129-144.
Suchen