IoT based technological support for moisture stress experimental study on plants with stress prediction using deep learning

Abstract

BACKGROUND:

The increased depletion of ground water resources poses the risk of higher moisture stress environment for agriculture crops. The rapid increase in the moisture stress situation imposes the need of efficient agricultural research on determining the impact of moisture stress on variety of crops.

OBJECTIVE:

The prime objective of the proposed work is building an IoT based Plant Phenotyping Device for moisture stress experimental study on variety of crops with deep learning model for stress response detection.

METHODS:

In this work, IoT technology is used for building a proposed system for conducting the moisture stress experiments on plants and adopting the image processing and convolution neural network based model for stress prediction.

RESULTS:

The accuracy of the proposed system was experimentally evaluated and empirical results were satisfactory in maintaining the desired level of moisture stress. Performance analysis of LeNet, AlexNet, customized AlexNet and GoogLeNet CNN models were carried out with hyper-parameters variations on the leaf images. GoogLeNet achieved a better validation accuracy of 96% among other models. The trained GoogLeNet model is used for predicting the moisture stress response and predicted results were matched with manual observation of stress response.

SIGNIFICANCE:

The affirmative results of proposed system would increases its adoption for in-house precision agriculture and also for conducting various moisture stress experiments on variety of crops. The confirmative detection of moisture stress tolerance level of plant provides knowledge on minimum level of water requirement for plant growth, which in-turn save the water by avoiding excess watering to plants.

Keywords

IoT sensors Raspberry Pi moisture stress deep learning

1 Introduction

Moisture stress has been one of the most common abiotic stresses affecting variety of crops across the world [1, 2]. The rapid increase of dry land and shortage of water poses new challenges for handling the drought stress in a better way to minimize crop loss [3]. Agricultural researchers around the world are conducting drought stress related experiments on variety of crops to get a better insight into drought stress to mitigate the stress induced diseases. The early detection of moisture stress responses on plants using farmer affordable technology has been a need of the hour.

The depletion of water sources push the need of growing crops by providing the adequate level of water and avoiding the over-supply of water to plants. Moisture stress tolerance level characteristic of a plant need to be carefully analyzed to identify the minimum water requirements of a plant and accordingly supply adequate water. The significance of the need of analyzing the moisture stress tolerance of plant is illustrated by considering the following scenario of assuming Plant A is having a higher moisture stress tolerance level compared to another plant B. Due to the higher moisture stress tolerance level of Plant A, a lesser water would be enough for Plant A compared to Plant B. Even-though variation in water supplies to plants, if the both the plants provides similar level of yield, then the problem statement arises on effective identification of moisture stress tolerance level plant and providing minimum required water and consequently save the water. The cost effective & precised water supply system to grow plant by providing adequate water has been a challenging task.

The moisture stress tolerance plant gene identification has been a requirement of the farmers of drought affected areas. The usage of non-stress tolerance plant gene would cause huge loss to farmers of drought affected areas. The lack of low cost device for précised mode of moisture stress experimental study poses the further risk of wrong determination of plant genes for stress tolerance.

IoT (Internet of Things), is a next level of WSN (Wireless Sensor Network) technology, which is comprised of an internet connected micro-controller attached with sensors, which in turn pushes the sensed data to the central data repository. In recent days, the ease of use and popularity of IoT and WSN technologies have increased its scope in precision agriculture [4–7]. Efficient technology based plant phenotype devices has been a need of agricultural scientists across the world to carryout various abiotic stress experiments on plants to assess the stress responses [8, 9].

Even though there are considerable technological advancements, the lack of problem specific automation for a précised way for conducting drought stress experiments leads to erroneous results. The advancement of the Internet of Things and Wireless Sensor Network technologies have opened up a space for innovative solutions for conducting scientific experiments with high precision [10]. The scientific study of moisture stress on plants needs a precised way of maintaining the moisture stress at a desired level, which is a laborious work in manual mode [11]. In contrast to this, latest technologies like IoT/WSN technology can be employed for maintaining the desired level of moisture stress and also automated way of collecting the stress responses in real time [12].

DCNN (Deep learning based Convolution Neural Network) has been a promising technology for remote monitoring of plant growth and a non-destructive way of assessing plant disease or stress responses [13–16]. In order to carry out the moisture stress experiment, different plants need to be kept at different levels of moisture stress such as highly stressed, moderate stressed and unstressed.

Hence, the precised way of conducing scientific experiments on moisture stress on plants needs a versatile device for automated pumping of water to maintain the desired level of moisture stress and real - time measurement of moisture stress, plant growth parameters and stress responses. In addition to maintaining the desired level of moisture stress, the sensor data needs to be analyzed for extracting deeper insights on stress behavior and building a possible data analytics based prediction model for stress response [17].

Even though IoT systems are in discussion for its adoption to agriculture purposes, one of the major issues in realization IoT for field deployment depends on providing non-interrupted regulated power supply. The fluctuating power supply would cause an erroneous sensor reading and poses the risk of reducing the accuracy of the system [18, 19]. Another major requirement is the need for an efficient IoT platform for plotting the real time graph of continuous data and central data repositories for data analytics.

Convolution neural network based deep learning models emerged as one of the non-destructive way of determining leaf diseases [20, 21]. Plant stress prediction has been a challenging issues and many researches were carried out using Deep learning approaches for predicting the plant stress in non destructive way [22]. Many comparative analysis based work is carried out on applying various Deep learning based CNN models for predicting the biotic stress on paddy crop [23]. The recent research study of preparing the dataset to apply deep learning models to predict the water stress on maize crops highlighted the use of DCNN for abiotic stress response prediction & provided a scope for possible improvisation [24].

This work discusses the design and development of an IoT based plant phenotyping device for conducting moisture stress experiments in a precised way with predictive analytics support. The proposed system adopts the Deep learning based Convolution Neural Network model for stress response assessment along with IoT platform for remote data access support.

The structure of the paper is as follows, the Section 2 deals with providing literature survey of existing work related to the proposed system. The Section 3 provides the overall architectural diagram & details of the proposed system. The Section 4 provides the detailed description about experimental setup carried out to conduct the experiment in précised way. The Section 5 discusses the results & discussion of the experimental work carried out in this work.

2 Related work

The following Table 1 depicts the related work carried out on building IoT based plant Phenotyping system with description about the various approaches used and experimental study carried out.

Table 1
Related work on IoT based plant phenotyping system

Study Approach/method used Key factors

[25] Open source platform is built by making use of distributed sensor and camera vision technology to collect the imagery and sensory data about the plant growth. Cost of setup is high

[26] Do it yourself approach is applied and built a plant handling system for plant phenotyping Lack of conduction of real time experimental study on plant phenotyping

[27] Combined approach is used to reduce the complexity of CNN Models used in Plant Pheotyping device for quantitative analysis. Real field experimental study of adopting the combined approach is not done.

[10] IoT based growth chamber is built to grow plant under different drought stress conditions. •Linear Regression Analysis is used assess the stress response

•Lack of imaging/computer vision based technology for assessing the stress response

[28] Low cost Raspberry pi based phenotyping robot is built and used the depth camera for detecting the height of plant. •Height of plant is measured using depth camera.

•Disease identification as stress response is not done.

[29] Arduino and weight sensor based setup is built and carried out experimental study to determine the water balance issue on plant growth. •RealSense camera is used and applied optical flow method to measure the plant behaviour based on water stress on plants.

•Automated deep learning module for detecting the stress response is not used.

[30] UAV Drone based image acquisition system is proposed and CNN models were built to detect the crop types and growth status High cost due to UAV drone

[31] Raspberry pi based IoT setup is built to grow small size plants by collecting specific environmental parameters. Stress related experiments were not conducted

[32] The greenhouse setup based experimental study is carried out to determine the water status on wheat crop using deep learning based study on images with various formats Water irrigation is done in manual mode to provide the water stress to plants.

Study	Approach/method used	Key factors
[25]	Open source platform is built by making use of distributed sensor and camera vision technology to collect the imagery and sensory data about the plant growth.	Cost of setup is high
[26]	Do it yourself approach is applied and built a plant handling system for plant phenotyping	Lack of conduction of real time experimental study on plant phenotyping
[27]	Combined approach is used to reduce the complexity of CNN Models used in Plant Pheotyping device for quantitative analysis.	Real field experimental study of adopting the combined approach is not done.
[10]	IoT based growth chamber is built to grow plant under different drought stress conditions.	•Linear Regression Analysis is used assess the stress response
		•Lack of imaging/computer vision based technology for assessing the stress response
[28]	Low cost Raspberry pi based phenotyping robot is built and used the depth camera for detecting the height of plant.	•Height of plant is measured using depth camera.
		•Disease identification as stress response is not done.
[29]	Arduino and weight sensor based setup is built and carried out experimental study to determine the water balance issue on plant growth.	•RealSense camera is used and applied optical flow method to measure the plant behaviour based on water stress on plants.
		•Automated deep learning module for detecting the stress response is not used.
[30]	UAV Drone based image acquisition system is proposed and CNN models were built to detect the crop types and growth status	High cost due to UAV drone
[31]	Raspberry pi based IoT setup is built to grow small size plants by collecting specific environmental parameters.	Stress related experiments were not conducted
[32]	The greenhouse setup based experimental study is carried out to determine the water status on wheat crop using deep learning based study on images with various formats	Water irrigation is done in manual mode to provide the water stress to plants.

In-line to the above related works the primary survey clarifies that most of the work carried out either on building a IoT enabled plant phenotyping system excluding Deep Learning CNN based stress detection or vice versa. Also limited emphasis is given on providing the cost efficient end product for experimental study on moisture stress. Hence the design and development of low cost IoT based Deep Learning CNN enabled plant phenotyping setup for drought stress experimental study has been considered as the primary research focus in this work.

3 Proposed system

Figure 1 shows the architectural block diagram of the proposed system of WSN based IoT device for plant phenotyping for moisture stress experiment with IoT Platform and DCNN model for stress response assessment.

Fig. 1

Architectural block diagram of proposed system.

The proposed system uses three methods for measuring the moisture content of the soil, gravimetric approach using weight sensor, commercially available capacitance soil moisture sensor, Low cost in-house built multi-probe resistive based soil moisture sensor. The results obtained in three methods are compared. The low cost multi-probe sensor has outperformed commercially available capacitance sensor. The result obtained by multi probe sensor is in line with the gravimetric approach. For real time application gravimetric approach cant applied to a field, where as low cost multi-probe can be used. In the proposed system, the Raspberry pi microcontroller is programmed for reading the load-cell value for measuring the weight of pot using python language. Due to the non-availability of an analog pin in raspberry pi, the arduino microcontroller is used to connect to the soil moisture sensor as shown in Fig. 1. The arduino microcontroller is programmed using embedded c programming to measure the soil moisture readings. The readings of the arduino were pushed to raspberry pi using serial port due to the non-availability of WiFi module in arduino. Each raspberry pi is programmed to receive the soil moisture sensor reading via serial input from arduino and load cell value via hx711 ADC. Also, each raspberry pi is programmed to push the received data to the ThingsBoard platform for obtaining real time data visualization and a copy to a MYSQL database server for further data analysis. Before pushing data to server, each raspberry pi programmatically verify the current level of moisture content based on the weight value and actuates the water pump accordingly to maintain the desired level of moisture stress.

The Raspberry pi NOIR V2 RGB camera module is used to capture real-time images of a plant and its leaves. The python application is deployed in Raspberry Pi attached with a camera for the remote access to capture real-time images. The captured images were given as input to the DCNN based trained model running on NVIDIA DIGITS Web Application Framework for stress response assessment.

4 Experimental setup

The proper functioning of the proposed system is tested by conducting the moisture stress experiment on plant and capturing the data accordingly for verification. The hot air oven approach is used to dry the soil and gravimetric approach is used to measure the soil water holding capacity of soil. The procedures used in this experimental study are as follows

1.
The Hot air oven is used to dry the soil at 95 degree temperature for a duration of 48 Hours
2.
The weight of the dry soil is Obtained, represented as D__Weight (Dry_SoilWeight)
3.
The empty pot weight, represented as EPWeight (Empty_PotWeight)
4.
Measuring the Pot Plate weight, represented as PlWeight(PlateWeight)
5.
The dry soil is poured to empty Pot and measuring the weight of the dry soil withpot, represented as DSPWeight $DSP{Weight} = D{weight} + EP{Weight}$ (1)
6.
Water is added to the pot until the soil becomes complete wet, correspondingly the amount of water added is noted down represented as Wateradded.
7.
The excess water from the pot is drained out depending upon its water holding capacity, the amount of water obtained in drain, represented as Waterdrain
8.
Now Water holding capacity, $WHC = Water{added} - Water{drain}$ (2)
9.
Once the soil reaches the maximum water holding capacity, measuring the wet soil weight at 100% moisture level is represented as, WSPWeight100 % $\begin{matrix} WSP{Weight}{100 %} = D{Weight} + EP{Weight} \\ + WHC considering 1 ml = \sim 1 gm \end{matrix}$ (3)
10.
Pot weight at 50% of Soil Moisture level is represented as $\begin{matrix} WHC{Weight}{50 %} = DSP{Weight} \\ + (WHC * 50 / 100) \end{matrix}$ (4)
11.
Pot weight at 70% of Soil Moisture level is represented as $\begin{matrix} WHC{Weight}{70 %} = DSP{Weight} \\ + (WHC * 70 / 100) \end{matrix}$ (5)

Upon computing the WHC{Weight _ 50 %} & WHC_Weight__{Weight _ 70 %}, the pots are kept on the proposed device and sensors are attached accordingly. The respective raspberry Pi microcontroller of the proposed device programmatically verifies the weight to the desired level of weight for maintaining the desired level of soil moisture content level. If the current weight is less than the WHC_{_ weight _ 50 %} in the first device, then the automatic pumping of water would be done to bring back the weight to maintain the 50% of soil moisture level. Similarly, if the current weight is less than the WHC_{_ weight _ 70 %} in the second device, then the automatic pumping of water would be carried out to bring back the weight to maintain the 70% of soil moisture level.
4.1 Implementation

The proposed device is comprised of two Raspberry Pi and two arduino micro-controllers, which are programmed to measure the real time weight, soil moisture level and actuate the pump for watering to maintain the desired level of moisture stress. The Embedded C program is written for arduino programming to read the upper & lower probe voltage analog reading and to push the same to raspberry pi via serial connection. The Python program is written for raspberry pi to measure the Load Cell weight value, and to measure the reading via arduino serial connection for upper & lower probe voltage of soil moisture. The measured weight value, Upper & lower probe voltage readings are pushed to the Thingsboard IoT Platform and the copy is stored in Mysql Database running on local server. Raspberry Pi device is also programmed to Turn-On grow lights at 9.00 AM and Turn-Off at 10.00 PM every day and also a python based web application is developed for turning On/Off the pump in manual mode. The plant images were captured using a camera connected to Raspberry pi and preprocessed using a graph cut segmentation process for extracting the individual leaf images. The segmented leaf images were fed into a CNN based trained model to detect the Moisture Stress response on plants.

Two experimental studies were carried out; the first one with the leafy vegetables and the second experiment is carried out on a tomato plant to evaluate the accuracy of the proposed device to maintain the desired level of stress and predict the stress response.

4.2 Dataset

The GoogLeNet CNN (Convolution Neural Network) based Deep Learning model is trained with leafy vegetables and tomato leaf images to predict stress response and disease detection. In this study, the appearance of disease symptoms on plant leaves is considered as an early indication of stress response.

The author [15] generated a image dataset of stressed and healthy soybean leaflets comprised of 65,760 images in their work of applying DCNN model for predicting the plant stress. In this work, author Sambudda Ghosal et al. generated dataset is downloaded from https://github.com/SCSLabISU/xPLNet and used for first experimental study of predicting the stress response on leafy vegetable crop. The dataset consists of 9 categories as shown in Fig. 2(a).

Fig. 2

(a): Leafy vegetables Leaf Images with different catego-ries of diseases (b): Tomato plant leaf images with different categories of diseases leaf images.

The leafy vegetables and tomato plants were grown in the proposed device under 50% and 70% stress and raspberry pi connected camera is used to capture the real - time images. The captured images were fed into trained CNN Model for detecting the moisture stress response.

The plantvillage tomato leaf dataset is used for building the DCNN model for predicting the stress response as occurrence of diseases. The dataset comprised of 10000 images, categorized into 9 categories as shown in Fig. 2(b). The dataset is downloaded from below kaggle website link https://www.kaggle.com/datasets/charuchaudhry/pla ntvillage-tomato-leaf-dataset. The captured plant images were processed using a Graph Cut segmentation algorithm using Matlab tool to segment the individual leaf image out of the entire plant image. The extracted leaf image is fed into the trained CNN Model for detecting the Moisture Stress Response on plants.

5 Results and discussion

Figure 3(a) shows the in-house built proposed system in accordance with the architectural diagram given in Fig. 1. The proposed system comprises of 2 IoT devices for plant phenotyping for maintaining the moisture stress level of 50% and 70% respectively. Figure 3(b) shows the experimental set-up under grow light conditions.

Fig. 3

Proposed system setup.

The proposed system uses an IoT enabled in-house fabricated weighing scale for adopting a gravimetric approach for precisely maintaining the desired level of moisture stress as shown in Fig. 4(a). The in-house fabricated weighing scale is comprised of 20KG Load-cell with ADC HX711 attached to raspberry pi. Figure 4(c) shows the low-cost in-house built resistive based multi-probe soil moisture sensor for detecting the soil moisture at multiple depths of a soil [33].

The multi-probe soil moisture sensor is used for experimental evaluation is cost effective and can be deployable in field for conducting moisture stress experiments. Accordingly, Fig. 4(b) shows the use of a multi-probe sensor along with a capacitance sensor and gravimetric approach for comparative analysis. In accordance to the set-up shown in Fig. 4(b), the experimental study was carried out to detect the non-precise reading of moisture level in a commercially available capacitance soil moisture sensor.

Fig. 4

(a) - In-house built Weighing scale of 20KG load-cell with hx711 ADC controlled by Raspberry Pi, (b)Installation of multi-probe sensor & commercial capacitance soil moisture sensor along with gravimetric approach for comparative study, (c) low cost in-house built resistive based soil moisture sensor for detecting moisture at different depth of soil.

Inline to this experimental study, the water is poured into pot from the top and an increase in weight value and soil moisture value were noticed. Correspondingly, as shown in Fig. 5(a), the weight of soil with pot reaches 1722 grams upon reaching the 100% water holding capacity, which is a 100% moisture state based on gravimetric approach. In contrast to the above, upon reaching the weight of around 1425 grams, capacitance soil moisture sensor reading shows the 100% moisture due to water accumulation around the sensor probe even though water is not there in the lower bottom surface of the soil.

Figure 5(b) shows the results of in-house built resistive based soil moisture sensor for detecting moisture at different depths of a soil. The upper and lower probe voltage values are the indicators of the presence of moisture in the soil in the probe level. As shown in Fig. 5(b), the sensor is able to detect the moisture level only at the upper surface of thesoil upon pouring 100 ml of water. Results also verify the moisture level at the bottom surface of the soil upon pouring another 200 ml of water and confirming the status of water reaching the bottom surface of the soil.

Fig. 5

(a) Non-precise readings of Capacitance Soil Moisture Sensor determination of wrong measurement of 100% moisture level; (b) Upper & lower probe voltage of multi-probe sensor readings with respect to pot weight readings.

Figure 6(a) shows the real time weight (in Grams) of soil with Pot 1 placed on the proposed IoT Device, which shows real-time moisture stress level with respect to gravimetric approach. In accordance to the Equation (4), the computed weight to maintain the 50% moisture stress level for soil with Pot 1 was around 1170 Grams. The result in Fig. 6(a) shows the increase in weight due to automated pumping of water upon decrease in weight value below the threshold value of 1170 grams. The experimental results confirm the accuracy of the device in maintaining the desired weight level of around 1170 grams for 50% moisture stress with obtained mean weight of 1177 grams. Correspondingly, Fig. 6(b) shows the status of the Pump goes automatically to the On Status in accordance to the drop of 50% Moisture stress and Pump goes automatically to the Off status upon reaching the threshold weight of 1170 grams. Similarly Fig. 7(a) shows real-time weight (Grams) of soil with Pot 2 kept on the proposed IoT Device and Fig. 7(b) shows the pump On/Off status to maintain the weight of 1028 Grams for a desired moisture stress level of 70%. Thingsboard open-source IoT platform is used for getting the real time graph of the moisture level, pump status. The mysql database is used for data storage for futurist data analytics. This experimental setup is used to measure the moisture consumption by plants for its growth. The results in Fig. 6(b) and Fig.7(b) provides the data about the total amount of water consumed by the respective plant, which could be further analyzed to determine the optimal water requirements for the plants. The determination of optimal water requirement would provide a scope of intelligence for avoiding the supply of excess amount of water, which in turn saves the water without compromising the crop yield.

Fig. 6

(a). Device 1 Weight in grams indicating the Moisture content level with respect to gravimetric approach and mean weight of ∼1177 confirms the maintenance of 50% Water holding capacity weight of 1170 Grams; (b) Device 1 Pump ON/OFF Status to maintain 70% stressed plant.

Fig. 7

(a). Device 2 Weight in Grams indicating the Moisture content level with respect to gravimetric approach and mean weight of ∼1039 confirms the maintenance of 50% Water holding capacity weight of 1028Grams; (b) Device 2 Pump ON/OFF Status to maintain 70% stressed plant.

5.1 Moisture stress experiment on leafy vegetable plant

Figure 8 shows the real time images of 50% and 70% stressed leafy vegetables plants respectively captured via remote access enabled camera, which is attached to the proposed system

Fig. 8

Images of 50% and 70% Stressed plants captured via remote access enabled Raspberry Pi attached NoIR Camera.

The comparative analysis of various state of the art CNN models are evaluated using the plant leaf stress dataset comprised of 44391 training and 14793 validation images, which is comprises of eight different categories of stressed leaf images and one category of healthy leaf images as shown in Fig. 2(a). The experimental evaluation of validation accuracy for different CNN Models with respect to hyper-parameter adjustment is shown in Table 2. The hyperparameters such as learning rate & optimizer is tuned to obtain the better accuracy. The base learning rate is set to 0.01 & 0.001 with a step down decay of 33% of total number of epochs and gamma value of 0.1 as multiplication factor for step down decay. All the models are trained for 30 epochs. The small jump in the loss value after 10 epochs in LeNet and AlexNet boosted for further tuning of Alexnet with different combination of tuning as shown in Table 2. The GoogleNet with a constant learning rate of 0.01 achieved an accuracy of 93%, which boosted for trying the step down decay of 33% with base learning rate as 0.01& resulting in increasing the accuracy of 96% as shown in Fig. 9.

Table 2

CNN Model validation accuracy with varying Learning rate and optimizer using xPLNet leaf stress dataset

Convolution Neural Network Model	Framework	Base Learning Rate	Validation Accuracy	Optimizer
LeNet	Tensorflow	0.01	87%	SGD
AlexNet	Tensorflow	0.001	91%	SGD
AlexNet with HSV Augmentation: Hue: 0.03, Saturation: 0.05, Value: 0.06	Tensorflow	0.001	72%	SGD
AlexNet with Adaptive gradient Optimizer	Tensorflow	0.001	90%	AdaGrad
AlexNet with additional convolution and pooling layer	Tensorflow	0.001	90	SGD
AlexNet with whiting	Tensorflow	0.001	81%	SGD
AlexNet with Additional FC Layer	Tensorflow	0.001	91.25	SGD
GoogLeNet	Caffe	0.01	96%	SGD

The accuracy is computed by using the following equation $Accuracy = \frac{Number of Correct Prediction}{Total Number of Prediction}$ (6)

Fig. 9

GoogLeNet CNN trained model accuracy tested on xPLNet Leaf dataset.

The extracted leaf image is given to the DCNN trained Model in the NVIDIA DIGITS framework for classification of leaf stress.

Table 3 shows the efficacy of CNN models used in this with hyper-parameters tuned over the other customized CNN model built and trained with same xPLNet soyabean leaf images dataset for stress detections. Even though CNN models are trained with soyabean leaf images for stress detection, the stress features out of leaf texture extracted in CNN model was affirmative for the trained model to determine the stress response on leafy crop leaflets used in this work.

Table 3

Comparative performance analysis of proposed system used CNN models for leafy crop stress detection with other existing work CNN model used on same dataset

Study	Description	Dataset Used	Accuracy
[15]	Deep learning based customized CNN Model is built and evaluated for stress identification in plants	xPLNet –Soyabean leaf images and comprised of 65,760 images	94%
In this current Study	Hyperparameters tuned CNN Models are applied	xPLNet –Soyabean leaf images and comprised of 65,760 images	96%

The real time captured images of plant under stress is preprocessed using graph cut segmentation to extract the leaf portion of the plant image and the result of the segmentation as shown in Fig. 10.

Fig. 10

Image segmentation for extracting the leaf portion of an image.

As shown in Fig. 11 DCNN classifies the given leaf image into non healthy leaf category and confirms the stress response on plant. Also the real time plant images are used to note down the plant growth rate parameters as number of leaf.

Fig. 11

Classification of 50% stressed plant leaf image using DCNN.

Figure 12(a) shows the complete dry leaf curl after 8 days of stress accumulation on 50% stressed plant. Similarly Fig. 12(b) shows the mid stage of leaf curl stress response after 8 days of stress accumulation on 70% stressed plant. Figure 12(c) & Fig. 12(d) shows the progress in stress response in week 3 and confirms the reliability of proposed device in persevering the desired stress level of 50% and 70% moisture stress respectively.

Fig. 12

Leafy Vegetable plant under 50% and 70% moisture stress showing the stress responses in week 2 and week3 of experimental study.

5.2 Moisture stress experiment on Tomato plants

The second experiment was conducted on tomato plant, where in 2 plants are taken. Plant A is grown under 50% stress and Plant B is grown under 70% Stress. The IoT based plant phenotyping device was able to maintain the stress on plants. The stress response were noticed and its shown in Figs. 13 and 14.

Fig. 13

Tomato plants under 50% stress experimental study and observed stress responses.

Fig. 14

Tomato plants under 70% stress experimental study and observed stress responses.

Figure 15(a) and (b) shows the output of the Graph cut based segmentation of the 50% moisture stressed plant A and 70% moisture stressed plant B respectively. The segmentation output of binary & masked image clearly shows the number of leaf in plant A is fewer than the Plant B.

Fig. 15

(a). Graph Cut applied segmented image of plant A during week 3; (b) - Graph Cut applied segmented image of plant B during week 3.

Figure 16 shows the number of leaf count during the experimental study and observed higher reduction of leaf count in 50% stressed plant compared 70% stressed plant further concludes the accuracy of the proposed system on maintaining the stress level.

Fig. 16

Number of leaves count in plant A and Plant B during experimental study.

Figure 17 shows the thermal images captured of Plant A and Plant B along with RGB images in the 3rd week after maintaining the stress level of 50% & 70% respectively.

Fig. 17

Plant RGB Images and corresponding Thermal Images.

Table 4 shows the thermal images with higher temperature value of around 29 Degree in the plant A compared to plant B temperature of 27 Degree. This shows that, the system is able to maintain the desired stress level

Table 4

Observed Elevated Temperature in plant leaf due to automated maintenance of moisture stress using proposed system

Plant	Stress Level	Observed Temperature
Plant A	50%	29 Degree
Plant B	70%	27 Degree

GoogLeNet DCNN model is trained with tomato leaf disease dataset comprised of 10000 trained images and 1000 validation images categorized into 10 categories as shown in Fig. 2(b). The trained GoogLeNet DCNN Model achieved a promising accuracy of 97% as shown in below Fig. 18 and trained model is used for predicting the stress response on tomato plants.

Fig. 18

GoogLeNet Deep learning trained model accuracy tested on Plant-Village Tomato disease dataset.

The comparison Table 5 shows the efficacy of the CNN model used in this work. Even though third approach accuracy is near to the approach used in this work, CNN approach used in this study is better suitable due to its less complexity and less number of parameter compared to third approach.

Table 5

Comparative performance analysis of proposed system used CNN models for tomato crop stress detection with other existing works CNN models used on same dataset

Study	Approach Used &Key factor	Dataset Used	Accuracy
[34]	Customized CNN Models is build comprised of 3 Convolution Layers (33 Kernel), Max Pooling (22) and 1 Fully connected layer and Relu Activation Function is used	PlantVillage Tomato Dataset- comprised of 10000 images	91.2%
[35]	MobileNet	7176 tomato leaf images belongs to 10 different classes are used to train the model	90.3
[36]	Modifed U-Net and ResNet is used - Too many parameter and very complex model	PlantVillage Tomato Dataset- comprised of 10000 images	97%
In this current Study	Hyperparameters tuned GooglNet Less number of parameters	PlantVillage Tomato Dataset- comprised of 10000 images	97.41%

Table 6

Comparative analysis of proposed IoT based plant phenotyping device with other existing works

Study	Key factors/limitation of existing work	Proposed System excellence over identified limitation of existing works.
[25]	Cost of setup is high	Low Cost setup is done using Raspberry Pi and low cost in-house built soil moisture sensor for field deployment
[26]	Lack of conduction of real time experimental study on plant phenotyping	Two real time moisture experimental study is carried out to confirm the accuracy of the system
[27]	Real field experimental study of adopting the combined approach is not done.	Combined 2 technologies such as IoT for precise irrigation and DCNN for stress prediction.
[10]	•Linear Regression Analysis is used to assess the stress response	Latest advanced computer vision and CNN based approaches were used for stress level prediction.
	•Lack of imaging/computer vision based technology for assessing the stress response.
[28]	•Height of plant is measured using depth camera.	Deep learning based CNN model is built and used for stress prediction as identification of diseases due to drought stress
	•Disease identification as stress response is not done.
[29]	RealSense camera is used and applied optical flow method to measure the plant behaviour based on water stress on plants.	Automated DCNN module is used for stress response detection
	•Automated deep learning module for detecting the stress response is not used.
[30]	High cost due to UAV drone	Low cost due to using Raspberry Pi attached NOIR cameras.
[31]	Stress related experiments were not conducted	In this study, 2 drought experimental study carried out on leafy and tomato crop.
[32]	Water irrigation is done in manual mode to provide the water stress to plants.	IoT based precise irrigation module is built and proposed device also ensures on maintaining the desired level moisture stress.

Figures 19 and 20 shows the stress response predicted upon feeding the stressed plant leaf images into trained DCNN models at different weeks. Figure 19(a) and (b) shows the moisture stress prediction by DCNN model by classifying the 50% stressed plant leaf images into 5^th category, which is a non healthy category of a leaf. Similarly the Fig. 20(a) & (b) shows 70% stressed plant B’s leaf classified into non healthy category. The result shows the DCNN model classification of leaf images into non healthy category proves the presence of moisture stress response. The experimental results conclude the accuracy of proposed system on maintaining the desired level of stress and validate the accuracy of trained model to detect the presence of stress response. The trained model would be useful in predicting the stress response and engaging the necessary actions well in advance to mitigate the progress of stress to entire plants.

Fig. 19

Plant A under 50% Stress prediction using GoogLeNet trained model. (a) Plant A stress classification in week 3 Tomoto new disease caffe Googlent image classification model. (b) Plant A under 50% stress predication using GoogLeNet trained model.

Fig. 20

Plant B under 70% Stress prediction using GoogLeNet trained model. (a) Plant B stress classification in week 3 Tomoto new disease caffe Googlent image classification model. (b) Plant B stress classification in week 4 Tomoto new disease caffe Googlent image classification model.

Table 6 shows the comparative analysis of the proposed IoT based plant phenotyping system with other existing related system. The proposed device outperforms existing work in terms of maintaining the moisture stress in précised level and provides low cost setup combining both IoT and Deep learning based CNN technology for ease of stress response detection.

6 Conclusion

Due to lack of low cost précised device for moisture stress experimental study, the study of moisture stress tolerance level of a plant and early detection of moisture stress has been a challenging task. The major challenge of maintaining the plant under desired level of drought stress has been a difficult task. Also the feasible solution for early detection of moisture stress response is the need of the hour due to rapid increase of drought land. The précised way of conducting the moisture stress experimental study is important for determining the correct plant genes with various level of moisture stress tolerance.

In this work, low cost IoT enabled plant phenotyping device is designed and developed to conduct the moisture stress experimental study and response detection using deep learning. The built device was able to maintain the desired moisture stress level of 50% to a Plant A and 70% for another Plant B respectively for the complete duration of experimental study. IoT capability of a built device provided features of remote access and monitoring of data. The various state-of-the-art Deep Learning models were trained with hyperparameter tuning to obtain the trained model for early detection of stress response. Among the trained models, the GoogLeNet CNN Model achieved a validation accuracy of 96%.

Further research on leaf dataset used for building the CNN models, the remote accessible camera may provide the future scope of work for generating leaf image dataset for variety of crops and building the higher efficient CNN models.

Author contribution

Mr.Sunil Kumar S: Literature survey, Design and Implementaion. Dr.Venkatramana Bhat: Problem formulation, System Design and experimental result analysis.

Acknowledgements

Authors like to thank VGST, Department. of IT, BT, and S&T, Government of Karnataka, Bangalore, for sanctioning research grant for the Department of Computer Science and Engineering, Mangalore Institute of Technology and Engineering, Moodabidri, under KFIST= L2" grant schema vide GRD No. 339 for setting up an advanced research lab to conduct this research work. Authors also would like to thank the management of Mangalore Institute of Technology and Engineering for providing the infrastructure facility to carry out this research.

References

Iqbal

M.S.

Singh

A.K.

Ansari

M.I.

, Effect of drought stress on crop production, in, New Frontiers in Stress Management for Durable Agriculture. Springer, Singapore (2020), pp.35–47. doi: 10.1007/978-981-15-1322-0_3.

Seleiman

M.F.

, et al. Plants Drought Stress Impacts on Plants and Different Approaches to Alleviate Its Adverse Effects (2021). doi:10.3390/plants.

Yadav

Jogawat

Rahman

M.S.

Narayan

O.P.

, Secondary metabolites in the drought stress tolerance of crop plants: A review,, Gene Reports 23 (2021). doi:10.1016/j.genrep.2021.101040.

Akhter

Sofi

S.A.

, Precision agriculture using IoT data analytics and machine learning,, Journal of King Saud University - Computer and Information Sciences (2021). doi:10.1016/j.jksuci.2021.05.013.

Akhund

T.M.N.U.

Newaz

N.T.

Zaman

Sultana

Barros

Whaiduzzaman

, IoT-Based Low-Cost Automated Irrigation System for Smart Farming, in, Lecture Notes in Networks and SystemsSpringer Science and Business Media Deutschland GmbH (2022), pp.83–91. doi: 10.1007/978-981-16-6309-3_9.

Anguraj

D.K.

Mandhala

V.N.

Bhattacharyya

hoon Kim,

, Hybrid neural network classification for irrigation control in WSN based precision agriculture,, Journal of Ambient Intelligence and Humanized Computing (2021). doi:10.1007/s12652-020-02704-6.

Huerta

M.K.

Garcia-Cedeno

Guillermo

J.C.

Clotet

, Wireless sensor networks applied to precision agriculture: A worldwide literature review with emphasis on Latin America, IEEE Geoscience and Remote Sensing Magazine 9(4) (2021), 209–222. doi:10.1109/MGRS.2020.3044235.

Al-Turjman

, The road towards plant phenotyping via WSNs: An overview, Computers and Electronics in Agriculture 161 (2019)4–13. doi:10.1016/j.compag.2018.09.018.

Fan

Zhang

Wen

Guo

, The future of Internet of Things in agriculture: Plant high-throughput phenotypic platform,, Journal of Cleaner Production 280 (2021). doi:10.1016/j.jclepro.2020.123651.

10.

Numajiri

, et al. iPOTs: Internet of Things-based pot system controlling optional treatment of soil water condition for plant phenotyping under drought stress, Plant Journal 107(5) (2021), 1569–1580. doi:10.1111/tpj.15400.

11.

Liyanage

D.K.

Chathuranga

Mori

B.A.

Thilakarathna

M.S.

, A simple, semi-automated, gravimetric method to simulate drought stress on plants, Agronomy 12(2) (2022). doi:10.3390/agronomy12020349.

12.

Kim

J.Y.

Abdel-Haleem

Luo

Szczepanek

Open-Source Electronics for Plant Phenotyping and Irrigation in Controlled 2 Environment 3. Available: https://ssrn.com/abstract=4068599.

13.

Chandel

N.S.

Chakraborty

S.K.

Rajwade

Y.A.

Dubey

Tiwari

M.K.

Jat

, Identifying crop water stress using deep learning models,, Neural Computing and Applications (2020). doi:10.1007/s00521-020-05325-4.

14.

Gao

Luo

Zhang

, Deep learning application in plant stress imaging: A review, AgriEngineering 2(3) (2020), 430–446. doi:10.3390/agriengineering2030029.

15.

Ghosal

Blystone

Singh

A.K.

Ganapathysubramanian

Singh

Sarkar

, An explainable deep machine vision framework for plant stress phenotyping, Proceedings of the National Academy of Sciences of the United States of America 115(18) (2018), 4613–4618. doi:10.1073/pnas.1716999115.

16.

Noon

S.K.

Amjad

Qureshi

M.A.

Mannan

, Use of deep learning techniques for identification of plant leaf stresses: A review,, Sustainable Computing: Informatics and Systems 28 (2020). doi:10.1016/j.suscom.2020.100443.

17.

Gupta

Nahar

Classification and yield prediction in smart agriculture system using IoT, Journal of Ambient Intelligence and Humanized Computing (2022), doi:10.1007/s12652-021-03685-w.

18.

Gaddam

Wilkin

Angelova

Gaddam

, Detecting sensor faults, anomalies and outliers in the internet of things: A survey on the challenges and solutions, Electronics (Switzerland) 9(3) (2020). doi:10.3390/electronics9030511.

19.

Okafor

N.U.

Alghorani

Delaney

D.T.

, Improving data quality of low-cost IoT sensors in environmental monitoring networks using data fusion and machine learning approach, ICT Express 6(3) (2020), 220–228. doi:10.1016/j.icte.2020.06.004.

20.

Hassan

S.M.

Maji

A.K.

Jasiñski

Leonowicz

Jasiñska

, Identification of Plant-Leaf Diseases Using CNN and transfer-learning approach, Electronics 10(12) (2021), 1388. doi:10.3390/electronics10121388.

21.

Tan

Jiang

, Review on Convolutional Neural Network (CNN) applied to plant leaf disease classification, Agriculture 11(8) (2021), 707. doi:10.3390/agriculture11080707.

22.

Azimi

Kaur

Gandhi

T.K.

, A deep learning approach to measure stress level in plants due to nitrogen deficiency,, Measurement: Journal of the International Measurement Confederation 173 (2021). doi:10.1016/j.measurement.2020.108650.

23.

Malvade

N.N.

Yakkundimath

Saunshi

Elemmi

M.C.

Baraki

, A comparative analysis of paddy crop biotic stress classification using pre-trained deep neural networks, Artificial Intelligence in Agriculture 6 (2022), 167–175. doi:10.1016/j.aiia.2022.09.001.

24.

Zhuang

Wang

Jiang

, Learned features of leaf phenotype to monitor maize water status in the fields, Computers and Electronics in Agriculture 172 (2020), 105347. doi:10.1016/j.compag.2020.105347.

25.

Reynolds

Ball

Bauer

Davey

Griffiths

Zhou

, CropSight: a scalable and open-source information management system for distributed plant phenotyping and IoT-based crop management, GigaScience 8(3) (2019). doi:10.1093/gigascience/giz009.

26.

Tanabata

Kodama

Hashiguchi

Inomata

Tanaka

Isobe

, Development of a plant conveyance system using an AGV and a self-designed plant-handling device: A case study of DIY plant phenotyping, Breed. Sci. 72(1) (2022), 85–95. doi:10.1270/jsbbs.21070.

27.

Atanbori

French

A.P.

Pridmore

T.P.

, Towards infield, live plant phenotyping using a reduced-parameter CNN, Machine Vision and Applications 31(1-2) (2020), 2. doi:10.1007/s00138-019-01051-7.

28.

Ito

, et al. Development of plant phenotyping platform using low-cost IoT devices and its performance evaluation, in Proceedings of the Forum Math-for-Industry 2019, Vol. 36, R. McKibbin, G. Wake, and O. Saeki, Eds., in Mathematics for Industry, vol. 36. Singapore: Springer Nature Singapore (2022), pp. 109–116. doi: 10.1007/978-981-19-1154-5_6.

29.

Hamidah

, et al. Water usage monitoring system for plants based on a weight sensor and imagery motion analysis, in Proceedings of the Forum Math-for-Industry 2019, vol. 36, R. McKibbin, G. Wake, and O. Saeki, Eds., in Mathematics for Industry, vol. 36. Springer Nature Singapore, Singapore (2022), 99–108, doi:10.1007/978-981-19-1154-5_5.

30.

Basak

Saha

Halder

Hati

A.J.

Banerjee

Krishnamurthy

Deep learning-based plant phenotyping framework: Analysis of crop life cycle data for indian farmers to develop a smart agri-field management system, in Proceedings of the NIELIT’s International Conference on Communication, Electronics and Digital Technology, vol. 676, S.N. Singh, S. Mahanta, and Y.J. Singh, Eds., in Lecture Notes in Networks and Systems, vol. 676. Singapore: Springer Nature Singapore (2023), pp. 163–181. doi: 10.1007/978-981-99-1699-3_11.

31.

Hadabas

Hovari

Vass

Kertesz

IoLT smart pot: An IoT-cloud solution for monitoring plant growth in greenhouses, in Proceedings of the 9th International Conference on Cloud Computing and Services Science. SCITEPRESS - Science and Technology Publications, Heraklion, Crete, Greece (2019), pp. 144–152. doi:. doi: 10.5220/0007755801440152.

32.

Elsherbiny

Zhou

Qiu

, A novel hybrid deep network for diagnosing water status in wheat crop using IoT-based multimodal data, Computers and Electronics in Agriculture 203 (2022), 107453. doi:10.1016/j.compag.2022.107453.

33.

Sunil Kumar,

Aithal

Venkatramana Bhat,

Design, calibration, and experimental study of low-cost resistivitybased soil moisture sensor for detecting moisture at different depths of a soil, in Advances in Artificial Intelligence and Data Engineering, vol. 1133, N.N. Chiplunkar and T. Fukao, Eds., in Advances in Intelligent Systems and Computing, vol. 1133. Springer Nature Singapore, Singapore (2021), pp. 1383–1397. doi: 10.1007/978-981-15-3514-7_104.

34.

Agarwal

Singh

Arjaria

Sinha

Gupta

, ToLeD: Tomato leaf disease detection using convolution neural network, , Procedia Computer Science 167 (2020), 293–301. doi:10.1016/j.procs.2020.03.225.

35.

Elhassouny

Smarandache

, Smart mobile application to recognize tomato leaf diseases using Convolutional Neural Networks, in, 2019 International Conference of Computer Science and Renewable Energies (ICCSRE), IEEE, Agadir, Morocco (2019), pp.1–4. doi: 10.1109/ICCSRE.2019.8807737.

36.

Wspanialy

Moussa

, A detection and severity estimation system for generic diseases of tomato greenhouse plants, Computers and Electronics in Agriculture 178 (2020), 105701. doi:10.1016/j.compag.2020.105701.