Similar to other sectors, present-day agriculture relies on new advances in different fields such as machine learning, computer vision, robotics, botany, etc. In the modern world, new scopes have been introduced to agriculture, either directly or indirectly, to meet human needs, preserve the natural and environments and resources for the future. As an example, the sustainability of growth is dependent on a drop in cost under a particular threshold, and modernization of agriculture, in different aspects, is a demand to accelerate the process toward an acceptable growth. In order to improve agricultural productivity and increase benefits, one necessity is to transition from traditional methods to modern methods and availability of smart machines. In this way, it is feasible to build systems based on automation and control concepts and utilize precise algorithms for carrying out different tasks with fewer hands-on farms and protecting natural resources for the next generations. Hence, experts in robotics and electrical engineering are also involved with new aspects of agriculture and farming.
Accordingly, while researchers have been forced to compete for increasing precision and profitability in agricultural activities and improve present methods with respect to natural environments, it is also necessary to serve on new major fronts: accurate mitigation of weeds in fields, optimum water consumption, reducing labor costs and number of workers, 24-hour remote control of fields, etc. Hence, it is necessary to provide more useful information about plant species and apply the extracted information for further purposes. Accurate recognition of plants is an essential part of such information. This task cannot be neglected as it supports not only farmers but also botanists and environmentalists.
By considering the workplaces of farmers and botanists, it is feasible to divide the workspaces into two main subsets: controlled environments like laboratories with static conditions and uncontrolled environments like outdoor environments with dynamic conditions. Despite the importance of plant recognition, a considerable number of works has been proposed for recognizing plant species in stationary conditions based on constant background, light condition, the position of leaves, presence of single leaves, etc. In the real world, such constraints and assumptions do not lead to promising results. Therefore, consideration of other factors is essential to build efficient systems for natural plant recognition.
In this research, both workspaces have been considered to develop well-mechanized plant recognition systems. This work employs the modern combined methods for local feature extraction and precise recognition of plant species. To fulfill the goals in the controlled environment, six different plant recognition systems are developed and evaluated by conducting various experiments. It is noteworthy that the modern combined methods have been adopted as the foundation of the first phase of the natural plant recognition systems in the uncontrolled environment. However, the story changes in outdoor environments and there is no fixed condition for taking images of plants and leaves.
In uncontrolled environments, environmental and non-environmental factors affect the photographing process. Light intensity and illumination are two crucial environmental factors that have an impact on images, and these factors may vary over time. Images taken from one particular scene or object are not the same if it is captured in the morning or the evening. Furthermore, weather affects the color intensity in outdoor environments as the color of leaves depends on temperature, light and water supply, and changes to these factors are also inevitable with the change of month and season. Non-environmental factors like background and distance have also effects on the performance of plant recognition systems. Backgrounds of images of natural plants taken in outdoor environments are generally more complicated in comparison to backgrounds in controlled environments. Meanwhile, the distance, either short or long, between camera and plant is undoubtedly another big challenge in uncontrolled environments. In addition, there is no certainty that the images contain only one single leaf or there will be a number of leaves within images.
To develop a more efficient natural plant recognition system, we ride the wave of the tsunami of deep learning and build a novel plant recognition system based on a convolutional neural network. Due to the promising result and the superior performance, the system is then deployed as the main core of a mobile real-time system. To evaluate the system, a mobile robot and a semi-robot have been equipped with cameras to navigate and explore the outdoor environment in two different years, 2017 and 2018. While exploring, an image is captured and automatically processed by the deep natural plant recognition system to visualize the species of the target plant as a real-time system. The final results show that the real-time mobile plant recognition system can identify natural plant species independently of the used camera, distance, time of day and other environmental and non-environmental factors in uncontrolled environments.