Dissertation Defence: High-throughput wheat plant phenotyping using data analysis techniques

Amirhossein Zaji, supervised by Dr. Zheng Liu, will defend their dissertation titled “High-throughput wheat plant phenotyping using data analysis techniques” in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Civil Engineering.

An abstract for Amirhossein’s dissertation is included below.

Examinations are open to all members of the campus community as well as the general public.

Please email zheng.liu@ubc.ca to receive the zoom link for this defence.

---

ABSTRACT

Wheat plant phenotyping is a fundamental aspect of plant breeding and crop management, as it allows for the identification and characterization of key plant characteristics that are vital for crop productivity and sustainability. However, traditional methods of wheat plant phenotyping are often time-consuming, labor-intensive, and susceptible to human error. This makes it challenging to accurately and efficiently analyze large numbers of wheat plants, hindering the potential of plant breeding and crop management programs.

Computer vision and Deep Learning (DL) provide a solution to these issues by offering more efficient and accurate methods to analyze and interpret images of wheat plants. These technologies utilize advanced algorithms and machine learning techniques to automatically identify and quantify specific plant characteristics, such as plant height, leaf area, and disease symptoms. This can greatly enhance the efficiency and accuracy of plant breeding and crop management programs. The use of computer vision and DL in wheat plant phenotyping has the potential to revolutionize the way that plant breeding and crop management programs are conducted, ultimately leading to more productive and sustainable wheat crops. However, the data preparation part of DL models, including tasks such as data collection, annotation, and preprocessing, can be extremely challenging and consume a significant amount of time.

The goal of this thesis is to investigate and develop strategies to minimize the time and effort required for data preparation in the application of DL models for high-throughput wheat plant phenotyping. The data preparation process, which includes image acquisition, annotation, and preprocessing, can be time-consuming and labor-intensive, and can be a major bottleneck for the widespread adoption of high-throughput phenotyping. To address this issue, the proposed research aims to investigate the use of more accessible sensors, such as low-cost cameras, for image acquisition, as well as to develop more efficient and automated methods for annotation and preprocessing. Additionally, we proposed an augmentation algorithm that can reduce the number of training samples needed for accurate DL models. Furthermore, this research will explore the use of transfer learning and other techniques to reduce the amount of labeled data required for training DL models. The ability to effectively and efficiently prepare data is a crucial step in the utilization of DL models, as the quality and quantity of the data directly impacts the performance of the model. Furthermore, reducing the data preparation time and effort can greatly increase the accessibility and scalability of these models for researchers and practitioners in the field of plant phenotyping.

The first objective of this research is to propose and evaluate a state-of-the-art framework for localizing and counting wheat spikes using dotted annotation datasets in conjunction with Gaussian Density Map (GDM) and Constant Density Map (CDM) generation algorithms. Dotted annotation is the fastest annotation technique for localizing and counting wheat spikes when compared to bounding box and polygon annotation. Hybrid UNet architectures, including VGG16-UNet, ResNet34-UNet, ResNet50-UNet, and ResNeXt-UNet are employed as the computational component. The proposed models are evaluated using the ACID dataset and demonstrate a significant improvement in spike localization and counting compared to previous research studies. Additionally, the generalizability of the proposed models is tested in real-world scenarios using the Global Wheat Head Dataset (GWHD) dataset.

The second objective of this research is to propose a novel augmentation algorithm, Automatic Object Level Augmentation (AutoOLA), that can significantly reduce the required number of training samples in DL models. The algorithm decouples different objects in the wheat images, including spikes, leaves, stems, and backgrounds, and augments them independently based on their own optimized augmentation policy. An evolutionary optimization algorithm is used to automatically design the types of transformations and their magnitudes for each decoupled object. The results demonstrate that the proposed AutoOLA technique could improve wheat spike counting performance by up to 60 percent.

The third objective of this research is to determine plant height at plant-level resolutions using stereovision cameras. Stereovision cameras are more accessible, easy to use plant height measurement sensor. To obtain the necessary datasets, the authors conducted an experiment using OAKD and D455 stereovision cameras. The collected images needed to be annotated to train and evaluate wheat spike localization using DL models. To expedite this process, we thoroughly annotated the images of one camera and transferred that to the images of the other camera using a designed active learning model. Then, by using a well-known DL architecture referred to as Mask R-CNN, we determined the location of the spikes. Finally, we used the depth layer of the cameras to estimate the height of each detected spikes using two different approaches of Instant Segmentation (IS) and Object Detection (OD). The results of this study provide valuable insights into the use of DL and computer vision methods for phenotyping wheat plants at high-throughput resolutions.

In conclusion, this thesis aimed to develop strategies to minimize the time and effort required in the applications of DL models for plant phenotyping. The proposed research focused on investigating the use of more accessible sensors, such as low-cost cameras, for image acquisition, as well as developing more efficient and automated methods for annotation and preprocessing, including a proposed augmentation algorithm. The research also explored the use of transfer learning and other techniques to reduce the amount of labeled data required for training DL models. By reducing the data preparation time and effort, this research has the potential to increase the accessibility and scalability of DL models for plant phenotyping, thus greatly advancing the field of plant breeding and genetics and improving the efficiency and sustainability of crop production.