Deep learning (DL) methods have transformed the way we extract plant traits – both under laboratory as well as field conditions. Evidence suggests that “well-trained” DL models can significantly simplify and accelerate trait extraction as well as expand the suite of extractable traits. Training a DL model typically requires the availability of copious amounts of annotated data; however, creating large-scale annotated dataset requires non-trivial efforts, time, and resources. This has become a major bottleneck in deploying DL tools in practice. Self-supervised learning (SSL) methods give exciting solution to this problem, as these methods use unlabeled data to produce pretrained models for subsequent fine-tuning on labeled data, and have demonstrated superior transfer learning performance on down-stream classification tasks. We investigated the application of SSL methods for plant stress classification using few labels. Plant stress classification is a fundamentally challenging problem in that (1) disease classification may depend on abnormalities in a small number of pixels, (2) high data imbalance across different classes, and (3) there are fewer annotated and available plant stress images than in other domains. We compared four different types of SSL methods on two different plant stress datasets. We report that pre-training on unlabeled plant stress images significantly outperforms transfer learning methods using random initialization for plant stress classification. In summary, SSL based model initialization and data curation improves annotation efficiency for plant stress classification tasks.

Mung bean [Vigna radiata (L.) Wilczek] is a drought-tolerant, short-duration crop, and a rich source of protein and other valuable minerals, vitamins, and antioxidants. The main objectives of this research were (1) to study the root traits related with the phenotypic and genetic diversity of 375 mung bean genotypes of the Iowa (IA) diversity panel and (2) to conduct genome-wide association studies of root-related traits using the Automated Root Image Analysis (ARIA) software. We collected over 9,000 digital images at three-time points (days 12, 15, and 18 after germination). A broad sense heritability for days 15 (0.22–0.73) and 18 (0.23–0.87) was higher than that for day 12 (0.24–0.51). We also reported root ideotype classification, i.e., PI425425 (India), PI425045 (Philippines), PI425551 (Korea), PI264686 (Philippines), and PI425085 (Sri Lanka) that emerged as the top five in the topsoil foraging category, while PI425594 (unknown origin), PI425599 (Thailand), PI425610 (Afghanistan), PI425485 (India), and AVMU0201 (Taiwan) were top five in the drought-tolerant and nutrient uptake “steep, cheap, and deep” ideotype. We identified promising genotypes that can help diversify the gene pool of mung bean breeding stocks and will be useful for further field testing. Using association studies, we identified markers showing significant associations with the lateral root angle (LRA) on chromosomes 2, 6, 7, and 11, length distribution (LED) on chromosome 8, and total root length-growth rate (TRL_GR), volume (VOL), and total dry weight (TDW) on chromosomes 3 and 5. We discussed genes that are potential candidates from these regions. We reported beta-galactosidase 3 associated with the LRA, which has previously been implicated in the adventitious root development via transcriptomic studies in mung bean. Results from this work on the phenotypic characterization, root-based ideotype categories, and significant molecular markers associated with important traits will be useful for the marker-assisted selection and mung bean improvement through breeding.

The rise of high-throughput phenotyping (HTP) has led to a dramatic increase in the ability to rapidly – and accurately – phenotype various organisms including plants. However, methods for efficiently managing, processing, analyzing, and sharing HTP data have not caught up to this new-found ability to collect big data which in turn introduces a whole host of new challenges. To address these, we have architected and implemented a multi-faceted infrastructure of webservices to further unify and automate the entire data collection process. We have integrated CyVerse and Slack into our IoHTP, two commonly used tools within plant science labs. CyVerse is a cloud-based data storage & management solution and Slack allows for bilateral instant-message communication with the HTP machines to keep researchers in touch with their autonomous experiments. Next, we are also developing our own website for administering jobs remotely to any number of (possibly geographically distributed) HTP machines. This innovative open-source approach has the potential to further advance high-throughput phenotyping worldwide by allowing interdisciplinary experts, namely in the plant and computer sciences, to collaborate more effectively and efficiently.

For field workers around the world, wheat trials are often synonymous with wheat heads counting: a tedious but important task to measure this important yield component. Deep Learning has been a promising solution to automate the acquisition of wheat head density from a high-throughput phenotyping system, but it has been shown to be sensitive to changing acquisition conditions, also known as “domain change.” In response, an international collaboration built the “Global Wheat Head Dataset” in 2020 and 2021, a collection of 6515 images acquired during 47 different acquisition sessions in 12 countries. In addition to these datasets, two data competitions were held in 2020 (Kaggle, over 2,200 competitors) and 2021 (AIcrowd, over 400 competitors). The winning solutions are expected to be usable in plant phenotyping pipelines to robustly assess wheat spike density. We tested this hypothesis by evaluating the 2021 winning solution on an independent dataset consisting of images measured both in the field and in the image by a human, taken with the same acquisition protocol. We use triple collocation analysis to demonstrate that the predicted density appears to be more reliable than the human density measured in the field and in the image. Furthermore, we demonstrate that Global Wheat Head Dataset can be used to estimate wheat ear density from a drone.

Improving photosynthesis has been considered critical to increasing crop yield to meet food demands from a growing population. To achieve this goal, high-throughput phenotyping techniques are highly needed to explore both natural and genetic variation in photosynthetic performance among crop cultivars. Due to the non-invasive nature of hyperspectral imaging, there is an increasing use of hyperspectral imaging for phenotyping of photosynthesis or photosynthetic physiology. The use of hyperspectral sensors has resulted in the accumulation of large amounts of data, shifting the research efforts into efficiently mining spectral information for high-throughput phenotyping. In this presentation, we will introduce data pipelines developed to leverage proximal sensing platforms and data sources including both reflectance spectra and solar-induced fluorescence (SIF) for quantifying photosynthetic performance at the canopy level. Photosynthetic performance was represented by the maximum carboxylation rate (Vcmax) and the maximum electron transport rate (Jmax). The experiments were conducted using eleven tobacco cultivars grown in field conditions during 2017 and 2018 at Energy Farm at University of Illinois. Time-synchronized hyperspectral images from 400 to 900 nm and irradiance measurements of sunlight under clear-sky conditions were collected for capturing reflectance spectra and SIF (and SIF related parameters). Within 30 minutes of spectral measurements, ground-truth Vcmax and Jmax were obtained from portable leaf gas exchange system. Our results suggested both reflectance spectra and SIF can provide accurate estimations of Vcmax and Jmax. The presented data pipelines have potential to relieve bottleneck in phenotyping of photosynthesis for breeding cultivars of enhanced photosynthesis.

An autonomous mobile ground-control point (AMGCP) was redesigned and refined for improved collaborative operation with an unmanned aerial vehicle (UAV) to enable calibration of image mosaics from multispectral (MS) and thermal cameras. The AMGCP has built-in reflectance panels and electronically controlled thermal panels that provide high and low reflectance and temperature references that can be used for calibration of reflectance measurements in MS images and temperature measurements in thermal-infrared images. The AMGCP also has an onboard temperature sensor that enables image-based temperature measurements to be compared to ambient temperature so that canopy temperature depression (CTD) can be calculated. The collaborative robotic system consists of the AMGCP and a UAV that have real-time kinematic (RTK) geographic positioning system (GPS) receivers onboard so their precise position can be determined in real time. The system also includes wireless communication capability between the AMGCP and UAV so they can transmit their position and other data to each other during a mission, in which the AMGCP positions itself at multiple locations under the flight path of the UAV, providing multiple instances of reflectance and temperature references in image mosaics collected by the UAV. Testing has shown that reflectance measurements can be calibrated to less than 1% reflectance error, and canopy temperatures of crop plants can be calibrated to within 1.0 C, enabling consistently accurate measurements to be made efficiently and without human intervention in various fields and regions and at different times and dates. This system is also suited to accurate measurement of CTD to facilitate genetic selection relative to various stresses and resilience characteristics like drought tolerance.

ABSTRACT Counting maize tassels in field conditions is predominantly done manually. Recently, computer-vision based methods have been utilized to detect tassels from images captured by UAV transects or poled-mounted cameras [1], [2], [3]. Once tassels are detected, deep-learning based local regression methods, Tasselnet, have been used to estimate in-field tassel counts [4]. However, field images are mostly captured over a period of time. Consequently, the input images in the foregoing Tasselnet technique are not independent but often form unequal sequences of correlated images. As such, the temporal sequence of images offers information about the growth trajectory of the plants. We propose a hybrid model that (a) utilizes convolutional neural network-based tassel localization in images, and (b) drives the local count of tassels utilizing the plant growth trajectory learned from the time-series of images. The resulting model can also handle important auxiliary information, obtained from in-field sensors (for example: soil moisture, air temperature etc.), that impacts plant growth and tassel counts. We implement our methodology on benchmark dataset [4] and compare our results with the SOTA Tasselnet [5]. Our initial results suggest that our technique is computationally viable and can produce accurate point estimates of tassel counts along with interval estimates capturing the precision of our estimates. Keywords: Computer vision, Convolutional neural networks, Deep learning, Maize tassels, Time-series. REFERENCES [1] Shi, Y., Alzadjali, A., Alali, M., Veeranampalayam-Sivakumar, A. N., Deogun, J., Scott, S., & Schnable, J. (2021). Maize tassel detection from UAV imagery using deep learning. Dryad. https://doi.org/10.5061/dryad.r2280gbcg. [2] Mirnezami, S. V., Srinivasan, S., Zhou, Y., Schnable, P. S., & Ganapathysubramanian, B. (2021). Detection of the Progression of Anthesis in Field-Grown Maize Tassels: A Case Study. Plant Phenomics, 2021, 4238701. doi:10.34133/2021/4238701. [3] Shete, S., Srinivasan, S., & Gonsalves, T. A. (2020). TasselGAN: An Application of the Generative Adversarial Model for Creating Field-Based Maize Tassel Data. Plant Phenomics, 2020, 8309605. doi:10.34133/2020/8309605. [4] Lu, H., Cao, Z., Xiao, Y., Zhuang, B., & Shen, C. (2017). TasselNet: counting maize tassels in the wild via local counts regression network. Plant Methods, 13(1), 79. doi:10.1186/s13007-017-0224-0. [5] Xiong, H., Cao, Z., Lu, H., Madec, S., Liu, L., & Shen, C. (2019). TasselNetv2: in-field counting of wheat spikes with context-augmented local regression networks. Plant Methods, 15(1), 150. doi:10.1186/s13007-019-0537-2.

Fusiform rust is a disease caused by the fungal pathogen Cronartium quercuum f. sp. fusiforme (Cqf). It is considered one of the most damaging and economically important diseases for loblolly pine (Pinus taeda L.), causing millions of dollars in damage and loss of products each year. Evaluating trees for disease resistance includes inoculation trials – these trials require artificial inoculation of the pathogen followed by visual inspection for disease incidence. Visual inspection can lead to incorrect classification due to human error or escaped susceptible (i.e. a susceptible individual with no symptoms). Here, we plan to use vibrational spectroscopy tools to improve the accuracy of phenotypic values. Vibrational spectroscopy tools allow for a user to obtain a single, comprehensive reading based on the chemical constituents of sample. Because pine trees mostly rely on chemical-based defenses, the relationship between chemical makeup and resistance is promising. We plan to collect spectra from 40 different loblolly pine families (20 with lower rust incidence and 20 higher rust incidence) over five different progeny test sites in the southeastern US, totaling 400 trees. We will use a handheld near-infrared (NIR) spectrometer for a real-time, in-field reading on phloem and needle tissue. In addition, phloem and needle tissue will be analyzed by a benchtop Fourier-transformed infrared (FT-IR) spectrometer. Using multivariate analyses and machine learning algorithms, spectral readings can be mined for patterns associated with fusiform rust disease resistance or susceptibility, which can be used to predict the phenotype of untested trees. The results of the two tools and two tissue types will be compared to evaluate the best method for identifying phenotype in the system. This chemical fingerprinting and classification approach to phenotyping loblolly pines will provide a more objective, efficient, and more accurate way to identify disease resistance in the field, thereby creating more robust forest stands against fusiform rust.

Predicting the composition of soybean seeds while the plants are growing in the field is very important to understand how different genotypes, field condition and environment influence different seed composition parameters. Knowing this information at global scale is even more important to understand the dynamics of food insecurity and the interaction of seed composition with global environmental changes. This study aims to develop a machine learning-based soybean seed composition model from the fusion of PlanetScope, Sentinel and Landsat satellite images. Although satellite images provide global coverage throughout the year, it suffers from coarser spatial resolution. However, PlanetScope provides four-band (i.e., red, green, blue, and near infrared) multispectral imageries at approximately 3m spatial resolution daily. Alternatively, Sentinel-2B and Landsat-8 have coarser spatial resolution (10 - 30m), they provide enriched spectral resolution. Therefore, the objectives of this study are to 1) fuse the PlanetScope image with corresponding Landsat and Sentinel images, 2) evaluate several machine learning algorithms (e.g., partial least squares, support vector machine, random forest, and deep neural network) to predict protein and oil content of soybean seeds from the fused satellite images. Two soybean fields were established in 2020 and 2021 at Bradford, MO to perform the experiment. Corresponding PlanetScope, Sentinel, and Landsat images were downloaded and processed for the entire growth seasons. Current results indicate that deep neural network provide the best performance in predicting both protein and oil content of soybean. Future step is to assess different fusion algorithms and predict seed composition at regional or global scale.

Developing crops with better root system architecture and nitrogen use efficiency can lead to resilient crops capable of sustaining productivity in both optimum and stress environments. The increase in soybean global demand and its use in biodiesel and new soy-based products calls for new soybean cultivars that have higher yields, better nutritional values or desirable traits for specific use. The soybean germplasm collection at the USDA is a valuable resource in discovering novel allelic variations. This collection has not been screened extensively for the root nodule and root system architecture traits. We have already established root phenotyping platforms in our laboratory and we are proposing to screen a diverse pool of the USDA soybean collection for nodules and root system architecture traits. Availability of SNP data for this collection will let us run genome-wide analysis and identify QTLs responsible for different root traits. We will then use hairy root transformation to knockout/down some of the candidate genes in the loci identified in earlier reports and new QTLs identified in this project. This project will help us in deciphering the phenotypic variation in soybean root traits. Ultimately, better understanding of the regulatory mechanism controlling these traits can help us in developing resilient crops and a sustainable cropping system.

We present several methods for improving plant reconstruction from multiple 3D observations. Producing 3D data useful for plant phenotyping requires proximal sensing (e.g. line scanner, depth camera) at multiple incident angles (φ) and often with multiple passes. These resulting individual point clouds must then be assembled into a single point cloud for analysis. Our interest in improving the registration of individual plants is focused specifically on observations made within field settings which present additional challenges over laboratory 3D scans, where background, overlap and light conditions can be controlled. To develop these methods, we use several season’s worth of data from the University of Arizona’s Field Scanalyzer located in Maricopa, Arizona. Our approach prioritizes: (1) plant completeness, (2) noise reduction, (3) temporal similarity and (4) computational efficiency. The first priority is accomplished simply by prioritizing individual point clouds that contain the majority of the individual plant. 3D field scanning can result in component point clouds that are from near-identical φ and cover the same portions of the individual plant. This results in both additional noise and uncertainties due to small georeferencing errors and plant movement between scans. Thus, we remove the data that is furthest in time with non-unique φ in order to achieve priorities 2 and 3. Our method results in small scene reconstruction which has low memory and computational demands. In order to improve registration further, we investigate iterative closest point (ICP) registration fitting using weights defined by crop height distributions and semantic segmentation point labeling.

We quantify the shape of hooked hairs which is a newly observed phenotype of epidermal cell extensions [1] in the common bean genotype L88-57 (Phaseolus vulgaris). The hooked hairs emerge below-ground before the root hairs and have a distinct ‘hooking’ morphology. We generated a dataset capturing their full distribution under the microscope within 3-5 days of germination. We quantify their shape by a novel computational pipeline that can automatically phenotype morphology. Our phenotyping pipeline quantifies traits like length, curvature, perimeter, area, and ‘hooking.’ Our objective is to quantify the response of hooked hairs to nutrient-stress to determine their function in common bean during early development. We used the pipeline for analyzing our dataset of hydroponically grown beans and observed statistically significant responses compared to the control for length, curvature, perimeter, and area to nitrogen (p<0.001**) and phosphorus (p<0.001**) stress treatments. The calculation of ‘hooking’ for our dataset is still ongoing. We are simultaneously developing a landmark-free method for the two-dimensional shape analysis of our dataset and believe that our phenotyping efforts will enable the high-throughput analysis of morphological root hair traits for any plant species.

Agricultural water resources are threatened by climatic variability and increased competition for available freshwater resources. In order to mitigate the effect of climate change on cotton production, breeders are increasing their efforts on improving drought tolerance in this essential fiber crop. To achieve this, effective screening of diverse germplasm is needed to identify useful genetic variation that can be utilized for crop improvement. Within the last decade, unmanned aerial vehicles (UAVs) have led to the ability to quickly and reliably image large areas while simultaneously decreasing temporal effects associated with a large time window for data collection. This technology allows researchers to scale their phenotyping efforts, enabling studies that utilize mapping and monitoring efforts such as plant water stress detection. In this study, we used UAV-based thermal imagery to screen a diverse population of over 350 different genotypes of cotton in order to locate varieties that exhibit cooler canopies. This diversity panel was grown under two contrasting levels of irrigation, well-watered and water-limited, with data collection flights occurring weekly for three months during the season. The thermal images were clipped to plot boundaries, soil and plant pixels were segmented, and average temperatures were extracted to identify potential drought tolerant varieties. The objectives of this study were to (i) demonstrate that UAV-based thermal imagery, along with our calibration methods, can be used to render accurate plant canopy temperature values and (ii) identify cotton genotypes that outperform others in a drought-stressed environment.

Anthropogenic factors such as climate change, harsh agricultural practices, and mining have contributed to increases in soil salinization and heavy metal contamination. Highly saline environments drastically lower yield for crop species and elevated levels of toxic metals like cadmium are carcinogenic in the environment. Some plants that have evolved in high-salinity habitats or in soils with heavy metals could be used to remediate contaminated soils. Halophytes are plants with various adaptations that allow them to survive and reproduce in saline conditions. General mechanisms for salt tolerance/uptake in halophytes are hypothesized to help deal with other stresses like heavy metals. Plants with these traits could be utilized to extract salt and heavy metals from affected soils in a process called phytoremediation. We plan to develop Sea Rocket (Cakile maritima) in the Mustard family as a model system to understand mechanisms of salt (NaCl) and cadmium uptake and tolerance, as it has been shown to accumulate both. As part of this study, we will hydroponically grow C. maritima in different stress treatments using salt and cadmium. As the plants uptake the pollutants, the conductivity of the solution will change. We will develop an automated pipeline to track these changes in real-time using conductivity sensors. In addition, we will sample root and leaf tissues at various time points to measure salt and cadmium uptake using ICP-OES elemental analysis. This data will provide insights into salt and cadmium uptake/tolerance and paves a path toward efficient and viable solutions improving phytoremediation approaches.

Multispectral imaging with unmanned aircraft systems (UAS) is a promising high-throughput phenotyping technology that has been shown to help understand the causal mechanisms associated with crop productivity. This imaging technology can accurately predict complex agronomic traits like grain yield within a given generation, creating the potential to fast-track selections in plant breeding and increase genetic gains. The objective of this study was to determine the effectiveness and efficiency of prediction on grain yield in an abnormal drought year across locations within a breeding program. Eleven spectral reflectance indices (SRI) including NDRE, NWI, NDVI, and percent canopy cover were used to evaluate Washington State University winter wheat breeding lines between 2018 and 2021. Data was collected using a DJI Inspire 2 drone, equipped with a Sentera Quad Multispectral Sensor, and collected at the heading date. Lines were observed from single location, single replication preliminary yield trials to multi-location, replicated advanced yield trials. Lines advanced in the breeding program were evaluated across 13 different location-year trials. The calculated SRIs and canopy cover were used individually and in combination as fixed effects in mixed model prediction for grain yield under drought conditions. Models were independently validated with 2021 data. Across locations, SRIs are shown to improve the prediction performance for grain yield under abnormal drought conditions by as much as 40% in the case of NDRE. This research is vital for plant breeders to understand the utility of UAS imaging in variety improvement when dealing with abnormal growing seasons.

Knowing how many plants have germinated in a farmer’s or researcher’s field is central to crop management and research. Plants are commonly counted by walking the field and counting the plants in each row. This technique is labor-intensive, slow, and error-prone. Automating stand counts using RGB imagery from Unmanned Aerial Vehicles (UAV) is an obvious solution. We propose DeepMaizeCounter, a robust computational system that provides accurate stand counts for research and production fields from imagery captured by freely flown, inexpensive drones and processed with an inexpensive computer. DeepMaizeCounter exploits mosaics computed from RGB videos, using a YOLOv4 model that is trained to recognize seedling maize plants in the V2–V10 growth stages (approximately 10–40cm in height), singly or in groups of two and three plants, and determines its accuracy on these classes of seedlings. We evaluated DeepMaizeCounter against in-field and on-frame manual stand counts for a number of different maize lines in both nursery and production fields. DeepMaizeCounter can reliably distinguish corn from weeds and other grasses, counting only the maize. The network is light and able to run 175 test frames in 6 seconds, or 29 frames per second. This opens the prospect that DeepMaizeCounter can eventually be deployed on cheap platforms for real-time counting.

Cosegmentation is a recent and rapidly emerging and rapidly growing extension of segmentation, which aims to detect the common object(s) in a group of images. Current cosegmentation methods are ideal and effective only for certain dataset types with limited features that still produce errors making it difficult to obtain detailed metrics of object parts. We propose to build a unified, trainable framework that incorporates multiple features of a high-throughput dataset’s segmented images from multiple algorithms using cosegmentation. Specifically, we propose a novel Cosegmentation for Plant Phenotyping Network (CoPPNet) that utilizes a Fully Convolutional Neural Network with a K-Means Clustering feedback loop for optimal temporal loss. The results from this study will set the benchmark for a novel advancement in computer vision segmentation accuracy and plant phenomics to better understand a plant’s environmental interactions for maximal resilience and yield.

Traditionally, walnut kernel color is scored by a human on a four-point color scale ranging from extra light to Amber, developed by the California DFA. Two important reasons for using a high throughput machine vision system for kernel phenotyping is to obtain better quantitative resolution to use in molecular breeding and to avoid errors in human vision phenotyping. RGB and LAB values give much more depth to qualitative calculations, and machine data is far more consistent than human scoring. Previous studies on walnut kernel imaging utilized a thresholding-based approach to segment walnut kernels from their background, however, detection accuracy can be improved. In this study, we make changes to detection methods, primarily using PyTorch computer vision processing and an improved thresholding method to better segment walnut kernels. The PyTorch CNN pipeline allows us to use specific photos to train a model, and the model can segment photos without the need to figure out image thresholds. Our proposed thresholding method uses the magick package in R instead of the ImageJ macros that were previously used. After taking photos with a CVS, we used the CNN model as well as an R script to identify and segment kernels from the background. Our preliminary data shows that with enough training, the CNN model is more robust in edge cases where we have overlapping kernels or shifted images. This paper will focus on exploring the differences between the three thresholding methods, and use the best method for future breeding projects.