ABSTRACT Counting maize tassels in field conditions is predominantly done manually. Recently, computer-vision based methods have been utilized to detect tassels from images captured by UAV transects or poled-mounted cameras [1], [2], [3]. Once tassels are detected, deep-learning based local regression methods, Tasselnet, have been used to estimate in-field tassel counts [4]. However, field images are mostly captured over a period of time. Consequently, the input images in the foregoing Tasselnet technique are not independent but often form unequal sequences of correlated images. As such, the temporal sequence of images offers information about the growth trajectory of the plants. We propose a hybrid model that (a) utilizes convolutional neural network-based tassel localization in images, and (b) drives the local count of tassels utilizing the plant growth trajectory learned from the time-series of images. The resulting model can also handle important auxiliary information, obtained from in-field sensors (for example: soil moisture, air temperature etc.), that impacts plant growth and tassel counts. We implement our methodology on benchmark dataset [4] and compare our results with the SOTA Tasselnet [5]. Our initial results suggest that our technique is computationally viable and can produce accurate point estimates of tassel counts along with interval estimates capturing the precision of our estimates. Keywords: Computer vision, Convolutional neural networks, Deep learning, Maize tassels, Time-series. REFERENCES [1] Shi, Y., Alzadjali, A., Alali, M., Veeranampalayam-Sivakumar, A. N., Deogun, J., Scott, S., & Schnable, J. (2021). Maize tassel detection from UAV imagery using deep learning. Dryad. https://doi.org/10.5061/dryad.r2280gbcg. [2] Mirnezami, S. V., Srinivasan, S., Zhou, Y., Schnable, P. S., & Ganapathysubramanian, B. (2021). Detection of the Progression of Anthesis in Field-Grown Maize Tassels: A Case Study. Plant Phenomics, 2021, 4238701. doi:10.34133/2021/4238701. [3] Shete, S., Srinivasan, S., & Gonsalves, T. A. (2020). TasselGAN: An Application of the Generative Adversarial Model for Creating Field-Based Maize Tassel Data. Plant Phenomics, 2020, 8309605. doi:10.34133/2020/8309605. [4] Lu, H., Cao, Z., Xiao, Y., Zhuang, B., & Shen, C. (2017). TasselNet: counting maize tassels in the wild via local counts regression network. Plant Methods, 13(1), 79. doi:10.1186/s13007-017-0224-0. [5] Xiong, H., Cao, Z., Lu, H., Madec, S., Liu, L., & Shen, C. (2019). TasselNetv2: in-field counting of wheat spikes with context-augmented local regression networks. Plant Methods, 15(1), 150. doi:10.1186/s13007-019-0537-2.

Developing crops with better root system architecture and nitrogen use efficiency can lead to resilient crops capable of sustaining productivity in both optimum and stress environments. The increase in soybean global demand and its use in biodiesel and new soy-based products calls for new soybean cultivars that have higher yields, better nutritional values or desirable traits for specific use. The soybean germplasm collection at the USDA is a valuable resource in discovering novel allelic variations. This collection has not been screened extensively for the root nodule and root system architecture traits. We have already established root phenotyping platforms in our laboratory and we are proposing to screen a diverse pool of the USDA soybean collection for nodules and root system architecture traits. Availability of SNP data for this collection will let us run genome-wide analysis and identify QTLs responsible for different root traits. We will then use hairy root transformation to knockout/down some of the candidate genes in the loci identified in earlier reports and new QTLs identified in this project. This project will help us in deciphering the phenotypic variation in soybean root traits. Ultimately, better understanding of the regulatory mechanism controlling these traits can help us in developing resilient crops and a sustainable cropping system.

The expanding geographic range of Phyllachora maydis, the fungus that induces Tar Spot infection on corn foliage, is increasingly threatening a Michigan industry that contributes over $1 billion to the state’s economy annually. Advances in machine learning now enable quantification of crop infection presence and severity using powerful object detection packages such as Tensorflow, Keras, and more. Tensorflow, specifically, has developed Application Programming Interface (API) tools to connect powerful object detection capabilities with streamlined usability. Foliar infection of maize by P. maydis is often difficult to detect early. Visible lesions initially appear tiny, ambiguous, and sparse, making them difficult to identify with the naked eye. Both farmers and breeders of corn desperately need better tools that allow early, definitive detection of lesions and provide more time for management decisions. This tool must verify presence of P. maydis and quantify infection severity as quickly as possible to allow growers the most options for treatment. I propose a combination of supervised machine learning using Tensorflow for custom object detection, and containerized application-development software such as Docker to create a user interface accessible on desktop or mobile devices. This application will be developed by weaving the transferrable infrastructure of Docker with the powerful machine learning platforms Tensorflow and Tensorflow Lite, thereby allowing users to analyze images using their preferred operating system. By implementing both complementary Tensorflow platforms, farmers and breeders will be afforded the choice of either capturing and analyzing one image at a time, or detecting lesions continuously in real-time.

Scene understanding methodologies (e.g., classification, segmentation, and anomaly detection) can be costly when operating on a per-pixel basis. As remote sensing applications rely on imagery with greater resolution, pixels regularly contain similar information to their neighbors. In recent years, scene understanding approaches have leveraged superpixel algorithms to partition imagery into small homogeneous regions for a variety of tasks. Instead of performing operations on millions of pixels, thousands, or in some cases hundreds of superpixels can be used as inputs into various pipelines. Most superpixel algorithms rely solely on RGB color information to produce superpixel maps. However, when multiple co-registered modalities are available, as is the case with the National Ecological Observatory Network (NEON) tree crown dataset, it is possible to combine information from multiple sources to produce a single shared map. In this work we combine airborne hyperspectral imagery, LiDAR point clouds, and RGB imagery to obtain superpixel maps and compare the oversegmentation results to those obtained by fusing individual maps produced from each modality. Superpixels are computed using the Simple Non-Iterative Clustering (SNIC) algorithm. Oversegmentation maps are scored using standard evaluation approaches. We present results on a subset of the National Ecological Observatory Network imagery dataset.

Hyperspectral based prediction of nutrient content in maize leaves Hyperspectral imaging is a promising method to predict crop traits in a high-throughput manner and unlock quantitative genetic studies. A single hyperspectral image can be used to predict several unrelated traits at once using spectral data from 350nm - 2500nm. Researchers have successfully modeled different physiological traits in maize such as vegetative Nitrogen content but the effect of different development stages, genotypes, and treatments on modelling power remains unclear. Here, I explore the ability to model leaf macro- and micro- nutrient content and leaf water content from hyperspectral transmittance data collected with a LeafSpec imaging device. I will compare three different machine learning algorithms; Partial Least Squares Regression, Random Forest and a Convolutional Neural Net to model nutrient content collected from twenty hybrids throughout the 2020 field season in fertilized and not fertilized blocks. Genotypes and development stages excluded from model training are used to externally validate models. Sulfur, Nitrogen, Calcium, Copper, and Iron leaf concentrations were the most amenable nutrients to prediction with coefficient of determination scores from 0.78 - 0.73, respectively. Models trained on samples from a collection of time points were able to accurately predict new time points and genotypes. The findings demonstrate the ability to predict nutrient content in field grown maize over a variety of developmental stages, genotypes, and treatments from a handheld hyperspectral imaging device.

Multispectral imaging with unmanned aircraft systems (UAS) is a promising high-throughput phenotyping technology that has been shown to help understand the causal mechanisms associated with crop productivity. This imaging technology can accurately predict complex agronomic traits like grain yield within a given generation, creating the potential to fast-track selections in plant breeding and increase genetic gains. The objective of this study was to determine the effectiveness and efficiency of prediction on grain yield in an abnormal drought year across locations within a breeding program. Eleven spectral reflectance indices (SRI) including NDRE, NWI, NDVI, and percent canopy cover were used to evaluate Washington State University winter wheat breeding lines between 2018 and 2021. Data was collected using a DJI Inspire 2 drone, equipped with a Sentera Quad Multispectral Sensor, and collected at the heading date. Lines were observed from single location, single replication preliminary yield trials to multi-location, replicated advanced yield trials. Lines advanced in the breeding program were evaluated across 13 different location-year trials. The calculated SRIs and canopy cover were used individually and in combination as fixed effects in mixed model prediction for grain yield under drought conditions. Models were independently validated with 2021 data. Across locations, SRIs are shown to improve the prediction performance for grain yield under abnormal drought conditions by as much as 40% in the case of NDRE. This research is vital for plant breeders to understand the utility of UAS imaging in variety improvement when dealing with abnormal growing seasons.

The rise of high-throughput phenotyping (HTP) has led to a dramatic increase in the ability to rapidly – and accurately – phenotype various organisms including plants. However, methods for efficiently managing, processing, analyzing, and sharing HTP data have not caught up to this new-found ability to collect big data which in turn introduces a whole host of new challenges. To address these, we have architected and implemented a multi-faceted infrastructure of webservices to further unify and automate the entire data collection process. We have integrated CyVerse and Slack into our IoHTP, two commonly used tools within plant science labs. CyVerse is a cloud-based data storage & management solution and Slack allows for bilateral instant-message communication with the HTP machines to keep researchers in touch with their autonomous experiments. Next, we are also developing our own website for administering jobs remotely to any number of (possibly geographically distributed) HTP machines. This innovative open-source approach has the potential to further advance high-throughput phenotyping worldwide by allowing interdisciplinary experts, namely in the plant and computer sciences, to collaborate more effectively and efficiently.

An autonomous mobile ground-control point (AMGCP) was redesigned and refined for improved collaborative operation with an unmanned aerial vehicle (UAV) to enable calibration of image mosaics from multispectral (MS) and thermal cameras. The AMGCP has built-in reflectance panels and electronically controlled thermal panels that provide high and low reflectance and temperature references that can be used for calibration of reflectance measurements in MS images and temperature measurements in thermal-infrared images. The AMGCP also has an onboard temperature sensor that enables image-based temperature measurements to be compared to ambient temperature so that canopy temperature depression (CTD) can be calculated. The collaborative robotic system consists of the AMGCP and a UAV that have real-time kinematic (RTK) geographic positioning system (GPS) receivers onboard so their precise position can be determined in real time. The system also includes wireless communication capability between the AMGCP and UAV so they can transmit their position and other data to each other during a mission, in which the AMGCP positions itself at multiple locations under the flight path of the UAV, providing multiple instances of reflectance and temperature references in image mosaics collected by the UAV. Testing has shown that reflectance measurements can be calibrated to less than 1% reflectance error, and canopy temperatures of crop plants can be calibrated to within 1.0 C, enabling consistently accurate measurements to be made efficiently and without human intervention in various fields and regions and at different times and dates. This system is also suited to accurate measurement of CTD to facilitate genetic selection relative to various stresses and resilience characteristics like drought tolerance.

Crop leaf angle is a crucial feature of plant architecture which influences photosynthetic efficiency and yield. Therefore high-throughput mapping of leaf angle is of extreme interest for both precision agriculture and crop breeding operations. In this study, we propose a UAV-based hybrid approach by combining a radiative transfer model (PROSAIL) and deep neural networks to estimate leaf angle from multi-angular hyperspectral and LiDAR data. PROSAIL can simulate canopy hyperspectral reflectance from a given list of parameters, where Average Leaf Inclination Angle (ALIA) is one of the canopy parameters. The goal is to develop a deep learning-based inversion function that takes UAV hyperspectral reflectance and other PROSAIL parameters as input and estimate ALIA of Maize as output. The other PROSAIL parameters will be estimated using several machine learning pipelines from UAV hyperspectral and LiDAR information. We also propose a multi-angular reflectance scheme where each image pixel will generate multiple simulated and observed reflectance from the overlapping regions using different angles (i.e., solar zenith angle, viewing zenith angle, and relative azimuth angle between the sun and sensor) involved in the PROSAIL simulation. An automated Python-based tool was developed that can calculate all three PROSAIL angles for a given hyperspectral data cube and generate the simulated reflectance for every vegetation pixels per experimental plot. Since the proposed method incorporates both crop information (i.e., PROSAIL) and a data-driven approach (i.e., deep learning), the method can be easily transferable for other study areas and crops, and it can rely on less ground truth data.

Agricultural water resources are threatened by climatic variability and increased competition for available freshwater resources. In order to mitigate the effect of climate change on cotton production, breeders are increasing their efforts on improving drought tolerance in this essential fiber crop. To achieve this, effective screening of diverse germplasm is needed to identify useful genetic variation that can be utilized for crop improvement. Within the last decade, unmanned aerial vehicles (UAVs) have led to the ability to quickly and reliably image large areas while simultaneously decreasing temporal effects associated with a large time window for data collection. This technology allows researchers to scale their phenotyping efforts, enabling studies that utilize mapping and monitoring efforts such as plant water stress detection. In this study, we used UAV-based thermal imagery to screen a diverse population of over 350 different genotypes of cotton in order to locate varieties that exhibit cooler canopies. This diversity panel was grown under two contrasting levels of irrigation, well-watered and water-limited, with data collection flights occurring weekly for three months during the season. The thermal images were clipped to plot boundaries, soil and plant pixels were segmented, and average temperatures were extracted to identify potential drought tolerant varieties. The objectives of this study were to (i) demonstrate that UAV-based thermal imagery, along with our calibration methods, can be used to render accurate plant canopy temperature values and (ii) identify cotton genotypes that outperform others in a drought-stressed environment.

Fusiform rust is a disease caused by the fungal pathogen Cronartium quercuum f. sp. fusiforme (Cqf). It is considered one of the most damaging and economically important diseases for loblolly pine (Pinus taeda L.), causing millions of dollars in damage and loss of products each year. Evaluating trees for disease resistance includes inoculation trials – these trials require artificial inoculation of the pathogen followed by visual inspection for disease incidence. Visual inspection can lead to incorrect classification due to human error or escaped susceptible (i.e. a susceptible individual with no symptoms). Here, we plan to use vibrational spectroscopy tools to improve the accuracy of phenotypic values. Vibrational spectroscopy tools allow for a user to obtain a single, comprehensive reading based on the chemical constituents of sample. Because pine trees mostly rely on chemical-based defenses, the relationship between chemical makeup and resistance is promising. We plan to collect spectra from 40 different loblolly pine families (20 with lower rust incidence and 20 higher rust incidence) over five different progeny test sites in the southeastern US, totaling 400 trees. We will use a handheld near-infrared (NIR) spectrometer for a real-time, in-field reading on phloem and needle tissue. In addition, phloem and needle tissue will be analyzed by a benchtop Fourier-transformed infrared (FT-IR) spectrometer. Using multivariate analyses and machine learning algorithms, spectral readings can be mined for patterns associated with fusiform rust disease resistance or susceptibility, which can be used to predict the phenotype of untested trees. The results of the two tools and two tissue types will be compared to evaluate the best method for identifying phenotype in the system. This chemical fingerprinting and classification approach to phenotyping loblolly pines will provide a more objective, efficient, and more accurate way to identify disease resistance in the field, thereby creating more robust forest stands against fusiform rust.

We present several methods for improving plant reconstruction from multiple 3D observations. Producing 3D data useful for plant phenotyping requires proximal sensing (e.g. line scanner, depth camera) at multiple incident angles (φ) and often with multiple passes. These resulting individual point clouds must then be assembled into a single point cloud for analysis. Our interest in improving the registration of individual plants is focused specifically on observations made within field settings which present additional challenges over laboratory 3D scans, where background, overlap and light conditions can be controlled. To develop these methods, we use several season’s worth of data from the University of Arizona’s Field Scanalyzer located in Maricopa, Arizona. Our approach prioritizes: (1) plant completeness, (2) noise reduction, (3) temporal similarity and (4) computational efficiency. The first priority is accomplished simply by prioritizing individual point clouds that contain the majority of the individual plant. 3D field scanning can result in component point clouds that are from near-identical φ and cover the same portions of the individual plant. This results in both additional noise and uncertainties due to small georeferencing errors and plant movement between scans. Thus, we remove the data that is furthest in time with non-unique φ in order to achieve priorities 2 and 3. Our method results in small scene reconstruction which has low memory and computational demands. In order to improve registration further, we investigate iterative closest point (ICP) registration fitting using weights defined by crop height distributions and semantic segmentation point labeling.

Anthropogenic factors such as climate change, harsh agricultural practices, and mining have contributed to increases in soil salinization and heavy metal contamination. Highly saline environments drastically lower yield for crop species and elevated levels of toxic metals like cadmium are carcinogenic in the environment. Some plants that have evolved in high-salinity habitats or in soils with heavy metals could be used to remediate contaminated soils. Halophytes are plants with various adaptations that allow them to survive and reproduce in saline conditions. General mechanisms for salt tolerance/uptake in halophytes are hypothesized to help deal with other stresses like heavy metals. Plants with these traits could be utilized to extract salt and heavy metals from affected soils in a process called phytoremediation. We plan to develop Sea Rocket (Cakile maritima) in the Mustard family as a model system to understand mechanisms of salt (NaCl) and cadmium uptake and tolerance, as it has been shown to accumulate both. As part of this study, we will hydroponically grow C. maritima in different stress treatments using salt and cadmium. As the plants uptake the pollutants, the conductivity of the solution will change. We will develop an automated pipeline to track these changes in real-time using conductivity sensors. In addition, we will sample root and leaf tissues at various time points to measure salt and cadmium uptake using ICP-OES elemental analysis. This data will provide insights into salt and cadmium uptake/tolerance and paves a path toward efficient and viable solutions improving phytoremediation approaches.

Predicting the composition of soybean seeds while the plants are growing in the field is very important to understand how different genotypes, field condition and environment influence different seed composition parameters. Knowing this information at global scale is even more important to understand the dynamics of food insecurity and the interaction of seed composition with global environmental changes. This study aims to develop a machine learning-based soybean seed composition model from the fusion of PlanetScope, Sentinel and Landsat satellite images. Although satellite images provide global coverage throughout the year, it suffers from coarser spatial resolution. However, PlanetScope provides four-band (i.e., red, green, blue, and near infrared) multispectral imageries at approximately 3m spatial resolution daily. Alternatively, Sentinel-2B and Landsat-8 have coarser spatial resolution (10 - 30m), they provide enriched spectral resolution. Therefore, the objectives of this study are to 1) fuse the PlanetScope image with corresponding Landsat and Sentinel images, 2) evaluate several machine learning algorithms (e.g., partial least squares, support vector machine, random forest, and deep neural network) to predict protein and oil content of soybean seeds from the fused satellite images. Two soybean fields were established in 2020 and 2021 at Bradford, MO to perform the experiment. Corresponding PlanetScope, Sentinel, and Landsat images were downloaded and processed for the entire growth seasons. Current results indicate that deep neural network provide the best performance in predicting both protein and oil content of soybean. Future step is to assess different fusion algorithms and predict seed composition at regional or global scale.

The field is not always easy for plant phenotyping using conventional phenotyping platforms due to the limited accessibility and regulated aviation area. Smartphone-triggered ground images were collected on wheat field that has a limited access to monitor growth conditions of four wheat varieties, Shinyoung (SY), Joseong (JS), Taewoo (TW), and Cheongwoo (CW). For field mapping during the growing season, six sets of the raw RGB images were acquired by a smartphone camera in an oblique view angle and processed to transform into nadir view images. A series of algorithms were developed to process the skewed tile images to straighten into the nadir images, align the deskewed images, and stitch them into a field image by detecting crop rows using Hough Transformation. Open-source software, iStitch, was developed to automate the algorithms in a batch process. Plot-level metrics were extracted to analyze plant growth of the wheat varieties using a gridding method for vegetation and leaf area indexes. The processed images resulted in the successful transformation and consistency of algorithms on image alignment and stitching. Plot-level analysis indicated that SY variety performed superior to the other varieties in plant quality and quantity and significantly different from TW variety in canopy coverage. The proposed approach of the stitching and gridding was applied on the skewed images acquired by a smartphone camera but can be directly used for other applications of plant phenotyping on images acquired by a camera on a mobile platform or a grid of stationary cameras in greenhouse or outdoor fields.

Knowing how many plants have germinated in a farmer’s or researcher’s field is central to crop management and research. Plants are commonly counted by walking the field and counting the plants in each row. This technique is labor-intensive, slow, and error-prone. Automating stand counts using RGB imagery from Unmanned Aerial Vehicles (UAV) is an obvious solution. We propose DeepMaizeCounter, a robust computational system that provides accurate stand counts for research and production fields from imagery captured by freely flown, inexpensive drones and processed with an inexpensive computer. DeepMaizeCounter exploits mosaics computed from RGB videos, using a YOLOv4 model that is trained to recognize seedling maize plants in the V2–V10 growth stages (approximately 10–40cm in height), singly or in groups of two and three plants, and determines its accuracy on these classes of seedlings. We evaluated DeepMaizeCounter against in-field and on-frame manual stand counts for a number of different maize lines in both nursery and production fields. DeepMaizeCounter can reliably distinguish corn from weeds and other grasses, counting only the maize. The network is light and able to run 175 test frames in 6 seconds, or 29 frames per second. This opens the prospect that DeepMaizeCounter can eventually be deployed on cheap platforms for real-time counting.

Cosegmentation is a recent and rapidly emerging and rapidly growing extension of segmentation, which aims to detect the common object(s) in a group of images. Current cosegmentation methods are ideal and effective only for certain dataset types with limited features that still produce errors making it difficult to obtain detailed metrics of object parts. We propose to build a unified, trainable framework that incorporates multiple features of a high-throughput dataset’s segmented images from multiple algorithms using cosegmentation. Specifically, we propose a novel Cosegmentation for Plant Phenotyping Network (CoPPNet) that utilizes a Fully Convolutional Neural Network with a K-Means Clustering feedback loop for optimal temporal loss. The results from this study will set the benchmark for a novel advancement in computer vision segmentation accuracy and plant phenomics to better understand a plant’s environmental interactions for maximal resilience and yield.

Traditionally, walnut kernel color is scored by a human on a four-point color scale ranging from extra light to Amber, developed by the California DFA. Two important reasons for using a high throughput machine vision system for kernel phenotyping is to obtain better quantitative resolution to use in molecular breeding and to avoid errors in human vision phenotyping. RGB and LAB values give much more depth to qualitative calculations, and machine data is far more consistent than human scoring. Previous studies on walnut kernel imaging utilized a thresholding-based approach to segment walnut kernels from their background, however, detection accuracy can be improved. In this study, we make changes to detection methods, primarily using PyTorch computer vision processing and an improved thresholding method to better segment walnut kernels. The PyTorch CNN pipeline allows us to use specific photos to train a model, and the model can segment photos without the need to figure out image thresholds. Our proposed thresholding method uses the magick package in R instead of the ImageJ macros that were previously used. After taking photos with a CVS, we used the CNN model as well as an R script to identify and segment kernels from the background. Our preliminary data shows that with enough training, the CNN model is more robust in edge cases where we have overlapping kernels or shifted images. This paper will focus on exploring the differences between the three thresholding methods, and use the best method for future breeding projects.