With the increasing volume of data generated in agriculture, developing guidelines for appropriate data sharing and management is essential. Founded in 2015, the AgBioData Consortium aims to identify the current major issues in data curation and management and ensure more Findable, Accessible, Interoperable, and Reusable (FAIR) data. In 2021 we received funding from the National Science Foundation for a Research Coordination Network grant, whose main goals are: (1) make recommendations and implementation plans for FAIR data in agriculture; (2) expand the AgBioData network by recruiting key stakeholders in agricultural research; (3) provide educational material to train researchers on FAIR data sharing; and (4) develop a roadmap for a sustainable genomic, genetic and breeding (GGB) Database Ecosystem. We have started by launching nine working groups covering different aspects of data management in agriculture and recruitment. We also plan new groups focused on sharing and archiving new types of data, such as those generated with high-throughput phenotyping platforms. As plant phenomics becomes a major part of agriculture, we foresee opportunities to collectively identify best data management practices in current and future phenomics repositories to ensure that phenomics data is FAIR from the start. We are welcoming new members with expertise in the field. If you are interested in joining, please visit our website (https://www.agbiodata.org).

Improving photosynthesis has been considered critical to increasing crop yield to meet food demands from a growing population. To achieve this goal, high-throughput phenotyping techniques are highly needed to explore both natural and genetic variation in photosynthetic performance among crop cultivars. Due to the non-invasive nature of hyperspectral imaging, there is an increasing use of hyperspectral imaging for phenotyping of photosynthesis or photosynthetic physiology. The use of hyperspectral sensors has resulted in the accumulation of large amounts of data, shifting the research efforts into efficiently mining spectral information for high-throughput phenotyping. In this presentation, we will introduce data pipelines developed to leverage proximal sensing platforms and data sources including both reflectance spectra and solar-induced fluorescence (SIF) for quantifying photosynthetic performance at the canopy level. Photosynthetic performance was represented by the maximum carboxylation rate (Vcmax) and the maximum electron transport rate (Jmax). The experiments were conducted using eleven tobacco cultivars grown in field conditions during 2017 and 2018 at Energy Farm at University of Illinois. Time-synchronized hyperspectral images from 400 to 900 nm and irradiance measurements of sunlight under clear-sky conditions were collected for capturing reflectance spectra and SIF (and SIF related parameters). Within 30 minutes of spectral measurements, ground-truth Vcmax and Jmax were obtained from portable leaf gas exchange system. Our results suggested both reflectance spectra and SIF can provide accurate estimations of Vcmax and Jmax. The presented data pipelines have potential to relieve bottleneck in phenotyping of photosynthesis for breeding cultivars of enhanced photosynthesis.

Developing crops with better root system architecture and nitrogen use efficiency can lead to resilient crops capable of sustaining productivity in both optimum and stress environments. The increase in soybean global demand and its use in biodiesel and new soy-based products calls for new soybean cultivars that have higher yields, better nutritional values or desirable traits for specific use. The soybean germplasm collection at the USDA is a valuable resource in discovering novel allelic variations. This collection has not been screened extensively for the root nodule and root system architecture traits. We have already established root phenotyping platforms in our laboratory and we are proposing to screen a diverse pool of the USDA soybean collection for nodules and root system architecture traits. Availability of SNP data for this collection will let us run genome-wide analysis and identify QTLs responsible for different root traits. We will then use hairy root transformation to knockout/down some of the candidate genes in the loci identified in earlier reports and new QTLs identified in this project. This project will help us in deciphering the phenotypic variation in soybean root traits. Ultimately, better understanding of the regulatory mechanism controlling these traits can help us in developing resilient crops and a sustainable cropping system.

Annotation plays an essential and important role in the performance of machine learning approaches. However, manually labeling objects accurately is often time-consuming, especially when processing large image data. To improve this process, we developed a deep learning-based interactive annotation approach with application to hyperspectral imagery. Specifically, a deep network with a metric-based loss function will be pre-trained on a large training set. Encoding layers in the network are used for dimensionality reduction as well as boosting discriminability among pixels belonging to different classes in the embedding space. Then, when new images are provided, the user can provide (fast, easy, weak) annotations to indicate objects of interest and fine-tune the pre-trained network to the task at hand. In this way, the fine-tuned network can adapt for the new category and task. Annotations can be iteratively modified and corrected. Once the model is fine-tuned by quickly annotating a few images, it can then be generalized to a large image dataset of similar objects. Experiments on the MUUFL Gulfport hyperspectral imaging dataset show that our approach outperforms classical active learning methods based on support vector machine in average Intersect of Union (from around 65% to around 80%) with significant improvement on computing speeds (from 30-200 seconds to less than 10 seconds)

Scene understanding methodologies (e.g., classification, segmentation, and anomaly detection) can be costly when operating on a per-pixel basis. As remote sensing applications rely on imagery with greater resolution, pixels regularly contain similar information to their neighbors. In recent years, scene understanding approaches have leveraged superpixel algorithms to partition imagery into small homogeneous regions for a variety of tasks. Instead of performing operations on millions of pixels, thousands, or in some cases hundreds of superpixels can be used as inputs into various pipelines. Most superpixel algorithms rely solely on RGB color information to produce superpixel maps. However, when multiple co-registered modalities are available, as is the case with the National Ecological Observatory Network (NEON) tree crown dataset, it is possible to combine information from multiple sources to produce a single shared map. In this work we combine airborne hyperspectral imagery, LiDAR point clouds, and RGB imagery to obtain superpixel maps and compare the oversegmentation results to those obtained by fusing individual maps produced from each modality. Superpixels are computed using the Simple Non-Iterative Clustering (SNIC) algorithm. Oversegmentation maps are scored using standard evaluation approaches. We present results on a subset of the National Ecological Observatory Network imagery dataset.

Plant growth is the product of gene by environment (GxE) interactions during plant development. Effective characterization of plant growth under various conditions provides insight into genetic components of plant development and mechanisms of stress resilience. While the emergence of high-through phenotyping facilities provides new avenues to further understand plant development and stress responses, the large costs of such facilities are hindering the study of dynamic growth processes. To democratize high-throughput plant phenotyping, we developed three sets of image-based phenotyping devices utilizing Raspberry Pi computers and low-weight/low-cost materials to continuously monitor shoot and root growth. The process is further automated by our workflows including data collection and statistical analysis. Our devices and workflows are customizable to image a wide variety of plants and tissues. To validate our system, we measured growth of Arabidopsis rosettes, tomato roots, and characterized the relationship between cowpea growth and evapotranspiration. These results demonstrate the variety of applications for Raspberry Pi based phenotyping. Importantly, this low-cost system is ideal for studying the genetics of plant growth and identifying new components of abiotic stress tolerance in a wide variety of species.

Agronomic technological advancements provide more precise means to establish methodologies that can estimate yield response in many different ways. We are experimenting with a new image-based technique to predict the yield response of Canola using the rate of ground cover accumulation. Our trial was composed of row spacing and seeding rate as factors that influence the growth and the spatial distribution to evaluate its influence on the yield. Using the Visible Band Difference Vegetation Index (VDVI) from digital images, we estimated the ground cover and modelled the change over time. We regressed ground cover accumulation and integrated the function to calculate the area under the curve to regress against yield. Preliminary analysis indicates that the green ground cover accumulation overtime is sufficiently correlated with the yield (F=168.1, p=2.2e-16, R2=0.4694). Further, our results suggest the amount of green ground cover accumulation over time is dependent on the seeding density and row spacing. The analysis shows the higher seeding densities, 40 plants/m2 and above, acquire biomass rapidly, and the most stable yield predictions with ground cover are likely reached at similar plant densities. The most stable yield predictions in-relation to row spacing obtained from either 0.3m, 0.45m or 0.6m spacing (R-squares 0.94, 0.93, and 0.89, respectively). We are further experimenting to understand what growth period of the crop is most suitable for ground cover based yield predictions. Our primary target is to develop a high throughput image-based methodology to estimate the yield response using the ground cover accumulation rate for on-farm precision agronomy.

An autonomous mobile ground-control point (AMGCP) was redesigned and refined for improved collaborative operation with an unmanned aerial vehicle (UAV) to enable calibration of image mosaics from multispectral (MS) and thermal cameras. The AMGCP has built-in reflectance panels and electronically controlled thermal panels that provide high and low reflectance and temperature references that can be used for calibration of reflectance measurements in MS images and temperature measurements in thermal-infrared images. The AMGCP also has an onboard temperature sensor that enables image-based temperature measurements to be compared to ambient temperature so that canopy temperature depression (CTD) can be calculated. The collaborative robotic system consists of the AMGCP and a UAV that have real-time kinematic (RTK) geographic positioning system (GPS) receivers onboard so their precise position can be determined in real time. The system also includes wireless communication capability between the AMGCP and UAV so they can transmit their position and other data to each other during a mission, in which the AMGCP positions itself at multiple locations under the flight path of the UAV, providing multiple instances of reflectance and temperature references in image mosaics collected by the UAV. Testing has shown that reflectance measurements can be calibrated to less than 1% reflectance error, and canopy temperatures of crop plants can be calibrated to within 1.0 C, enabling consistently accurate measurements to be made efficiently and without human intervention in various fields and regions and at different times and dates. This system is also suited to accurate measurement of CTD to facilitate genetic selection relative to various stresses and resilience characteristics like drought tolerance.

Knowing how many plants have germinated in a farmer’s or researcher’s field is central to crop management and research. Plants are commonly counted by walking the field and counting the plants in each row. This technique is labor-intensive, slow, and error-prone. Automating stand counts using RGB imagery from Unmanned Aerial Vehicles (UAV) is an obvious solution. We propose DeepMaizeCounter, a robust computational system that provides accurate stand counts for research and production fields from imagery captured by freely flown, inexpensive drones and processed with an inexpensive computer. DeepMaizeCounter exploits mosaics computed from RGB videos, using a YOLOv4 model that is trained to recognize seedling maize plants in the V2–V10 growth stages (approximately 10–40cm in height), singly or in groups of two and three plants, and determines its accuracy on these classes of seedlings. We evaluated DeepMaizeCounter against in-field and on-frame manual stand counts for a number of different maize lines in both nursery and production fields. DeepMaizeCounter can reliably distinguish corn from weeds and other grasses, counting only the maize. The network is light and able to run 175 test frames in 6 seconds, or 29 frames per second. This opens the prospect that DeepMaizeCounter can eventually be deployed on cheap platforms for real-time counting.

Plant growth and development is impacted by the ability to capture resources including sunlight, determined in part by the arrangement of plant parts throughout the canopy. This is a very complex trait to describe, but has a major impact on downstream traits such as biomass or grain yield per acre. Though some is known about genetic factors contributing to leaf angle, maturity, and leaf size and number, these discrete traits do not encompass the structural complexity of the canopy. In addition, modeling and prediction for plant developmental traits using genomics or phenomics are usually conducted separately. We have developed proof-of-concept models that incorporate spatio-temporal factors from drone-acquired LiDAR features in a maize diversity panel to predict plant growth and development over time to improve our understanding of the biology of canopy formation and development. Briefly, voxel models for probability of beam penetration into the foliage were generated from 3D LiDAR scans collected at seven dates throughout crop canopy development. From the same plots, key architectural features of the maize canopy were measured by hand: stand count; plant, tassel, and flag leaf height; anthesis and silking dates; ear leaf, total leaf, and largest leaf number; and largest leaf length and width. We develop a self-supervised autoencoding neural network architecture that separately encodes plant temporal growth patterns for individual genotypes and plant spatial distributions for each plot. Then, leveraging the resulting latent space encoding of the LiDAR scans, we train and demonstrate accurate prediction of hand-measured crop traits.

Current methods of root sampling typically only obtain small or incomplete sections of root systems and do not capture their true complexity. To facilitate the visualization and analysis of entire, full sized root systems of crop plants, mesocosm growth containers were developed with an internal volume of 45 ft3 (1.27 m3). Mesocosms allow for unconstrained root growth, excavation and preservation of 3-dimensional RSA, and modularity that facilitates the use of a variety of sensors. Sensors arrays monitoring matric potential, temperature and CO2 levels are buried in a grid formation at depths of 1.25, 2.75, & 4.25 ft to assess environmental fluxes at regular intervals. Additionally, 3-dimensional water availability can be measured using ERT inside of root mesocosms. Methods of 3D data visualization of fluxes were developed to allow for comparison with root architectural traits. Following harvest, the recovered root system can be digitally reconstructed through photogrammetry, which is an inexpensive method requiring only an appropriate studio space and a digital camera. Initial metrics inferred from the 3D models include root system biomass (occupied voxels), volume, flatness, convex hull volume and solidity with depth. Root systems are finally dissected and biomass measurements are made in a 3-dimensional matrix of the growth zone, while the crown is saved for X-ray CT analysis.

Improving nutrient and water uptake in crops is one of the major challenges to sustain a fast-growing population that faces increasingly nutrient limited soils. Root hairs, which are specialized epidermal cells, are important drivers of nutrient and water uptake from the soil. Microscopy provides a mean to record root hairs as digital images. However, due to their geometry and complex spatial arrangements quantifying root hairs in microscopy images manually remains a bottleneck. Manual selection of representative root hairs can result in inaccurate estimations of root hair traits and misrepresentation of root hair functions. We present a method to quantify phenotypes automatically by measuring all individual root hairs in digital microscopy images. Our method uses random forests classification to separate root hair from the parent root and the image background. We define metrics to evaluate segments of root hairs that intersect or form blobs of two or more root hairs. Using simulated annealing for combinatorial optimization, we reconstruct individual root hairs by resolving intersections in a globally optimal way. As a result, we measure root hair length, its distribution, and root hair density in each image. We validate our method on examples of three maize cultivars under phosphorus, nitrogen, and potassium stress. Results show that our measurements of root hair traits strongly correlate with manually measured validation data in mean root hair length (Pearson-correlation: 0.74 to 0.88, p<.001), as well as in root hair density (Pearson-correlation: 0.65 to 0.84, p<.001). We show that our method distinguishes subtle differences between genotypes and treatments based on the extracted traits and believe that our study paves a way towards identifying the genetic control of root hair traits and increased agricultural production.

Vegetation indices (VIs) derived from multi-spectral imaging (MSI) can be used to collect non-destructive phenotypes that could be used to better understand development curves and interactions with environmental factors throughout the growing season. To investigate the amount of variation present in VIs derived from MSI and their relationship with important end-of-season traits, genetic and residual (co)variances for the VIs and their genetic and residual correlations with grain yield and grain moisture were estimated using maize data collected as part of the Genomes to Fields (G2F) initiative. One of the VIs considered in this study was normalized difference vegetation index (NDVI). In addition to NDVI, cumulative NDVI (cNDVI) was used as a phenotype to explore methods to simultaneously fit multiple phenotypes from MSI collected throughout the growing season. The potential of random regression models were investigated using either linear Splines or Legendre polynomial functions. Low to moderately high heritability estimates (0.10 – 0.35) was observed for NDVI values at each of the time points within years, indicating that there exists a reasonable amount of genetic variation. Moreover, strong genetic and residual correlations were found between grain yield and NDVI. Finally, it was found that using random regression with either of the functions converged using all time points and show a potential to be used as an alternative to multi-trait models.

Preharvest seed composition estimation using satellite imaging provides critical information for food security planning and management at a regional scale. As one of the staple crops, soybean plays important role in U.S. economic development. Estimating soybean seed composition is the precondition for improving seed quality and meal content at scale, therefore, maintaining U.S. soy competence in international markets. Traditionally, soybean seed compositions are measured after harvest via wet chemistry analysis, which is time-consuming and expensive. This study presents very first, to the best of our knowledge, satellite remote sensing of soybean seed composition. We demonstrate that WorldView-3 satellite imagery and machine/deep learning is powerful tool to predict seed composition from standing crops.

High-throughput phenotyping and genotyping have provided a vast source of information for evaluating the genetic merit of different breeding materials, but their implementation has been limited in alfalfa due to the complexity of the genome and the perennial nature of the crop. Vegetative indices (VIs) collected from an unmanned aerial vehicle (UAV) equipped with multi-spectral camera can be used to study forage growth and development throughout each growth cycle. Random regression models could be implemented to fit such longitudinal phenotypes like VIs collected over time to estimate growth curves, to access genetic variation in growth and the relations of VIs to end-use traits like forage yield and quality. The main objectives of this project are (1) to incorporate aerial high-throughput phenotyping to predict performance and genetic merit of the breeding materials, (2) to fit longitudinal random regression model to estimate genotype-specific growth curves, and (3) to develop a genotyping approach to estimate genetic relationships between alfalfa populations. The imaging of the alfalfa experimental trials was done every ~ 4.3 days throughout the growing season. The Vegetative indices (VIs) close to the harvest date were extracted and used to fit multi-traits models to evaluate the genetic correlations between VIs and forage biomass yield. The VIs considered were Normalized Vegetative index (NDVI), Green NDVI, Red Edge NDVI, simple ratio of Near Infrared to Red (NIR), and Digital Surface Map (DSM). The preliminary results showed highest correlation of Green NDVI and biomass yield (0.4053, 0.7875, and 0.6779), followed by Rededge NDVI and biomass yield (0.417, 0.7898, and 0.6417) for the first, second and third cuttings respectively for the experimental trial located at Helfer, Ithaca. Heritability estimates ranging from 0.03 to 0.75 was observed indicating the presence of genetic variation in these VIs. Pairwise Fst values estimated from population-level genotyping approach was found to be efficient estimates of genetic relatedness between populations. Random regression models with a linear spline function and legendre polynomials including other environmental trials are under evaluation to see the potentiality of these models to fit VIs from multiple time points.

The expanding geographic range of Phyllachora maydis, the fungus that induces Tar Spot infection on corn foliage, is increasingly threatening a Michigan industry that contributes over $1 billion to the state’s economy annually. Advances in machine learning now enable quantification of crop infection presence and severity using powerful object detection packages such as Tensorflow, Keras, and more. Tensorflow, specifically, has developed Application Programming Interface (API) tools to connect powerful object detection capabilities with streamlined usability. Foliar infection of maize by P. maydis is often difficult to detect early. Visible lesions initially appear tiny, ambiguous, and sparse, making them difficult to identify with the naked eye. Both farmers and breeders of corn desperately need better tools that allow early, definitive detection of lesions and provide more time for management decisions. This tool must verify presence of P. maydis and quantify infection severity as quickly as possible to allow growers the most options for treatment. I propose a combination of supervised machine learning using Tensorflow for custom object detection, and containerized application-development software such as Docker to create a user interface accessible on desktop or mobile devices. This application will be developed by weaving the transferrable infrastructure of Docker with the powerful machine learning platforms Tensorflow and Tensorflow Lite, thereby allowing users to analyze images using their preferred operating system. By implementing both complementary Tensorflow platforms, farmers and breeders will be afforded the choice of either capturing and analyzing one image at a time, or detecting lesions continuously in real-time.

Cosegmentation is a recent and rapidly emerging and rapidly growing extension of segmentation, which aims to detect the common object(s) in a group of images. Current cosegmentation methods are ideal and effective only for certain dataset types with limited features that still produce errors making it difficult to obtain detailed metrics of object parts. We propose to build a unified, trainable framework that incorporates multiple features of a high-throughput dataset’s segmented images from multiple algorithms using cosegmentation. Specifically, we propose a novel Cosegmentation for Plant Phenotyping Network (CoPPNet) that utilizes a Fully Convolutional Neural Network with a K-Means Clustering feedback loop for optimal temporal loss. The results from this study will set the benchmark for a novel advancement in computer vision segmentation accuracy and plant phenomics to better understand a plant’s environmental interactions for maximal resilience and yield.

Focusing on non-destructive and automated acquisition of plant phenotypic parameters,this extended abstract proposed an end-to-end deep RNN based network structure for single perspective sparse raw point cloud regression task called DRN. It has been proven to achieve accuracy improvements in PointNet++ and PonitCNN when it comes to regression of lettuce plant height. We believe DRN structure is suitable for feature extraction from plant point cloud data and regression of spatial distance related plant phenotypes like plant height.