In this paper, Long Short-Term Memory (LSTM) networks and multilayered feedforward neural networks (FFNNs) were combined with first principles equations in a hybrid workflow to describe human embryonic kidney 293 (HEK293) culture dynamics. Experimental data of 27 extracellular state variables in 20 fed-batch HEK293 cultures were collected in a parallel high throughput 250 mL cultivation system. The adaptive moment estimation method (ADAM) with stochastic regularization and cross-validation were employed for deep learning. A total of 784 hybrid models with varying deep neural network (DNN) architectures, depths, layers sizes and node activation functions were compared. In most scenarios, hybrid LSTM models outperformed hybrid FFNN models in terms of training and testing error. Hybrid LSTM models revealed to be less sensitive to data resampling than FFNN hybrid models. As disadvantages, Hybrid LSTM models are in general more complex (higher number of parameters) and have a higher computation cost than FFNN hybrid models. The hybrid model with the highest prediction accuracy consisted in a LSTM network with 7 internal states connected in series with dynamic material balance equations. This hybrid model correctly predicted the dynamics of the 27 state variables (R 2=0.93 in the test data set), including biomass, key substrates, amino acids and metabolic by-products for around 10 cultivation days.

Due to the complexity of biological processes, developing model-based strategies for monitoring, optimization and control is nontrivial. Hybrid neural models, combining mechanistic modeling with artificial neural networks, have been reported as powerful tools for bioprocess applications. In this paper, a systematic literature review is presented focused on the application of hybrid neural models to bioprocesses by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) over the last 30 years. This analysis showed that hybrid neural modeling has covered a wide range of microbial processes, animal cells, mixed microbial cultures, and enzyme biocatalysis. Hybrid neural models have been mainly applied for predictive modeling/process analysis, process monitoring/software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, process analytical technology, quality-by-design, and more recently, digital twins. Hybrid modeling experienced a decline in the number of publications after a peak in 2004 and is now surging again. A “model scale” research gap was identified, which will likely narrow by a better integration with deep learning and systems biology in the near future. The biopharma sector is currently a major driver but applications to biologics quality attributes (e.g. glycosylation), new modalities and downstream unit operations are significant research gaps.

Flux balance analysis is currently the standard method to compute metabolic fluxes in genome-scale networks. Several variations employing diverse objective functions and/or constraints have been published. Here we propose a hybrid semi-parametric version of flux balance analysis that combines mechanistic-level constraints (parametric) with empirical constraints (non-parametric), at the genome-scale. A CHO dataset with 27 measured exchange fluxes obtained from 21 reactor experiments served to evaluate the method. The reduced CHO genome-scale model comprehended 686 metabolites, 788 reactions and 210 degrees of freedom. The experimental flux dataset could be compressed to 6 principal components retaining 93.7% of explained variance. The conjugation of both types of constraints is coded as a linear program with comparable computational cost as standard flux balance analysis. The hybrid flux balance analysis showed a significant reduction in the specific growth rate prediction error in comparison to the non-hybrid version. The hybrid method was eventually used to design a metabolically efficient feed to extend cell expansion from 9.87 Mcell/ml to 22.48 Mcell/ml at the point of induction with minimal accumulation of byproducts. It is concluded that the predictive advantage of the hybrid method resulted from the statistical abstraction of regulatory mechanisms, which were absent in the standard flux balance analysis.