Deep hybrid modeling of a HEK293 process: combining Long Short-Term
Memory (LSTM) networks with first principles equations
Abstract
In this paper, Long Short-Term Memory (LSTM) networks and multilayered
feedforward neural networks (FFNNs) were combined with first principles
equations in a hybrid workflow to describe human embryonic kidney 293
(HEK293) culture dynamics. Experimental data of 27 extracellular state
variables in 20 fed-batch HEK293 cultures were collected in a parallel
high throughput 250 mL cultivation system. The adaptive moment
estimation method (ADAM) with stochastic regularization and
cross-validation were employed for deep learning. A total of 784 hybrid
models with varying deep neural network (DNN) architectures, depths,
layers sizes and node activation functions were compared. In most
scenarios, hybrid LSTM models outperformed hybrid FFNN models in terms
of training and testing error. Hybrid LSTM models revealed to be less
sensitive to data resampling than FFNN hybrid models. As disadvantages,
Hybrid LSTM models are in general more complex (higher number of
parameters) and have a higher computation cost than FFNN hybrid models.
The hybrid model with the highest prediction accuracy consisted in a
LSTM network with 7 internal states connected in series with dynamic
material balance equations. This hybrid model correctly predicted the
dynamics of the 27 state variables (R 2=0.93 in the
test data set), including biomass, key substrates, amino acids and
metabolic by-products for around 10 cultivation days.