AI systems increasingly shape critical decisions across personal and societal domains. While empirical risk minimization (ERM) drives much of the AI's success, it typically prioritizes accuracy over trustworthiness, often resulting in biases, opacity, and other adverse effects. This paper discusses how key requirements for trustworthy AI can be translated into design choices for the components of ERM. We hope to provide actionable guidance for building AI systems that meet emerging standards for trustworthiness of AI.

We propose networked exponential families for non-parametric machine learning from massive network-structured datasets (“big data over networks”). High-dimensional data points are interpreted as the realizations of a random process distributed according to some exponential family. Networked exponential families allow to jointly leverage the information contained in high-dimensional data points and their network structure. For data points representing individuals, we obtain perfectly personalized models which enable high-precision medicine or more general recommendation systems.We learn the parameters of networked exponential families, using the network Lasso which implicitly pools (or clusters) the data points according to the intrinsic network structure and a local likelihood function. Our main theoretical result characterizes how the accuracy of network Lasso depends on the network structure and the information geometry of the node-wise exponential families. The network Lasso can be implemented as highly scalable message-passing over the data network. Such message passing is appealing for federated machine learning relying on edge computing. The proposed method is also privacy preserving in the sense that no raw data but only parameter (estimates) are shared among different nodes.

We present a model-agnostic federated learning method for decentralized data with an intrinsic network structure.The network structure reflects similarities between the (statistics of) local datasets and, in turn, their associated local models. Our method is an instance of empirical risk minimization, using a regularization term that is constructed from the network structure of data. In particular, we require well-connected local models, forming clusters, to yield similar predictions on a common test set. In principle our method can be applied to any collection of local models. The only restriction put on these local models is that they allow for efficient implementation of regularized empirical risk minimization (training). Such implementations might be available in the form of high-level programming frameworks such as scikit-learn, Keras or PyTorch.

we characterize the network structure of data such that federated mulit-task learning is possible.