Transfer learning in optimization: from surrogate modeling adaptation to morphology-aware control of soft robotics

Abstract

Machine learning and optimization are now widely used to address challenging problems in science and engineering. Their practical use, however, is often limited by the difficulty of collecting sufficient data. In many applications, each data point may require an expensive numerical simulation, a physical experiment, or a substantial amount of computation. Repeating this process whenever a new problem is encountered can therefore be prohibitively costly. Transfer learning offers a possible solution by allowing information obtained from previous, related problems to be reused. The main difficulty lies in determining which parts of the previously acquired knowledge remain useful and how they should be adapted to a new task.

This thesis examines transfer learning in optimization from this perspective, with particular emphasis on problems in which the source and target tasks are related but not identical. Two settings are studied. The first concerns surrogate-based black-box optimization, where a model built for one objective function is reused for another related objective function. The second concerns the co-design of soft robots, where changes in morphology can also change the structure of the corresponding control problem. Despite the differences between these applications, both involve the same fundamental question: how can useful information be retained when the problem to which it is applied changes?

The first part of the thesis focuses on transferring surrogate models between related optimization problems. The central idea is to describe the difference between source and target problems through transformations of the input space. The initial formulation considers affine transformations composed of rotation and translation and uses a small number of target evaluations to adapt a surrogate trained on the source problem. The method is first developed for differentiable surrogate models, with Gaussian process regression as the main example, and is later generalized to non-differentiable models such as random forests through derivative-free optimization. Its performance is studied on both synthetic benchmark functions and real-world optimization tasks. The framework is subsequently extended to include nonlinear input warping based on the Beta cumulative distribution function, allowing more complicated relationships between source and target landscapes to be modeled. The results show that adapting an existing surrogate can be advantageous when target data are scarce and the relationship between the two problems can be captured adequately. They also show that greater flexibility in the transformation model comes at a cost, since more expressive mappings can be harder to estimate reliably from limited observations. The second part of the thesis considers knowledge transfer in soft robotic co-design. In this setting, changes to the robot body can alter the number and arrangement of sensors and actuators, which makes direct reuse of conventional fixed-dimensional controllers difficult. To cope with these structural changes, a controller based on Graph Attention Networks is developed. A robot is represented as a graph, allowing the policy to operate on structural relationships rather than on a fixed ordering of inputs and outputs. This representation supports the inheritance of controller knowledge when new morphologies are generated during evolution. Together with evolutionary optimization and reinforcement learning, the proposed approach allows offspring robots to reuse and adapt information learned by their parents. Experiments on EvoGym show that this graph-based representation leads to more consistent and effective policy transfer than conventional multi-layer perceptron controllers. Taken together, the studies examine two related forms of knowledge reuse in optimization. In surrogate modeling, transfer is achieved by learning how one objective landscape is transformed into another. In soft robotics, transfer is enabled by using a representation that can remain applicable when the physical structure of the system changes. The findings suggest that successful transfer depends less on reusing an entire model unchanged than on identifying which information can remain valid across tasks and providing an appropriate mechanism for adapting the rest. By doing so, both surrogate models and control policies can be reused more efficiently, reducing the need to start from scratch for every new optimization problem.