UPCommons. Portal del coneixement obert de la UPC
Publication
Contribución de congreso (ConferenceOutput)
CAPS: Cost-Aware ML Pipeline Selection
We present CAPS, a novel system for cost-aware pipeline selection in automated machine learning (AutoML). CAPS’ approach is orthogonal to the search strategies used by existing AutoML frameworks, enabling seamless integration with them. This integration benefits AutoML systems by reducing computational waste—time spent evaluating inefficient pipelines or exceeding time or memory constraints—through lightweight performance and cost estimation. To incorporate cost-awareness into AutoML, CAPS represents a set of pipelines as a directed hypergraph, estimates execution times for each function, and formulates the optimal subset selection as a constrained prize-collecting optimization problem. CAPS employs a simple yet effective greedy algorithm to approximate this optimal subset. Our evaluation shows that when CAPS prioritizes cost, it reduces waste up to 4×—saving up to 23 hours in 50 hours of pipeline evaluation. When CAPS balances the trade-off between performance and cost, it consistently achieves performance improvements of 5% to 15% over the state of the art within the same time budget across datasets. It also demonstrates that CAPS is compatible with complementary efficiency techniques such as early stopping and warm-starting, achieving speedups of up to 4.7×., This work has been partially supported by the H2020-MSCA-ITN2020 DEDS project (GA 955895), the EU Horizon Europe research and innovation programmes DataGEMS (GA 101188416), WiseFood (GA 101181895), and CREXDATA (GA 101092749), MLEvol (PID2024-156019OB-I00/AEI/10.13039/501100011033) , TALC (PID2023-152841/OA-I00/AEI/10.13039/501100011033) and by ERDF, EU. The computation infrastructure has been provided by the Consortium des Équipements de Calcul Intensif (CÉCI), funded by the Fonds de la Recherche Scientifique de Belgique (F.R.S.-FNRS) under Grant No. 2.5020.11 and by the Walloon Region., Peer Reviewed