NFL定理在離散化Lipschitz函數集合上之探討

Abstract

No-Free-Lunch(NFL)定理指出當對所有問題作平均時，所有最佳化演算法的表現都是一致的，意即各個最佳化演算法的總體效能並無法定義孰優孰劣。然而NFL定理並不意味著泛用型最佳化演算法(general-purpose optimizers)無用武之地，只要問題存在有可供演算法利用的結構，仍有可能找到一最佳化演算法在某一問題集合上具有優勢。這份論文提出了一個問題的集合，稱之為離散化Lipschitz函數集合(discrete Lipschitz class，DLC)，且此一集合可視為透過規範搜尋空間鄰近區域的差值來模擬連續性。此論文探討了DLC和NFL定理間之關係，並證明一最佳化演算法subthreshold-seeker之推廣形式可在DLC上效能勝於random search。同時，此論文也設計了一抽樣測試法透過實驗驗證在一更實際之架構下subthreshold-seeker在DLC上之表現明顯優於random search。因此，這份論文說明了儘管最佳化演算法並無法同時對所有問題都有優異的效能，但仍然有可能在一廣泛且深具意義的問題集合上取得優勢。

The No-Free-Lunch theorem states that all algorithms have the identical performance on average over all functions and there is no algorithm able to outperform others on all problems. However, such a result does not imply that search heuristics or optimization algorithms are futile if we are more cautious with the applicability of these methods and the search space. In this paper, within the No-Free-Lunch framework, we firstly introduce the discrete Lipschitz class by transferring the Lipschitz functions, i.e., functions with bounded slope, as a measure to fulfill the notion of continuity in discrete functions. We then investigate the properties of the discrete Lipschitz class, generalize an algorithm called subthreshold-seeker for optimization, and show that the generalized subthreshold-seeker outperforms random search on this class. Finally, we propose a tractable sampling-test scheme to empirically demonstrate the superiority of the generalized subthreshold-seeker under practical configurations. This study concludes that there exists algorithms outperforming random search on the discrete Lipschitz class in both theoretical and practical aspects and indicates that the effectiveness of search heuristics may not be universal but still general in some broad sense.