加強式學習控制系統應用於金融市場操作

Abstract

本論文視證券市場操作為一控制問題，由於股票價格模型建立困難或具有時變性質，且傳統控制理論皆基於完整受控系統動態行為的數學模式發展，所以大部份方法對證券市場的問題將很難獲得解決。為了克服上述問題，希望尋找一種不需受控模型並可隨時變受控系統改變的控制方法。加強式學習符合這項要求，在學習過程中不需要環境（市場）模型或優秀的教導者指導，而且控制器將隨環境變化自我改善表現，最後經由學習過程自動歸納出對應的操作策略。在實證部份，將這項方法應用於台灣證券市場，因證券價格訊號盤整期間長和頻率變化大，實驗結果顯示操作反應有延遲現象，不過長期獲率仍令人滿意，驗證加強式學習於金融市場操作具初步成果。

This thesis considers stock market speculation as a control problem. It is difficult to build price models of stocks because of the time-variant property of stocks. Further, conventional control theorem is largely based on the dynamic behavior of mathematical model of the plant underlying control. This implies that most known control methods are not able to solve the stock speculation problem. To deal with the proposed problem, we wish to find a control method that does not require a mathematical model of the controlled plant, and will adjust the scheme itself while the plant is time-varying. The reinforcement learning suits for these requirements. In the learning system, the controller improves its behavior by adjusting internal parameters according to the interaction with environment. In our experiments, we apply the proposed method to Taiwan stock market. Due to the consolidation of stock and its high versatility, the experimental results display the delayed response of operation, but the long-period profits are satisfactory. We conclude that reinforcement learning for financial market speculation has the preliminary success.