Browsing by Subject "reinforcement learning"

Sort by: Order: Results:

Now showing items 1-4 of 4
  • Huertas, Andres (Helsingin yliopisto, 2020)
    Investment funds are continuously looking for new technologies and ideas to enhance their results. Lately, with the success observed in other fields, wealth managers are taking a closes look at machine learning methods. Even if the use of ML is not entirely new in finance, leveraging new techniques has proved to be challenging and few funds succeed in doing so. The present work explores de usage of reinforcement learning algorithms for portfolio management for the stock market. It is well known the stochastic nature of stock and aiming to predict the market is unrealistic; nevertheless, the question of how to use machine learning to find useful patterns in the data that enable small market edges, remains open. Based on the ideas of reinforcement learning, a portfolio optimization approach is proposed. RL agents are trained to trade in a stock exchange, using portfolio returns as rewards for their RL optimization problem, thus seeking optimal resource allocation. For this purpose, a set of 68 stock tickers in the Frankfurt exchange market was selected, and two RL methods applied, namely Advantage Actor-Critic(A2C) and Proximal Policy Optimization (PPO). Their performance was compared against three commonly traded ETFs (exchange-traded funds) to asses the algorithm's ability to generate returns compared to real-life investments. Both algorithms were able to achieve positive returns in a year of testing( 5.4\% and 9.3\% for A2C and PPO respectively, a European ETF (VGK, Vanguard FTSE Europe Index Fund) for the same period, reported 9.0\% returns) as well as healthy risk-to-returns ratios. The results do not aim to be financial advice or trading strategies, but rather explore the potential of RL for studying small to medium size stock portfolios.
  • Hartmann, Hendrik; Pauli, Larissa K.; Janssen, Lieneke K.; Huhn, Sebastian; Ceglarek, Uta; Horstmann, Annette (2020)
    Obesity is associated with alterations in dopaminergic transmission and cognitive function. Rodent studies suggest that diets rich in saturated fat and refined sugars (HFS), as opposed to diets diets low in saturated fat and refined sugars (LFS), change the dopamine system independent of excessive body weight. However, the impact of HFS on the human brain has not been investigated. Here, we compared the effect of dietary dopamine depletion on dopamine-dependent cognitive task performance between two groups differing in habitual intake of dietary fat and sugar. Specifically, we used a double-blind within-subject cross-over design to compare the effect of acute phenylalanine/tyrosine depletion on a reinforcement learning and a working memory task, in two groups that are on opposite ends of the spectrum of self-reported HFS intake (low vs high intake: LFS vs HFS group). We tested 31 healthy young women matched for body mass index (mostly normal weight to overweight) and IQ. Depletion of peripheral precursors of dopamine reduced the working memory specific performance on the operation span task in the LFS, but not in the HFS group (P = 0.016). Learning from positive- and negative-reinforcement (probabilistic selection task) was increased in both diet groups after dopamine depletion (P = 0.049). As a secondary exploratory research question, we measured peripheral dopamine precursor availability (pDAP) at baseline as an estimate for central dopamine levels. The HFS group had a significantly higher pDAP at baseline compared to the LFS group (P = 0.025). Our data provide the first evidence indicating that the intake of HFS is associated with changes in dopamine precursor availability, which is suggestive of changes in central dopamine levels in humans. The observed associations are present in a sample of normal to overweight participants (ie, in the absence of obesity), suggesting that the consumption of a HFS might already be associated with altered behaviours. Alternatively, the effects of HFS diet and obesity might be independent.
  • Malo, Pekka; Tahvonen, Olli; Suominen, Antti; Back, Philipp; Viitasaari, Lauri (2021)
    We solve a stochastic high-dimensional optimal harvesting problem by using reinforcement learning algorithms developed for agents who learn an optimal policy in a sequential decision process through repeated experience. This approach produces optimal solutions without discretization of state and control variables. Our stand-level model includes mixed species, tree size structure, optimal harvest timing, choice between rotation and continuous cover forestry, stochasticity in stand growth, and stochasticity in the occurrence of natural disasters. The optimal solution or policy maps the system state to the set of actions, i.e., clear-cutting, thinning, or no harvest decisions as well as the intensity of thinning over tree species and size classes. The algorithm repeats the solutions for deterministic problems computed earlier with time-consuming methods. Optimal policy describes harvesting choices from any initial state and reveals how the initial thinning versus clear-cutting choice depends on the economic and ecological factors. Stochasticity in stand growth increases the diversity of species composition. Despite the high variability in natural regeneration, the optimal policy closely satisfies the certainty equivalence principle. The effect of natural disasters is similar to an increase in the interest rate, but in contrast to earlier results, this tends to change the management regime from rotation forestry to continuous cover management.
  • Kropotov, Ivan (Helsingin yliopisto, 2020)
    Reinforcement learning (RL) is a basic machine learning method, which has recently gained in popularity. As the field matures, RL methods are being applied on progressively more complex problems. This leads to need to design increasingly more complicated models, which are difficult to train and apply in practice. This thesis explores one potential way of solving the problem with large and slow RL models, which is using a modular approach to build the models. The idea behind this approach is to decompose the main task into smaller subtasks and have separate modules each of which concentrates on solving a single subtask. In more detail, the proposed agent will be built using the Q-decomposition algorithm, which provides a simple and robust algorithm for building modular RL agents. The problem we use as an example of usefulness of the modular approach is a simplified version of the video game Doom and we design a RL agent that learns to play it. The empirical results indicate that the proposed model is able to learn to play the simplified version of Doom on a reasonable level, but not perfectly. Additionally, we show that the proposed model might suffer from usage of too simple models for solving the subtasks. Nevertheless, taken as a whole the results and the experience of designing the agent show that the modular approach for RL is a promising way forward and warrants further exploration.