… To provide for learning when to select macro-actions, we extend the notion of the optimal
action-value function, Q*, to include macro-actions. That is, for each state s and macroaction m …

… We present a method for automatically constructing macro-actions from scratch from
primitive actions during the reinforcement learning process. The overall idea is to reinforce the …

… We show that using all of the detected macro-actions are not … wrong macro-actions may easily
worsen learning performance… In Section 2, we review background of reinforcement learning…

… macro actions to form a synergistic macro action ensemble, in which synergism exhibits
when the constituent macro actions … Ozawa, “A reinforcement learning model using macroactions …

… macro actions / options itself instead of a human supervisor programming them. Especially in
Reinforcement Learning, … a way of finding options in Reinforcement Learning based on the …

… Evaluations on benchmark problems and a larger domain demonstrate the advantage of
learning with macro-actions over primitive-actions and the scalability of our approaches. …

… Then, we provide a model of the environment permitting macro actions, which is a special
case of Semi-… In this work, we employ the former surrogate objective due to its better empirical …

… of macro actions with better performance on the learning task… shown that the welldefined
macro actions can help to reduce … [Roles of macro-action, Empirical Analysis, deep with macro]. …

… However, after the empirical analysis, we noticed a few shortcomings in our choice of penalty
per action switch, p. Let b denote the number of action switches on a path. We expect that …

… Empirical results demonstrate the superiority of our approaches in large multi-agent … to
allow agents to asynchronously learn and execute high-level policies over macro-actions. …