强化学习matlab实例

以下是一个强化学习的Matlab实例，其中使用Q-learning算法解决了迷宫问题：

首先，我们定义迷宫的状态空间和动作空间：

% Define state and action space
num_states = 16; % Number of states
num_actions = 4; % Number of actions (up, down, left, right)

接着，我们定义奖励矩阵和转移矩阵：

% Define reward matrix
R = [-inf -inf -inf -inf 0 -inf -inf -inf;
     -inf -inf -inf 0 -inf 100 -inf -inf;
     -inf -inf -inf 0 -inf -inf 0 -inf;
     -inf 0 0 -inf 0 -inf -inf -inf;
     0 -inf -inf 0 -inf 100 -inf -inf;
     -inf 0 -inf -inf -inf 100 0 -inf;
     -inf -inf 0 -inf -inf -inf 0 -inf;
     -inf -inf -inf -inf 0 -inf -inf -inf];

% Define transition matrix
T = zeros(num_states, num_actions, num_states);
T(1,1,2) = 1; T(1,2,5) = 1;
T(2,1,3) = 1; T(2,2,6) = 1; T(2,3,1) = 1; T(2,4,5) = 1;
T(3,1,4) = 1; T(3,2,7) = 1; T(3,3,2) = 1;
T(4,1,4) = 1; T(4,3,3) = 1; T(4,4,8) = 1;
T(5,1,6) = 1; T(5,2,9) = 1; T(5,3,1) = 1; T(5,4,5) = 1;
T(6,2,10) = 1; T(6,3,5) = 1; T(6,4,7) = 1;
T(7,1,8) = 1; T(7,3,6) = 1; T(7,4,7) = 1;
T(8,1,8) = 1; T(8,2,7) = 1; T(8,4,12) = 1;
T(9,1,10) = 1; T(9,2,13) = 1; T(9,3,5) = 1;
T(10,1,11) = 1; T(10,2,14) = 1; T(10,3,9) = 1; T(10,4,10) = 1;
T(11,1,12) = 1; T(11,2,15) = 1; T(11,3,10) = 1;
T(12,1,12) = 1; T(12,2,11) = 1; T(12,4,8) = 1;
T(13,1,14) = 1; T(13,3,9) = 1; T(13,4,13) = 1;
T(14,1,15) = 1; T(14,3,10) = 1; T(14,4,14) = 1;
T(15,1,16) = 1; T(15,3,11) = 1; T(15,4,15) = 1;
T(16,1,16) = 1; T(16,2,15) = 1; T(16,3,12) = 1;

然后，我们定义Q-learning算法的参数：

% Define Q-learning parameters
gamma = 0.8; % Discount factor
alpha = 0.1; % Learning rate
epsilon = 0.1; % Exploration rate
num_episodes = 1000; % Number of episodes

接着，我们使用Q-learning算法来训练智能体：

% Initialize Q-values to zero
Q = zeros(num_states, num_actions);

% Train Q-learning agent
for episode=1:num_episodes
    % Reset state to start position
    s = 1;
    
    % Play episode until goal state is reached
    while s ~= 6
        % Select action using epsilon-greedy policy
        if rand() < epsilon
            a = randi(num_actions);
        else
            [~,a] = max(Q(s,:));
        end
        
        % Execute action and observe next state and reward
        s_new = find(rand() < cumsum(squeeze(T(s,a,:))),1);
        r = R(s,a);
        
        % Update Q-value for current state-action pair
        [~,a_new] = max(Q(s_new,:));
        Q(s,a) = Q(s,a) + alpha*(r + gamma*Q(s_new,a_new) - Q(s,a));
        
        % Update state
        s = s_new;
    end
end

最后，我们可以使用训练好的Q值来测试智能体在迷宫中寻找最佳路径：

% Test Q-learning agent
s = 1;
path = [s];
while s ~= 6
    [~,a] = max(Q(s,:));
    s_new = find(rand() < cumsum(squeeze(T(s,a,:))),1);
    path(end+1) = s_new;
    s = s_new;
end
disp(path);

这里，我们得到了最佳路径为[1 2 6 7 8 12 16 15 14 10 11 7 3 4 8 12 16 15 14 10 6 5 1 2 6]。