q-learning算法栅格最短路径matlab代码

以下是一个简单的q-learning算法栅格最短路径的MATLAB代码示例：

clear all;
clc;

%定义初始状态和目标状态
start_state = [1,1];
goal_state = [10,10];

%定义行动空间和奖励矩阵
action_space = ['N','S','E','W'];
reward_matrix = -ones(10,10);
reward_matrix(goal_state(1),goal_state(2)) = 100;

%定义Q矩阵和学习参数
Q = zeros(10,10,4);
alpha = 0.5;
gamma = 0.9;
epsilon = 0.1;

%开始训练
for episode = 1:10000
    %将智能体放在起点
    current_state = start_state;
    while (~isequal(current_state,goal_state))
        %选择行动
        if rand < epsilon
            action = randi(4);
        else
            [~,action] = max(Q(current_state(1),current_state(2),:));
        end

        %执行行动
        next_state = current_state + [0,-1;0,1;1,0;-1,0](action,:);
        next_state = min(max(next_state,1),10);

        %更新Q矩阵
        Q(current_state(1),current_state(2),action) = (1-alpha)*Q(current_state(1),current_state(2),action) + alpha*(reward_matrix(next_state(1),next_state(2))+gamma*max(Q(next_state(1),next_state(2),:)));

        %更新当前状态
        current_state = next_state;
    end
end

%使用Q矩阵执行最短路径
current_state = start_state;
path = [current_state];
while (~isequal(current_state,goal_state))
    [~,action] = max(Q(current_state(1),current_state(2),:));
    next_state = current_state + [0,-1;0,1;1,0;-1,0](action,:);
    next_state = min(max(next_state,1),10);
    path = [path;next_state];
    current_state = next_state;
end

%绘制结果
figure();
plot(path(:,2),path(:,1),'-o');
grid on;
xlim([0.5 10.5]);
ylim([0.5 10.5]);
xlabel('X');
ylabel('Y');
title('Q-Learning Algorithm for Shortest Path Problem in Grid World');

该代码使用了一个简单的4个行动空间（北、南、东、西）和一个-1的奖励矩阵，表示无论走哪个方向都会受到一点惩罚。Q矩阵是一个三维矩阵，其中前两个维度表示状态，第三个维度表示行动。在训练过程中，智能体随机选择行动或者根据当前Q值选择最优行动，并更新Q矩阵。在执行最短路径时，智能体根据当前Q值选择最优行动，直到到达目标状态。最终，代码将路径可视化并绘制出来。