Algorithm 1: The online LyDROO algorithm for solving (P1). input : Parameters V , {γi, ci}Ni=1, K, training interval δT , Mt update interval δM ; output: Control actions 􏰄xt,yt􏰅Kt=1; 1 Initialize the DNN with random parameters θ1 and empty replay memory, M1 ← 2N; 2 Empty initial data queue Qi(1) = 0 and energy queue Yi(1) = 0, for i = 1,··· ,N; 3 fort=1,2,...,Kdo 4 Observe the input ξt = 􏰄ht, Qi(t), Yi(t)􏰅Ni=1 and update Mt using (8) if mod (t, δM ) = 0; 5 Generate a relaxed offloading action xˆt = Πθt 􏰀ξt􏰁 with the DNN; 6 Quantize xˆt into Mt binary actions 􏰄xti|i = 1, · · · , Mt􏰅 using the NOP method; 7 Compute G􏰀xti,ξt􏰁 by optimizing resource allocation yit in (P2) for each xti; 8 Select the best solution xt = arg max G 􏰀xti , ξt 􏰁 and execute the joint action 􏰀xt , yt 􏰁; { x ti } 9 Update the replay memory by adding (ξt,xt); 10 if mod (t, δT ) = 0 then 11 Uniformly sample a batch of data set {(ξτ , xτ ) | τ ∈ St } from the memory; 12 Train the DNN with {(ξτ , xτ ) | τ ∈ St} and update θt using the Adam algorithm; 13 end 14 t ← t + 1; 15 Update {Qi(t),Yi(t)}N based on 􏰀xt−1,yt−1􏰁 and data arrival observation 􏰒At−1􏰓N using (5) and (7). i=1 i i=1 16 end，算法的输入输出是什么

输入：参数V，{γi,ci}Ni=1，K，训练间隔δT，Mt更新间隔δM。
输出：控制动作􏰄xt,yt􏰅Kt=1。

其中，V是一个参数集合，{γi,ci}是关于每个用户i的重要性权重和计算任务的数据量，K是迭代次数，δT是训练间隔，δM是Mt更新间隔。算法的输出是K次迭代后的控制动作􏰄xt,yt􏰅Kt=1。