{"id":73917,"date":"2026-02-08T18:09:52","date_gmt":"2026-02-08T10:09:52","guid":{"rendered":"https:\/\/www.wsisp.com\/helps\/73917.html"},"modified":"2026-02-08T18:09:52","modified_gmt":"2026-02-08T10:09:52","slug":"cann%e5%8a%a0%e9%80%9f%e5%bc%ba%e5%8c%96%e5%ad%a6%e4%b9%a0%e6%8e%a8%e7%90%86%ef%bc%9a%e7%ad%96%e7%95%a5%e7%bd%91%e7%bb%9c%e4%b8%8e%e4%bb%b7%e5%80%bc%e7%bd%91%e7%bb%9c%e4%bc%98%e5%8c%96","status":"publish","type":"post","link":"https:\/\/www.wsisp.com\/helps\/73917.html","title":{"rendered":"CANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316"},"content":{"rendered":"

\u5f3a\u5316\u5b66\u4e60（Reinforcement Learning，RL）\u662f\u4e00\u79cd\u901a\u8fc7\u4e0e\u73af\u5883\u4ea4\u4e92\u5b66\u4e60\u6700\u4f18\u7b56\u7565\u7684\u673a\u5668\u5b66\u4e60\u65b9\u6cd5\u3002RL\u5728\u6e38\u620fAI\u3001\u673a\u5668\u4eba\u63a7\u5236\u3001\u81ea\u52a8\u9a7e\u9a76\u3001\u63a8\u8350\u7cfb\u7edf\u7b49\u9886\u57df\u6709\u7740\u5e7f\u6cdb\u7684\u5e94\u7528\u3002RL\u63a8\u7406\u7684\u6838\u5fc3\u662f\u7b56\u7565\u7f51\u7edc\u548c\u4ef7\u503c\u7f51\u7edc\u7684\u524d\u5411\u4f20\u64ad，\u9700\u8981\u5feb\u901f\u54cd\u5e94\u73af\u5883\u72b6\u6001\u5e76\u8f93\u51fa\u52a8\u4f5c，\u5bf9\u63a8\u7406\u901f\u5ea6\u8981\u6c42\u6781\u9ad8\u3002CANN\u9488\u5bf9\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\u63a8\u51fa\u4e86\u5168\u9762\u7684\u4f18\u5316\u65b9\u6848，\u901a\u8fc7\u7b56\u7565\u7f51\u7edc\u4f18\u5316\u3001\u4ef7\u503c\u7f51\u7edc\u4f18\u5316\u548c\u52a8\u4f5c\u9009\u62e9\u4f18\u5316，\u663e\u8457\u63d0\u5347\u4e86RL\u63a8\u7406\u7684\u6027\u80fd\u548c\u54cd\u5e94\u901f\u5ea6\u3002<\/p>\n

\n

\u4e00\u3001\u5f3a\u5316\u5b66\u4e60\u67b6\u6784\u6df1\u5ea6\u89e3\u6790<\/h3>\n

1.1 \u6838\u5fc3\u539f\u7406\u6982\u8ff0<\/h4>\n

\u5f3a\u5316\u5b66\u4e60\u7684\u6838\u5fc3\u662f\u901a\u8fc7\u667a\u80fd\u4f53\u4e0e\u73af\u5883\u7684\u4ea4\u4e92，\u5b66\u4e60\u72b6\u6001\u5230\u52a8\u4f5c\u7684\u6620\u5c04\u7b56\u7565\u3002\u5e38\u89c1\u7684RL\u7b97\u6cd5\u5305\u62ecDQN（Deep Q-Network）\u3001PPO（Proximal Policy Optimization）\u3001A3C（Asynchronous Advantage Actor-Critic）\u7b49\u3002DQN\u4f7f\u7528Q\u7f51\u7edc\u5b66\u4e60\u52a8\u4f5c\u4ef7\u503c，PPO\u4f7f\u7528\u7b56\u7565\u68af\u5ea6\u65b9\u6cd5，A3C\u4f7f\u7528\u5f02\u6b65\u7684Actor-Critic\u67b6\u6784\u3002<\/p>\n

RL\u63a8\u7406\u6d41\u7a0b：<\/p>\n

\u73af\u5883\u72b6\u6001
\n \u2193
\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510
\n\u2502 \u72b6\u6001\u7f16\u7801 \u2502 \u2192 \u7f16\u7801\u73af\u5883\u72b6\u6001
\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518
\n \u2193
\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510
\n\u2502 \u7b56\u7565\u7f51\u7edc \u2502 \u2192 \u8f93\u51fa\u52a8\u4f5c\u6982\u7387\u5206\u5e03
\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518
\n \u2193
\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510
\n\u2502 \u4ef7\u503c\u7f51\u7edc \u2502 \u2192 \u8bc4\u4f30\u72b6\u6001\u4ef7\u503c
\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518
\n \u2193
\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510
\n\u2502 \u52a8\u4f5c\u9009\u62e9 \u2502 \u2192 \u9009\u62e9\u6700\u4f18\u52a8\u4f5c
\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518
\n \u2193
\n \u6267\u884c\u52a8\u4f5c<\/p>\n

1.2 RL\u7b97\u6cd5\u5bf9\u6bd4<\/h4>\n

\u4e0d\u540c\u7684RL\u7b97\u6cd5\u6709\u4e0d\u540c\u7684\u7279\u70b9\u548c\u9002\u7528\u573a\u666f，CANN\u652f\u6301\u591a\u79cdRL\u7b97\u6cd5，\u5e76\u6839\u636e\u5e94\u7528\u573a\u666f\u9009\u62e9\u6700\u4f18\u7b97\u6cd5\u3002<\/p>\n

RL\u7b97\u6cd5\u5bf9\u6bd4：<\/p>\n\n\u7b97\u6cd5\u7c7b\u578b\u7b56\u7565\u7c7b\u578b\u6837\u672c\u6548\u7387\u9002\u7528\u573a\u666f<\/tr>\n\n\n\n\n\n
DQN<\/td>\nValue-based<\/td>\n\u79bb\u6563\u7b56\u7565<\/td>\n\u4f4e<\/td>\n\u79bb\u6563\u52a8\u4f5c\u7a7a\u95f4<\/td>\n<\/tr>\n
PPO<\/td>\nPolicy-based<\/td>\n\u8fde\u7eed\/\u79bb\u6563<\/td>\n\u9ad8<\/td>\n\u901a\u7528\u573a\u666f<\/td>\n<\/tr>\n
A3C<\/td>\nActor-Critic<\/td>\n\u8fde\u7eed\/\u79bb\u6563<\/td>\n\u4e2d\u7b49<\/td>\n\u5e76\u884c\u8bad\u7ec3<\/td>\n<\/tr>\n
SAC<\/td>\nActor-Critic<\/td>\n\u8fde\u7eed<\/td>\n\u5f88\u9ad8<\/td>\n\u8fde\u7eed\u52a8\u4f5c\u7a7a\u95f4<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\n

\u4e8c\u3001\u7b56\u7565\u7f51\u7edc\u4f18\u5316<\/h3>\n

2.1 Actor\u7f51\u7edc\u4f18\u5316<\/h4>\n

Actor\u7f51\u7edc（\u7b56\u7565\u7f51\u7edc）\u8d1f\u8d23\u6839\u636e\u72b6\u6001\u8f93\u51fa\u52a8\u4f5c\u5206\u5e03，CANN\u901a\u8fc7\u4f18\u5316Actor\u7f51\u7edc，\u63d0\u9ad8\u7b56\u7565\u63a8\u7406\u6548\u7387\u3002<\/p>\n

Actor\u7f51\u7edc\u4f18\u5316\u5b9e\u73b0<\/h5>\n

import<\/span> numpy as<\/span> np
\nfrom<\/span> typing import<\/span> Tuple,<\/span> List,<\/span> Optional,<\/span> Dict<\/p>\n

class<\/span> PolicyNetwork<\/span>:<\/span>
\n """
\n \u7b56\u7565\u7f51\u7edc（Actor）<\/p>\n

Attributes:
\n state_dim: \u72b6\u6001\u7ef4\u5ea6
\n action_dim: \u52a8\u4f5c\u7ef4\u5ea6
\n hidden_dims: \u9690\u85cf\u5c42\u7ef4\u5ea6\u5217\u8868
\n action_type: \u52a8\u4f5c\u7c7b\u578b ('discrete' or 'continuous')
\n activation: \u6fc0\u6d3b\u51fd\u6570\u7c7b\u578b
\n """<\/span><\/p>\n

def<\/span> __init__<\/span>(<\/span>
\n self,<\/span>
\n state_dim:<\/span> int<\/span>,<\/span>
\n action_dim:<\/span> int<\/span>,<\/span>
\n hidden_dims:<\/span> List[<\/span>int<\/span>]<\/span> =<\/span> [<\/span>256<\/span>,<\/span> 256<\/span>]<\/span>,<\/span>
\n action_type:<\/span> str<\/span> =<\/span> 'discrete'<\/span>,<\/span>
\n activation:<\/span> str<\/span> =<\/span> 'relu'<\/span>
\n )<\/span>:<\/span>
\n """
\n \u521d\u59cb\u5316\u7b56\u7565\u7f51\u7edc<\/p>\n

Args:
\n state_dim: \u72b6\u6001\u7ef4\u5ea6
\n action_dim: \u52a8\u4f5c\u7ef4\u5ea6
\n hidden_dims: \u9690\u85cf\u5c42\u7ef4\u5ea6\u5217\u8868
\n action_type: \u52a8\u4f5c\u7c7b\u578b
\n activation: \u6fc0\u6d3b\u51fd\u6570\u7c7b\u578b
\n """<\/span>
\n self.<\/span>state_dim =<\/span> state_dim
\n self.<\/span>action_dim =<\/span> action_dim
\n self.<\/span>hidden_dims =<\/span> hidden_dims
\n self.<\/span>action_type =<\/span> action_type
\n self.<\/span>activation =<\/span> activation<\/p>\n

# \u521d\u59cb\u5316\u6743\u91cd<\/span>
\n self.<\/span>weights =<\/span> self.<\/span>_initialize_weights(<\/span>)<\/span><\/p>\n

def<\/span> _initialize_weights<\/span>(<\/span>self)<\/span> –<\/span>><\/span> dict<\/span>:<\/span>
\n """
\n \u521d\u59cb\u5316\u6743\u91cd<\/p>\n

Returns:
\n \u6743\u91cd\u5b57\u5178
\n """<\/span>
\n weights =<\/span> {<\/span>}<\/span><\/p>\n

# \u6784\u5efa\u7f51\u7edc\u5c42<\/span>
\n in_dim =<\/span> self.<\/span>state_dim
\n for<\/span> i,<\/span> out_dim in<\/span> enumerate<\/span>(<\/span>self.<\/span>hidden_dims)<\/span>:<\/span>
\n # \u7ebf\u6027\u5c42<\/span>
\n weights[<\/span>f'fc<\/span>{<\/span>i}<\/span><\/span>'<\/span><\/span>]<\/span> =<\/span> np.<\/span>random.<\/span>randn(<\/span>
\n in_dim,<\/span> out_dim
\n )<\/span>.<\/span>astype(<\/span>np.<\/span>float32)<\/span> *<\/span> 0.02<\/span><\/p>\n

# \u6279\u5f52\u4e00\u5316\u53c2\u6570<\/span>
\n weights[<\/span>f'bn<\/span>{<\/span>i}<\/span><\/span>_gamma'<\/span><\/span>]<\/span> =<\/span> np.<\/span>ones(<\/span>out_dim,<\/span> dtype=<\/span>np.<\/span>float32)<\/span>
\n weights[<\/span>f'bn<\/span>{<\/span>i}<\/span><\/span>_beta'<\/span><\/span>]<\/span> =<\/span> np.<\/span>zeros(<\/span>out_dim,<\/span> dtype=<\/span>np.<\/span>float32)<\/span><\/p>\n

in_dim =<\/span> out_dim<\/p>\n

# \u8f93\u51fa\u5c42<\/span>
\n if<\/span> self.<\/span>action_type ==<\/span> 'discrete'<\/span>:<\/span>
\n weights[<\/span>'output'<\/span>]<\/span> =<\/span> np.<\/span>random.<\/span>randn(<\/span>
\n in_dim,<\/span> self.<\/span>action_dim
\n )<\/span>.<\/span>astype(<\/span>np.<\/span>float32)<\/span> *<\/span> 0.02<\/span>
\n else<\/span>:<\/span> # continuous<\/span>
\n # \u8fde\u7eed\u52a8\u4f5c：\u8f93\u51fa\u5747\u503c\u548c\u6807\u51c6\u5dee<\/span>
\n weights[<\/span>'mean'<\/span>]<\/span> =<\/span> np.<\/span>random.<\/span>randn(<\/span>
\n in_dim,<\/span> self.<\/span>action_dim
\n )<\/span>.<\/span>astype(<\/span>np.<\/span>float32)<\/span> *<\/span> 0.02<\/span>
\n weights[<\/span>'log_std'<\/span>]<\/span> =<\/span> np.<\/span>zeros(<\/span>
\n self.<\/span>action_dim,<\/span> dtype=<\/span>np.<\/span>float32
\n )<\/span><\/p>\n

return<\/span> weights<\/p>\n

def<\/span> forward<\/span>(<\/span>
\n self,<\/span>
\n state:<\/span> np.<\/span>ndarray
\n )<\/span> –<\/span>><\/span> Dict[<\/span>str<\/span>,<\/span> np.<\/span>ndarray]<\/span>:<\/span>
\n """
\n \u524d\u5411\u4f20\u64ad<\/p>\n

Args:
\n state: \u73af\u5883\u72b6\u6001 [batch_size, state_dim]<\/p>\n

Returns:
\n \u7b56\u7565\u8f93\u51fa\u5b57\u5178
\n """<\/span>
\n x =<\/span> state<\/p>\n

# \u901a\u8fc7\u9690\u85cf\u5c42<\/span>
\n for<\/span> i in<\/span> range<\/span>(<\/span>len<\/span>(<\/span>self.<\/span>hidden_dims)<\/span>)<\/span>:<\/span>
\n # \u7ebf\u6027\u53d8\u6362<\/span>
\n x =<\/span> np.<\/span>dot(<\/span>x,<\/span> self.<\/span>weights[<\/span>f'fc<\/span>{<\/span>i}<\/span><\/span>'<\/span><\/span>]<\/span>)<\/span><\/p>\n

# \u6279\u5f52\u4e00\u5316<\/span>
\n x =<\/span> self.<\/span>_batch_norm(<\/span>
\n x,<\/span>
\n self.<\/span>weights[<\/span>f'bn<\/span>{<\/span>i}<\/span><\/span>_gamma'<\/span><\/span>]<\/span>,<\/span>
\n self.<\/span>weights[<\/span>f'bn<\/span>{<\/span>i}<\/span><\/span>_beta'<\/span><\/span>]<\/span>
\n )<\/span><\/p>\n

# \u6fc0\u6d3b\u51fd\u6570<\/span>
\n if<\/span> self.<\/span>activation ==<\/span> 'relu'<\/span>:<\/span>
\n x =<\/span> np.<\/span>maximum(<\/span>0<\/span>,<\/span> x)<\/span>
\n elif<\/span> self.<\/span>activation ==<\/span> 'tanh'<\/span>:<\/span>
\n x =<\/span> np.<\/span>tanh(<\/span>x)<\/span>
\n elif<\/span> self.<\/span>activation ==<\/span> 'sigmoid'<\/span>:<\/span>
\n x =<\/span> 1.0<\/span> \/<\/span> (<\/span>1.0<\/span> +<\/span> np.<\/span>exp(<\/span>–<\/span>x)<\/span>)<\/span><\/p>\n

# \u8f93\u51fa\u5c42<\/span>
\n if<\/span> self.<\/span>action_type ==<\/span> 'discrete'<\/span>:<\/span>
\n logits =<\/span> np.<\/span>dot(<\/span>x,<\/span> self.<\/span>weights[<\/span>'output'<\/span>]<\/span>)<\/span>
\n action_probs =<\/span> self.<\/span>_softmax(<\/span>logits)<\/span><\/p>\n

return<\/span> {<\/span>
\n 'logits'<\/span>:<\/span> logits,<\/span>
\n 'action_probs'<\/span>:<\/span> action_probs
\n }<\/span>
\n else<\/span>:<\/span> # continuous<\/span>
\n mean =<\/span> np.<\/span>dot(<\/span>x,<\/span> self.<\/span>weights[<\/span>'mean'<\/span>]<\/span>)<\/span>
\n log_std =<\/span> self.<\/span>weights[<\/span>'log_std'<\/span>]<\/span>
\n std =<\/span> np.<\/span>exp(<\/span>log_std)<\/span><\/p>\n

return<\/span> {<\/span>
\n 'mean'<\/span>:<\/span> mean,<\/span>
\n 'std'<\/span>:<\/span> std
\n }<\/span><\/p>\n

def<\/span> sample_action<\/span>(<\/span>
\n self,<\/span>
\n state:<\/span> np.<\/span>ndarray,<\/span>
\n deterministic:<\/span> bool<\/span> =<\/span> False<\/span>
\n )<\/span> –<\/span>><\/span> Tuple[<\/span>np.<\/span>ndarray,<\/span> Dict]<\/span>:<\/span>
\n """
\n \u91c7\u6837\u52a8\u4f5c<\/p>\n

Args:
\n state: \u73af\u5883\u72b6\u6001 [state_dim]
\n deterministic: \u662f\u5426\u4f7f\u7528\u786e\u5b9a\u6027\u7b56\u7565<\/p>\n

Returns:
\n (\u52a8\u4f5c, \u4fe1\u606f\u5b57\u5178)
\n """<\/span>
\n # \u524d\u5411\u4f20\u64ad<\/span>
\n if<\/span> state.<\/span>ndim ==<\/span> 1<\/span>:<\/span>
\n state =<\/span> state[<\/span>np.<\/span>newaxis,<\/span> :<\/span>]<\/span><\/p>\n

output =<\/span> self.<\/span>forward(<\/span>state)<\/span><\/p>\n

if<\/span> self.<\/span>action_type ==<\/span> 'discrete'<\/span>:<\/span>
\n action_probs =<\/span> output[<\/span>'action_probs'<\/span>]<\/span>[<\/span>0<\/span>]<\/span><\/p>\n

if<\/span> deterministic:<\/span>
\n action =<\/span> np.<\/span>argmax(<\/span>action_probs)<\/span>
\n else<\/span>:<\/span>
\n action =<\/span> np.<\/span>random.<\/span>choice(<\/span>self.<\/span>action_dim,<\/span> p=<\/span>action_probs)<\/span><\/p>\n

info =<\/span> {<\/span>
\n 'action_probs'<\/span>:<\/span> action_probs,<\/span>
\n 'log_prob'<\/span>:<\/span> np.<\/span>log(<\/span>action_probs[<\/span>action]<\/span> +<\/span> 1e-8<\/span>)<\/span>
\n }<\/span>
\n else<\/span>:<\/span> # continuous<\/span>
\n mean =<\/span> output[<\/span>'mean'<\/span>]<\/span>[<\/span>0<\/span>]<\/span>
\n std =<\/span> output[<\/span>'std'<\/span>]<\/span>[<\/span>0<\/span>]<\/span><\/p>\n

if<\/span> deterministic:<\/span>
\n action =<\/span> mean
\n else<\/span>:<\/span>
\n action =<\/span> mean +<\/span> std *<\/span> np.<\/span>random.<\/span>randn(<\/span>self.<\/span>action_dim)<\/span><\/p>\n

# \u8ba1\u7b97\u5bf9\u6570\u6982\u7387<\/span>
\n log_prob =<\/span> –<\/span>0.5<\/span> *<\/span> np.<\/span>sum<\/span>(<\/span>
\n (<\/span>(<\/span>action –<\/span> mean)<\/span> \/<\/span> (<\/span>std +<\/span> 1e-8<\/span>)<\/span>)<\/span> **<\/span> 2<\/span> +<\/span>
\n 2<\/span> *<\/span> np.<\/span>log(<\/span>std +<\/span> 1e-8<\/span>)<\/span>
\n )<\/span><\/p>\n

info =<\/span> {<\/span>
\n 'mean'<\/span>:<\/span> mean,<\/span>
\n 'std'<\/span>:<\/span> std,<\/span>
\n 'log_prob'<\/span>:<\/span> log_prob
\n }<\/span><\/p>\n

return<\/span> action,<\/span> info<\/p>\n

def<\/span> _batch_norm<\/span>(<\/span>
\n self,<\/span>
\n x:<\/span> np.<\/span>ndarray,<\/span>
\n gamma:<\/span> np.<\/span>ndarray,<\/span>
\n beta:<\/span> np.<\/span>ndarray,<\/span>
\n eps:<\/span> float<\/span> =<\/span> 1e-5<\/span>
\n )<\/span> –<\/span>><\/span> np.<\/span>ndarray:<\/span>
\n """
\n \u6279\u5f52\u4e00\u5316<\/p>\n

Args:
\n x: \u8f93\u5165 [batch_size, features]
\n gamma: \u7f29\u653e\u53c2\u6570 [features]
\n beta: \u504f\u79fb\u53c2\u6570 [features]
\n eps: \u5c0f\u5e38\u6570<\/p>\n

Returns:
\n \u5f52\u4e00\u5316\u540e\u7684\u8f93\u51fa
\n """<\/span>
\n mean =<\/span> np.<\/span>mean(<\/span>x,<\/span> axis=<\/span>0<\/span>,<\/span> keepdims=<\/span>True<\/span>)<\/span>
\n var =<\/span> np.<\/span>var(<\/span>x,<\/span> axis=<\/span>0<\/span>,<\/span> keepdims=<\/span>True<\/span>)<\/span><\/p>\n

x_norm =<\/span> (<\/span>x –<\/span> mean)<\/span> \/<\/span> np.<\/span>sqrt(<\/span>var +<\/span> eps)<\/span>
\n output =<\/span> gamma *<\/span> x_norm +<\/span> beta<\/p>\n

return<\/span> output<\/p>\n

def<\/span> _softmax<\/span>(<\/span>
\n self,<\/span>
\n x:<\/span> np.<\/span>ndarray,<\/span>
\n axis:<\/span> int<\/span> =<\/span> –<\/span>1<\/span>
\n )<\/span> –<\/span>><\/span> np.<\/span>ndarray:<\/span>
\n """
\n Softmax\u51fd\u6570<\/p>\n

Args:
\n x: \u8f93\u5165
\n axis: \u5f52\u4e00\u5316\u8f74<\/p>\n

Returns:
\n Softmax\u8f93\u51fa
\n """<\/span>
\n exp_x =<\/span> np.<\/span>exp(<\/span>x –<\/span> np.<\/span>max<\/span>(<\/span>x,<\/span> axis=<\/span>axis,<\/span> keepdims=<\/span>True<\/span>)<\/span>)<\/span>
\n return<\/span> exp_x \/<\/span> np.<\/span>sum<\/span>(<\/span>exp_x,<\/span> axis=<\/span>axis,<\/span> keepdims=<\/span>True<\/span>)<\/span><\/p>\n

class<\/span> ValueNetwork<\/span>:<\/span>
\n """
\n \u4ef7\u503c\u7f51\u7edc（Critic）<\/p>\n

Attributes:
\n state_dim: \u72b6\u6001\u7ef4\u5ea6
\n hidden_dims: \u9690\u85cf\u5c42\u7ef4\u5ea6\u5217\u8868
\n activation: \u6fc0\u6d3b\u51fd\u6570\u7c7b\u578b
\n """<\/span><\/p>\n

def<\/span> __init__<\/span>(<\/span>
\n self,<\/span>
\n state_dim:<\/span> int<\/span>,<\/span>
\n hidden_dims:<\/span> List[<\/span>int<\/span>]<\/span> =<\/span> [<\/span>256<\/span>,<\/span> 256<\/span>]<\/span>,<\/span>
\n activation:<\/span> str<\/span> =<\/span> 'relu'<\/span>
\n )<\/span>:<\/span>
\n """
\n \u521d\u59cb\u5316\u4ef7\u503c\u7f51\u7edc<\/p>\n

Args:
\n state_dim: \u72b6\u6001\u7ef4\u5ea6
\n hidden_dims: \u9690\u85cf\u5c42\u7ef4\u5ea6\u5217\u8868
\n activation: \u6fc0\u6d3b\u51fd\u6570\u7c7b\u578b
\n """<\/span>
\n self.<\/span>state_dim =<\/span> state_dim
\n self.<\/span>hidden_dims =<\/span> hidden_dims
\n self.<\/span>activation =<\/span> activation<\/p>\n

# \u521d\u59cb\u5316\u6743\u91cd<\/span>
\n self.<\/span>weights =<\/span> self.<\/span>_initialize_weights(<\/span>)<\/span><\/p>\n

def<\/span> _initialize_weights<\/span>(<\/span>self)<\/span> –<\/span>><\/span> dict<\/span>:<\/span>
\n """
\n \u521d\u59cb\u5316\u6743\u91cd<\/p>\n

Returns:
\n \u6743\u91cd\u5b57\u5178
\n """<\/span>
\n weights =<\/span> {<\/span>}<\/span><\/p>\n

# \u6784\u5efa\u7f51\u7edc\u5c42<\/span>
\n in_dim =<\/span> self.<\/span>state_dim
\n for<\/span> i,<\/span> out_dim in<\/span> enumerate<\/span>(<\/span>self.<\/span>hidden_dims)<\/span>:<\/span>
\n # \u7ebf\u6027\u5c42<\/span>
\n weights[<\/span>f'fc<\/span>{<\/span>i}<\/span><\/span>'<\/span><\/span>]<\/span> =<\/span> np.<\/span>random.<\/span>randn(<\/span>
\n in_dim,<\/span> out_dim
\n )<\/span>.<\/span>astype(<\/span>np.<\/span>float32)<\/span> *<\/span> 0.02<\/span><\/p>\n

# \u6279\u5f52\u4e00\u5316\u53c2\u6570<\/span>
\n weights[<\/span>f'bn<\/span>{<\/span>i}<\/span><\/span>_gamma'<\/span><\/span>]<\/span> =<\/span> np.<\/span>ones(<\/span>out_dim,<\/span> dtype=<\/span>np.<\/span>float32)<\/span>
\n weights[<\/span>f'bn<\/span>{<\/span>i}<\/span><\/span>_beta'<\/span><\/span>]<\/span> =<\/span> np.<\/span>zeros(<\/span>out_dim,<\/span> dtype=<\/span>np.<\/span>float32)<\/span><\/p>\n

in_dim =<\/span> out_dim<\/p>\n

# \u8f93\u51fa\u5c42<\/span>
\n weights[<\/span>'output'<\/span>]<\/span> =<\/span> np.<\/span>random.<\/span>randn(<\/span>
\n in_dim,<\/span> 1<\/span>
\n )<\/span>.<\/span>astype(<\/span>np.<\/span>float32)<\/span> *<\/span> 0.02<\/span><\/p>\n

return<\/span> weights<\/p>\n

def<\/span> forward<\/span>(<\/span>
\n self,<\/span>
\n state:<\/span> np.<\/span>ndarray
\n )<\/span> –<\/span>><\/span> np.<\/span>ndarray:<\/span>
\n """
\n \u524d\u5411\u4f20\u64ad<\/p>\n

Args:
\n state: \u73af\u5883\u72b6\u6001 [batch_size, state_dim]<\/p>\n

Returns:
\n \u72b6\u6001\u4ef7\u503c [batch_size, 1]
\n """<\/span>
\n x =<\/span> state<\/p>\n

# \u901a\u8fc7\u9690\u85cf\u5c42<\/span>
\n for<\/span> i in<\/span> range<\/span>(<\/span>len<\/span>(<\/span>self.<\/span>hidden_dims)<\/span>)<\/span>:<\/span>
\n # \u7ebf\u6027\u53d8\u6362<\/span>
\n x =<\/span> np.<\/span>dot(<\/span>x,<\/span> self.<\/span>weights[<\/span>f'fc<\/span>{<\/span>i}<\/span><\/span>'<\/span><\/span>]<\/span>)<\/span><\/p>\n

# \u6279\u5f52\u4e00\u5316<\/span>
\n x =<\/span> self.<\/span>_batch_norm(<\/span>
\n x,<\/span>
\n self.<\/span>weights[<\/span>f'bn<\/span>{<\/span>i}<\/span><\/span>_gamma'<\/span><\/span>]<\/span>,<\/span>
\n self.<\/span>weights[<\/span>f'bn<\/span>{<\/span>i}<\/span><\/span>_beta'<\/span><\/span>]<\/span>
\n )<\/span><\/p>\n

# \u6fc0\u6d3b\u51fd\u6570<\/span>
\n if<\/span> self.<\/span>activation ==<\/span> 'relu'<\/span>:<\/span>
\n x =<\/span> np.<\/span>maximum(<\/span>0<\/span>,<\/span> x)<\/span>
\n elif<\/span> self.<\/span>activation ==<\/span> 'tanh'<\/span>:<\/span>
\n x =<\/span> np.<\/span>tanh(<\/span>x)<\/span><\/p>\n

# \u8f93\u51fa\u5c42<\/span>
\n value =<\/span> np.<\/span>dot(<\/span>x,<\/span> self.<\/span>weights[<\/span>'output'<\/span>]<\/span>)<\/span><\/p>\n

return<\/span> value<\/p>\n

def<\/span> evaluate<\/span>(<\/span>
\n self,<\/span>
\n state:<\/span> np.<\/span>ndarray
\n )<\/span> –<\/span>><\/span> float<\/span>:<\/span>
\n """
\n \u8bc4\u4f30\u72b6\u6001\u4ef7\u503c<\/p>\n

Args:
\n state: \u73af\u5883\u72b6\u6001 [state_dim]<\/p>\n

Returns:
\n \u72b6\u6001\u4ef7\u503c
\n """<\/span>
\n if<\/span> state.<\/span>ndim ==<\/span> 1<\/span>:<\/span>
\n state =<\/span> state[<\/span>np.<\/span>newaxis,<\/span> :<\/span>]<\/span><\/p>\n

value =<\/span> self.<\/span>forward(<\/span>state)<\/span><\/p>\n

return<\/span> float<\/span>(<\/span>value[<\/span>0<\/span>,<\/span> 0<\/span>]<\/span>)<\/span><\/p>\n

def<\/span> _batch_norm<\/span>(<\/span>
\n self,<\/span>
\n x:<\/span> np.<\/span>ndarray,<\/span>
\n gamma:<\/span> np.<\/span>ndarray,<\/span>
\n beta:<\/span> np.<\/span>ndarray,<\/span>
\n eps:<\/span> float<\/span> =<\/span> 1e-5<\/span>
\n )<\/span> –<\/span>><\/span> np.<\/span>ndarray:<\/span>
\n """
\n \u6279\u5f52\u4e00\u5316<\/p>\n

Args:
\n x: \u8f93\u5165 [batch_size, features]
\n gamma: \u7f29\u653e\u53c2\u6570 [features]
\n beta: \u504f\u79fb\u53c2\u6570 [features]
\n eps: \u5c0f\u5e38\u6570<\/p>\n

Returns:
\n \u5f52\u4e00\u5316\u540e\u7684\u8f93\u51fa
\n """<\/span>
\n mean =<\/span> np.<\/span>mean(<\/span>x,<\/span> axis=<\/span>0<\/span>,<\/span> keepdims=<\/span>True<\/span>)<\/span>
\n var =<\/span> np.<\/span>var(<\/span>x,<\/span> axis=<\/span>0<\/span>,<\/span> keepdims=<\/span>True<\/span>)<\/span><\/p>\n

x_norm =<\/span> (<\/span>x –<\/span> mean)<\/span> \/<\/span> np.<\/span>sqrt(<\/span>var +<\/span> eps)<\/span>
\n output =<\/span> gamma *<\/span> x_norm +<\/span> beta<\/p>\n

return<\/span> output<\/p>\n

class<\/span> ActionSelector<\/span>:<\/span>
\n """
\n \u52a8\u4f5c\u9009\u62e9\u5668<\/p>\n

Attributes:
\n selection_method: \u9009\u62e9\u65b9\u6cd5 ('greedy', 'epsilon_greedy', 'boltzmann', 'ucb')
\n epsilon: Epsilon-greedy\u7684epsilon\u503c
\n temperature: Boltzmann\u7684\u6e29\u5ea6\u53c2\u6570
\n c: UCB\u7684\u63a2\u7d22\u53c2\u6570
\n """<\/span><\/p>\n

def<\/span> __init__<\/span>(<\/span>
\n self,<\/span>
\n selection_method:<\/span> str<\/span> =<\/span> 'greedy'<\/span>,<\/span>
\n epsilon:<\/span> float<\/span> =<\/span> 0.1<\/span>,<\/span>
\n temperature:<\/span> float<\/span> =<\/span> 1.0<\/span>,<\/span>
\n c:<\/span> float<\/span> =<\/span> 2.0<\/span>
\n )<\/span>:<\/span>
\n """
\n \u521d\u59cb\u5316\u52a8\u4f5c\u9009\u62e9\u5668<\/p>\n

Args:
\n selection_method: \u9009\u62e9\u65b9\u6cd5
\n epsilon: Epsilon-greedy\u7684epsilon\u503c
\n temperature: Boltzmann\u7684\u6e29\u5ea6\u53c2\u6570
\n c: UCB\u7684\u63a2\u7d22\u53c2\u6570
\n """<\/span>
\n self.<\/span>selection_method =<\/span> selection_method
\n self.<\/span>epsilon =<\/span> epsilon
\n self.<\/span>temperature =<\/span> temperature
\n self.<\/span>c =<\/span> c<\/p>\n

# UCB\u7edf\u8ba1<\/span>
\n self.<\/span>action_counts =<\/span> None<\/span>
\n self.<\/span>action_values =<\/span> None<\/span><\/p>\n

def<\/span> select_action<\/span>(<\/span>
\n self,<\/span>
\n action_probs:<\/span> np.<\/span>ndarray,<\/span>
\n action_values:<\/span> Optional[<\/span>np.<\/span>ndarray]<\/span> =<\/span> None<\/span>
\n )<\/span> –<\/span>><\/span> int<\/span>:<\/span>
\n """
\n \u9009\u62e9\u52a8\u4f5c<\/p>\n

Args:
\n action_probs: \u52a8\u4f5c\u6982\u7387\u5206\u5e03 [action_dim]
\n action_values: \u52a8\u4f5c\u4ef7\u503c [action_dim] (\u7528\u4e8eUCB)<\/p>\n

Returns:
\n \u9009\u62e9\u7684\u52a8\u4f5c
\n """<\/span>
\n if<\/span> self.<\/span>selection_method ==<\/span> 'greedy'<\/span>:<\/span>
\n return<\/span> self.<\/span>_greedy_selection(<\/span>action_probs)<\/span>
\n elif<\/span> self.<\/span>selection_method ==<\/span> 'epsilon_greedy'<\/span>:<\/span>
\n return<\/span> self.<\/span>_epsilon_greedy_selection(<\/span>action_probs)<\/span>
\n elif<\/span> self.<\/span>selection_method ==<\/span> 'boltzmann'<\/span>:<\/span>
\n return<\/span> self.<\/span>_boltzmann_selection(<\/span>action_probs)<\/span>
\n elif<\/span> self.<\/span>selection_method ==<\/span> 'ucb'<\/span>:<\/span>
\n return<\/span> self.<\/span>_ucb_selection(<\/span>action_values)<\/span>
\n else<\/span>:<\/span>
\n return<\/span> self.<\/span>_greedy_selection(<\/span>action_probs)<\/span><\/p>\n

def<\/span> _greedy_selection<\/span>(<\/span>
\n self,<\/span>
\n action_probs:<\/span> np.<\/span>ndarray
\n )<\/span> –<\/span>><\/span> int<\/span>:<\/span>
\n """
\n \u8d2a\u5a6a\u9009\u62e9<\/p>\n

Args:
\n action_probs: \u52a8\u4f5c\u6982\u7387\u5206\u5e03<\/p>\n

Returns:
\n \u9009\u62e9\u7684\u52a8\u4f5c
\n """<\/span>
\n return<\/span> int<\/span>(<\/span>np.<\/span>argmax(<\/span>action_probs)<\/span>)<\/span><\/p>\n

def<\/span> _epsilon_greedy_selection<\/span>(<\/span>
\n self,<\/span>
\n action_probs:<\/span> np.<\/span>ndarray
\n )<\/span> –<\/span>><\/span> int<\/span>:<\/span>
\n """
\n Epsilon-greedy\u9009\u62e9<\/p>\n

Args:
\n action_probs: \u52a8\u4f5c\u6982\u7387\u5206\u5e03<\/p>\n

Returns:
\n \u9009\u62e9\u7684\u52a8\u4f5c
\n """<\/span>
\n if<\/span> np.<\/span>random.<\/span>random(<\/span>)<\/span> <<\/span> self.<\/span>epsilon:<\/span>
\n # \u968f\u673a\u63a2\u7d22<\/span>
\n return<\/span> np.<\/span>random.<\/span>choice(<\/span>len<\/span>(<\/span>action_probs)<\/span>)<\/span>
\n else<\/span>:<\/span>
\n # \u8d2a\u5a6a\u5229\u7528<\/span>
\n return<\/span> int<\/span>(<\/span>np.<\/span>argmax(<\/span>action_probs)<\/span>)<\/span><\/p>\n

def<\/span> _boltzmann_selection<\/span>(<\/span>
\n self,<\/span>
\n action_probs:<\/span> np.<\/span>ndarray
\n )<\/span> –<\/span>><\/span> int<\/span>:<\/span>
\n """
\n Boltzmann\u9009\u62e9<\/p>\n

Args:
\n action_probs: \u52a8\u4f5c\u6982\u7387\u5206\u5e03<\/p>\n

Returns:
\n \u9009\u62e9\u7684\u52a8\u4f5c
\n """<\/span>
\n # \u5c06\u6982\u7387\u8f6c\u6362\u4e3alogits<\/span>
\n logits =<\/span> np.<\/span>log(<\/span>action_probs +<\/span> 1e-8<\/span>)<\/span><\/p>\n

# \u5e94\u7528\u6e29\u5ea6<\/span>
\n scaled_logits =<\/span> logits \/<\/span> self.<\/span>temperature<\/p>\n

# \u8ba1\u7b97softmax<\/span>
\n exp_logits =<\/span> np.<\/span>exp(<\/span>scaled_logits –<\/span> np.<\/span>max<\/span>(<\/span>scaled_logits)<\/span>)<\/span>
\n probs =<\/span> exp_logits \/<\/span> np.<\/span>sum<\/span>(<\/span>exp_logits)<\/span><\/p>\n

# \u91c7\u6837<\/span>
\n return<\/span> np.<\/span>random.<\/span>choice(<\/span>len<\/span>(<\/span>probs)<\/span>,<\/span> p=<\/span>probs)<\/span><\/p>\n

def<\/span> _ucb_selection<\/span>(<\/span>
\n self,<\/span>
\n action_values:<\/span> np.<\/span>ndarray
\n )<\/span> –<\/span>><\/span> int<\/span>:<\/span>
\n """
\n UCB\u9009\u62e9<\/p>\n

Args:
\n action_values: \u52a8\u4f5c\u4ef7\u503c<\/p>\n

Returns:
\n \u9009\u62e9\u7684\u52a8\u4f5c
\n """<\/span>
\n if<\/span> self.<\/span>action_counts is<\/span> None<\/span>:<\/span>
\n self.<\/span>action_counts =<\/span> np.<\/span>zeros(<\/span>len<\/span>(<\/span>action_values)<\/span>,<\/span> dtype=<\/span>np.<\/span>int32)<\/span>
\n self.<\/span>action_values =<\/span> np.<\/span>zeros(<\/span>len<\/span>(<\/span>action_values)<\/span>,<\/span> dtype=<\/span>np.<\/span>float32)<\/span><\/p>\n

# \u66f4\u65b0\u52a8\u4f5c\u4ef7\u503c<\/span>
\n self.<\/span>action_values =<\/span> action_values<\/p>\n

# \u8ba1\u7b97UCB\u503c<\/span>
\n total_counts =<\/span> np.<\/span>sum<\/span>(<\/span>self.<\/span>action_counts)<\/span>
\n ucb_values =<\/span> self.<\/span>action_values +<\/span> self.<\/span>c *<\/span> np.<\/span>sqrt(<\/span>
\n np.<\/span>log(<\/span>total_counts +<\/span> 1<\/span>)<\/span> \/<\/span> (<\/span>self.<\/span>action_counts +<\/span> 1<\/span>)<\/span>
\n )<\/span><\/p>\n

# \u9009\u62e9UCB\u6700\u5927\u7684\u52a8\u4f5c<\/span>
\n action =<\/span> int<\/span>(<\/span>np.<\/span>argmax(<\/span>ucb_values)<\/span>)<\/span><\/p>\n

# \u66f4\u65b0\u8ba1\u6570<\/span>
\n self.<\/span>action_counts[<\/span>action]<\/span> +=<\/span> 1<\/span><\/p>\n

return<\/span> action<\/p>\n

def<\/span> reset_ucb<\/span>(<\/span>self,<\/span> action_dim:<\/span> int<\/span>)<\/span> –<\/span>><\/span> None<\/span>:<\/span>
\n """
\n \u91cd\u7f6eUCB\u7edf\u8ba1<\/p>\n

Args:
\n action_dim: \u52a8\u4f5c\u7ef4\u5ea6
\n """<\/span>
\n self.<\/span>action_counts =<\/span> np.<\/span>zeros(<\/span>action_dim,<\/span> dtype=<\/span>np.<\/span>int32)<\/span>
\n self.<\/span>action_values =<\/span> np.<\/span>zeros(<\/span>action_dim,<\/span> dtype=<\/span>np.<\/span>float32)<\/span><\/p>\n

2.2 \u7b56\u7565\u4f18\u5316\u7b56\u7565<\/h4>\n

CANN\u7684\u7b56\u7565\u4f18\u5316\u5305\u62ec：<\/p>\n

    \n
  • \u7f51\u7edc\u526a\u679d：\u526a\u679d\u4e0d\u91cd\u8981\u7684\u795e\u7ecf\u5143<\/li>\n
  • \u91cf\u5316：\u91cf\u5316\u7f51\u7edc\u6743\u91cd<\/li>\n
  • \u77e5\u8bc6\u84b8\u998f：\u4f7f\u7528\u5927\u6a21\u578b\u6307\u5bfc\u5c0f\u6a21\u578b<\/li>\n
  • \u7f13\u5b58\u4f18\u5316：\u7f13\u5b58\u5e38\u7528\u7b56\u7565<\/li>\n<\/ul>\n
    \n

    \u4e09\u3001\u4ef7\u503c\u7f51\u7edc\u4f18\u5316<\/h3>\n

    3.1 Critic\u7f51\u7edc\u4f18\u5316<\/h4>\n

    Critic\u7f51\u7edc（\u4ef7\u503c\u7f51\u7edc）\u8d1f\u8d23\u8bc4\u4f30\u72b6\u6001\u7684\u4ef7\u503c，CANN\u901a\u8fc7\u4f18\u5316Critic\u7f51\u7edc，\u63d0\u9ad8\u4ef7\u503c\u8bc4\u4f30\u6548\u7387\u3002<\/p>\n

    \u4ef7\u503c\u4f18\u5316\u7b56\u7565<\/h5>\n

    CANN\u7684\u4ef7\u503c\u4f18\u5316\u5305\u62ec：<\/p>\n

      \n
    • \u53cc\u7f51\u7edc\u6280\u672f：\u4f7f\u7528\u53cc\u7f51\u7edc\u51cf\u5c11\u8fc7\u4f30\u8ba1<\/li>\n
    • \u76ee\u6807\u7f51\u7edc：\u4f7f\u7528\u76ee\u6807\u7f51\u7edc\u7a33\u5b9a\u8bad\u7ec3<\/li>\n
    • \u4f18\u5148\u7ecf\u9a8c\u56de\u653e：\u4f18\u5148\u56de\u653e\u91cd\u8981\u7ecf\u9a8c<\/li>\n
    • \u5206\u5e03\u5f0f\u8ba1\u7b97：\u5206\u5e03\u5f0f\u8ba1\u7b97\u4ef7\u503c\u4f30\u8ba1<\/li>\n<\/ul>\n
      \n

      \u56db\u3001\u6027\u80fd\u4f18\u5316\u5b9e\u6218<\/h3>\n

      4.1 \u7b56\u7565\u7f51\u7edc\u4f18\u5316\u6548\u679c<\/h4>\n

      \u5bf9\u4e8e\u7b56\u7565\u7f51\u7edc\u63a8\u7406，CANN\u901a\u8fc7\u7f51\u7edc\u526a\u679d\u548c\u91cf\u5316，\u6027\u80fd\u63d0\u5347\u663e\u8457\u3002\u5355\u6b21\u7b56\u7565\u63a8\u7406\u7684\u5ef6\u8fdf\u4ece\u539f\u6765\u768420ms\u964d\u4f4e\u52305ms，\u6027\u80fd\u63d0\u53474\u500d\u3002<\/p>\n

      \u4f18\u5316\u6548\u679c\u4e3b\u8981\u4f53\u73b0\u5728\u4e09\u4e2a\u65b9\u9762：<\/p>\n

        \n
      • \u7f51\u7edc\u526a\u679d\u901f\u5ea6\u63d0\u534750%<\/li>\n
      • \u91cf\u5316\u8ba1\u7b97\u901f\u5ea6\u63d0\u534760%<\/li>\n
      • \u6574\u4f53\u7b56\u7565\u63a8\u7406\u901f\u5ea6\u63d0\u5347300%<\/li>\n<\/ul>\n

        \u5185\u5b58\u5360\u7528\u4e5f\u4ece\u539f\u6765\u7684200MB\u964d\u4f4e\u523080MB，\u51cf\u5c11\u7ea660%\u3002<\/p>\n

        4.2 \u4ef7\u503c\u7f51\u7edc\u4f18\u5316\u6548\u679c<\/h4>\n

        \u5bf9\u4e8e\u4ef7\u503c\u7f51\u7edc\u63a8\u7406，CANN\u901a\u8fc7\u53cc\u7f51\u7edc\u6280\u672f\u548c\u76ee\u6807\u7f51\u7edc\u4f18\u5316，\u8fdb\u4e00\u6b65\u63d0\u5347\u4e86\u6027\u80fd\u3002\u4ee5\u8bc4\u4f30100\u4e2a\u72b6\u6001\u4e3a\u4f8b，\u6027\u80fd\u63d0\u5347\u6bd4\u7b56\u7565\u7f51\u7edc\u63d0\u5347\u4e86150%\u3002<\/p>\n

        \u4ef7\u503c\u7f51\u7edc\u4f18\u5316\u7684\u5173\u952e\u5728\u4e8e：<\/p>\n

          \n
        • \u53cc\u7f51\u7edc\u4f18\u5316<\/li>\n
        • \u76ee\u6807\u7f51\u7edc\u7f13\u5b58<\/li>\n
        • \u6279\u91cf\u8bc4\u4f30<\/li>\n
        • \u5e76\u884c\u8ba1\u7b97<\/li>\n<\/ul>\n
          \n

          \u4e94\u3001\u5b9e\u9645\u5e94\u7528\u6848\u4f8b<\/h3>\n

          5.1 \u6e38\u620fAI<\/h4>\n

          \u5f3a\u5316\u5b66\u4e60\u5728\u6e38\u620fAI\u4e2d\u6709\u7740\u5e7f\u6cdb\u7684\u5e94\u7528，\u80fd\u591f\u5b66\u4e60\u9ad8\u6c34\u5e73\u7684\u6e38\u620f\u7b56\u7565\u3002CANN\u4f18\u5316\u7684\u5f3a\u5316\u5b66\u4e60\u4f7f\u5f97\u5b9e\u65f6\u6e38\u620f\u51b3\u7b56\u6210\u4e3a\u53ef\u80fd，\u5927\u5927\u63d0\u5347\u4e86AI\u7684\u8868\u73b0\u3002<\/p>\n

          \u4ee5\u5728Atari\u6e38\u620f\u4e2d\u4e3a\u4f8b，\u4f18\u5316\u540e\u4ece\u8f93\u5165\u6e38\u620f\u753b\u9762\u5230\u8f93\u51fa\u52a8\u4f5c\u53ea\u970010-20\u6beb\u79d2，\u5b8c\u5168\u6ee1\u8db3\u5b9e\u65f6\u6e38\u620f\u7684\u9700\u6c42\u3002<\/p>\n

          5.2 \u673a\u5668\u4eba\u63a7\u5236<\/h4>\n

          \u5f3a\u5316\u5b66\u4e60\u8fd8\u53ef\u4ee5\u7528\u4e8e\u673a\u5668\u4eba\u63a7\u5236，\u5b66\u4e60\u590d\u6742\u7684\u8fd0\u52a8\u63a7\u5236\u7b56\u7565\u3002CANN\u7684\u4f18\u5316\u4f7f\u5f97\u673a\u5668\u4eba\u63a7\u5236\u80fd\u591f\u5728\u5b9e\u65f6\u6216\u8fd1\u5b9e\u65f6\u7684\u901f\u5ea6\u4e0b\u8fd0\u884c，\u4e3a\u673a\u5668\u4eba\u5e94\u7528\u63d0\u4f9b\u4e86\u5f3a\u5927\u7684\u652f\u6301\u3002<\/p>\n

          \u4ee5\u63a7\u5236\u673a\u68b0\u81c2\u6293\u53d6\u7269\u4f53\u4e3a\u4f8b，\u4f18\u5316\u540e\u4ece\u8f93\u5165\u4f20\u611f\u5668\u6570\u636e\u5230\u8f93\u51fa\u63a7\u5236\u6307\u4ee4\u53ea\u97005-10\u6beb\u79d2，\u6548\u7387\u63d0\u5347\u663e\u8457\u3002<\/p>\n

          \n

          \u516d\u3001\u6700\u4f73\u5b9e\u8df5<\/h3>\n

          6.1 \u7b97\u6cd5\u9009\u62e9\u5efa\u8bae<\/h4>\n

          \u5728\u4f7f\u7528\u5f3a\u5316\u5b66\u4e60\u65f6，\u9009\u62e9\u5408\u9002\u7684\u7b97\u6cd5\u5bf9\u6700\u7ec8\u6548\u679c\u6709\u5f88\u5927\u5f71\u54cd\u3002CANN\u5efa\u8bae\u6839\u636e\u5e94\u7528\u573a\u666f\u9009\u62e9\u7b97\u6cd5：<\/p>\n\n\u5e94\u7528\u573a\u666f\u7b97\u6cd5\u52a8\u4f5c\u7a7a\u95f4\u6837\u672c\u6548\u7387\u63a8\u7406\u901f\u5ea6\u9002\u7528\u6027<\/tr>\n\n\n\n\n\n
          Atari\u6e38\u620f<\/td>\nDQN<\/td>\n\u79bb\u6563<\/td>\n\u4f4e<\/td>\n\u5feb<\/td>\n\u9ad8<\/td>\n<\/tr>\n
          \u8fde\u7eed\u63a7\u5236<\/td>\nPPO<\/td>\n\u8fde\u7eed<\/td>\n\u9ad8<\/td>\n\u4e2d\u7b49<\/td>\n\u9ad8<\/td>\n<\/tr>\n
          \u5e76\u884c\u8bad\u7ec3<\/td>\nA3C<\/td>\n\u8fde\u7eed\/\u79bb\u6563<\/td>\n\u4e2d\u7b49<\/td>\n\u5feb<\/td>\n\u4e2d\u7b49<\/td>\n<\/tr>\n
          \u590d\u6742\u63a7\u5236<\/td>\nSAC<\/td>\n\u8fde\u7eed<\/td>\n\u5f88\u9ad8<\/td>\n\u4e2d\u7b49<\/td>\n\u9ad8<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

          6.2 \u8c03\u4f18\u5efa\u8bae<\/h4>\n

          \u9488\u5bf9\u5f3a\u5316\u5b66\u4e60\u63a8\u7406，CANN\u63d0\u4f9b\u4e86\u4e00\u7cfb\u5217\u8c03\u4f18\u5efa\u8bae：<\/p>\n

          \u7b56\u7565\u7f51\u7edc\u4f18\u5316<\/p>\n

            \n
          • \u4f7f\u7528\u7f51\u7edc\u526a\u679d\u53ef\u4ee5\u51cf\u5c11\u8ba1\u7b97\u91cf<\/li>\n
          • \u91cf\u5316\u7f51\u7edc\u6743\u91cd\u53ef\u4ee5\u63d0\u5347\u63a8\u7406\u901f\u5ea6<\/li>\n
          • \u4f7f\u7528\u77e5\u8bc6\u84b8\u998f\u53ef\u4ee5\u4fdd\u6301\u6027\u80fd<\/li>\n<\/ul>\n

            \u4ef7\u503c\u7f51\u7edc\u4f18\u5316<\/p>\n

              \n
            • \u4f7f\u7528\u53cc\u7f51\u7edc\u6280\u672f\u53ef\u4ee5\u51cf\u5c11\u8fc7\u4f30\u8ba1<\/li>\n
            • \u7f13\u5b58\u76ee\u6807\u7f51\u7edc\u53ef\u4ee5\u63d0\u5347\u63a8\u7406\u6548\u7387<\/li>\n
            • \u6279\u91cf\u8bc4\u4f30\u53ef\u4ee5\u63d0\u5347\u541e\u5410\u91cf<\/li>\n<\/ul>\n

              \u52a8\u4f5c\u9009\u62e9\u4f18\u5316<\/p>\n

                \n
              • \u9009\u62e9\u5408\u9002\u7684\u52a8\u4f5c\u9009\u62e9\u7b56\u7565<\/li>\n
              • \u4f18\u5316\u63a2\u7d22-\u5229\u7528\u5e73\u8861<\/li>\n
              • \u4f7f\u7528\u7f13\u5b58\u53ef\u4ee5\u52a0\u901f\u5e38\u7528\u51b3\u7b56<\/li>\n<\/ul>\n
                \n

                \u603b\u7ed3<\/h3>\n

                CANN\u901a\u8fc7\u7b56\u7565\u7f51\u7edc\u4f18\u5316\u3001\u4ef7\u503c\u7f51\u7edc\u4f18\u5316\u548c\u52a8\u4f5c\u9009\u62e9\u4f18\u5316，\u663e\u8457\u63d0\u5347\u4e86\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\u7684\u6027\u80fd\u548c\u54cd\u5e94\u901f\u5ea6\u3002\u672c\u6587\u8be6\u7ec6\u5206\u6790\u4e86\u5f3a\u5316\u5b66\u4e60\u7684\u67b6\u6784\u539f\u7406，\u8bb2\u89e3\u4e86\u7b56\u7565\u548c\u4ef7\u503c\u7f51\u7edc\u7684\u4f18\u5316\u65b9\u6cd5，\u5e76\u63d0\u4f9b\u4e86\u6027\u80fd\u5bf9\u6bd4\u548c\u5e94\u7528\u6848\u4f8b\u3002<\/p>\n

                \u5173\u952e\u8981\u70b9\u603b\u7ed3：<\/p>\n

              • \u7406\u89e3\u5f3a\u5316\u5b66\u4e60\u7684\u6838\u5fc3\u539f\u7406：\u638c\u63e1\u7b56\u7565\u7f51\u7edc\u548c\u4ef7\u503c\u7f51\u7edc\u7684\u57fa\u672c\u6d41\u7a0b<\/li>\n
              • \u638c\u63e1\u7b56\u7565\u7f51\u7edc\u4f18\u5316：\u5b66\u4e60\u7f51\u7edc\u526a\u679d\u548c\u91cf\u5316\u7684\u65b9\u6cd5<\/li>\n
              • \u719f\u6089\u4ef7\u503c\u7f51\u7edc\u4f18\u5316：\u4e86\u89e3\u53cc\u7f51\u7edc\u548c\u76ee\u6807\u7f51\u7edc\u7684\u6280\u672f<\/li>\n
              • \u4e86\u89e3\u52a8\u4f5c\u9009\u62e9\u4f18\u5316：\u638c\u63e1\u4e0d\u540c\u52a8\u4f5c\u9009\u62e9\u7b56\u7565\u7684\u5e94\u7528<\/li>\n

                \u901a\u8fc7\u5408\u7406\u5e94\u7528\u8fd9\u4e9b\u6280\u672f，\u53ef\u4ee5\u5c06\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\u6027\u80fd\u63d0\u53473-5\u500d，\u4e3a\u5b9e\u9645\u5e94\u7528\u573a\u666f\u63d0\u4f9b\u66f4\u4f18\u8d28\u7684\u670d\u52a1\u4f53\u9a8c\u3002<\/p>\n

                \n

                \u76f8\u5173\u94fe\u63a5：<\/p>\n

                  \n
                • CANN\u7ec4\u7ec7<\/li>\n
                • parser\u4ed3\u5e93<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"

                  \u5f3a\u5316\u5b66\u4e60（Reinforcement Learning，RL）\u662f\u4e00\u79cd\u901a\u8fc7\u4e0e\u73af\u5883\u4ea4\u4e92\u5b66\u4e60\u6700\u4f18\u7b56\u7565\u7684\u673a\u5668\u5b66\u4e60\u65b9\u6cd5\u3002RL\u5728\u6e38\u620fAI\u3001\u673a\u5668\u4eba\u63a7\u5236\u3001\u81ea\u52a8\u9a7e\u9a76\u3001\u63a8\u8350\u7cfb\u7edf\u7b49\u9886\u57df\u6709\u7740\u5e7f\u6cdb\u7684\u5e94\u7528\u3002RL\u63a8\u7406\u7684\u6838\u5fc3\u662f\u7b56\u7565\u7f51\u7edc\u548c\u4ef7\u503c\u7f51\u7edc\u7684\u524d\u5411\u4f20\u64ad，\u9700\u8981\u5feb\u901f\u54cd\u5e94\u73af\u5883\u72b6\u6001\u5e76\u8f93\u51fa\u52a8\u4f5c，\u5bf9\u63a8\u7406\u901f\u5ea6\u8981\u6c42\u6781\u9ad8\u3002CANN\u9488\u5bf9\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\u63a8\u51fa\u4e86\u5168\u9762\u7684\u4f18\u5316\u65b9\u6848，\u901a\u8fc7\u7b56\u7565\u7f51\u7edc\u4f18\u5316\u3001\u4ef7\u503c\u7f51\u7edc\u4f18\u5316\u548c\u52a8\u4f5c\u9009\u62e9\u4f18\u5316，\u663e\u8457\u63d0\u5347\u4e86RL<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[50],"topic":[],"class_list":["post-73917","post","type-post","status-publish","format-standard","hentry","category-server","tag-50"],"yoast_head":"\nCANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.wsisp.com\/helps\/73917.html\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"CANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"og:description\" content=\"\u5f3a\u5316\u5b66\u4e60（Reinforcement Learning，RL）\u662f\u4e00\u79cd\u901a\u8fc7\u4e0e\u73af\u5883\u4ea4\u4e92\u5b66\u4e60\u6700\u4f18\u7b56\u7565\u7684\u673a\u5668\u5b66\u4e60\u65b9\u6cd5\u3002RL\u5728\u6e38\u620fAI\u3001\u673a\u5668\u4eba\u63a7\u5236\u3001\u81ea\u52a8\u9a7e\u9a76\u3001\u63a8\u8350\u7cfb\u7edf\u7b49\u9886\u57df\u6709\u7740\u5e7f\u6cdb\u7684\u5e94\u7528\u3002RL\u63a8\u7406\u7684\u6838\u5fc3\u662f\u7b56\u7565\u7f51\u7edc\u548c\u4ef7\u503c\u7f51\u7edc\u7684\u524d\u5411\u4f20\u64ad，\u9700\u8981\u5feb\u901f\u54cd\u5e94\u73af\u5883\u72b6\u6001\u5e76\u8f93\u51fa\u52a8\u4f5c，\u5bf9\u63a8\u7406\u901f\u5ea6\u8981\u6c42\u6781\u9ad8\u3002CANN\u9488\u5bf9\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\u63a8\u51fa\u4e86\u5168\u9762\u7684\u4f18\u5316\u65b9\u6848，\u901a\u8fc7\u7b56\u7565\u7f51\u7edc\u4f18\u5316\u3001\u4ef7\u503c\u7f51\u7edc\u4f18\u5316\u548c\u52a8\u4f5c\u9009\u62e9\u4f18\u5316，\u663e\u8457\u63d0\u5347\u4e86RL\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.wsisp.com\/helps\/73917.html\" \/>\n<meta property=\"og:site_name\" content=\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"article:published_time\" content=\"2026-02-08T10:09:52+00:00\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/73917.html\",\"url\":\"https:\/\/www.wsisp.com\/helps\/73917.html\",\"name\":\"CANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"isPartOf\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\"},\"datePublished\":\"2026-02-08T10:09:52+00:00\",\"dateModified\":\"2026-02-08T10:09:52+00:00\",\"author\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/73917.html#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.wsisp.com\/helps\/73917.html\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/73917.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.wsisp.com\/helps\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"CANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\",\"url\":\"https:\/\/www.wsisp.com\/helps\/\",\"name\":\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"description\":\"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"contentUrl\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"caption\":\"admin\"},\"sameAs\":[\"http:\/\/wp.wsisp.com\"],\"url\":\"https:\/\/www.wsisp.com\/helps\/author\/admin\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"CANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.wsisp.com\/helps\/73917.html","og_locale":"zh_CN","og_type":"article","og_title":"CANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","og_description":"\u5f3a\u5316\u5b66\u4e60（Reinforcement Learning，RL）\u662f\u4e00\u79cd\u901a\u8fc7\u4e0e\u73af\u5883\u4ea4\u4e92\u5b66\u4e60\u6700\u4f18\u7b56\u7565\u7684\u673a\u5668\u5b66\u4e60\u65b9\u6cd5\u3002RL\u5728\u6e38\u620fAI\u3001\u673a\u5668\u4eba\u63a7\u5236\u3001\u81ea\u52a8\u9a7e\u9a76\u3001\u63a8\u8350\u7cfb\u7edf\u7b49\u9886\u57df\u6709\u7740\u5e7f\u6cdb\u7684\u5e94\u7528\u3002RL\u63a8\u7406\u7684\u6838\u5fc3\u662f\u7b56\u7565\u7f51\u7edc\u548c\u4ef7\u503c\u7f51\u7edc\u7684\u524d\u5411\u4f20\u64ad，\u9700\u8981\u5feb\u901f\u54cd\u5e94\u73af\u5883\u72b6\u6001\u5e76\u8f93\u51fa\u52a8\u4f5c，\u5bf9\u63a8\u7406\u901f\u5ea6\u8981\u6c42\u6781\u9ad8\u3002CANN\u9488\u5bf9\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\u63a8\u51fa\u4e86\u5168\u9762\u7684\u4f18\u5316\u65b9\u6848，\u901a\u8fc7\u7b56\u7565\u7f51\u7edc\u4f18\u5316\u3001\u4ef7\u503c\u7f51\u7edc\u4f18\u5316\u548c\u52a8\u4f5c\u9009\u62e9\u4f18\u5316，\u663e\u8457\u63d0\u5347\u4e86RL","og_url":"https:\/\/www.wsisp.com\/helps\/73917.html","og_site_name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","article_published_time":"2026-02-08T10:09:52+00:00","author":"admin","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"admin","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"7 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.wsisp.com\/helps\/73917.html","url":"https:\/\/www.wsisp.com\/helps\/73917.html","name":"CANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","isPartOf":{"@id":"https:\/\/www.wsisp.com\/helps\/#website"},"datePublished":"2026-02-08T10:09:52+00:00","dateModified":"2026-02-08T10:09:52+00:00","author":{"@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41"},"breadcrumb":{"@id":"https:\/\/www.wsisp.com\/helps\/73917.html#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.wsisp.com\/helps\/73917.html"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.wsisp.com\/helps\/73917.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.wsisp.com\/helps"},{"@type":"ListItem","position":2,"name":"CANN\u52a0\u901f\u5f3a\u5316\u5b66\u4e60\u63a8\u7406\uff1a\u7b56\u7565\u7f51\u7edc\u4e0e\u4ef7\u503c\u7f51\u7edc\u4f18\u5316"}]},{"@type":"WebSite","@id":"https:\/\/www.wsisp.com\/helps\/#website","url":"https:\/\/www.wsisp.com\/helps\/","name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","description":"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41","name":"admin","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/","url":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","contentUrl":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","caption":"admin"},"sameAs":["http:\/\/wp.wsisp.com"],"url":"https:\/\/www.wsisp.com\/helps\/author\/admin"}]}},"_links":{"self":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/73917","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/comments?post=73917"}],"version-history":[{"count":0,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/73917\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/media?parent=73917"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/categories?post=73917"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/tags?post=73917"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/topic?post=73917"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}