Erlang中文论坛

标题: 异步gen_server进行port访问时性能严重下降的原因和应对方法... [打印本页]

作者: dreamxyp    时间: 2013-10-21 00:48
标题: 异步gen_server进行port访问时性能严重下降的原因和应对方法...
在进行项目开发时,有这么一个需求:一个生产者不断产生消息,并将消息异步投递给消费者,异步投递不受任何限制,消费者进程内部将消息写入文件或者通过套接字发送该消息。
看到这个需求,我想也没想,熟练地写下了如下代码:

API:
  1. start_link(FileName, DsgHost, DstPort) ->
  2.     gen_server:start_link({local, ?SERVER}, ?MODULE, [FileName, DsgHost, DstPort], []).


  3. deliver_to_file(Msg) -> deliver_to(file, Msg).
  4. deliver_to_net(Msg) -> deliver_to(net, Msg).

  5. deliver_to(Type, Msg) ->
  6.     gen_server:cast(?SERVER, {deliver_to, Type, Msg}).
复制代码
gen_server callback:
  1. init([FileName, DsgHost, DstPort]) ->
  2.     {ok, FD} = file:open(FileName,
  3.         [binary, raw, read, write, append]),
  4.     {ok, Socket} = gen_tcp:connect(DsgHost, DstPort,
  5.         [inet, binary, {active, false}, {packet, 4}, {reuseaddr, true}]),
  6.     {ok, #state{file = FD, socket = Socket}}.



  7. handle_cast({deliver_to, Type, Msg}, State) ->
  8.     handle_msg(Type, Msg, State),
  9.     {noreply, State};


  10. handle_msg(Type, Msg, #state{file = File, socket = Socket}) when is_binary(Msg) ->
  11.     case Type of
  12.         file -> file:write(File, Msg);
  13.         net -> gen_tcp:send(Socket, Msg)
  14.     end;
  15. handle_msg(Type, Msg, State) ->
  16.     handle_msg(Type, term_to_binary(Msg), State).
复制代码

压测函数编写如下:
  1. deliver_to_with_len_and_times(Type, Len, Times) ->
  2.     Msg = makeup_log_by_len(Len),
  3.     deliver_to_times(Type, Msg, Times).

  4. deliver_to_times(Type, Msg, Times) when Times > 0 ->
  5.     deliver_to(Type, Msg);
  6. deliver_to_times(_Type, _Msg, 0) ->
  7.     ok.

  8. makeup_log_by_len(Len) ->
  9.     list_to_binary(lists:duplicate(Len, $a)).
复制代码

函数主要目的为异步地向consumer投递Times个长度为Len的消息。
这段看似简单的代码,却在压测的时候产生了意想不到的效果:当投递消息个数较小时(<1000),handle_msg的处理qps上万,但是随着投递消息数目的增加(>10000),性能却急剧下降,最后仅有300+,使用vmstat、ifstat和top工具观察,磁盘写入和网络发送数据量均不大,但是cpu消耗却不小,这是肿么回事呢?

未完待续...


作者: 茶叶feng    时间: 2018-10-8 10:03
没后续了?




欢迎光临 Erlang中文论坛 (https://bbs.erldoc.com/) Powered by Discuz! X3.3