docs: update doc for voice

support group chat in one seesion

.github/ISSUE_TEMPLATE.md

@@ -1,7 +1,7 @@
 ### 前置确认
 
-1. 运行于国内网络环境，未开代理
-2.  python 已安装：版本在 3.7 ~ 3.10 之间，依赖已安装
+1. 网络能够访问openai接口 [#351](https://github.com/zhayujie/chatgpt-on-wechat/issues/351)
+2. python 已安装：版本在 3.7 ~ 3.10 之间，依赖已安装
 3. 在已有 issue 中未搜索到类似问题
 4. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
 

.gitignore

@@ -6,3 +6,4 @@ venv*
 config.json
 QR.png
 nohup.out
+tmp

README.md

@@ -10,12 +10,16 @@
 - [x] **多账号：** 支持多微信账号同时运行
 - [x] **图片生成：** 支持根据描述生成图片，并自动发送至个人聊天或群聊
 - [x] **上下文记忆**：支持多轮对话记忆，且为每个好友维护独立的上下会话
+- [x] **语音识别：** 支持接收和处理语音消息，通过文字或语音回复
 
 
 # 更新日志
->**2023.03.02：** 接入[ChatGPT API](https://platform.openai.com/docs/guides/chat) (gpt-3.5-turbo)，默认使用该模型进行对话，需升级openai依赖 (`pip3 install --upgrade openai`)
 
->**2023.02.20：** 增加 [python-wechaty](https://github.com/wechaty/python-wechaty) 支持，Pad协议相对稳定，不易封号，但Token收费，可申请七天体验Token (使用参考[#244](https://github.com/zhayujie/chatgpt-on-wechat/pull/244)，contributed by [ZQ7](https://github.com/ZQ7))
+>**2023.03.09：** 基于 `whisper API` 实现对微信语音消息的解析和回复，添加配置项 `"speech_recognition":true` 即可启用。(contributed by [wanggang1987](https://github.com/wanggang1987) in [#385](https://github.com/zhayujie/chatgpt-on-wechat/pull/385))
+
+>**2023.03.02：** 接入[ChatGPT API](https://platform.openai.com/docs/guides/chat) (gpt-3.5-turbo)，默认使用该模型进行对话，需升级openai依赖 (`pip3 install --upgrade openai`)。网络问题参考 [#351](https://github.com/zhayujie/chatgpt-on-wechat/issues/351)
+
+>**2023.02.20：** 增加 [python-wechaty](https://github.com/wechaty/python-wechaty) 作为可选渠道，使用Pad协议相对稳定，但Token收费 (使用参考[#244](https://github.com/zhayujie/chatgpt-on-wechat/pull/244)，contributed by [ZQ7](https://github.com/ZQ7))
 
 >**2023.02.09：** 扫码登录存在封号风险，请谨慎使用，参考[#58](https://github.com/AutumnWhj/ChatGPT-wechat-bot/issues/158)
 
@@ -48,7 +52,7 @@
 
 ### 1. OpenAI账号注册
 
-前往 [OpenAI注册页面](https://beta.openai.com/signup) 创建账号，参考这篇 [教程](https://freepac.siterubix.com/chatgpt%e6%b3%a8%e5%86%8c%e6%95%99%e7%a8%8b%ef%bc%88%e5%ae%8c%e6%95%b4%e6%8c%87%e5%8d%97%ef%bc%89/) 可以通过虚拟手机号来接收验证码。创建完账号则前往 [API管理页面](https://beta.openai.com/account/api-keys) 创建一个 API Key 并保存下来，后面需要在项目中配置这个key。
+前往 [OpenAI注册页面](https://beta.openai.com/signup) 创建账号，参考这篇 [教程](https://www.pythonthree.com/register-openai-chatgpt/) 可以通过虚拟手机号来接收验证码。创建完账号则前往 [API管理页面](https://beta.openai.com/account/api-keys) 创建一个 API Key 并保存下来，后面需要在项目中配置这个key。
 
 > 项目中使用的对话模型是 davinci，计费方式是约每 750 字 (包含请求和回复) 消耗 $0.02，图片生成是每张消耗 $0.016，账号创建有免费的 $18 额度，使用完可以更换邮箱重新注册。
 
@@ -89,6 +93,7 @@ cp config-template.json config.json
 # config.json文件内容示例
 { 
   "open_ai_api_key": "YOUR API KEY",                          # 填入上面创建的 OpenAI API KEY
+  "proxy": "127.0.0.1:7890",                                  # 代理客户端的ip和端口
   "single_chat_prefix": ["bot", "@bot"],                      # 私聊时文本需要包含该前缀才能触发机器人回复
   "single_chat_reply_prefix": "[bot] ",                       # 私聊时自动回复的前缀，用于区分真人
   "group_chat_prefix": ["@bot"],                              # 群聊时包含该前缀则会触发机器人回复
@@ -111,8 +116,13 @@ cp config-template.json config.json
 + 默认只要被人 @ 就会触发机器人自动回复；另外群聊天中只要检测到以 "@bot" 开头的内容，同样会自动回复（方便自己触发），这对应配置项 `group_chat_prefix`
 + 可选配置: `group_name_keyword_white_list`配置项支持模糊匹配群名称，`group_chat_keyword`配置项则支持模糊匹配群消息内容，用法与上述两个配置项相同。（Contributed by [evolay](https://github.com/evolay))
 
-**3.其他配置**
+**3.语音识别**
++ 添加 `"speech_recognition": true` 将开启语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，目前只支持私聊 (注意由于语音消息无法匹配前缀，一旦开启将对所有语音自动回复)；
++ 添加 `"voice_reply_voice": true` 将开启语音回复语音，但是需要配置对应语音合成平台的key，由于itchat协议的限制，只能发送语音mp3文件，若使用wechaty则回复的是微信语音。
 
+**4.其他配置**
+
++ `proxy`：由于目前 `openai` 接口国内无法访问，需配置代理客户端的地址，详情参考  [#351](https://github.com/zhayujie/chatgpt-on-wechat/issues/351)
 + 对于图像生成，在满足个人或群组触发条件外，还需要额外的关键词前缀来触发，对应配置 `image_create_prefix `
 + 关于OpenAI对话及图片接口的参数配置（内容自由度、回复字数限制、图片大小等），可以参考 [对话接口](https://beta.openai.com/docs/api-reference/completions) 和 [图像接口](https://beta.openai.com/docs/api-reference/completions)  文档直接在 [代码](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/bot/openai/open_ai_bot.py) `bot/openai/open_ai_bot.py` 中进行调整。
 + `conversation_max_tokens`：表示能够记忆的上下文最大字数（一问一答为一组对话，如果累积的对话字数超出限制，就会优先移除最早的一组对话）
@@ -140,6 +150,7 @@ touch nohup.out                                   # 首次运行需要新建日
 nohup python3 app.py & tail -f nohup.out          # 在后台运行程序并通过日志输出二维码
 ```
 扫码登录后程序即可运行于服务器后台，此时可通过 `ctrl+c` 关闭日志，不会影响后台程序的运行。使用 `ps -ef | grep app.py | grep -v grep` 命令可查看运行于后台的进程，如果想要重新启动程序可以先 `kill` 掉对应的进程。日志关闭后如果想要再次打开只需输入 `tail -f nohup.out`。
+scripts/目录有相应的脚本可以调用
 
 > **注意：** 如果 扫码后手机提示登录验证需要等待5s，而终端的二维码再次刷新并提示 `Log in time out, reloading QR code`，此时需参考此 [issue](https://github.com/zhayujie/chatgpt-on-wechat/issues/8) 修改一行代码即可解决。
 

bot/chatgpt/chat_gpt_bot.py

@@ -3,70 +3,96 @@
 from bot.bot import Bot
 from config import conf
 from common.log import logger
+from common.expired_dict import ExpiredDict
 import openai
 import time
 
-user_session = dict()
+if conf().get('expires_in_seconds'):
+    all_sessions = ExpiredDict(conf().get('expires_in_seconds'))
+else:
+    all_sessions = dict()
 
 # OpenAI对话模型API (可用)
 class ChatGPTBot(Bot):
     def __init__(self):
         openai.api_key = conf().get('open_ai_api_key')
+        proxy = conf().get('proxy')
+        if proxy:
+            openai.proxy = proxy
 
     def reply(self, query, context=None):
         # acquire reply content
         if not context or not context.get('type') or context.get('type') == 'TEXT':
             logger.info("[OPEN_AI] query={}".format(query))
-            from_user_id = context['from_user_id']
+            session_id = context['session_id']
             if query == '#清除记忆':
-                Session.clear_session(from_user_id)
+                Session.clear_session(session_id)
                 return '记忆已清除'
+            elif query == '#清除所有':
+                Session.clear_all_session()
+                return '所有人记忆已清除'            
 
-            new_query = Session.build_session_query(query, from_user_id)
-            logger.debug("[OPEN_AI] session query={}".format(new_query))
+            session = Session.build_session_query(query, session_id)
+            logger.debug("[OPEN_AI] session query={}".format(session))
 
             # if context.get('stream'):
             #     # reply in stream
-            #     return self.reply_text_stream(query, new_query, from_user_id)
+            #     return self.reply_text_stream(query, new_query, session_id)
 
-            reply_content = self.reply_text(new_query, from_user_id, 0)
-            logger.debug("[OPEN_AI] new_query={}, user={}, reply_cont={}".format(new_query, from_user_id, reply_content))
-            if reply_content:
-                Session.save_session(query, reply_content, from_user_id)
-            return reply_content
+            reply_content = self.reply_text(session, session_id, 0)
+            logger.debug("[OPEN_AI] new_query={}, session_id={}, reply_cont={}".format(session, session_id, reply_content["content"]))
+            if reply_content["completion_tokens"] > 0:
+                Session.save_session(reply_content["content"], session_id, reply_content["total_tokens"])
+            return reply_content["content"]
 
         elif context.get('type', None) == 'IMAGE_CREATE':
             return self.create_img(query, 0)
 
-    def reply_text(self, query, user_id, retry_count=0):
+    def reply_text(self, session, session_id, retry_count=0) ->dict:
+        '''
+        call openai's ChatCompletion to get the answer
+        :param session: a conversation session
+        :param session_id: session id
+        :param retry_count: retry count
+        :return: {}
+        '''
         try:
             response = openai.ChatCompletion.create(
                 model="gpt-3.5-turbo",  # 对话模型的名称
-                messages=query,
+                messages=session,
                 temperature=0.9,  # 值在[0,1]之间，越大表示回复越具有不确定性
-                max_tokens=1200,  # 回复最大的字符数
+                #max_tokens=4096,  # 回复最大的字符数
                 top_p=1,
                 frequency_penalty=0.0,  # [-2,2]之间，该值越大则更倾向于产生不同的内容
                 presence_penalty=0.0,  # [-2,2]之间，该值越大则更倾向于产生不同的内容
             )
-            # res_content = response.choices[0]['text'].strip().replace('<|endoftext|>', '')
-            logger.info(response.choices[0]['message']['content'])
-            # log.info("[OPEN_AI] reply={}".format(res_content))
-            return response.choices[0]['message']['content']
+            # logger.info("[ChatGPT] reply={}, total_tokens={}".format(response.choices[0]['message']['content'], response["usage"]["total_tokens"]))
+            return {"total_tokens": response["usage"]["total_tokens"], 
+                    "completion_tokens": response["usage"]["completion_tokens"], 
+                    "content": response.choices[0]['message']['content']}
         except openai.error.RateLimitError as e:
             # rate limit exception
             logger.warn(e)
             if retry_count < 1:
                 time.sleep(5)
                 logger.warn("[OPEN_AI] RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.reply_text(query, user_id, retry_count+1)
+                return self.reply_text(session, session_id, retry_count+1)
             else:
-                return "提问太快啦，请休息一下再问我吧"
+                return {"completion_tokens": 0, "content": "提问太快啦，请休息一下再问我吧"}
+        except openai.error.APIConnectionError as e:
+            # api connection exception
+            logger.warn(e)
+            logger.warn("[OPEN_AI] APIConnection failed")
+            return {"completion_tokens": 0, "content":"我连接不到你的网络"}
+        except openai.error.Timeout as e:
+            logger.warn(e)
+            logger.warn("[OPEN_AI] Timeout")
+            return {"completion_tokens": 0, "content":"我没有收到你的消息"}
         except Exception as e:
             # unknown exception
             logger.exception(e)
-            Session.clear_session(user_id)
-            return "请再问我一次吧"
+            Session.clear_session(session_id)
+            return {"completion_tokens": 0, "content": "请再问我一次吧"}
 
     def create_img(self, query, retry_count=0):
         try:
@@ -84,7 +110,7 @@ class ChatGPTBot(Bot):
             if retry_count < 1:
                 time.sleep(5)
                 logger.warn("[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.reply_text(query, retry_count+1)
+                return self.create_img(query, retry_count+1)
             else:
                 return "提问太快啦，请休息一下再问我吧"
         except Exception as e:
@@ -93,7 +119,7 @@ class ChatGPTBot(Bot):
 
 class Session(object):
     @staticmethod
-    def build_session_query(query, user_id):
+    def build_session_query(query, session_id):
         '''
         build query with conversation history
         e.g.  [
@@ -103,28 +129,54 @@ class Session(object):
             {"role": "user", "content": "Where was it played?"}
         ]
         :param query: query content
-        :param user_id: from user id
+        :param session_id: session id
         :return: query content with conversaction
         '''
-        session = user_session.get(user_id, [])
+        session = all_sessions.get(session_id, [])
         if len(session) == 0:
             system_prompt = conf().get("character_desc", "")
             system_item = {'role': 'system', 'content': system_prompt}
             session.append(system_item)
-            user_session[user_id] = session
+            all_sessions[session_id] = session
         user_item = {'role': 'user', 'content': query}
         session.append(user_item)
         return session
 
     @staticmethod
-    def save_session(query, answer, user_id):
-        session = user_session.get(user_id)
+    def save_session(answer, session_id, total_tokens):
+        max_tokens = conf().get("conversation_max_tokens")
+        if not max_tokens:
+            # default 3000
+            max_tokens = 1000
+        max_tokens=int(max_tokens)
+
+        session = all_sessions.get(session_id)
         if session:
             # append conversation
             gpt_item = {'role': 'assistant', 'content': answer}
             session.append(gpt_item)
 
-    @staticmethod
-    def clear_session(user_id):
-        user_session[user_id] = []
+        # discard exceed limit conversation
+        Session.discard_exceed_conversation(session, max_tokens, total_tokens)
+    
 
+    @staticmethod
+    def discard_exceed_conversation(session, max_tokens, total_tokens):
+        dec_tokens = int(total_tokens)
+        # logger.info("prompt tokens used={},max_tokens={}".format(used_tokens,max_tokens))
+        while dec_tokens > max_tokens:
+            # pop first conversation
+            if len(session) > 3:
+                session.pop(1)
+                session.pop(1)
+            else:
+                break    
+            dec_tokens = dec_tokens - max_tokens
+
+    @staticmethod
+    def clear_session(session_id):
+        all_sessions[session_id] = []
+
+    @staticmethod
+    def clear_all_session():
+        all_sessions.clear()

bridge/bridge.py

@@ -1,4 +1,5 @@
 from bot import bot_factory
+from voice import voice_factory
 
 
 class Bridge(object):
@@ -7,3 +8,9 @@ class Bridge(object):
 
     def fetch_reply_content(self, query, context):
         return bot_factory.create_bot("chatGPT").reply(query, context)
+
+    def fetch_voice_to_text(self, voiceFile):
+        return voice_factory.create_voice("openai").voiceToText(voiceFile)
+
+    def fetch_text_to_voice(self, text):
+        return voice_factory.create_voice("baidu").textToVoice(text)

channel/channel.py

@@ -11,7 +11,7 @@ class Channel(object):
         """
         raise NotImplementedError
 
-    def handle(self, msg):
+    def handle_text(self, msg):
         """
         process received msg
         :param msg: message object
@@ -29,3 +29,9 @@ class Channel(object):
 
     def build_reply_content(self, query, context=None):
         return Bridge().fetch_reply_content(query, context)
+
+    def build_voice_to_text(self, voice_file):
+        return Bridge().fetch_voice_to_text(voice_file)
+    
+    def build_text_to_voice(self, text):
+        return Bridge().fetch_text_to_voice(text)

channel/wechat/wechat_channel.py

@@ -3,12 +3,14 @@
 """
 wechat channel
 """
+
 import itchat
 import json
 from itchat.content import *
 from channel.channel import Channel
 from concurrent.futures import ThreadPoolExecutor
 from common.log import logger
+from common.tmp_dir import TmpDir
 from config import conf
 import requests
 import io
@@ -18,7 +20,7 @@ thread_pool = ThreadPoolExecutor(max_workers=8)
 
 @itchat.msg_register(TEXT)
 def handler_single_msg(msg):
-    WechatChannel().handle(msg)
+    WechatChannel().handle_text(msg)
     return None
 
 
@@ -28,6 +30,12 @@ def handler_group_msg(msg):
     return None
 
 
+@itchat.msg_register(VOICE)
+def handler_single_voice(msg):
+    WechatChannel().handle_voice(msg)
+    return None
+
+
 class WechatChannel(Channel):
     def __init__(self):
         pass
@@ -39,13 +47,37 @@ class WechatChannel(Channel):
         # start message listener
         itchat.run()
 
-    def handle(self, msg):
-        logger.debug("[WX]receive msg: " + json.dumps(msg, ensure_ascii=False))
+    def handle_voice(self, msg):
+        if conf().get('speech_recognition') != True :
+            return
+        logger.debug("[WX]receive voice msg: " + msg['FileName'])
+        thread_pool.submit(self._do_handle_voice, msg)
+
+    def _do_handle_voice(self, msg):
+        from_user_id = msg['FromUserName']
+        other_user_id = msg['User']['UserName']
+        if from_user_id == other_user_id:
+            file_name = TmpDir().path() + msg['FileName']
+            msg.download(file_name)
+            query = super().build_voice_to_text(file_name)
+            if conf().get('voice_reply_voice'):
+                self._do_send_voice(query, from_user_id)
+            else:
+                self._do_send_text(query, from_user_id)
+
+    def handle_text(self, msg):
+        logger.debug("[WX]receive text msg: " + json.dumps(msg, ensure_ascii=False))
+        content = msg['Text']
+        self._handle_single_msg(msg, content)
+
+    def _handle_single_msg(self, msg, content):
         from_user_id = msg['FromUserName']
         to_user_id = msg['ToUserName']              # 接收人id
         other_user_id = msg['User']['UserName']     # 对手方id
-        content = msg['Text']
         match_prefix = self.check_prefix(content, conf().get('single_chat_prefix'))
+        if "」\n- - - - - - - - - - - - - - -" in content:
+            logger.debug("[WX]reference query skipped")
+            return
         if from_user_id == other_user_id and match_prefix is not None:
             # 好友向自己发送消息
             if match_prefix != '':
@@ -57,9 +89,8 @@ class WechatChannel(Channel):
             if img_match_prefix:
                 content = content.split(img_match_prefix, 1)[1].strip()
                 thread_pool.submit(self._do_send_img, content, from_user_id)
-            else:
-                thread_pool.submit(self._do_send, content, from_user_id)
-
+            else :
+                thread_pool.submit(self._do_send_text, content, from_user_id)
         elif to_user_id == other_user_id and match_prefix:
             # 自己给好友发送消息
             str_list = content.split(match_prefix, 1)
@@ -70,7 +101,7 @@ class WechatChannel(Channel):
                 content = content.split(img_match_prefix, 1)[1].strip()
                 thread_pool.submit(self._do_send_img, content, to_user_id)
             else:
-                thread_pool.submit(self._do_send, content, to_user_id)
+                thread_pool.submit(self._do_send_text, content, to_user_id)
 
 
     def handle_group(self, msg):
@@ -87,7 +118,9 @@ class WechatChannel(Channel):
             content = context_special_list[1]
         elif len(content_list) == 2:
             content = content_list[1]
-
+        if "」\n- - - - - - - - - - - - - - -" in content:
+            logger.debug("[WX]reference query skipped")
+            return ""
         config = conf()
         match_prefix = (msg['IsAt'] and not config.get("group_at_off", False)) or self.check_prefix(origin_content, config.get('group_chat_prefix')) \
                        or self.check_contain(origin_content, config.get('group_chat_keyword'))
@@ -100,16 +133,30 @@ class WechatChannel(Channel):
                 thread_pool.submit(self._do_send_group, content, msg)
 
     def send(self, msg, receiver):
-        logger.info('[WX] sendMsg={}, receiver={}'.format(msg, receiver))
         itchat.send(msg, toUserName=receiver)
+        logger.info('[WX] sendMsg={}, receiver={}'.format(msg, receiver))
 
-    def _do_send(self, query, reply_user_id):
+    def _do_send_voice(self, query, reply_user_id):
         try:
             if not query:
                 return
             context = dict()
             context['from_user_id'] = reply_user_id
             reply_text = super().build_reply_content(query, context)
+            if reply_text:
+                replyFile = super().build_text_to_voice(reply_text)
+                itchat.send_file(replyFile, toUserName=reply_user_id)
+                logger.info('[WX] sendFile={}, receiver={}'.format(replyFile, reply_user_id))
+        except Exception as e:
+            logger.exception(e)
+
+    def _do_send_text(self, query, reply_user_id):
+        try:
+            if not query:
+                return
+            context = dict()
+            context['session_id'] = reply_user_id
+            reply_text = super().build_reply_content(query, context)
             if reply_text:
                 self.send(conf().get("single_chat_reply_prefix") + reply_text, reply_user_id)
         except Exception as e:
@@ -133,8 +180,8 @@ class WechatChannel(Channel):
             image_storage.seek(0)
 
             # 图片发送
-            logger.info('[WX] sendImage, receiver={}'.format(reply_user_id))
             itchat.send_image(image_storage, reply_user_id)
+            logger.info('[WX] sendImage, receiver={}'.format(reply_user_id))
         except Exception as e:
             logger.exception(e)
 
@@ -142,11 +189,19 @@ class WechatChannel(Channel):
         if not query:
             return
         context = dict()
-        context['from_user_id'] = msg['ActualUserName']
+        group_name = msg['User']['NickName']
+        group_id = msg['User']['UserName']
+        group_chat_in_one_session = conf().get('group_chat_in_one_session', [])
+        if ('ALL_GROUP' in group_chat_in_one_session or \
+                group_name in group_chat_in_one_session or \
+                self.check_contain(group_name, group_chat_in_one_session)):
+            context['session_id'] = group_id
+        else:
+            context['session_id'] = msg['ActualUserName']
         reply_text = super().build_reply_content(query, context)
         if reply_text:
             reply_text = '@' + msg['ActualNickName'] + ' ' + reply_text.strip()
-            self.send(conf().get("group_chat_reply_prefix", "") + reply_text, msg['User']['UserName'])
+            self.send(conf().get("group_chat_reply_prefix", "") + reply_text, group_id)
 
 
     def check_prefix(self, content, prefix_list):
@@ -163,3 +218,4 @@ class WechatChannel(Channel):
             if content.find(ky) != -1:
                 return True
         return None
+

channel/wechat/wechaty_channel.py

@@ -109,7 +109,7 @@ class WechatyChannel(Channel):
                     content = content.split(img_match_prefix, 1)[1].strip()
                     await self._do_send_group_img(content, room_id)
                 else:
-                    await self._do_send_group(content, room_id, from_user_id, from_user_name)
+                    await self._do_send_group(content, room_id, room_name, from_user_id, from_user_name)
 
     async def send(self, message: Union[str, Message, FileBox, Contact, UrlLink, MiniProgram], receiver):
         logger.info('[WX] sendMsg={}, receiver={}'.format(message, receiver))
@@ -128,7 +128,7 @@ class WechatyChannel(Channel):
             if not query:
                 return
             context = dict()
-            context['from_user_id'] = reply_user_id
+            context['session_id'] = reply_user_id
             reply_text = super().build_reply_content(query, context)
             if reply_text:
                 await self.send(conf().get("single_chat_reply_prefix") + reply_text, reply_user_id)
@@ -159,11 +159,17 @@ class WechatyChannel(Channel):
         except Exception as e:
             logger.exception(e)
 
-    async def _do_send_group(self, query, group_id, group_user_id, group_user_name):
+    async def _do_send_group(self, query, group_id, group_name, group_user_id, group_user_name):
         if not query:
             return
         context = dict()
-        context['from_user_id'] = str(group_id) + '-' + str(group_user_id)
+        group_chat_in_one_session = conf().get('group_chat_in_one_session', [])
+        if ('ALL_GROUP' in group_chat_in_one_session or \
+                group_name in group_chat_in_one_session or \
+                self.check_contain(group_name, group_chat_in_one_session)):
+            context['session_id'] = str(group_id)
+        else:
+            context['session_id'] = str(group_id) + '-' + str(group_user_id)
         reply_text = super().build_reply_content(query, context)
         if reply_text:
             reply_text = '@' + group_user_name + ' ' + reply_text.strip()

common/expired_dict.py

@@ -0,0 +1,23 @@
+from datetime import datetime, timedelta
+
+class ExpiredDict(dict):
+    def __init__(self, expires_in_seconds):
+        super().__init__()
+        self.expires_in_seconds = expires_in_seconds
+
+    def __getitem__(self, key):
+        value, expiry_time = super().__getitem__(key)
+        if datetime.now() > expiry_time:
+            del self[key]
+            raise KeyError("expired {}".format(key))
+        self.__setitem__(key, value)
+        return value
+
+    def __setitem__(self, key, value):
+        expiry_time = datetime.now() + timedelta(seconds=self.expires_in_seconds)
+        super().__setitem__(key, (value, expiry_time))
+    def get(self, key, default=None):
+        try:
+            return self[key]
+        except KeyError:
+            return default

common/tmp_dir.py

@@ -0,0 +1,20 @@
+
+import os
+import pathlib
+from config import conf
+
+
+class TmpDir(object):
+    """A temporary directory that is deleted when the object is destroyed.
+    """
+
+    tmpFilePath = pathlib.Path('./tmp/')
+    
+    def __init__(self):
+        pathExists = os.path.exists(self.tmpFilePath)
+        if not pathExists and conf().get('speech_recognition') == True:
+            os.makedirs(self.tmpFilePath)
+
+    def path(self):
+        return str(self.tmpFilePath) + '/'
+    

config-template.json

@@ -1,11 +1,14 @@
 {
   "open_ai_api_key": "YOUR API KEY",
-  "wechaty_puppet_service_token": "WECHATY PUPPET SERVICE TOKEN",
+  "proxy": "",
   "single_chat_prefix": ["bot", "@bot"],
   "single_chat_reply_prefix": "[bot] ",
   "group_chat_prefix": ["@bot"],
+  "group_chat_in_one_session": ["ChatGPT测试群"],
   "group_name_white_list": ["ChatGPT测试群", "ChatGPT测试群2"],
   "image_create_prefix": ["画", "看", "找"],
   "conversation_max_tokens": 1000,
-  "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。"
+  "speech_recognition": false,
+  "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。",
+  "expires_in_seconds": 3600
 }

docker/Dockerfile.alpine

@@ -3,7 +3,7 @@ FROM python:3.7.9-alpine
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
 
-ARG CHATGPT_ON_WECHAT_VER=1.0.2
+ARG CHATGPT_ON_WECHAT_VER
 
 ENV BUILD_PREFIX=/app \
     BUILD_OPEN_AI_API_KEY='YOUR OPEN AI KEY HERE'
@@ -12,35 +12,29 @@ RUN apk add --no-cache \
         bash \
         curl \
         wget \
-        gcc \
-        g++ \
-        ca-certificates \
-        openssh \ 
-        libffi-dev 
-
-RUN wget -t 3 -T 30 -nv -O chatgpt-on-wechat-${CHATGPT_ON_WECHAT_VER}.tar.gz \
-          https://github.com/zhayujie/chatgpt-on-wechat/archive/refs/tags/${CHATGPT_ON_WECHAT_VER}.tar.gz \
-    && tar -xzf chatgpt-on-wechat-${CHATGPT_ON_WECHAT_VER}.tar.gz \
-    && mv chatgpt-on-wechat-${CHATGPT_ON_WECHAT_VER} ${BUILD_PREFIX} \
-    && rm chatgpt-on-wechat-${CHATGPT_ON_WECHAT_VER}.tar.gz
+    && export BUILD_GITHUB_TAG=${CHATGPT_ON_WECHAT_VER:-`curl -sL "https://api.github.com/repos/zhayujie/chatgpt-on-wechat/releases/latest" | \
+        grep '"tag_name":' | \
+        sed -E 's/.*"([^"]+)".*/\1/'`} \
+    && wget -t 3 -T 30 -nv -O chatgpt-on-wechat-${BUILD_GITHUB_TAG}.tar.gz \
+            https://github.com/zhayujie/chatgpt-on-wechat/archive/refs/tags/${BUILD_GITHUB_TAG}.tar.gz \
+    && tar -xzf chatgpt-on-wechat-${BUILD_GITHUB_TAG}.tar.gz \
+    && mv chatgpt-on-wechat-${BUILD_GITHUB_TAG} ${BUILD_PREFIX} \
+    && rm chatgpt-on-wechat-${BUILD_GITHUB_TAG}.tar.gz \
+    && cd ${BUILD_PREFIX} \
+    && cp config-template.json ${BUILD_PREFIX}/config.json \
+    && sed -i "2s/YOUR API KEY/${BUILD_OPEN_AI_API_KEY}/" ${BUILD_PREFIX}/config.json \
+    && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
+    && pip install --no-cache   \
+        itchat-uos==1.5.0.dev0  \
+        openai                  \
+    && apk del curl wget
 
 WORKDIR ${BUILD_PREFIX}
 
-RUN cd ${BUILD_PREFIX} \
-    && cp config-template.json ${BUILD_PREFIX}/config.json \
-    && sed -i "2s/YOUR API KEY/${BUILD_OPEN_AI_API_KEY}/" ${BUILD_PREFIX}/config.json
-
-RUN /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache \
-      itchat-uos==1.5.0.dev0  \
-      openai \
-      wechaty
-
 ADD ./entrypoint.sh /entrypoint.sh
 
-RUN chmod +x /entrypoint.sh
-
-RUN adduser -D -h /home/noroot -u 1000 -s /bin/bash noroot \
+RUN chmod +x /entrypoint.sh \
+    && adduser -D -h /home/noroot -u 1000 -s /bin/bash noroot \
     && chown noroot:noroot ${BUILD_PREFIX}
 
 USER noroot

docker/Dockerfile.debian

@@ -3,42 +3,40 @@ FROM python:3.7.9
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
 
-ARG CHATGPT_ON_WECHAT_VER=1.0.2
+ARG CHATGPT_ON_WECHAT_VER
 
 ENV BUILD_PREFIX=/app \
     BUILD_OPEN_AI_API_KEY='YOUR OPEN AI KEY HERE'
 
-RUN apt-get update && \
-    apt-get install -y --no-install-recommends \
+RUN apt-get update  \
+    && apt-get install -y --no-install-recommends \
         wget \
-        curl && \
-    rm -rf /var/lib/apt/lists/*
-
-RUN wget -t 3 -T 30 -nv -O chatgpt-on-wechat-${CHATGPT_ON_WECHAT_VER}.tar.gz \
-          https://github.com/zhayujie/chatgpt-on-wechat/archive/refs/tags/${CHATGPT_ON_WECHAT_VER}.tar.gz \
-    && tar -xzf chatgpt-on-wechat-${CHATGPT_ON_WECHAT_VER}.tar.gz \
-    && mv chatgpt-on-wechat-${CHATGPT_ON_WECHAT_VER} ${BUILD_PREFIX} \
-    && rm chatgpt-on-wechat-${CHATGPT_ON_WECHAT_VER}.tar.gz
+        curl  \
+    && rm -rf /var/lib/apt/lists/* \
+    && export BUILD_GITHUB_TAG=${CHATGPT_ON_WECHAT_VER:-`curl -sL "https://api.github.com/repos/zhayujie/chatgpt-on-wechat/releases/latest" | \
+        grep '"tag_name":' | \
+        sed -E 's/.*"([^"]+)".*/\1/'`} \
+    && wget -t 3 -T 30 -nv -O chatgpt-on-wechat-${BUILD_GITHUB_TAG}.tar.gz \
+            https://github.com/zhayujie/chatgpt-on-wechat/archive/refs/tags/${BUILD_GITHUB_TAG}.tar.gz \
+    && tar -xzf chatgpt-on-wechat-${BUILD_GITHUB_TAG}.tar.gz \
+    && mv chatgpt-on-wechat-${BUILD_GITHUB_TAG} ${BUILD_PREFIX} \
+    && rm chatgpt-on-wechat-${BUILD_GITHUB_TAG}.tar.gz \
+    && cd ${BUILD_PREFIX} \
+    && cp config-template.json ${BUILD_PREFIX}/config.json \
+    && sed -i "2s/YOUR API KEY/${BUILD_OPEN_AI_API_KEY}/" ${BUILD_PREFIX}/config.json \
+    && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
+    && pip install --no-cache \
+        itchat-uos==1.5.0.dev0  \
+        openai 
 
 WORKDIR ${BUILD_PREFIX}
 
-RUN cd ${BUILD_PREFIX} \
-    && cp config-template.json ${BUILD_PREFIX}/config.json \
-    && sed -i "2s/YOUR API KEY/${BUILD_OPEN_AI_API_KEY}/" ${BUILD_PREFIX}/config.json 
-
-RUN /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache \
-      itchat-uos==1.5.0.dev0  \
-      openai \
-      wechaty
-
 ADD ./entrypoint.sh /entrypoint.sh
 
-RUN chmod +x /entrypoint.sh
-
-RUN groupadd -r noroot \
-   && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
-   && chown -R noroot:noroot ${BUILD_PREFIX} 
+RUN chmod +x /entrypoint.sh \
+    && groupadd -r noroot \
+    && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
+    && chown -R noroot:noroot ${BUILD_PREFIX} 
 
 USER noroot
 

docker/build.alpine.sh

@@ -1,10 +1,15 @@
 #!/bin/bash
 
-CHATGPT_ON_WECHAT_TAG=1.0.2
+# fetch latest release tag
+CHATGPT_ON_WECHAT_TAG=`curl -sL "https://api.github.com/repos/zhayujie/chatgpt-on-wechat/releases/latest" | \
+     grep '"tag_name":' | \
+     sed -E 's/.*"([^"]+)".*/\1/'`
 
+# build image
 docker build -f Dockerfile.alpine \
              --build-arg CHATGPT_ON_WECHAT_VER=$CHATGPT_ON_WECHAT_TAG \
              -t zhayujie/chatgpt-on-wechat .
 
+# tag image
 docker tag zhayujie/chatgpt-on-wechat zhayujie/chatgpt-on-wechat:$CHATGPT_ON_WECHAT_TAG-alpine 
            

docker/build.debian.sh

@@ -1,9 +1,14 @@
 #!/bin/bash
 
-CHATGPT_ON_WECHAT_TAG=1.0.2
+# fetch latest release tag
+CHATGPT_ON_WECHAT_TAG=`curl -sL "https://api.github.com/repos/zhayujie/chatgpt-on-wechat/releases/latest" | \
+     grep '"tag_name":' | \
+     sed -E 's/.*"([^"]+)".*/\1/'`
 
+# build image
 docker build -f Dockerfile.debian \
              --build-arg CHATGPT_ON_WECHAT_VER=$CHATGPT_ON_WECHAT_TAG \
              -t zhayujie/chatgpt-on-wechat .
 
+# tag image
 docker tag zhayujie/chatgpt-on-wechat zhayujie/chatgpt-on-wechat:$CHATGPT_ON_WECHAT_TAG-debian

docker/docker-compose.yaml

@@ -8,11 +8,12 @@ services:
     container_name: sample-chatgpt-on-wechat
     environment:
       OPEN_AI_API_KEY: 'YOUR API KEY'
-      WECHATY_PUPPET_SERVICE_TOKEN: 'WECHATY PUPPET SERVICE TOKEN'
+      OPEN_AI_PROXY: ''
       SINGLE_CHAT_PREFIX: '["bot", "@bot"]'
       SINGLE_CHAT_REPLY_PREFIX: '"[bot] "'
       GROUP_CHAT_PREFIX: '["@bot"]'
       GROUP_NAME_WHITE_LIST: '["ChatGPT测试群", "ChatGPT测试群2"]'
       IMAGE_CREATE_PREFIX: '["画", "看", "找"]'
       CONVERSATION_MAX_TOKENS: 1000
-      CHARACTER_DESC: '你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。'
+      CHARACTER_DESC: '你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。'
+      EXPIRES_IN_SECONDS: 3600

docker/entrypoint.sh

@@ -9,6 +9,7 @@ CHATGPT_ON_WECHAT_CONFIG_PATH=${CHATGPT_ON_WECHAT_CONFIG_PATH:-""}
 CHATGPT_ON_WECHAT_EXEC=${CHATGPT_ON_WECHAT_EXEC:-""}
 
 OPEN_AI_API_KEY=${OPEN_AI_API_KEY:-""}
+OPEN_AI_PROXY=${OPEN_AI_PROXY:-""}
 SINGLE_CHAT_PREFIX=${SINGLE_CHAT_PREFIX:-""}
 SINGLE_CHAT_REPLY_PREFIX=${SINGLE_CHAT_REPLY_PREFIX:-""}
 GROUP_CHAT_PREFIX=${GROUP_CHAT_PREFIX:-""}
@@ -16,6 +17,7 @@ GROUP_NAME_WHITE_LIST=${GROUP_NAME_WHITE_LIST:-""}
 IMAGE_CREATE_PREFIX=${IMAGE_CREATE_PREFIX:-""}
 CONVERSATION_MAX_TOKENS=${CONVERSATION_MAX_TOKENS:-""}
 CHARACTER_DESC=${CHARACTER_DESC:-""}
+EXPIRES_IN_SECONDS=${EXPIRES_IN_SECONDS:-""}
 
 # CHATGPT_ON_WECHAT_PREFIX is empty, use /app
 if [ "$CHATGPT_ON_WECHAT_PREFIX" == "" ] ; then
@@ -39,10 +41,13 @@ else
     echo -e "\033[31m[Warning] You need to set OPEN_AI_API_KEY before running!\033[0m"
 fi
 
-if [ "$WECHATY_PUPPET_SERVICE_TOKEN" != "" ] ; then
-    sed -i "3c   \"wechaty_puppet_service_token\": \"$WECHATY_PUPPET_SERVICE_TOKEN\"," $CHATGPT_ON_WECHAT_CONFIG_PATH
-else
-    echo -e "\033[31m[Info] You need to set WECHATY_PUPPET_SERVICE_TOKEN if you use wechaty!\033[0m"
+# use http_proxy as default
+if [ "$HTTP_PROXY" != "" ] ; then
+    sed -i "3c   \"proxy\": \"$HTTP_PROXY\"," $CHATGPT_ON_WECHAT_CONFIG_PATH
+fi
+
+if [ "$OPEN_AI_PROXY" != "" ] ; then
+    sed -i "3c   \"proxy\": \"$OPEN_AI_PROXY\"," $CHATGPT_ON_WECHAT_CONFIG_PATH
 fi
 
 if [ "$SINGLE_CHAT_PREFIX" != "" ] ; then
@@ -70,7 +75,11 @@ if [ "$CONVERSATION_MAX_TOKENS" != "" ] ; then
 fi
 
 if [ "$CHARACTER_DESC" != "" ] ; then
-    sed -i "10c   \"character_desc\": \"$CHARACTER_DESC\"" $CHATGPT_ON_WECHAT_CONFIG_PATH
+    sed -i "10c   \"character_desc\": \"$CHARACTER_DESC\"," $CHATGPT_ON_WECHAT_CONFIG_PATH
+fi
+
+if [ "$EXPIRES_IN_SECONDS" != "" ] ; then
+    sed -i "11c   \"expires_in_seconds\": $EXPIRES_IN_SECONDS" $CHATGPT_ON_WECHAT_CONFIG_PATH
 fi
 
 # go to prefix dir

docker/sample-chatgpt-on-wechat/.env

@@ -1,5 +1,5 @@
 OPEN_AI_API_KEY=YOUR API KEY
-WECHATY_PUPPET_SERVICE_TOKEN=WECHATY PUPPET SERVICE TOKEN
+OPEN_AI_PROXY=
 SINGLE_CHAT_PREFIX=["bot", "@bot"]
 SINGLE_CHAT_REPLY_PREFIX="[bot] "
 GROUP_CHAT_PREFIX=["@bot"]
@@ -7,6 +7,7 @@ GROUP_NAME_WHITE_LIST=["ChatGPT测试群", "ChatGPT测试群2"]
 IMAGE_CREATE_PREFIX=["画", "看", "找"]
 CONVERSATION_MAX_TOKENS=1000
 CHARACTER_DESC=你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。
+EXPIRES_IN_SECONDS=3600
 
 # Optional
 #CHATGPT_ON_WECHAT_PREFIX=/app

voice/baidu/baidu_voice.py

@@ -0,0 +1,36 @@
+
+"""
+baidu voice service
+"""
+import time
+from aip import AipSpeech
+from common.log import logger
+from common.tmp_dir import TmpDir
+from voice.voice import Voice
+from config import conf
+
+class BaiduVoice(Voice):
+    APP_ID = conf().get('baidu_app_id')
+    API_KEY = conf().get('baidu_api_key')
+    SECRET_KEY = conf().get('baidu_secret_key')
+    client = AipSpeech(APP_ID, API_KEY, SECRET_KEY)
+    
+    def __init__(self):
+        pass
+
+    def voiceToText(self, voice_file):
+        pass
+
+    def textToVoice(self, text):
+        result = self.client.synthesis(text, 'zh', 1, {
+            'spd': 5, 'pit': 5, 'vol': 5, 'per': 111
+        })
+        if not isinstance(result, dict):
+            fileName = TmpDir().path() + '语音回复_' + str(int(time.time())) + '.mp3'
+            with open(fileName, 'wb') as f:
+                f.write(result)
+            logger.info('[Baidu] textToVoice text={} voice file name={}'.format(text, fileName))
+            return fileName
+        else:
+            logger.error('[Baidu] textToVoice error={}'.format(result))
+            return None

voice/google/google_voice.py

@@ -0,0 +1,51 @@
+
+"""
+google voice service
+"""
+
+import pathlib
+import subprocess
+import time
+import speech_recognition
+import pyttsx3
+from common.log import logger
+from common.tmp_dir import TmpDir
+from voice.voice import Voice
+
+
+class GoogleVoice(Voice):
+    recognizer = speech_recognition.Recognizer()
+    engine = pyttsx3.init()
+
+    def __init__(self):
+        # 语速
+        self.engine.setProperty('rate', 125)
+        # 音量
+        self.engine.setProperty('volume', 1.0)
+        # 0为男声，1为女声
+        voices = self.engine.getProperty('voices')
+        self.engine.setProperty('voice', voices[1].id)
+
+    def voiceToText(self, voice_file):
+        new_file = voice_file.replace('.mp3', '.wav')
+        subprocess.call('ffmpeg -i ' + voice_file +
+                        ' -acodec pcm_s16le -ac 1 -ar 16000 ' + new_file, shell=True)
+        with speech_recognition.AudioFile(new_file) as source:
+            audio = self.recognizer.record(source)
+        try:
+            text = self.recognizer.recognize_google(audio, language='zh-CN')
+            logger.info(
+                '[Google] voiceToText text={} voice file name={}'.format(text, voice_file))
+            return text
+        except speech_recognition.UnknownValueError:
+            return "抱歉，我听不懂。"
+        except speech_recognition.RequestError as e:
+            return "抱歉，无法连接到 Google 语音识别服务；{0}".format(e)
+
+    def textToVoice(self, text):
+        textFile = TmpDir().path() + '语音回复_' + str(int(time.time())) + '.mp3'
+        self.engine.save_to_file(text, textFile)
+        self.engine.runAndWait()
+        logger.info(
+            '[Google] textToVoice text={} voice file name={}'.format(text, textFile))
+        return textFile

voice/openai/openai_voice.py

@@ -0,0 +1,27 @@
+
+"""
+google voice service
+"""
+import json
+import openai
+from config import conf
+from common.log import logger
+from voice.voice import Voice
+
+
+class OpenaiVoice(Voice):
+    def __init__(self):
+        openai.api_key = conf().get('open_ai_api_key')
+
+    def voiceToText(self, voice_file):
+        logger.debug(
+            '[Openai] voice file name={}'.format(voice_file))
+        file = open(voice_file, "rb")
+        reply = openai.Audio.transcribe("whisper-1", file)
+        text = reply["text"]
+        logger.info(
+            '[Openai] voiceToText text={} voice file name={}'.format(text, voice_file))
+        return text
+
+    def textToVoice(self, text):
+        pass

voice/voice.py

@@ -0,0 +1,16 @@
+"""
+Voice service abstract class
+"""
+
+class Voice(object):
+    def voiceToText(self, voice_file):
+        """
+        Send voice to voice service and get text
+        """
+        raise NotImplementedError
+
+    def textToVoice(self, text):
+        """
+        Send text to voice service and get voice
+        """
+        raise NotImplementedError

voice/voice_factory.py

@@ -0,0 +1,20 @@
+"""
+voice factory
+"""
+
+def create_voice(voice_type):
+    """
+    create a voice instance
+    :param voice_type: voice type code
+    :return: voice instance
+    """
+    if voice_type == 'baidu':
+        from voice.baidu.baidu_voice import BaiduVoice
+        return BaiduVoice()
+    elif voice_type == 'google':
+        from voice.google.google_voice import GoogleVoice
+        return GoogleVoice()
+    elif voice_type == 'openai':
+        from voice.openai.openai_voice import OpenaiVoice
+        return OpenaiVoice()
+    raise RuntimeError