Skip to content

Commit

Permalink
Merge pull request #13 from DjangoPeng/v0.5
Browse files Browse the repository at this point in the history
feat(v0.5): add email notifications, schedule and daemon service management.
  • Loading branch information
DjangoPeng authored Aug 25, 2024
2 parents 7f76b6e + a638217 commit 9e0b241
Show file tree
Hide file tree
Showing 18 changed files with 873 additions and 186 deletions.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -164,4 +164,5 @@ cython_debug/
# User-defined
daily_progress/*
src/jupyter/*.md
src/jupyter/daily_progress/*
src/jupyter/daily_progress/*
run/*
70 changes: 54 additions & 16 deletions README-EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,52 +24,90 @@ pip install -r requirements.txt

### 2. Configure the Application

Edit the `config.json` file to set up your GitHub token, notification settings, subscription file, and update interval:
Edit the `config.json` file to set up your GitHub token, Email settings(e.g.Tencent Exmail), subscription file, and update settings:


```json
{
"github_token": "your_github_token",
"notification_settings": {
"email": "[email protected]",
"slack_webhook_url": "your_slack_webhook_url"
"email": {
"smtp_server": "smtp.exmail.qq.com",
"smtp_port": 465,
"from": "[email protected]",
"password": "your_email_password",
"to": "[email protected]"
},
"slack_webhook_url": "your_slack_webhook_url",
"subscriptions_file": "subscriptions.json",
"update_interval": 86400
"github_progress_frequency_days": 1,
"github_progress_execution_time":"08:00"
}

```
**For security reasons:** It is recommended to configure the GitHub Token and Email Password using environment variables to avoid storing sensitive information in plain text, as shown below:

```shell
# GitHub
export GITHUB_TOKEN="github_pat_xxx"
# Email
export EMAIL_PASSWORD="password"
```

### 3. How to Run

GitHub Sentinel supports three different ways to run the application:
GitHub Sentinel supports the following three modes of operation:

#### A. Run as a Command-Line Tool

You can run the application interactively from the command line:
You can interactively run the application from the command line:

```sh
python src/command_tool.py
```

In this mode, you can manually input commands to manage subscriptions, retrieve updates, and generate reports.
In this mode, you can manually enter commands to manage subscriptions, retrieve updates, and generate reports.

#### B. Run as a Daemon Process with Scheduler
#### B. Run as a Background Service

To run the application as a background service (daemon) that regularly checks for updates:
To run the application as a background service (daemon), it will automatically update according to the configured schedule.

1. Ensure you have the `python-daemon` package installed:
You can use the daemon management script [daemon_control.sh](daemon_control.sh) to start, check the status, stop, and restart:

1. Start the service:

```sh
pip install python-daemon
$ ./daemon_control.sh start
Starting DaemonProcess...
DaemonProcess started.
```

2. Launch the daemon process:
- This will launch [./src/daemon_process.py], generating reports periodically as set in `config.json`, and sending emails.
- Service logs will be saved to `logs/DaemonProcess.log`, with historical logs also appended to `logs/app.log`.

2. Check the service status:

```sh
nohup python3 src/daemon_process.py > logs/daemon_process.log 2>&1 &
$ ./daemon_control.sh status
DaemonProcess is running.
```

- This will start the scheduler in the background, checking for updates at the interval specified in your `config.json`.
- Logs will be saved to the `logs/daemon_process.log` file.
3. Stop the service:

```sh
$ ./daemon_control.sh stop
Stopping DaemonProcess...
DaemonProcess stopped.
```

4. Restart the service:

```sh
$ ./daemon_control.sh restart
Stopping DaemonProcess...
DaemonProcess stopped.
Starting DaemonProcess...
DaemonProcess started.
```

#### C. Run as a Gradio Server

Expand Down
62 changes: 49 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,19 +24,32 @@ pip install -r requirements.txt

### 2. 配置应用

编辑 `config.json` 文件,以设置您的 GitHub 令牌、通知设置、订阅文件和更新间隔
编辑 `config.json` 文件,以设置您的 GitHub Token、Email 设置(以腾讯企微邮箱为例)、订阅文件和更新设置

```json
{
"github_token": "your_github_token",
"notification_settings": {
"email": "[email protected]",
"slack_webhook_url": "your_slack_webhook_url"
"email": {
"smtp_server": "smtp.exmail.qq.com",
"smtp_port": 465,
"from": "[email protected]",
"password": "your_email_password",
"to": "[email protected]"
},
"slack_webhook_url": "your_slack_webhook_url",
"subscriptions_file": "subscriptions.json",
"update_interval": 86400
"github_progress_frequency_days": 1,
"github_progress_execution_time":"08:00"
}
```
**出于安全考虑:** GitHub Token 和 Email Password 的设置均支持使用环境变量进行配置,以避免明文配置重要信息,如下所示:

```shell
# Github
export GITHUB_TOKEN="github_pat_xxx"
# Email
export EMAIL_PASSWORD="password"
```

### 3. 如何运行

Expand All @@ -52,25 +65,48 @@ python src/command_tool.py

在此模式下,您可以手动输入命令来管理订阅、检索更新和生成报告。

#### B. 作为后台进程运行(带调度器)
#### B. 作为后台服务运行

要将该应用作为后台服务(守护进程)运行,它将根据相关配置定期自动更新。

您可以直接使用守护进程管理脚本 [daemon_control.sh](daemon_control.sh) 来启动、查询状态、关闭和重启:

1. 启动服务:

要将该应用作为后台服务(守护进程)运行,它将定期检查更新:
```sh
$ ./daemon_control.sh start
Starting DaemonProcess...
DaemonProcess started.
```

- 这将启动[./src/daemon_process.py],按照 `config.json` 中设置的更新频率和时间点定期生成报告,并发送邮件。
- 本次服务日志将保存到 `logs/DaemonProcess.log` 文件中。同时,历史累计日志也将同步追加到 `logs/app.log` 日志文件中。

1. 确保您已安装 `python-daemon`
2. 查询服务状态

```sh
pip install python-daemon
$ ./daemon_control.sh status
DaemonProcess is running.
```

2. 启动后台进程
3. 关闭服务

```sh
nohup python3 src/daemon_process.py > logs/daemon_process.log 2>&1 &
$ ./daemon_control.sh stop
Stopping DaemonProcess...
DaemonProcess stopped.
```

- 这将启动后台调度器,按照 `config.json` 中指定的间隔定期检查更新。
- 日志将保存到 `logs/daemon_process.log` 文件中。
4. 重启服务:

```sh
$ ./daemon_control.sh restart
Stopping DaemonProcess...
DaemonProcess stopped.
Starting DaemonProcess...
DaemonProcess started.
```

#### C. 作为 Gradio 服务器运行

要使用 Gradio 界面运行应用,允许用户通过 Web 界面与该工具交互:
Expand Down
13 changes: 9 additions & 4 deletions config.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
{
"github_token": "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",
"notification_settings": {
"email": "[email protected]",
"slack_webhook_url": "your_slack_webhook_url"
"email": {
"smtp_server": "smtp.exmail.qq.com",
"smtp_port": 465,
"from": "[email protected]",
"password": "",
"to": "[email protected]"
},
"slack_webhook_url": "your_slack_webhook_url",
"subscriptions_file": "subscriptions.json",
"update_interval": 86400
"github_progress_frequency_days": 1,
"github_progress_execution_time":"08:00"
}
75 changes: 75 additions & 0 deletions daemon_control.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/bin/bash
# 守护进程控制脚本

# 定义守护进程 Python 脚本的路径
DAEMON_PATH="./src/daemon_process.py"
# 定义守护进程的名称
DAEMON_NAME="DaemonProcess"
# 定义日志文件的路径
LOG_FILE="./logs/$DAEMON_NAME.log"
# 定义守护进程的 PID 文件路径,用于存储进程号
PID_FILE="./run/$DAEMON_NAME.pid"

# 启动守护进程的函数
start() {
echo "Starting $DAEMON_NAME..."
# 使用 nohup 命令在后台运行 Python 脚本,并将输出重定向到日志文件
nohup python3 $DAEMON_PATH > $LOG_FILE 2>&1 &
# 将守护进程的 PID 写入文件
echo $! > $PID_FILE
echo "$DAEMON_NAME started."
}

# 停止守护进程的函数
stop() {
if [ -f $PID_FILE ]; then
# 如果 PID 文件存在,读取 PID
PID=$(cat $PID_FILE)
echo "Stopping $DAEMON_NAME..."
# 使用 kill 命令停止进程
kill $PID
echo "$DAEMON_NAME stopped."
# 删除 PID 文件
rm $PID_FILE
else
echo "$DAEMON_NAME is not running."
fi
}

# 检查守护进程状态的函数
status() {
if [ -f $PID_FILE ]; then
PID=$(cat $PID_FILE)
# 检查进程是否在运行
if ps -p $PID > /dev/null
then
echo "$DAEMON_NAME is running."
else
echo "$DAEMON_NAME is not running."
fi
else
echo "$DAEMON_NAME is not running."
fi
}

# 根据输入参数选择执行哪个函数
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart)
# 重启守护进程
stop
start
;;
*)
# 如果参数不符合预期,显示用法
echo "Usage: $0 {start|stop|status|restart}"
exit 1
esac
3 changes: 2 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ requests
openai
gradio
loguru
python-daemon
schedule
markdown2
2 changes: 0 additions & 2 deletions src/command_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

from config import Config # 从config模块导入Config类,用于配置管理
from github_client import GitHubClient # 从github_client模块导入GitHubClient类,用于GitHub API操作
from notifier import Notifier # 从notifier模块导入Notifier类,用于通知功能
from report_generator import ReportGenerator # 从report_generator模块导入ReportGenerator类,用于报告生成
from llm import LLM # 从llm模块导入LLM类,可能用于语言模型相关操作
from subscription_manager import SubscriptionManager # 从subscription_manager模块导入SubscriptionManager类,管理订阅
Expand All @@ -12,7 +11,6 @@
def main():
config = Config() # 创建配置实例
github_client = GitHubClient(config.github_token) # 创建GitHub客户端实例
notifier = Notifier(config.notification_settings) # 创建通知器实例
llm = LLM() # 创建语言模型实例
report_generator = ReportGenerator(llm) # 创建报告生成器实例
subscription_manager = SubscriptionManager(config.subscriptions_file) # 创建订阅管理器实例
Expand Down
22 changes: 13 additions & 9 deletions src/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,20 @@ def __init__(self):
self.load_config()

def load_config(self):
# 从环境变量获取GitHub Token
self.github_token = os.getenv('GITHUB_TOKEN')

# 尝试从环境变量获取配置或使用 config.json 文件中的配置作为回退
with open('config.json', 'r') as f:
config = json.load(f)

# 如果环境变量中没有GitHub Token,则从配置文件中读取
if not self.github_token:
self.github_token = config.get('github_token')

self.notification_settings = config.get('notification_settings')
# 使用环境变量或配置文件的 GitHub Token
self.github_token = os.getenv('GITHUB_TOKEN', config.get('github_token'))

# 初始化电子邮件设置
self.email = config.get('email', {})
# 使用环境变量或配置文件中的电子邮件密码
self.email['password'] = os.getenv('EMAIL_PASSWORD', self.email.get('password', ''))

self.subscriptions_file = config.get('subscriptions_file')
self.update_interval = config.get('update_interval', 24 * 60 * 60) # 默认24小时
# 默认每天执行
self.freq_days = config.get('github_progress_frequency_days', 1)
# 默认早上8点更新 (操作系统默认时区是 UTC +0,08点刚好对应北京时间凌晨12点)
self.exec_time = config.get('github_progress_execution_time', "08:00")
Loading

0 comments on commit 9e0b241

Please sign in to comment.